Backend bug-fix plan (October 2026): security first, then correctness

This plan fixes the backend defects confirmed by the architecture review (architecture-review-2026-10.md, items V5–V9 and V14–V19, plus Appendices B, C, E and G). Security comes first (#3972), then correctness (#3973–#3975 and

3978–#3980). It is planning only; nothing starts until Chris approves and answers the decisions

in §9.

Every item was re-checked against main@85e6facf7 (3 Oct 2026, 20:27) by reading the code. Line numbers below are from that commit. Markers: VERIFIED (I read the code path), CORRECTED (the review was wrong or out of date; the note says how), NOT VERIFIED (the note says why).

Already fixed and merged; listed only for completeness: - #3967 (invitation token, V1); - #3968 (change-password gate, V2); - #3970 (per-source password budget, V3); - #3964 (owner-only ownership transfer, V12).

The Swagger client secret (V4, #3992, draft #3966) belongs to the authentication migration session.

1. Scope

1a. Security (#3972)

ID	Problem	Impact	Evidence (main@85e6facf7)	Status
S1	No rate limiter anywhere in the backend	Every anonymous and credential endpoint can be hammered without limit	0 matches for AddRateLimiter/EnableRateLimiting/RequireRateLimiting in src/**/*.cs. Identity has hand-rolled per-feature throttles only (RecoveryEmailOutbox.cs:114, RegistrationNoticeThrottle.cs, PasswordAttemptStore.cs)	VERIFIED
S2	Anonymous contact and support forms send a confirmation to any address, using a caller-chosen Name and Subject	Open relay for phishing and spam, from SyRF's sending domain and on its SES quota	ApplicationController.cs:199-217 and :219-245, both [AllowAnonymous], call SendContactUsConfirmation(dto.Email, dto.Name, dto.Subject)	VERIFIED (V16)
S3	Email-verification codes can be brute-forced	An account holder can verify an address they don't own, then claim that address's pending invitations	Investigator.cs:121-152 (each resend appends a code), :272-277 (Verify matches any issued code), :280-288 (6 digits, 24 h); AccountController.cs:478-529 (no attempt count; a failed guess is never saved); NewEmailAddressAssociatedWithInvestigatorHandler.cs:20 → ClaimInvitationsForEmailAsync	VERIFIED (V17)
S3b	profile/tickets/email sends a code to any address, and also answers "already in use"	A signed-in mail cannon, and an address-enumeration oracle	AccountController.cs:478-505 (EmailAlreadyInUse at :489-493)	VERIFIED (new)
S3c	profile/email/confirm with an address that isn't pending returns 500	Noise; the error is an unhandled exception	Investigator.cs:140-142 throws InvalidOperationException	VERIFIED (new)
S4	Anonymous GET api/projects/keywords aggregates the keywords of every project	Private projects' keywords leak. The endpoint is an unauthenticated full-collection scan, and one null keyword makes ToDictionary throw, a 500 for everyone	ProjectController.cs:92-99 → ProjectRepository.cs:123-128 (no IsPublic filter). The only web caller is project-detail.effects.ts:781, which runs signed in	VERIFIED (V15)
S5	Anonymous email-lookup returns investigator GUIDs and whether an Identity user exists	Account enumeration plus identifier disclosure	AccountController.cs:160-172. Its intended read:investigator.ids policy names an "actions" scheme that is never registered (Program.cs:403-411), and the endpoint doesn't use that policy anyway. The only consumer is the Auth0 Action docs/auth0-actions/transform-token.js:66-78, which calls without a token	VERIFIED. CORRECTED: the deployed Action reads data.auth0Ids, a field the current response doesn't have (AccountCheckResponse.cs:6-16). Whether the deployed Action matches the copy in the repo is NOT VERIFIED (it lives in the Auth0 tenant)
S6	The API's anonymous signup proxy and the BFF login endpoints have no limit	Registration abuse through the API host	AccountController.cs:639-642 ([AllowAnonymous] signup proxy); BFF and e-auth controllers are class-level [AllowAnonymous]	VERIFIED
I1	Identity POST /api/account/signup and /Account/Register have no request limiter (G 5.3)	Each request does PBKDF2 hashing plus up to 3 pmInvestigator scans. A per-address notice throttle exists, but no per-client limit	AccountApiController.cs:75-76; Register.cshtml.cs:47. RegistrationNoticeThrottle bounds the mail sent to one address only	VERIFIED
I2	Credential changes need only the session cookie: adding or removing a passkey, disabling 2FA, resetting the authenticator (G 5.6)	A stolen session can entrench itself	Passkeys.cshtml.cs (class [Authorize], OnPostRegisterAsync at :89, no step-up); TwoFactor.cshtml.cs OnPostDisableAsync/OnPostResetAsync. The step-up infrastructure exists (Manage/StepUp.cshtml.cs) but only ExternalLogins.cshtml.cs uses it	VERIFIED
I3	Verification links are built from the Host header (V19)	Low exploitability: forwarded headers only honour the issuer host	Register.cshtml.cs:268, ExternalLogin.cshtml.cs:385	VERIFIED (V19)
I4	No index on NormalizedEmail/NormalizedUserName; no OpenIddict indexes (G 5.8)	Full scans on every sign-in and registration lookup; concurrent seeding can create duplicate clients	IdentityUserIndexInitializer.cs creates only SyrfUserId, Passkeys.CredentialId and Logins.*	CORRECTED: some indexes exist. The NormalizedEmail/NormalizedUserName and OpenIddict gaps are VERIFIED by grep; token pruning NOT VERIFIED
I5	Google first sign-in: an exception after CommitClaimAsync deletes the bound account (G 5.5, a #3935 relative)	The address is left permanently RefusedPreviouslyMapped	ExternalLogin.cshtml.cs:369-377 commits; the generic catch (Exception) at :407-411 runs CompensateExternalRegistrationAsync → RegistrationCompensation.DeleteAccountAsync (:424-435)	VERIFIED (by reading the code)
I6	PUT/DELETE api/account/profile are authorised with the OpenIddict server scheme (G 5.9)	Reported to always return 500 (ID0002)	AccountApiController.cs:511,641 use OpenIddictServerAspNetCoreDefaults, while AdminApiController.cs:23 uses the validation scheme	VERIFIED attribute; the 500 is NOT VERIFIED by a run

1b. Correctness

ID	Problem	Impact	Evidence	Issue / status
C1	Domain events are fire-and-forget	Handler failures vanish (invitation email, invitation claim, threshold recalculation). Handlers outlive the request, so ordering is lost and the scope they run in is already disposed	MongoUnitOfWorkBase.cs:730-741: the void DispatchEvents discards _eventManager.DispatchAsync((dynamic)…) (the dynamic call hides CS4014). EventManager.cs:21-31 has no try/catch. 13 IHandles<> implementations; about 35 DispatchEvents call sites through the sync IUnitOfWork.DispatchEvents (IUnitOfWork.cs:187)	#3973, VERIFIED (V5)
C2	Liveness runs every health check; readiness runs only ready-tagged ones	A dependency blip restarts pods instead of taking them out of rotation	SyrfHealthCheckExtensions.cs:34-43 (/health/live has no predicate). The API registers change-streams tagged live (Program.cs:167-170). Quartz's sql and DbContext checks have no tag (Quartz/Program.cs:58-59). API, PM and Quartz charts: liveness /health/live, timeoutSeconds: 1 (api/.chart/values.yaml:283-289, PM :247-253, Quartz :151-157). Identity already does it correctly (Identity/Program.cs:450-462)	#3974, VERIFIED (V9)
C3	Change-stream health never recovers on a quiet collection, and the retry budget is cumulative	Once 5 consecutive failures are counted, the check stays Unhealthy until an event arrives. Together with C2, pods are liveness-killed after the database has recovered. After 15 failures without any event in between, the shared stream terminates for good	MongoContext.cs:302-309: success is recorded only in the .Do on an event (:396-404), and attempt resets there too; ChangeStreamHealthTracker.cs (5-failure threshold)	#3974, VERIFIED (B2)
C3b	One subscriber exception ends the shared change stream	Live updates stop on that pod, while health still reports Healthy (a non-retryable error never calls RecordFailure)	MongoContext.cs:263 calls obs.OnNext inline; shouldRetry (:306) excludes non-Mongo exceptions	#3974. Mechanism VERIFIED; the "terminated Publish subject" consequence is NOT VERIFIED (needs a test)
C4	The Lamar convention scan overrides the factory-registered singletons	Every resolve of IRuntimeFeatureFlagProvider builds a new provider, whose constructor resets the shared FeatureFlags to deployed values. Non-production runtime overrides (staging and preview admin UI) are undone for interface consumers. IBffSessionTokenValidator loses its 5 s timeout and its pooled client in every environment	Program.cs:193-198 and :320-321; scan at :611-615; RuntimeFeatureFlagProvider.cs:29-34,83	#3975, VERIFIED (V8, reproduced by the review on Lamar 15.0.1)
C5	Forbid(StudyNotInProjectMessage) passes a message as a scheme name	500 instead of a refusal, whenever ReviewEligibilityPolicy is off (the default)	ReviewController.cs:101,315,477,605,677,734; Refuse at :1705-1706	#3978, VERIFIED (V14)
C6	PageSize = 0 means no limit; a negative FromRow gives a 500	An unbounded study query on request	StudyController.cs:64-89 → StudyRepository.cs:1286-1287 (Skip/Limit); IncomingTableParamsDto.cs:9-34 has no bounds	#3978, VERIFIED
C7	Anonymous RetrieveInvitation reads CurrentUserId	500 for anonymous callers, on an [AllowAnonymous] endpoint	ProjectController.cs:586-593; SyrfBaseController.cs:9-10 throws	#3978, VERIFIED
C8	Data-export job hub subscriptions are never cleaned up on disconnect	Subscription leak per connection	NotificationHub.cs:112-131 unsubscribes projects and studies, but not _dataExportJobSubscriptionManager (field at :78)	#3978, VERIFIED
C9	API and Quartz exit 0 after a fatal startup exception	Kubernetes and CI see a fatal crash as a clean exit	API Program.cs:815-822, Quartz Program.cs:118-125: catch { Log.Fatal } with no exit code	#3978, VERIFIED
C10	SignalR EnableDetailedErrors = true unconditionally	Exception messages reach clients in production	Program.cs:379-381	#3978, VERIFIED
C11	Developer exception page on Staging	Stack traces on internet-facing staging	Program.cs:767-769; staging runs ASPNETCORE_ENVIRONMENT: "Staging" (cluster-gitops/syrf/environments/staging/api/values.yaml:17)	#3978, VERIFIED
C12	AuthorizationHandler writes a 404 body itself, then the policy fails	"Response already started" exception; the client gets a truncated, non-JSON body with a literal {projectId}	AuthorizationHandler.cs:153-162 and :349-355	#3978, VERIFIED (reproduced by the review)
C13	The Study Library session filter hard-codes 2 reviewers	Single- and three-reviewer stages show wrong Completed and In-progress results	StudyRepository.cs:1653-1725 (>= 2/< 2 at :1665,1669,1712,1725). The eligibility filters already use scope.SessionCountTarget (ReviewEligibilityPoolFilters.cs:84,108,223) and Filters.HasMinCompletedStageSessions	#3979, VERIFIED (V18)
C14	ValueObject.Equals reads only the runtime type's fields, so it ignores the fields of a base class	Any two ProjectAgreementThresholds (and StudyAgreementMeasures) are equal. The guards at Project.cs:180,192 and ProjectManagementService.cs:1247 never trigger	ValueObject.cs:43-71 (GetFields on the runtime type) vs GetHashCode, which walks the hierarchy (:73-87); AgreementMeasure.cs:22-23 declares the fields	#3980, VERIFIED (V6)
C15	The agreement threshold is lost in MassTransit System.Text.Json deserialization	PM receives (null, 0). The consumer recalculates inclusion with it (UpdateStudyScreeningStatsConsumer.cs:94-95) and writes a junk (0,null) inclusion element	AgreementMeasure.cs:22-23 (protected setters, no [JsonConstructor])	#3980, VERIFIED (V7: repro, plus a production sample: 1 junk element in 3,000 studies)

A hidden coupling in C14/C15 (new): today CompleteInclusionInfoCalculation (Project.cs:187-196) passes only because equality is broken. The message carries (null,0), and broken equality says it matches the active job. If equality is fixed before serialization, every inclusion recalculation throws and the project latches in CalculatingInclusionInfo. Separately, StartInclusionInfoCalculation (:178-181) calls .Max() on a filtered sequence. That sequence becomes empty, and .Max() then throws, once equality really filters. Both must be fixed in one PR, serialization first.

2. MVP boundary, exclusions, flag decision

MVP (R0 + R1): - correct client IP; - a configurable rate limiter with an Observe mode and a kill switch; - the email relay closed; - verification codes capped; - anonymous keyword, email-lookup and invitation leaks closed; - liveness that no longer restarts pods for dependency blips.

R2–R3 deliver the remaining correctness fixes. Identity items (R-ID) run on the migration session's schedule.

Out of scope, and where each item is tracked:

Item	Tracked in
Transactional outbox, global MassTransit retry/redelivery, contract round-trip tests for every message	#3984 (persistence and messaging plan)
Non-upsert saves, Audit.Version bumps on direct Study writes, the RepositoryCache mutate-then-409 pattern, isolated reads	#3985 (persistence and messaging plan)
Batched saves dropping events on cancellation (B8), import-progress rollback (PM B7)	Persistence and messaging plan
Swagger client secret (V4)	#3992 / #3966, migration session
Retiring email-lookup and the Auth0 Actions	#3988, after Auth0 retirement (migration session)
ValueObject → records everywhere; primitive-only message contracts	#3988 / #3984 follow-up (see D7)
Explicit DI modules instead of convention scans (V7 composition item)	Architecture roadmap; this plan only pins the two broken singletons
S3 signature rate limit	#806

Flag decision:

PR	Flagged?	Mechanism and reason
PR-1 client IP	No	It restores correct RemoteIpAddress. Verified with an Observe-only log line before PR-2 enforces anything
PR-2 rate limiter	Yes	RateLimiting:Mode = Off / Observe / Enforce, with per-policy limits, as Helm values through env-mapping.yaml. It changes observable behaviour (429s). Off is the kill switch. Production starts in Observe (D2). Not a runtime flag: runtime overrides are non-production only (.claude/rules/feature-flags.md)
PR-3 anonymous surface	Partly	Contact-form confirmation and keywords: unflagged security fixes. Email-lookup: no code change unless D4
PR-4 verification codes	No	Security fix. Existing codes stay valid until expiry, so nothing breaks mid-flow
PR-5 health	No (GitOps-reversible)	Probe semantics change, but it is a chart and library change with a one-line revert. Proven on staging first
PR-6 domain events (observe)	No	Adds logging and metrics only; dispatch timing is unchanged
PR-6b domain events (await)	Yes	DomainEvents:DispatchMode = FireAndForget / Await. It changes request latency and failure semantics. Pending D6
PR-7 Lamar	No	Restores the singletons as documented. A container test pins it
PR-8 / PR-9 API defects	No	Each turns a 500 into the intended status. C10/C11 are environment hardening
PR-10 study filter	No	Restores intended filtering. Docs note (D8)
PR-11 value objects	No for code. The data cleanup is a separate, Chris-approved migration	The code fix is a hidden-coupling bug fix that must ship atomically

3. Common acceptance criteria (every PR)

#	Criterion	Verification
C1	The regression test is written first and fails on main, then passes	Red and green output in the PR body
C2	Only focused test projects run locally, niced (nice dotnet test <one project> --filter …); never the full solution on this host	Commands in the PR body; CI Test .NET green
C3	.claude/rules/* invariants for touched areas hold (identity, feature flags, repository cache)	PR review
C4	Docs ride in the same PR: /docs/ for engineers, /user-guide/ where users see a change (429 messages, the contact form)	PR review; validate-docs --skip-indexes
C5	The PR body states the flag decision (§2) and links the issue and finding IDs	PR review
C6	Reviews settled on the head (pr-review-settled.sh exit 0), the bot summary has no request changes, CI green	Settled-gate output
C7	Staging rollout check after merge for PRs marked ★; production rollout is a separate Chris-approved step	Staging check comment on the issue
C8	No new [AllowAnonymous] endpoint and no new unlimited anonymous endpoint (enforced by the PR-2 guard test from then on)	Guard test

4. Releases and pull requests

PROPOSAL marks every numeric value that needs Chris's confirmation.

R0: rate-limiting foundation (sequential: PR-1 → PR-2)

PR-1: trustworthy client IP (S1 prerequisite; effort S) ★

The API's ProxySettings:ProxyUsed defaults to false (appsettings.json:28-30). CORRECTED (orchestrator check): staging overrides it (cluster-gitops syrf/environments/staging/api/values.yaml:67-75, ProxyUsed=true, ForwardLimit=2, known network 10.96.0.0/14). Production has no override. So production's API may see the ingress pod's address for every caller, unless the ingress preserves source IPs. Per-IP partitioning would then turn into one global bucket, which is a self-inflicted lock-out. NOT VERIFIED live; this PR's first step is to measure, and then to carry staging's settings across to production through a Chris-approved production change.

Files: - Program.cs (forwarded-headers block :216-231, :762-765); - env-mapping.yaml / API values (the ProxySettings known networks); - a cluster-gitops values change for staging.

#	Acceptance criterion	Verification
1.1	Request through staging ingress → the logged client IP equals the caller's public IP, not an RFC1918 address	Staging rollout check (a temporary diagnostic log line, removed in PR-2)
1.2	A spoofed X-Forwarded-For from outside the known networks → ignored	Integration (WebApplicationFactory with ForwardedHeadersOptions)
1.3	ProxyUsed=false (local, E2E) → behaviour unchanged	Existing E2E smoke

PR-2: API rate limiter with Observe/Enforce and kill switch (S1, S6; effort M) ★

Files: - new RateLimiting/ folder: options, policy table, middleware, guard test; - Program.cs: one registration line and one app.Use… line, placed after UseForwardedHeaders and authentication; - env-mapping.yaml and the generated values.

No controller edits. Policies attach through an IApplicationModelConvention keyed by controller and action, so PR-3, PR-4, PR-8 and PR-9 can edit those controllers in parallel.

Approach: 1. Build on System.Threading.RateLimiting partitioned limiters, in a thin middleware rather than the stock UseRateLimiter, so that Observe logs and counts would-be rejections without rejecting. 2. Partition anonymous traffic by client IP (IPv6 normalised to /64) and authenticated traffic by SyRF user ID. 3. In-memory, per pod. A distributed store is deferred (D3).

Policies (all limits PROPOSAL):

Policy	Endpoints	Limit
contact	submit-general-inquiry, submit-support-request	3 per 10 min per IP
anon-lookup	email-lookup, email-lookup-p, keywords, invitations/{token}	20/min per IP
signup	API account/signup proxy	5 per hour per IP
auth	BFF/e-auth login and callback	30/min per IP
email-code	profile/tickets/email	5 per hour per user
email-code	profile/email/confirm	10 per hour per user
global-anon	Any other anonymous request	300/min per IP

Rejections return 429 + Retry-After as ProblemDetails, and increment a syrf.ratelimit.rejected{policy,mode} metric.

#	Acceptance criterion	Verification
2.1	Mode=Enforce, a 4th contact submission within 10 min from one IP → 429 with Retry-After; no email sent	Integration (fake email service)
2.2	Mode=Observe, the same burst → all succeed; one structured log and one metric per would-be rejection	Integration
2.3	Mode=Off → the middleware is not in the pipeline; behaviour identical to main	Integration + startup test
2.4	Two IPs → independent buckets; two users behind one IP on authenticated policies → independent buckets	Unit (partition key)
2.5	Every [AllowAnonymous] action in the API assembly → mapped to a named policy, or the guard test fails	Reflection guard test
2.6	Limits changed in values → applied on restart, with no code change	Options binding test
2.7	Hermetic E2E smoke with Mode=Enforce and default limits → no 429s in the normal journeys	E2E (e2e/run-local.sh --spec smoke)
2.8	Staging in Enforce for 7 days (PROPOSAL) → zero rejections of real users in logs; production deploys in Observe	Staging rollout check; production is a separate Chris-approved step

R1: close the anonymous and verification holes (parallel after PR-2's options exist; disjoint files)

PR-3: anonymous surface (S2, S4, C7; effort M)

Files: - ApplicationController.cs; - ProjectController.cs (GetKeywords, RetrieveInvitation only); - ProjectRepository.cs (GetKeywords, GetProjectWithPendingInvitationByTokenAsync); - the web contact and support components (429 message only).

Approach: 1. Contact forms: still alert the helpdesk, but stop sending the confirmation to a caller-supplied address (D4a; alternatively send it only when the caller is signed in and the address is one of theirs). Never echo the caller's Name or Subject into outbound mail. Cap field lengths. 2. Keywords: require authentication. Restrict to public projects plus the caller's projects, and skip null keywords. $unwind/$group server-side instead of ToList() over the collection. 3. RetrieveInvitation: take a nullable caller ID; anonymous → 401 (D4b). Keep the token-shaped 404 for an unknown token.

#	Acceptance criterion	Verification
3.1	Anonymous contact submission with any Email → the helpdesk alert is sent; nothing is sent to Email	Controller test with a fake email service
3.2	Name/Subject containing a URL or newlines → never appears in any outbound mail to a third party	Unit
3.3	Anonymous GET projects/keywords → 401	Integration
3.4	Signed-in user; a private project they aren't a member of has keyword K → K is absent	Integration (Testcontainers Mongo)
3.5	A project with a null keyword → 200, no exception	Integration
3.6	Anonymous GET projects/invitations/{token} → 401 (or the D4b outcome), never 500	Integration
3.7	The web contact form gets a 429 → it shows "too many requests, try again later"	Component spec
3.8	The user guide's contact-form page states that no confirmation email is sent (if D4a)	Docs

PR-4: email-verification hardening (S3, S3b, S3c, S5 option; effort M)

Files: Investigator.cs (NewEmailVerifier, VerificationCode), AccountController.cs (profile email endpoints; email-lookup only if D5 chooses it), and their tests.

Approach: 1. At most 1 live code per verifier: a resend replaces the code instead of appending one. 2. Persist a failed-attempt count. 5 failures (PROPOSAL) invalidate the verifier, and the user must request a new code. 3. Expiry 1 h (PROPOSAL, down from 24 h). 4. Constant-time compare. 5. The failure path saves the investigator through the existing SaveCurrentInvestigatorAsync concurrency handling. 6. An unknown pending address → 422 Invalid, not 500. 7. tickets/email returns the same 200 whether or not the address is in use; an in-use address gets a "this address already has an account" mail instead of a code (D5a).

Stored data: old documents with several codes stay readable. Verify keeps accepting any still-unexpired legacy code until it expires, so no migration is needed.

#	Acceptance criterion	Verification
4.1	Resend twice → only the newest code verifies	Unit (domain)
4.2	5 wrong guesses → the 6th guess, even with the right code, returns Invalid, and the verifier is gone	Unit + controller test
4.3	Wrong guesses → the count survives a reload (persisted)	Integration (Testcontainers Mongo)
4.4	Code older than 1 h → Expired	Unit with a fake clock
4.5	Confirm with an address that isn't pending → 422, not 500	Controller test
4.6	tickets/email for an address already in use → same status and body as for a free address	Controller test
4.7	Investigator documents with legacy multi-code verifiers → still deserialize, and the newest unexpired code verifies	Integration (BSON fixture)

R2: correctness, wave 1 (all parallel; disjoint files except one line each in API Program.cs)

PR-5: health probes and change-stream recovery (C2, C3, C3b; #3974; effort M) ★

Files: - SyrfHealthCheckExtensions.cs; - MongoContext.cs (retry, success recording, subscriber isolation); - ChangeStreamHealthTracker.cs; - API Program.cs:170 (tag ready); - Quartz Program.cs:58-59 (tag ready); - API, PM and Quartz .chart/values.yaml (readiness failureThreshold: 3, liveness timeoutSeconds: 3, both PROPOSAL).

Approach: 1. /health/live gets the predicate live, keeping the version JSON body that environment.effects.ts:178 reads. 2. Dependencies (change streams, SQL, MassTransit) report only to ready. Check in the PR whether MassTransit 8.4's own checks carry ready by default (NOT VERIFIED). 3. Record a success when the cursor opens, and reset attempt there. 4. Retry without a lifetime cap (the 180 s backoff ceiling is kept). 5. Wrap each subscriber so its exception is logged and doesn't reach the cursor loop. 6. A terminated stream marks itself Unhealthy and is rebuilt on the next subscribe.

#	Acceptance criterion	Verification
5.1	A ready-tagged check fails → /health/live 200, /health/ready 503	Integration (TestServer)
5.2	/health/live body → still includes the version fields	Integration; web environment.effects spec unchanged
5.3	Change stream fails 6 times, then reopens with no events → health returns to Healthy within one poll	Unit (Rx TestScheduler)
5.4	20 failures spread over reconnects with no events → the stream is still retrying	Unit (TestScheduler)
5.5	One subscriber throws on an event → other subscribers still receive later events	Integration (Testcontainers Mongo replica set)
5.6	Quartz SQL unavailable → Quartz pod becomes unready and is not restarted	Staging rollout check (scale the SQL connection off briefly; staging is not mission critical)
5.7	Rendered charts → liveness and readiness paths unchanged; thresholds as proposed	helm template diff in the PR

PR-6: domain events, observed dispatch (C1 part 1; #3973; effort S)

Files: EventManager.cs, MongoUnitOfWorkBase.cs (DispatchEvents).

Approach: dispatch the batch on one task chain, so handlers run sequentially in chronological order. Catch per handler, log with event type, handler and aggregate ID, and increment syrf.domain_event.handler_failed. Attach the chain to a tracked registry so that shutdown drains it (bounded at 10 s, PROPOSAL). The request still doesn't wait, so timing is unchanged.

#	Acceptance criterion	Verification
6.1	A handler throws → one error log with event type and handler name, metric +1, and later handlers for the same batch still run	Unit (Lamar test container)
6.2	Two events in one batch → handled in DateTimeEventOccurred order, never concurrently	Unit
6.3	Host shutdown with handlers in flight → drained or logged as abandoned after the bound	Unit
6.4	No DispatchAsync Task is discarded unobserved	Analyzer/grep guard test over MongoUnitOfWorkBase.cs

PR-6b: domain events, awaited (C1 part 2; flagged; effort L; pending D6)

Add IUnitOfWork.DispatchEventsAsync, and await it on the async save paths when DomainEvents:DispatchMode=Await. Handler failures are logged and not rethrown into the request (the save has already committed). This touches about 35 call sites and overlaps the outbox design (#3984), so D6 decides whether it ships here or folds into the outbox.

PR-7: Lamar singleton identity (C4; #3975; effort S)

Files: API Program.cs:611-615 (exclude the two types from the scan, or re-assert the singletons after it), plus a new container test.

#	Acceptance criterion	Verification
7.1	Resolve IRuntimeFeatureFlagProvider twice and resolve RuntimeFeatureFlagProvider → the same instance	Container test on the real API registry
7.2	A runtime override is applied, then a controller resolves the interface → the override is still visible	Integration
7.3	IBffSessionTokenValidator → typed-client instance with a 5 s timeout	Container test
7.4	Every interface the API registers with a factory singleton → not overridden by the scan	Generic guard test over the service collection

PR-8: API host hygiene (C9, C10, C11; #3978; effort S) ★

Files: API Program.cs (:381, :767-769, :815-822), Quartz Program.cs:118-125.

#	Acceptance criterion	Verification
8.1	Fatal startup exception → process exit code 1 (API and Quartz)	Process test (dotnet host with a broken config, run in CI only)
8.2	Production and Staging → SignalR detailed errors off; Development → on	Options test per environment
8.3	Staging unhandled exception → ProblemDetails without a stack trace	Integration (environment Staging)
8.4	Staging pod with a bad config → CrashLoopBackOff (not Completed)	Staging rollout check

PR-9: API controller defects (C5, C6, C8, C12; #3978; effort M)

Files: - ReviewController.cs (6 call sites); - IncomingTableParamsDto.cs + StudyController.cs (validation only; StudyRepository untouched); - NotificationHub.cs:112-131; - AuthorizationHandler.cs + a new IAuthorizationMiddlewareResultHandler.

#	Acceptance criterion	Verification
9.1	Study not in project, ReviewEligibilityPolicy off → 404 ProblemDetails (matching the flag-on ReviewWriteRefusal.NotFound mapping), never 500	Controller test per endpoint
9.2	PageSize 0, negative or above 500 (PROPOSAL) → 400; negative FromRow → 400	Controller test
9.3	The web Study Library's existing page sizes → unchanged results	Existing web specs + E2E Library smoke
9.4	Connection with export-job subscriptions disconnects → its subscriptions are removed	Hub unit test
9.5	Routed project doesn't exist → a complete 404 ProblemDetails naming the ID; no "response already started" log	Integration (WebApplicationFactory)
9.6	Web project guard on a missing project → still shows the not-found UI	E2E (hermetic)

R3: correctness, wave 2 (parallel with R2; disjoint files)

PR-10: Study Library session filter uses the stage target (C13; #3979; effort M)

Files: StudyRepository.cs (GetSessionFilter, CreateMongoFindQuery signatures), IStudyRepository.cs, and the callers that pass the stage's SessionCountTarget. Reuse Filters.HasMinCompletedStageSessions.

#	Acceptance criterion	Verification
10.1	Stage target 1; study with 1 completed session → appears under Completed, not In progress	Integration (Testcontainers Mongo)
10.2	Stage target 3; 2 completed → In progress	Integration
10.3	Target 2 → results identical to main (golden query test)	Integration
10.4	"Not yet started for me" / "unavailable" filters → use the same target	Integration
10.5	Study Library docs describe Completed as "reached the stage's reviewer target"	Docs

PR-11: agreement threshold serialization and ValueObject equality (C14, C15; #3980; effort M)

Files: AgreementMeasure.cs, ValueObject.cs, Project.cs (StartInclusionInfoCalculation .Max guard only), plus tests in SharedKernel.Tests and PM.Core.Tests.

D7 recommendation: fix ValueObject, not records. 28 types derive from ValueObject<T>. They use BSON class maps, private setters and ISupportInitialize, and records would compare List<> members by reference exactly as today, so records change risk without fixing more. Only AgreementMeasure's two subclasses inherit fields from a base, so the equality change is confined to them and to the value objects that contain them (ScreeningInfo, the inclusion jobs).

Order inside the PR: 1. [JsonConstructor] (or [JsonInclude] on the protected setters), with a round-trip test that uses MassTransit's own STJ options. Messages already in flight carry the values in JSON, because only deserialization dropped them, so no contract change is needed. 2. ValueObject.Equals walks the hierarchy, like GetHashCode. 3. .Max → a guarded DefaultIfEmpty. 4. List every equality call site on the affected types in the PR body.

#	Acceptance criterion	Verification
11.1	ProjectAgreementThreshold(0.333, 2) through MassTransit's STJ serializer → arrives as (0.333, 2)	Contract round-trip test
11.2	(0.333,2) vs (null,2) → not equal; equal values → equal, with matching hash codes	Unit
11.3	Inclusion recalculation end to end with a threshold change → completes, the job closes, no (0,null) element is written	Integration (MassTransit test harness + Testcontainers Mongo)
11.4	StartInclusionInfoCalculation with no completed job for the current threshold → Started, no exception	Unit
11.5	Stored pmProject and pmStudy documents → deserialize unchanged (BSON is unaffected by the STJ attributes)	Integration (BSON fixture)

Data note (#3980), read-only analysis first: - Production syrftest.pmStudy: 1 junk (0,null) inclusion element in a $sample of 3,000 (V7). A full count timed out and was deliberately not retried. - Next step, with no writes: 1. Count on the preview cluster's staging copy. 2. Then, with Chris's approval, run a production count in a quiet window, batched by project. 3. Report the projects affected. - Any cleanup is a separate Chris-approved migration: - dry-run first, listing the _ids; - $pull the elements matching {NumberScreened: 0, AbsoluteAgreementRatio: null}, plus $inc Audit.Version, because direct writes must bump the version (see #3985); - a pre-image export as rollback; CSUUID filters; - coordination with the materialised-statistics session, because pmStudy writes feed its change stream and folds.

R-ID: Identity items (coordinated with the authentication migration session; not planned on its behalf)

Identity is not the production IdP yet; the S09 staging switch is next. The migration session owns these files and decides the schedule. This plan supplies specs and acceptance criteria only.

Status and slots (reply from the authentication migration session, 2026-10-05). The session accepted every item and records each one as an M005 slice. The "Slot (auth session)" column is the session's decision and replaces this plan's original recommendation, which is kept in the last column for the record.

PR	Items	Files	Status (2026-10-05)	Slot (auth session)	Original recommendation
ID-1	Request limiter on signup, register, login, forgot-password and resend (I1), using the same Observe/Enforce options shape as PR-2	Identity Program.cs + new RateLimiting/	Accepted	After S09, before S10. It depends on backend PR-1 (client IP), without which Observe mode would record one shared ingress address	Before S09
ID-2	Profile endpoints authorised with the validation scheme (I6)	AccountApiController.cs:511,641	Accepted	Any time. The PR starts with a failing test that proves the 500	Any time; first confirm the 500 with a test
ID-3	One IdentityLinkBuilder from the issuer (I3/V19)	Register.cshtml.cs:268, ExternalLogin.cshtml.cs:385, VerificationLinkFactory.cs	Accepted; partly fixed: #4030 fixed part of the redirect work, and #4040 holds the leftovers	After S09, bundled with the redirect work (#4040)	After S09, or bundled with ID-4
ID-4	Google first sign-in: no deletion after a committed claim (I5)	ExternalLogin.cshtml.cs:365-411, RegistrationCompensation.cs	Accepted; in progress now (migration session). The PR keeps the account after a committed claim and pins the pre-commit compensation with tests	Now, before S09	Fix before S09
ID-5	Indexes on NormalizedEmail/NormalizedUserName and OpenIddict collections, plus pruning (I4)	IdentityUserIndexInitializer.cs	Accepted	Before S10. Non-unique indexes first; the unique index only after a duplicate check on both identity databases (staging and production)	Before the production cutover (S10)
ID-6	Step-up for passkey add/remove, 2FA disable/reset, and passkey removal on password reset (I2)	Passkeys.cshtml.cs, TwoFactor.cshtml.cs	Accepted	Between S09 and S10	After S09, before S10
ID-7	Provision PM only after email verification (the second half of G 5.3)	Registration services	Accepted as backlog	Backlog. A design change that needs Chris's decision before it is scheduled	Migration session's backlog

Each ID PR uses the same acceptance-criterion pattern as PR-2 and PR-4. For example, ID-1: - Mode=Enforce and the 6^th signup per hour from one IP (PROPOSAL) → 429, with the response shape unchanged otherwise (still non-enumerating); - Observe → logged only; - the uniform-202 test AnonymousSurface_IsExactlyTheAuditedNonEnumeratingSet still passes.

5. Order and critical path

flowchart LR
  PR1[PR-1 client IP] --> PR2[PR-2 rate limiter]
  PR2 --> PR3[PR-3 anon surface]
  PR2 --> PR4[PR-4 verification codes]
  PR5[PR-5 health]
  PR6[PR-6 events observed] --> PR6b[PR-6b events awaited, if D6]
  PR7[PR-7 Lamar]
  PR8[PR-8 host hygiene]
  PR9[PR-9 controller defects]
  PR10[PR-10 study filter]
  PR11[PR-11 value objects] --> MIG[data cleanup, Chris-approved]

• Critical path: PR-1 → PR-2 (staging Observe → Enforce) → PR-3 / PR-4.
• PR-3 and PR-4 can start coding in parallel with PR-2 (the policy names are fixed in this plan); they merge after it.
• Start immediately, in parallel, one worker per worktree: PR-1, PR-5, PR-6, PR-7, PR-10, PR-11. Their files are disjoint.
• Shared file: API Program.cs is touched by PR-1, PR-2, PR-5 (one line), PR-7 and PR-8, each in a different region. Merge order is PR-1 → PR-7 → PR-5 → PR-8 → PR-2, rebasing each (trivial). Bundling PR-7 and PR-8 is possible if Chris prefers fewer PRs.
• PR-9 is independent, but touches ReviewController.cs and NotificationHub.cs, which open PRs also touch (§8).

6. Issues

Issue	PRs
#3972	PR-1, PR-2, PR-3, PR-4; ID-1…ID-7 cross-linked for the migration session
#3973	PR-6, PR-6b
#3974	PR-5
#3975	PR-7
#3978	PR-8, PR-9, plus C7 in PR-3
#3979	PR-10
#3980	PR-11, plus a separate data-migration issue

On approval: - split #3972 into a security epic with child issues; - create the data-migration issue; - each issue copies its acceptance-criteria table from this plan.

7. Risks

Risk	Mitigation
Per-IP limits collapse to one bucket behind the ingress, locking everyone out	PR-1 first, proven on staging; PR-2 ships in Observe; Off kill switch through GitOps
Universities behind NAT share one IP	Authenticated policies partition by user; generous anonymous limits (PROPOSAL); Observe data sets the final values
In-memory limits are per pod (limit × replicas)	Acceptable for abuse damping. A distributed limiter (the shared Valkey from S08A) is D3
Fixing equality before serialization latches inclusion jobs	One PR, serialization first, end-to-end test 11.3
Liveness no longer restarts a wedged process	Change-stream self-healing (5.3–5.5) plus readiness removes it from traffic; PM keeps its single replica, so watch its readiness alerts
Awaiting events turns handler latency into request latency	PR-6b is flagged and decided separately (D6); PR-6 first gives failure visibility
The Auth0 Action breaks if email-lookup changes	No functional change to email-lookup before Auth0 retires (D5); rate-limited only. The migration session (2026-10-05) makes no change until the deployed Auth0 Action has been compared with docs/auth0-actions/transform-token.js, which needs a management token from Chris
Hidden consumers of SignalR detailed errors or the dev exception page	Web shows its own messages; staging check 8.4; Development keeps both
The Study filter change surprises users	Docs note; target-2 stages are unchanged (10.3)

8. Coordination with other active work

Open PRs touching this plan's files (gh pr list --state open, 3 Oct 2026):

File	Open PRs	Action
API Program.cs	#3966 (Swagger, migration), #3965, #3947, #3945, #3944, #3943, #3942, #3932 (notifications stack), #2469	Rebase; changes are regional
RuntimeFeatureFlagProvider.cs	#3965, #3944, #3942, #3932	PR-7 does not edit it
NotificationHub.cs	#3932, #2469	PR-9 edits OnDisconnectedAsync only; whichever lands second rebases
ReviewController.cs	#3939 (progressive batches), #3746 (review eligibility, paused)	PR-9 changes 6 one-line returns; coordinate with the review-eligibility owner
ProjectController.cs	#3941 (notifications), #2224, #2469	PR-3 edits two actions only
Project.cs / ProjectRepository.cs / StudyRepository.cs	#3941, #2934 (deletion kernel), #2572, #2224, #2858	PR-11 changes one guard in Project.cs; PR-10 changes one private method plus signatures
MongoContext.cs	#2934	PR-5 edits the change-stream block only
StudyController.cs	#3947	PR-9 edits the DTO bounds only
env-mapping.yaml	Most feature PRs	Generated sections; regenerate after rebase

Owning streams: - the authentication migration session (R-ID; #3992; email-lookup and Auth0 Actions); - materialised project statistics (the PR-11 data cleanup; pmStudy writes); - notifications/study attention (Program.cs, NotificationHub.cs); - review eligibility (paused; ReviewController.cs); - the persistence and messaging plan (#3984/#3985; PR-6b boundary).

On 3 Oct no open PR touched the Identity files named in R-ID. Since then the migration session has #4040 (the ID-3 redirect leftovers) open and ID-4 in progress (2026-10-05); re-check gh pr list before any R-ID work.

9. Decisions needed from Chris

• D1. Rate-limit mechanism: in-app limiter (recommended: per-endpoint, per-user partitions, testable in the hermetic stack), or ingress annotations in cluster-gitops (coarser, no per-user keys)? I haven't checked whether the ingress controller supports them.
• D2. Rollout: staging Enforce from day one and production Observe for 7 days, then Enforce as a separate approved step (recommended)?
• D3. Distributed limits: accept per-pod in-memory limits now (recommended), and revisit with the shared Valkey after S08A?
• D4a. Contact-form confirmation: stop sending it to the caller-supplied address (recommended), send it only to signed-in users' own addresses, or keep it behind a CAPTCHA (a new third-party dependency)?
• D4b. Anonymous invitation preview: 401 and send the user to sign in (recommended), or a minimal anonymous view (project name only)?
• D5. email-lookup: rate-limit only and leave it for Auth0 retirement (recommended), or ask the migration session to replace it now with a token-gated variant for the Action? Migration session, 2026-10-05: no change to email-lookup until the deployed Auth0 Action has been compared with docs/auth0-actions/transform-token.js; that comparison needs a management token from Chris.
• D5a. Make tickets/email non-enumerating (recommended)?
• D6. Awaited domain events (PR-6b): ship flagged here, or fold into the outbox work (#3984)? Recommended: fold into #3984, because PR-6 already removes the "silently lost" part.
• D7. Value objects: fix ValueObject.Equals (recommended) vs migrate to records?
• D8. Study filter: ship unflagged with a docs note (recommended)?
• D9. Identity: answered by the migration session, 2026-10-05 (§4 R-ID): ID-4 is in progress before S09; ID-2 any time; ID-1, ID-3 and ID-6 after S09 (ID-1 needs backend PR-1 first); ID-5 before S10; all recorded as M005 slices. Still open for Chris: ID-7 (provision PM only after email verification) is a design change on the session's backlog.
• D10. PROPOSAL values:
• rate limits (§4 PR-2 table);
• the 7-day staging soak;
• verification codes: 5 attempts, 1 h expiry;
• page size cap 500;
• probes: readiness failureThreshold 3, liveness timeoutSeconds 3;
• event drain 10 s;
• ID-1: 6 signups per hour per IP.
• D11. Data cleanup (#3980): approve a read-only full count (staging copy first, then production in a quiet window) before any migration is proposed?