Skip to content

Pre-deploy database migrations: delivery plan (October 2026)

The design and its reasons are in ADR-023. This plan says what gets built, in what order, how each part is verified and how the catch-up promotion uses it. Chris took the decisions in §10 on 2026-10-14: D1–D9 as recommended, D10 deferred. It is temporary: delete it once the rollout gates (§6) have passed and the how-to in §9 exists.

1. MVP boundary

In the MVP (code and chart, everything off by default):

  1. The runner library: the migration model, state collection, lease, checksums and lock file.
  2. The entry point: SYRF_MIGRATION_MODE in the API and PM images (status, up, down, verify).
  3. The first four migrations (§4).
  4. The app side: Migrations:PreDeploy, StartupIndexPolicy, the schema readiness check, and /health/live excluding it.
  5. The chart: syrf-common.migrations.job included by the API and PM charts (PreSync up; PostSync down for a rollback), and the generalised startupProbe, both behind values that default off.
  6. The tests in ADR-023 §6.

Not in the MVP (follow-ups in §8):

  • turning anything on in any environment (each needs Chris's approval in cluster-gitops);
  • the rehearsals (§6), which are rollout gates, not code;
  • Downs for any G4b finding other than SessionTally;
  • a separate least-privilege migration database user;
  • switching the preview DatabaseLifecycle index-init Job to up;
  • governing collections beyond pmStudy and pmProject.

Shortest critical path to the catch-up promotion: PR-B and PR-C merged → staging enabled and R1 and R3 passed → R2 timing accepted → production enabled inside the catch-up promotion PR.

2. The contract between the PRs

PR-B (.NET) and PR-C (chart) are built in parallel against this contract.

Entry point environment (read before any DI container is built):

Variable Values Notes
SYRF_MIGRATION_MODE status, up, down, verify, bind Unset means normal startup. bind (added in #4258) writes the environment binding record (ADR-023 §8)
SYRF_MIGRATION_TARGET integer ≥ 0 Required for down, rejected otherwise
SYRF_MIGRATION_ACTOR free text, at most 200 characters Recorded in appliedBy.actor; the chart passes argocd:<release name>:<image tag> (under helm template the Helm revision is always 1)
SYRF_MIGRATION_LOCK_WAIT_SECONDS integer, default 3600 How long a runner waits for another runner's lease
SYRF_MIGRATION_WRITER_CHECK true (default) or false down only: wait for no live syrf-api/syrf-projectmanagement instance in syrf_metadata
SYRF_MIGRATION_WRITER_WAIT_SECONDS integer, default 600 down only
SYRF_MIGRATION_BINDING_EXPECTED_ENVIRONMENT production, staging, preview:pr-<n> or local:<stack> Required by bind; read by up only when a pending migration requires the binding; compared by status when set
SYRF_MIGRATION_BINDING_EXPECTED_DATABASE database name As above
SYRF_MIGRATION_BINDING_EXPECTED_CLUSTER cluster fingerprint, <setName>@<host suffix> or standalone As above; status prints the live value

MongoDB settings come from the usual SYRF__ConnectionStrings__MongoConnection__* variables, and the version recorded in appliedBy from the usual GitVersion variables.

Exit codes: 0 success; 1 a migration failed, a checksum did not match or verify found a gap; 2 usage or configuration error, including every environment binding refusal (which writes nothing); 3 the lease or writer wait timed out.

App setting: SYRF__Migrations__PreDeploy=true turns on the app side (ADR-023 §5). Absent or any other value means off.

Chart values (in syrf-common defaults and both service charts):

migrations:
  enabled: false                # renders the PreSync Job and SYRF__Migrations__PreDeploy
  activeDeadlineSeconds: 5400
  lockWaitSeconds: 3600
  resources:
    requests: { cpu: 100m, memory: 256Mi }
    limits: { memory: 512Mi }
  rollback:
    targetVersion: null         # set (with enabled) to render the PostSync down Job instead of up
    image: {}                   # optional { repository, tag } override; default the service image
    skipWriterCheck: false
    writerWaitSeconds: 600
  binding:                      # added in #4258; each value reaches every migration Job only when set
    bind: false                 # with enabled and all three values: a PreSync bind Job before up
    expectedEnvironment: ""
    expectedDatabase: ""
    expectedCluster: ""

startupProbe:
  enabled: false                # preview + waitForDatabase keeps today's probe regardless
  initialDelaySeconds: 10
  periodSeconds: 10
  failureThreshold: 30
  timeoutSeconds: 1

Setting migrations.rollback.targetVersion while migrations.enabled is false fails the render with a message, so a rollback cannot silently do nothing.

Collections: syrf_migrations (state) and syrf_migrations_lock (lease), in the service database.

3. PRs

PR Content Model Depends on
PR-A #4177 ADR-023, this plan, CD plan cross-references docs none
PR-B #4179 src/libs/mongo/SyRF.Mongo.Common/Migrations (runner, state, lease, lock file) and its tests; the PM migrations in SyRF.ProjectManagement.Mongo.Data/Migrations; SYRF_MIGRATION_MODE in the API and PM Program.cs; StartupIndexPolicy, the schema check and the live predicate; Testcontainers, parity and architecture tests .NET the contract (§2)
PR-C #4180 syrf-common.migrations.job, the API and PM chart includes, the startupProbe generalisation, helm unittest suites chart the contract (§2)

PR-B and PR-C can merge in either order: with everything off, neither changes behaviour. PR-C's Job is only useful once PR-B's entry point is in the image, and both stay off until §6.

4. The first migrations

Version Id Up Down
1 0001_pmStudy_ProjectId_WorkloadShareBucket pmStudy: { ProjectId: 1, WorkloadShareBucket: 1 }, name ProjectId_1_WorkloadShareBucket_1 Drop the index
2 0002_pmStudy_ProjectId_BulkUpdateLock_OperationId pmStudy: { ProjectId: 1, "BulkUpdateLock.OperationId": 1 }, name ProjectId_1_BulkUpdateLock.OperationId_1, partialFilterExpression: { "BulkUpdateLock.OperationId": { $exists: true } } Drop the index
3 0003_pmProject_bulk_pdf_upload_sweep_status pmProject: { "BulkPdfUploadJobs.Status": 1 }, name bulk_pdf_upload_sweep_status Drop the index
4 0004_pmStudy_SessionTally_reservation_fields No-op (outcome: noop); main writes and tolerates the fields Batched $unset of ExtractionInfo.SessionTallies.$[].NumberOfSlotReservations, …NumberOfDirtySlotReservations, …TotalAllocatedSessionCount, …TotalEngagedSessionCount on documents whose tallies hold any of them; then a count that must be zero
5 0005_pmDeletionOperation_ActiveSweepProjectId_unique pmDeletionOperation: { ActiveSweepProjectId: 1 }, name ActiveSweepProjectId_1_unique, unique: true, partialFilterExpression: { ActiveSweepProjectId: { $exists: true } } (one sweeping deletion per project; ADR-024) Drop the index
6 0006_pmDeletionOperation_State_LeaseExpiresAtUtc pmDeletionOperation: { State: 1, "Lease.ExpiresAtUtc": 1 }, name State_1_Lease.ExpiresAtUtc_1 (the deletion runner's scan) Drop the index
7 0007_pmDeletionLedger_OperationId_AtUtc pmDeletionLedger: { OperationId: 1, AtUtc: 1 }, name OperationId_1_AtUtc_1 Drop the index
8 0008_pmDeletionOperation_ProjectId_State pmDeletionOperation: { ProjectId: 1, State: 1 }, name ProjectId_1_State_1 (the pending-deletion query; ADR-024 slice S3) Drop the index
9 0009_runtimeFeatureFlagAudit_mutationId_unique runtimeFeatureFlagAudit: { mutationId: 1 }, name mutationId_1_unique, unique: true, partialFilterExpression: { mutationId: { $exists: true } } (one audit entry per mutation ID; runtime feature releases §6.4). Requires the environment binding Drop the index
10 0010_runtimeFeatureFlagAudit_storeEpoch_revision_unique runtimeFeatureFlagAudit: { storeEpoch: 1, revision: 1 }, name storeEpoch_1_revision_1_unique, unique: true, partialFilterExpression: { storeEpoch: { $exists: true } } (one audit entry per revision of a policy lineage). Requires the environment binding Drop the index
11 0011_runtimePolicyHostAck_service_lastSeenAtUtc runtimePolicyHostAck: { service: 1, lastSeenAtUtc: 1 }, name service_1_lastSeenAtUtc_1 (the fleet view's per-service scan; runtime feature releases §6.2, §6.4). Requires the environment binding Drop the index
12 0012_runtimeFeatureRelease_environmentId_status runtimeFeatureRelease: { environmentId: 1, status: 1 }, name environmentId_1_status_1. Requires the environment binding Drop the index
13 0013_runtimeFeatureReleaseEvent_mutationId_type_unique runtimeFeatureReleaseEvent: { mutationId: 1, type: 1 }, name mutationId_1_type_1_unique, unique: true (one event of each type per mutation ID). Requires the environment binding Drop the index
14 0014_runtimeFeatureReleaseEvent_releaseId_recordedAtUtc runtimeFeatureReleaseEvent: { releaseId: 1, recordedAtUtc: 1 }, name releaseId_1_recordedAtUtc_1. Requires the environment binding Drop the index
15 0015_runtimeFeatureReleaseEvent_environmentId_featureId_recordedAtUtc runtimeFeatureReleaseEvent: { environmentId: 1, featureId: 1, recordedAtUtc: 1 }, name environmentId_1_featureId_1_recordedAtUtc_1. Requires the environment binding Drop the index
16 0016_runtimeFeatureReleaseOutbox_dedupKey_unique runtimeFeatureReleaseOutbox: { dedupKey: 1 }, name dedupKey_1_unique, unique: true (one item per deduplication key; no TTL). Requires the environment binding Drop the index
17 0017_runtimeFeatureReleaseOutbox_state_nextAttemptAtUtc runtimeFeatureReleaseOutbox: { state: 1, nextAttemptAtUtc: 1 }, name state_1_nextAttemptAtUtc_1 (the dispatcher's scan). Requires the environment binding Drop the index
18 0018_runtimeWorkloadObservation_observedAtUtc runtimeWorkloadObservation: { observedAtUtc: 1 }, name observedAtUtc_1. Requires the environment binding Drop the index
19 0019_runtimeEnvironmentReport_environmentId_collectionEndedAtUtc runtimeEnvironmentReport: { environmentId: 1, collectionEndedAtUtc: 1 }, name environmentId_1_collectionEndedAtUtc_1. Requires the environment binding Drop the index
20 0020_runtimeEnvironmentReport_deleteAfterUtc_ttl runtimeEnvironmentReport: { deleteAfterUtc: 1 }, name deleteAfterUtc_1_ttl, expireAfterSeconds: 0 (OD-20 retention: the writer sets deleteAfterUtc 30 days after collection). Requires the environment binding Drop the index

The catch-up rollback target is version 3: it undoes the SessionTally fields and keeps indexes 1–3 (ADR-023 §4). Migrations 5–20 come after it and are crossed by that rollback too. That is safe because of the rule the lock-file test pins: every migration after version 4 must be an index migration. A Down restores the database to its state before the Up (an index the Up created is dropped, an index it adopted is kept), so 104eca9 is left a database it has already run on (ADR-023 amendment of 2026-10-16). Any other kind of migration after version 4 needs the rollback target revisited first.

Migrations 9 to 20 require the environment binding (ADR-023 §8, #4272, #4281).

  • Binding first. Every environment that enables the Job must bind first: in the same cluster-gitops change with migrations.binding.bind: true and the three expected values, or by running bind before.
  • Without a binding. up stops before 0009 with exit 2 and applies nothing in that run, 0001–0008 included. The PreSync Job then fails, and the previous release keeps serving.
  • Forward rule. 9 and 10 are partial on fields no release up to this one writes. The API's RuntimeFeatureFlagAuditIndexForwardRuleTests prove it. 11–20 index new collections that no release up to this one reads or writes; RuntimeReleaseCollectionsForwardRuleTests in the PM data tests prove it. The rule is in runtime feature releases §6.4.

5. The catch-up promotion

  1. PR-B and PR-C merged. Nothing changes anywhere: the toggle is off.
  2. Staging (R1). A cluster-gitops PR sets migrations.enabled: true for API and PM in staging, and binds staging in the same change (migrations.binding.bind: true, expectedEnvironment: staging, expectedDatabase: syrf_staging, and expectedCluster copied from the live fingerprint that status prints), because 9 to 20 require the binding. The staging indexes 1–3 already exist (startup built them), so they record adopted; 4 records noop; 5–20 build (or, if already present, adopt) their indexes on new or small collections. Staging's new pods pass the schema readiness check.
  3. Rollback drill on staging (R3). The R-1 and R-2 procedure (§7) against staging, rolling API and PM back to the production baseline build (104eca9 today, or whatever production runs when R3 happens), then forward again. Only that build proves the Down fixes A3: any build after #2467 reads the four fields anyway. If staging data has other main-only shapes the baseline cannot read, that is the G4b gap of step 6, and closing it is a prerequisite for R3. Staging is not mission critical, so the short outage is acceptable there.
  4. Timing (R2). Up against a temporary cluster restored from a production snapshot (D3), run from the migration entry point, with a synthetic read load to record latency. Recorded: the collection sizes, each build's duration, p95 latency during the builds, and whether a killed and restarted runner attaches to an in-progress build. Go or no-go for createIndexes (D2).
  5. Production. The catch-up promotion PR sets migrations.enabled: true for API and PM next to the new chart and image tags, with the production binding (migrations.binding.bind: true, expectedEnvironment: production, expectedDatabase: syrftest, and the production cluster's live fingerprint). Chris approves that specific PR. PreSync binds, builds the three indexes while 104eca9 serves, records the SessionTally no-op, builds the deletion, audit and runtime release indexes (5–20, new or small collections), and then the Sync phase deploys main. If the Job fails, including a binding refusal, production stays on 104eca9.
  6. The G4b reader (CD plan PR-11) runs against 104eca9 before the promotion (catch-up promotion only: it backs the one planned rollback the deployments amendment keeps). Every failure it finds other than SessionTally either gets its own Down migration or is resolved another way before production is enabled.

6. Rehearsals and rollout gates

Gate Where Pass condition Approval
R1 Staging The bind Job and both PreSync Jobs complete; runtimeEnvironmentBinding holds staging, syrf_staging and the live fingerprint; syrf_migrations shows 1–3 adopted, 4 noop, 5–20 applied (or adopted), with appliedBy set; new pods become ready Chris approves the staging cluster-gitops PR
R2 Temporary cluster from a production snapshot (D3) Build durations and latency recorded; Chris accepts them, or chooses the out-of-band fallback (ADR-023 §3) Chris approves the snapshot restore and its cost
R3 Staging R-1 removes the four fields (count zero); R-2 deploys the production baseline build (104eca9 today), which serves studies saved under main without FormatException; rolling forward again succeeds. Prerequisite (catch-up rollback only): G4b shows no other baseline-unreadable shape in staging data Chris approves the two staging PRs

No gate runs against production. Production is enabled only inside the catch-up promotion PR that Chris approves.

7. Rollback runbook

For a rollback that needs a data Down (the catch-up case: target version 3).

  1. R-1, drain and down. One cluster-gitops PR, in the production environment folders for API and PM: keep the current chart and image tags; set replicaCount: 0; set migrations.rollback.targetVersion: 3. Chris approves; an administrator merges.
  2. ArgoCD syncs both Applications: the Deployments scale to zero, then each PostSync Job runs down to version 3. One Job does the work and the other waits for the lease and finds nothing to do. Each Job waits until syrf_metadata lists no live API or PM instance before changing data.
  3. Check the Job logs. The Down crosses every migration above version 3, newest first, so expect Rolled back 0020_runtimeEnvironmentReport_deleteAfterUtc_ttl and so on down to Rolled back 0011_runtimePolicyHostAck_service_lastSeenAtUtc (the runtime release indexes, in reverse order), then Rolled back 0010_runtimeFeatureFlagAudit_storeEpoch_revision_unique, Rolled back 0009_runtimeFeatureFlagAudit_mutationId_unique, Rolled back 0008_pmDeletionOperation_ProjectId_State, Rolled back 0007_pmDeletionLedger_OperationId_AtUtc, Rolled back 0006_pmDeletionOperation_State_LeaseExpiresAtUtc, Rolled back 0005_pmDeletionOperation_ActiveSweepProjectId_unique (each preceded by a "dropped index" or "was already absent" line; where the Up adopted an index that already existed, nothing is dropped and the line ends "the adopted change was kept"), then Rolled back 0004_pmStudy_SessionTally_reservation_fields with remaining count 0, and finally Rolled back to schema version 3. If a crashed pod left a stale instance and the writer wait timed out, confirm both Deployments show zero pods in ArgoCD, then re-run R-1 with migrations.rollback.skipWriterCheck: true.
  4. R-2, old build. One cluster-gitops PR: API and PM chart and image tags back to the previous release, replicaCount restored, migrations.rollback removed. Chris approves; an administrator merges.
  5. Verify: the old pods are ready and a study saved under main opens without an error (Sentry shows no FormatException from SessionTally).

For a rollback that needs no data Down, skip steps 1–3: R-2 alone is the rollback, and the migrations stay applied (their changes are compatible with the old build by rule 5 of ADR-023 §1).

8. Acceptance criteria

MVP: M1–M18 and M22. Rollout gates before the catch-up promotion: M19–M21.

# Criterion Verification
M1 up applies pending migrations in version order and records status, checksum, durationMs, outcome and appliedBy for each Testcontainers test
M2 A second up changes nothing; a run interrupted in the middle of a batched migration finishes on the next up Testcontainers test with a fault-injected runner
M3 An applied migration whose recorded checksum differs from the binary's makes up exit 1 without running anything Testcontainers test
M4 The lock-file test fails when an entry changes, is removed, or a version is inserted before the last one Negative unit tests
M5 On seeded pmStudy and pmProject: after Up the three indexes match their declared specification in listIndexes; after Down to 0 they are gone; after Up again they are back. The four tally fields are gone after Down of version 4 and reappear when a study is saved Testcontainers test (Up, Down, Up)
M6 An existing identical index is recorded as adopted, and a Down keeps it (history outcome kept; ADR-023 amendment of 2026-10-16); an index with the same name and a different specification makes up exit 1 and leaves the existing index in place Testcontainers test
M7 down refuses a target above the current version and refuses to cross an Irreversible migration, naming it and its reason Unit tests
M8 Two concurrent runners never apply a migration twice: the second waits for the lease; an expired lease is taken over Testcontainers test
M9 down waits while syrf_metadata lists a live API or PM instance, proceeds when none is listed, exits 3 after the writer wait, and skips the wait when the check is off Testcontainers test
M10 The entry point honours the §2 contract (modes, target rules, exit codes), builds no DI container and needs only the MongoDB and GitVersion settings Unit tests on the command class; a local run against the local Docker MongoDB, output in the PR body
M11 With PreDeploy off, startup creates exactly the indexes it creates on main today Index manifest test against an empty Testcontainers database
M12 With PreDeploy on, startup creates no migration-owned index and no index on a governed collection outside StartupIndexBaseline; a fixture that adds one fails Architecture test with a negative fixture
M13 Each migration-owned index has the same name, keys and options as the repository's own definition Parity test
M14 With PreDeploy on, /health/ready reports schema Unhealthy before up and Healthy after, and a later mismatch or read failure reports Degraded (still ready); /health/live returns 200 throughout API and PM endpoint tests
M15 With default values neither chart renders a migration Job or SYRF__Migrations__PreDeploy, and the production-values render of API and PM is unchanged helm unittest; render diff before and after, attached to PR-C
M16 With migrations.enabled: true the chart renders a PreSync Job (BeforeHookCreation) running the service image with SYRF_MIGRATION_MODE=up, the MongoDB and GitVersion environment, backoffLimit: 0, the deadline, the actor and no service-account token; and the Deployment gets SYRF__Migrations__PreDeploy=true helm unittest
M17 With a rollback target the chart renders a PostSync down Job with the target, the image override and the writer-check settings, and no up Job; a target with enabled: false fails the render helm unittest
M18 startupProbe is absent by default outside preview, preview with waitForDatabase renders exactly today's probe, and startupProbe.enabled renders the configured values helm unittest
M19 R1 passes on staging Job logs and the syrf_migrations documents, linked from the rollout issue
M20 R3 passes on staging against the production baseline build The two staging PRs, Job logs and a manual check, on the baseline build, of a study saved under main
M21 R2 timings are recorded and Chris has chosen createIndexes or the fallback Rehearsal report linked from the rollout issue
M22 No environment has migrations.enabled or startupProbe.enabled until Chris approves it cluster-gitops values; the CD plan G2b production render golden

Unknown-field tolerance is a backstop (T1–T5)

Chris, 2026-10-14: "ok, but should have a system where this is never relied upon! Explicit migrations, rollbacks etc." Unknown-field tolerance ([BsonExtraElements], ignore-extra-elements; CD plan G4a) stays, but only as a backstop. Nothing may rely on it to make a rollback work. See ADR-023 Status.

T1 is the existing process; T2, T2a and T3–T5 are planned, not implemented.

Since the forward-only amendment (2026-10-17), production data only moves forward and data migrations are Irreversible by default. The later deployments amendment (2026-10-17) drops the guarantee that the previous build can run against the current database: a rollback is a new release planned when needed, and compatibility is kept only for the rollout window. T2a, T3 and T4 (G4b) are therefore no longer merge gates, except for the one catch-up rollback to 104eca9, which keeps step 6 of §5 and R3. #4282 is on hold and will shrink to whatever the rollout window needs.

# Criterion Verification
T1 A PR that adds or changes a persisted field an older build cannot read adds a migration with a Down (or Irreversible with a reason) and a lock-file entry, and the rollback runbook (§7) names the rollback target The existing PmMigrations and runner tests and migrations.lock.json (M4); PR review against the ADR-023 §1 rules
T2 Superseded 2026-10-17 as a general gate (deployments amendment); applies only to the catch-up rollback to 104eca9. Planned. The API and PM hosts count and log every document read with unknown elements on a governed collection: a counter (collection and type labels) plus a rate-limited warning naming the collection, type and element names, never values. Detection needs capture: every type reachable from a governed collection, nested value objects included (SessionTally is the case), captures unknown elements ([BsonExtraElements] or an equivalent serializer convention that counts and logs them). Silent IgnoreExtraElements drops them and cannot be counted Unit test: a document with an extra element, on a root and on a nested value object, yields counter 1 and one warning; a clean document yields neither
T2a Superseded 2026-10-17 as a general gate (deployments amendment); applies only to the catch-up rollback to 104eca9. Planned. The G4a coverage test fails for a type reachable from a governed collection that uses silent IgnoreExtraElements (or no tolerance), including a nested value object Negative fixture: a nested governed type with IgnoreExtraElements fails; the same type with capture passes
T3 Superseded 2026-10-17 as a general gate (deployments amendment); applies only to the catch-up rollback to 104eca9. Planned. After any rollback, the T2 counter staying above zero is treated as a defect (the rollback relied on tolerance), not as success. The R-2 verification step and R3 check it R3 report on staging shows the counter at zero after R-2; the runbook step names the metric
T4 Superseded 2026-10-17 as a general gate (deployments amendment); applies only to the catch-up rollback to 104eca9. Planned. The N-1 read check (CD plan G4b, the old-build corpus reader, decision D3 option (b)) is the enforcement gate: it reads data written by main with the production build's model and fails the PR on a shape the baseline cannot read, so the shape needs a migration Down (T1) or recorded known debt with a named rollback target CD plan G4.1–G4.5 (§4.4); with today's baseline it is red on SessionTally until migration version 4 covers it
T5 Superseded 2026-10-17 as a general gate (deployments amendment); applies only to the catch-up rollback to 104eca9. Planned. T2, T2a, T3 and T4 are named in the §3 PR list or CD plan Phase 3 with an owner before the catch-up promotion; none is implemented by PR-A to PR-C Issue links recorded in the rollout issue

9. Follow-ups (not in the MVP)

  • A how-to, "Write a database migration", once PR-B lands: when a change needs one, the rules in ADR-023 §1, the lock file, and the required tests.
  • T2, T2a, T3 and T4 (§8): superseded as merge gates by the deployments amendment (2026-10-17). Only the G4b run against 104eca9, for the catch-up rollback, remains; G4a is on hold (#4282).
  • Downs for any further G4b findings against the production baseline.
  • A separate migration database user, so the application user can lose createIndex and dropIndex. Needs a camarades-infrastructure (Atlas) change.
  • Switch the preview DatabaseLifecycle post-seed index-init Job to SYRF_MIGRATION_MODE=up and enable the toggle in previews, so previews exercise the same path.
  • Build both pmStudy indexes in one createIndexes if R2 shows two scans matter.
  • Extend the governed list by the threshold Chris chooses (D7).
  • Retire SYRF_INDEX_INIT_MODE once previews use up for governed collections.
  • #4182: the legacy-tally fallback in Filters.cs, before activeReviewerTrackingEnabled is enabled anywhere with legacy tallies (ADR-023 Context).
  • Optionally pass $ARGOCD_APP_NAME/$ARGOCD_APP_REVISION as Helm parameters from the cluster-gitops ApplicationSet, so appliedBy.actor records the exact sync revision.

10. Decisions for Chris (decided 2026-10-14)

Decided by Chris on 2026-10-14: he agreed with every recommendation (D1–D9 as written) and deferred D10.

# Decision Recommendation Decision (Chris, 2026-10-14)
D1 Where the migration step runs An ArgoCD PreSync Job in both the API and PM charts, using the service image (ADR-023 §2) Accepted as recommended
D2 How the large index builds run createIndexes from the Job, gated by R2; the Atlas rolling build in the UI plus "adopt" as the fallback (ADR-023 §3) Accepted as recommended
D3 Where R2 runs A temporary M20 cluster restored from a recent production snapshot in the production Atlas project, deleted after the run. Production data never leaves the production project Accepted as recommended
D4 Rollback for the catch-up promotion The two-PR drain-and-down procedure (§7), accepting downtime during a rollback. A 104eca9 compatibility image stays the fallback if R3 fails Accepted as recommended
D5 Enable order Staging (R1, R3), then production inside the catch-up promotion PR Accepted as recommended
D6 Startup probe in production Enable with failureThreshold: 30 (about five minutes) for API and PM; matches CD plan D14 Accepted as recommended
D7 Which collections are governed pmStudy and pmProject now; add any collection over 1 GB or 1 million documents in production, measured in R2 Accepted as recommended
D8 Readiness gate or crash when a migration is missing Readiness gate (ADR-023 §5) Accepted as recommended
D9 Catch-up rollback target Version 3: undo SessionTally, keep the three indexes Accepted as recommended
D10 Should the rollback Down also remove Study.SlotReservations? The baseline build never maintains them, so after a roll-forward a stale reservation would count as a held slot Yes, as a separate migration with a no-op Up, ordered before the SessionTally one, once Chris agrees. Moot while activeReviewerTrackingEnabled stays off everywhere Deferred. Moot while activeReviewerTrackingEnabled stays off everywhere; revisit before that flag is turned on anywhere

11. Overlap with the continuous delivery plan

  • G5 (PR-6). PR-C delivers G5's startup probe, default off; enabling it and the probe values stay with D6 and CD plan D14. G5's startup index manifest golden and source scan remain PR-6's scope; PR-B's architecture test covers the governed collections, and StartupIndexPolicy is the helper G5 names.
  • G4a (PR-3) and G4b (PR-11). Superseded 2026-10-17 by the deployments amendment: G4b is no longer an enforcement gate, and G4a is on hold (#4282). G4b's run against the production baseline remains the input to step 6 of §5, for the catch-up rollback only.
  • Phase 3 item 7. This plan is that item.
  • D10 (RollingUpdate). Independent. A failed PreSync Job leaves the Deployment untouched under either strategy. The strategy matters only for the readiness backstop: if a new pod ever starts without its migrations, RollingUpdate keeps the old pods serving, while Recreate has already removed them.