Data model: options, evidence and recommendation¶
Part of the temporary reversible deletion plan. Counts come from main at 35407997e,
excluding test projects. Storage shapes are proposals for the deletion ADR (slice S0) to fix.
1. What the model has to achieve¶
- A deleted search's Studies are invisible to every ordinary read: Library, screening and annotation pools, allocation, reconciliation, exports, statistics, search counts.
- Writes to them are refused, so nothing lands on a deleted Study and a restore is exact.
- Restore returns exactly what was kept.
- It works for very large searches and projects without hitting Atlas transaction limits.
- It survives a crash at any point and resumes where it stopped.
- Every FEAT-024 statistics family stays exact (see statistics).
- It fits ADR-020 (study locks), the ADR-019 fold, Bulk PDF clean-up, and a rollback.
- It can later carry the integrated plan's canonical parts (R0, R2a, R3a) without being redone.
2. Evidence: how Study data is read and written today¶
| What | Count or fact | Source |
|---|---|---|
| Study repository | 9 partial files, 4,984 lines, 109 calls on the Study collection | Mongo.Data/Repositories/StudyRepository*.cs |
| Repository interface | About 130 declared members | Core/Interfaces/IStudyRepository.cs |
| Shared Study filters | 35 Study filter builders; aggregation pipelines in StudyStats.cs |
Mongo.Data/Filters.cs, StudyStats.cs |
| Other code opening the Study collection directly | 20 files: the statistics source readers and calculators, the fold store and worker, the bulk-update writer, the risk-of-bias store, StageReviewSettingsController, StageAllocationRegimeRepository, the pending-index tools |
rg "GetCollection<Study" |
| Unit-of-work Study calls | 179 .Studies. calls in 31 non-test files, of which about 37 load one Study by id |
rg "\.Studies\." |
| How reads are scoped | Project-scoped reads filter on Study.ProjectId; id loads in the review paths check study.ProjectId != projectId before acting (seven checks in ReviewController, two in NotificationHub, two in AnnotationFoldTargets, one in ActivityClaimRevokedConsumer) |
e.g. ReviewController.cs:97 |
| Indexes | {ProjectId}, {ProjectId, _id}, several ProjectId-prefixed compound indexes, and the fold's partial {ProjectId, PendingStatistics.OldestAtUtc}. None on {ProjectId, SystematicSearchId}. Building any new index scans all of pmStudy (about 10.7 GB in production) and is a deliberate production operation |
StudyRepository.cs:3052-3057; StudyRepository.PendingStatisticsIndex.cs |
| Write enforcement | Every Study write path honours the ADR-020 lock predicate through the registered IAggregateWriteGuard<Study>; generic saves add it through GetWriteFilter; direct writes add StudyBulkUpdateLockGuard.Unlocked; StudyWriteLockArchitectureTests fails the build on an unguarded write |
StudyBulkUpdateLockGuard.cs; AggregateWriteGuards.cs; MongoExtensions.cs:262-326 |
| Optimistic concurrency | Whole-Study saves filter on Audit.Version (current or 0); upserting saves turn a guarded miss into the guard's typed refusal |
MongoExtensions.cs:262-326, 513-525 |
| Transaction limits | ADR-020 rejected one transaction per job because of Atlas lifetime and size limits; it uses 100 Studies per transaction | ADR-020 "Alternatives", "Transaction size" |
| Search link | Project.SystematicSearchIds and SystematicSearch.ProjectIds; a search document can be linked to more than one project, but each Study belongs to one project. Project.RemoveSystematicSearch also removes the search's import-job entry |
Project.cs:810-815; SystematicSearch.cs:140-143 |
| Project access | AuthorizationHandler loads the routed project for every Project* and Stage* policy (legacy and enforced paths), and the SignalR handler does the same for hub calls |
AuthorizationHandler.cs:149-162 |
| Fold | A save in fold mode appends a pending entry to the Study it writes; only the fold worker removes entries; the worker finds pending Studies through the ProjectId-keyed index |
statistics-reference.md "Async point fold" |
3. The options¶
| A. Marker and a filter on every read | B. Search-level exclusion list | C. Move Studies to a holding collection | D. Detach and mark (recommended) | E. Unlink the search only | |
|---|---|---|---|---|---|
| Idea | Stamp Deletion on each Study; add Deletion == null to every read |
Record deleted search ids on the Project; add SystematicSearchId $nin to every read |
Copy each Study to pmDeletedStudy and delete it from pmStudy; copy back on restore |
Change each Study's ProjectId to a holding id owned by the deletion, stamp a marker remembering the original project, unlink the search |
Remove the project↔search link and keep a tombstone |
| Reads to change | Every project-scoped read: most of the 109 repository calls, 35 filters, the aggregation pipelines, 20 direct-collection files, plus id loads | The same set, and each must also load the Project's list first | None for project-scoped reads; about 37 id loads would now get "not found" and must handle it | None for project-scoped reads; id loads that check ProjectId refuse naturally; the rest are audited (slice S1) |
None, but it does not hide anything: Studies keep ProjectId and stay in every pool |
| Writes | Guard extension (ADR-020 seam) | Every writer must check the list | Writes miss (documents gone); upserting saves could re-create a moved Study | Guard extension (ADR-020 seam); the re-home bumps Audit.Version so stale copies fail |
Unchanged |
| Atomicity for the user | Per-Study; sweep | One document write; instant | Per batch, needs a multi-document transaction per batch | Per-Study, each update atomic and idempotent; sweep | One transaction |
| Size limits | Sweep in batches; no transaction needed | None | Copies whole documents (some Studies are large); each batch transaction carries full documents; Atlas size and 60-second lifetime limits apply | Sweep of small field updates; no multi-document transaction; no size limit | None |
| Resumable | Yes, by filter | Yes | Yes, but a crash between copy and delete leaves duplicates to reconcile | Yes: the update filter excludes Studies already moved; the cursor is recorded | Yes |
| Restore | Unset the marker (sweep) | Remove the id from the list | Copy back (sweep, same limits) | Reverse the re-home (sweep) | Relink |
| Statistics | Every family query must filter; any miss serves a wrong count | Same, and the fold and its index ignore the list | Population changes as with D; pending entries move with the document | Population changes exactly as the old immediate delete did; detached Studies stay outside every scope of the project for as long as they are deleted | Wrong: deleted Studies still counted |
| Leak risk (a deleted Study appearing somewhere) | High: one forgotten filter among 100+ | High, same reason | Low for reads; high for code that assumes a loaded Study exists | Low: only reads that ignore ProjectId can see them; they are few and listed in S1 |
Total |
| Rollback to an older binary | Older binaries see deleted Studies in every view | Same | Older binaries cannot see moved Studies (safe), but cannot restore them | Older binaries cannot see re-homed Studies through any project-scoped read (safe). Unknown fields land in Entity.ExtraElements ([BsonExtraElements]), so they load and re-save a marked Study without error and keep the marker (see rollout) |
— |
| Prior attempt | The August WIP (79 modified files, about 8,600 added lines, not merged) | — | — | — | Today's code unlinks, then hard-deletes |
4. Recommendation¶
Searches: option D, "detach and mark". It is the only option that hides deleted Studies from every project-scoped read without changing those reads, while reusing an enforcement seam that is already complete and policed by an architecture test. It writes small field updates, so it has no transaction size limit and needs no new index. It is idempotent by construction. Because nothing is ever erased (D2), detached Studies simply stay where they are, indefinitely: delete and restore are the only statistics events, and if an erasure policy were ever approved it would be a delete scoped to one holding id.
Projects: a state on the Project document, no Study writes. Every project-scoped route, hub call and
background job already starts from the Project, so one state field behind the central access gate hides
the whole project. Restore is a single write. Statistics have a ready-made switch
(ProjectStatisticsControl.IsProjectDeleted) that the readers, the fold and the rebuilds already honour.
Per-Study markers for project deletion (the integrated plan's composite guard, AC-R2a-53) remain the R0
and R2a work for canonical collections; for legacy writers the gate is enough, because the only possible
late write is one already authorised a moment before the deletion, and the data it writes is kept and
restored like everything else.
Why not reuse the August design. It chose option A with a 24-hour grace period followed by physical
deletion, which Chris has now ruled out. The ideas worth keeping are its lifecycle value object
(monotonic states, exact-token claim with expiry and takeover), the value-free record idea
(pmDeletedResource), the rule that an absent or failed lookup is never deletion evidence, and its
analysis of Bulk PDF uploads during deletion. Its code is about six weeks behind main (it predates the
FEAT-024 fold, ADR-020 locks and the authority programme) and should not be resurrected.
5. Detailed design¶
5.1 The Study deletion marker¶
Study.Deletion (nullable, omitted when null, so untouched Studies are byte-identical):
| Field | Meaning |
|---|---|
OperationId |
The deletion operation that moved this Study |
OriginalProjectId |
Where it returns on restore |
MarkedAtUtc |
When it moved |
While marked, Study.ProjectId holds the operation's holding id: a fresh GUID minted for that
operation and never used for a Project. All moved Studies of one operation share it, so restore walks
them through the existing {ProjectId, _id} index without a new index.
5.2 The composite write guard¶
The registered IAggregateWriteGuard<Study> becomes a composite: "not locked by a bulk update" and
"not deletion-marked". Every generic save, every direct write that uses StudyBulkUpdateLockGuard.Unlocked
today, and every snapshot write that checks BulkUpdateLock gains the second condition through the same
seam. A miss caused by the marker throws a typed StudyDeletedException (HTTP 410, problem type
study-deleted; hub denial reason StudyDeleted). StudyWriteLockArchitectureTests is extended to
require the composite predicate. One writer is allowed to cross it: the deletion runner, which sets and
clears the marker, always with $inc Audit.Version. It goes on the allow-list with its reason, like the
fold bookkeeping today.
With no marker in the database the predicate matches every Study, so this slice changes no behaviour.
5.3 The search link¶
At deletion the search is unlinked through the existing domain methods (Project.RemoveSystematicSearch
and SystematicSearch.RemoveProjectIdFromSearch). Because RemoveSystematicSearch also removes the
search's import-job entry, the operation keeps a snapshot of that entry. There is no list position to keep:
Project.SystematicSearchIds is a set. Stage search filters are left untouched and not snapshotted (as built
in S4a). A stage's SystematicSearchFilter is stored but applied nowhere today: no pool, query or statistics path
reads it (Stage.IsUsingSearchFilter has no caller; the Library's search filter is a separate, per-request
StudyFilters.SystematicSearchFilter). So a stored id that names the deleted search is simply kept, and is still
there after a restore. The only editor is the stage settings card "Select Studies for Review", shown only with
studyFiltersConfigurable on (default off, and not turned on in any environment's configuration). It is a placeholder: it never
loads or shows the stored ids, its search options carry no value, and its Save replaces systematicSearchFilter
with the empty value of an unbound control. So saving that card clears the whole stored filter, whether or not a
search is deleted, and a cleared id does not come back on restore; this is existing behaviour, not something the
deletion adds. Restore re-adds the link and the import-job entry. A search linked to other projects keeps those links; only this project's
association is detached. A deleted search with no other project is a document with no links; nothing in
the code deletes unlinked searches today (the only deletions are in the two orphaned
ProjectManagementService methods, which stay as the flag-off branch until deletionLifecycle is retired
(#4172), and preview seeding), so it stays for as long as the deletion lasts. The reversible deletion never
deletes the search document.
5.4 The deletion operation (pmDeletionOperation)¶
One document per deletion. It is the durable intent, the progress record and the lock.
| Field | Meaning |
|---|---|
_id |
Operation id; also the statistics correlation (§ statistics) |
Scope |
Search or Project |
ProjectId, SearchId |
The target |
HoldingProjectId |
Search scope only |
State |
See below |
RequestedBy, RequestedAtUtc, Reason |
Who, when, why (reason optional) |
RestoreWindowDays, RestoreWindowEndsAtUtc |
Snapshotted when the deletion completes; the "Can be restored until" date. After it, only application administrators may restore |
LinkSnapshot |
Search scope: the import-job entry. There is no list position, and stage filters are not touched or snapshotted (§5.3). As built (S4a-2): { ProjectListedSearch, SearchListedProject, SearchImportJob }: whether each side listed the other, and the project's SearchImportJobs entry for the search exactly as stored (raw BSON, so fields this build does not know survive; restore deserializes it). Captured when the operation leaves Requested, in the same write that moves it to Deleting |
Cursor, Counts |
Sweep progress: last _id processed; Studies moved, skipped (pending entries), sessions kept. As built (S4a-2): SweepCursor (last _id the current pass walked; null at the start of a pass), StudiesMoved (exact: recounted under the holding id after a takeover and before completion), StudiesSkipped (left behind in the current pass), SweepRetryingSinceUtc (a pass ended with Studies left behind and nothing has moved since), plus StatisticsFenceAdmittedAtUtc and StatisticsFenceHeld (the fence step passed; false when statistics are off) and SearchUnlinkedAtUtc. Every field is additive: absent on an S3 record, read as "not started", and ignored by an older build. "Sessions kept" is not counted |
Lease |
{ Token, ExpiresAtUtc }: exact-token, renewable, taken over after expiry (ADR-020's 2-minute lease pattern) |
CancelledJobs, AwaitedJobs |
Uploads, imports and bulk study updates the deletion cancelled, and the work it is waiting for, each with its last observed state (in-flight work). As built (S3 seam): one list, InFlight, of { Kind, JobId, Status (CancelRequested or Waiting), Reason, ObservedState } as the last in-flight review found it, plus InFlightWaitingSinceUtc (the blocked alert measures from here) and InFlightClearedAtUtc (nothing outstanding; the next step may run). Kinds, statuses and reasons are stored as strings so an older build reads a newer one's values. The cancelled jobs themselves are recorded in the ledger (UploadCancelledForDeletion and so on), one record per job |
RestoredBy, RestoredAtUtc |
The restore, when it happens |
BlockedReason |
For example WaitingForImport, BulkPdfProcessingNotFinished, PendingStatisticsNotDrained, ProjectSweepBusy, StatisticsFenceBusy (S4a: another owner holds the fence; the operation stays Requested), StudiesLockedByBulkUpdate (S4a: the sweep's remaining Studies are locked), StatisticsFenceReleaseFailed (S4a: every Study is moved but the fence release failed; the operation stays Deleting and retries), or CorruptRecord: the record cannot be read as an operation, so the runner quarantined it and no longer offers it (S3a). Stored as a string, so an older build reads a reason a newer one added |
States.
Search: Requested ──► Deleting ──► Deleted ──► Restoring ──► Restored (terminal)
Project: Requested ──► Deleted ──► Restored (terminal)
There is no terminal "erased" state: Deleted lasts until a restore, however long that is. Whether the
users' restore window is still open is computed from RestoreWindowEndsAtUtc at read time, so the end of
the window needs no job.
Requested is where in-flight work is handled: the item's Bulk PDF uploads, imports and bulk study
updates are cancelled, and work that cannot be cancelled or does not belong to the item is waited for
(in-flight work). A search whose import never completed ends here as a cancelled
import entry with no Studies. It also exists because the statistics
fence is keyed by the operation id: the operation is recorded first, the fence is admitted when the
waits are over, then the first source write happens. If the fence is held by another owner at that
moment, the operation stays Requested and tries again on the next tick. A project has no Deleting step
because nothing is swept. A partial unique index allows one sweeping operation per project
(Deleting or Restoring), which serialises sweeps within a project and keeps the statistics fence
single-owner; further requests wait in Requested.
A crashed sweep is resumed by whoever takes over the expired lease. A sweep never gives up silently: if it cannot finish, it stays visible as "Deleting…" or "Restoring…" with its reason, and an engineer can lead it forward or run the reverse sweep, which works on partial progress because both directions select by operation id.
5.5 The deletion ledger¶
pmDeletionLedger, append-only, never updated, no TTL. One record per action (the kinds are listed in
behaviour §10): record id, kind, operation id, scope,
project id, search id, actor id (or system), time, counts. No names, titles, reasons or other free
text (the optional reason stays on the operation record), so the ledger can be read by support and
audit, and kept for ever, without exposing any content.
The integrated plan proposes pmProjectDeletionLedger with ProjectDeletionRecord and
ProjectRestorationRecord (ACD §3.3, marked PROPOSAL). This plan generalises it to searches; the ADR
fixes the final name.
5.6 The Project deletion state¶
Project.Deletion (nullable): State (Deleted), OperationId, RequestedBy, DeletedAtUtc,
RestoreWindowEndsAtUtc. It is a top-level field so that canonical conversion can carry it
(BC-R38: deleted projects convert in their deleted state), and its values match the ACD's
Project.deletionState (Active is the absence of the field).
5.7 Sequences¶
Delete a search.
- API: authorise (
RemoveSearch); check the flag; insert the operation asRequestedwith a lease; appendSearchDeletionRequested; answer202 Accepted. From now on new imports, uploads and bulk study updates in the project are refused.
As built (S4a, syrf#4239). SearchDeletionRequests (project-management core, used by both search
DELETE routes): fail closed (the typed 503) while the deletion indexes are missing; answer an unfinished
(Requested or Deleting) deletion of the same search as the idempotent repeat; 404 when the search
is not in the project; otherwise insert the operation as Requested without a lease (the runner's
first tick takes it; nothing needs it before then), append SearchDeletionRequested with the requester
as actor, publish IRunDeletionLifecycleCommand (best effort; the schedule is the backstop where
project-management's flag is on) and answer 202 Accepted with a DeletionOperationDto. The import-job
route records the same search-scope deletion, including for an import-job entry with no search
document. Details and the concurrent-request race: rollout §2.
2. Project-management takes over (a command; the recurring schedule is the backstop). In-flight work:
cancel the search's import, its cancellable Bulk PDF uploads and any version 2 bulk study update whose
plan touches its Studies; wait until each is terminal and has released its fence, and until no other
import, bulk update, recalculation, export or sweep of the project is running
(in-flight work). The ADR-020 check ThrowIfBulkUpdateInProgressAsync is part
of this wait, not a refusal.
3. Admit the statistics fence, keyed by the operation id, in its own transaction (when statistics are on
for the project). If another owner holds it, stay Requested and retry on the next tick. A crash
anywhere here is resumed by lease takeover, and every step is idempotent.
4. One short transaction: state Deleting, unlink the search, snapshot the link. The search leaves the
list now.
5. Sweep in batches of 500 (configurable), walking {ProjectId: p, _id > cursor} and updating Studies
that match {ProjectId: p, SystematicSearchId: s, BulkUpdateLock: null, Deletion: null,
PendingStatistics.OldestAtUtc: absent}: set ProjectId = holding, set Deletion, clear
SlotReservations, $inc Audit.Version. Record the cursor and counts; renew the lease.
6. Repeat until no Study of {ProjectId: p, SystematicSearchId: s} remains. Studies skipped because they
had pending fold entries are retried after the fold drains them (normally under a second; a bounded
wait raises BlockedReason = PendingStatisticsNotDrained for the operator).
7. One short transaction: state Deleted, set RestoreWindowEndsAtUtc, append SearchDeleted with the
final counts; then release the statistics fence as Stale with a client invalidation and publish the
SignalR SearchDeleted event.
As built (slice S4a-2). There is no cross-collection transaction; every step is a conditional write under the operation's lease and is idempotent, so a crash anywhere is finished by whoever takes the lease over.
- Steps 2 to 4,
Requested. When the in-flight review in a tick finds nothing outstanding for a search deletion, the same tick (so that review is fresh) reads the link snapshot, then admits the fence (ISearchDeletionStatisticsFence, keyed by the operation id, passing the tick's lease token so the admission's own transaction checks, throughIDeletionOperationStore.GuardAdmissionAsync, that this runner still holds the lease and the operation is stillRequested). Superseded (it is not): nothing is written and the current owner carries on. Busy: the operation staysRequested, blockedStatisticsFenceBusywith its first time kept, and asks again next tick. Admitted, or not requested because statistics are off: one write records the review, the fence step and the snapshot and moves the operation toDeleting, taking the project's sweep slot. If another operation holds the slot (ProjectSweepBusy), the fence is released again (a failed release is logged; the operation staysRequestedand its next admission is a retry under the same id) and the operation staysRequested. A project deletion keeps its S3 behaviour. - Unlink, start of every
Deletingtick until done. The search leaves the project (Project.RemoveSystematicSearch, which also drops its import-job entry) and the project leaves the search (SystematicSearch.RemoveProjectIdFromSearch), each a versioned save of an isolated copy, project first, retried three times against concurrent writers and otherwise left for the next tick.SearchUnlinkedAtUtcis written only when both sides are done. The search document is never deleted, another project's link is untouched, andStage.SystematicSearchFilteris left as it is. The search leaves the list here, before any Study moves. - Step 5, the sweep. Candidates are
{ProjectId: p, SystematicSearchId: s, _id > cursor}sorted by_id, limitBatchSize, hinted to the existing{ProjectId: 1, _id: 1}index (no blocking sort, no new index). One pipelineupdateManyon those ids, filteredProjectId: p, SystematicSearchId: s,StudyWriteGuard.WritableandPendingStatistics: null, setsProjectIdto the holding id andDeletion = {OperationId, OriginalProjectId, MarkedAtUtc}, emptiesSlotReservations, recomputes the stored session tallies as a save with no reservations would (reservation-only tallies dropped, reservation counts zero) and movesAudit.Version(a Study with noAuditgets the complete audit of a first save). The moved ids are read back by_id, their current reviewer presences are ended (EndReason = SearchDeleted, version moved; STATUS gap 3), and the cursor and counts are written. The lease is renewed before every batch and a tick stops after half the lease. A batch that skipped Studies signals the fold for the project. - Step 6, passes. When a pass reaches the end with Studies left behind, the cursor resets for the next tick's
pass and
SweepRetryingSinceUtcstarts. AfterPendingStatisticsWaitMinutes(default 15) without a Study moved, the operation is blockedPendingStatisticsNotDrained(orStudiesLockedByBulkUpdate), keeps retrying every tick, and is unblocked as soon as a batch moves a Study. - Takeover. At the start of each tick the Studies under the holding id are counted. More than recorded means a
runner died between a batch's Study write and its progress write: that batch's presences are closed (the
holding ids after the cursor) and
StudiesMovedis set to the real number, so counts are exact and nothing moves twice (a moved Study no longer matches the candidate filter). - Step 7, completion. When none of the search's Studies is left in the project: release the fence first (a
crash after it only releases again). If the release fails (after the fence's own bounded retries) the operation
stays
Deleting, blockedStatisticsFenceReleaseFailed, with no ledger record, and the next tick releases again: aDeletedoperation is never run again, so completing would leave the families fenced for good. Then appendSearchDeletedwithStudyCount = StudiesMoved(deterministic id; appended before the state write because aDeletedoperation is never run again), then writeDeleted, which snapshots the window and frees the sweep slot. - Step 7, the event (S4a's PR 4, syrf#4241). Once the
Deletedwrite is recorded, the stage publishes oneSearchDeletedNoticethroughISearchDeletedNotifier(bounded to 10 seconds; best effort: a failure is logged at Warning, changes nothing and is not retried, since aDeletedoperation never runs again). A completion write that is not recorded, or a failed fence release (which keeps the operationDeleting), announces nothing; whoever completes it announces it. Project-management'sBusSearchDeletedNotifierpublishesISearchDeletedEventthroughIBus(identifiers, the search's name when it can be read,StudiesMoved,DeletedAtUtc,RestoreWindowEndsAtUtc). The API'sSearchDeletedConsumerfans it out per pod as the hub methodSearchDeletedto the project's subscribers who still hold project View (rollout §2); the web reloads the project, refreshes a study list on screen and shows "Search "" was deleted."
Restore a search. The mirror image: authorise (RemoveSearch within the window, application
administrators at any time), refuse as listed in behaviour §4, wait
for running project-wide jobs as a deletion does, CAS the operation to Restoring (not a statistics
source write), admit the fence keyed by a restoration id derived from the operation, sweep
{ProjectId: holding, Deletion.OperationId: op} back to ProjectId = original (clearing the marker and
bumping the version), then for the returned Studies (a) re-persist the derived screening fields under the
current agreement settings through the ordinary version-guarded save (the same mechanism as the FEAT-024
re-persist-drift operation) and (b) remove answers to questions no longer defined, then relink the
search and its import-job entry, state Restored, release Stale, publish SearchRestored.
Delete a project. Authorise (owner); insert the operation as Requested and answer 202. Handle
in-flight work for the whole project as in steps 2 and 3 above (cancel every import, Bulk PDF upload and
bulk study update; wait for what cannot be cancelled, recalculations, exports and any search deletion or
restore). Then one
transaction: operation Deleted, set Project.Deletion with RestoreWindowEndsAtUtc, set
ProjectStatisticsControl.IsProjectDeleted (when a control exists), append ProjectDeleted. Publish
ProjectDeleted. No fence: no Study changes.
Restore a project. Authorise (owner within the window, application administrators at any time); one
transaction: clear Project.Deletion, clear IsProjectDeleted, operation Restored, ledger
ProjectRestored. The kept statistics rows are served again as they pass the usual checks; families that
changed while the project was deleted were already marked Stale by those changes
(statistics §6). Then release any reservation
recorded before the deletion through the ordinary release path, and request a backfill of every family
(it rebuilds only scopes that are not current).
5.8 Scale¶
The re-home touches one small field set per Study, so a batch of 500 is a quick single updateMany. A
100,000-Study search is 200 batches. The walk uses the {ProjectId, _id} index; it scans the project's
other Studies too, which costs one pass over the project's index entries in total. The largest production
search and project sizes are not known here, because production must not be queried without approval
(open question Q1); the FEAT-024 stats-large-v1 dataset at --scale 4 (100,000 Studies) is the
benchmark for slice S4 (see testing §6).
5.9 Places that can still see a moved Study¶
Reads that ignore ProjectId can still see re-homed Studies: id loads that do not compare the project,
and any global scan. Slice S1 lists every one (starting from the 37 id loads and the 20 direct-collection
files), makes each either compare ProjectId or treat a marked Study as not found, and adds a test per
path. Preview seeding (DeleteStudiesFromSystematicSearch) is exempt, as it is for ADR-020.