Specification: Training, assessment and the inference beta¶
Planning specification. This page turns the owner-session decisions of 4 and 5 October 2026 on
training (S4) and the first classification and inference capability (S5) into a buildable design.
Owner decisions recorded here are planning approval only. Brief approval and implementation
authorisation are separate per-gate decisions (D1-04) and remain on hold (owner, 5 October 2026).
No work has started, no gate has passed, nothing is enabled in production, and no automatic membership
change is authorised. The
owner-session consolidation
§5 ("Training and inference") wins over older package text where they conflict; the
register (S4 expansion, S4
manual assessment and retry, S5 behaviour, S5 beta opt-in) and the
condensed packages keep the
rationale. Names follow the brief's naming registry and are PROPOSALs for the F1a naming ADR; storage
is PROPOSAL for the F1a storage ADR. Tracker brief row: T-TI-00.
Companion specifications: reconciliation and screening (RS: accepted results that collective inference reads), review domain and versioning (RD: form and profile versions), stage pools, steps and history (SP: steps and pool events), access, communications and deletion (AC: groups and membership) and, since 6 October 2026, classification, populations and references (CPR: lane C1's explicit records, concepts and mappings, which the beta evaluates). 6 October 2026: DQ-09 is decided as option (a): rule authoring and publication sit inside the beta, and CPR's earlier proposal of rule authoring in C1 (option ©) is not adopted (§12; TI-R38).
1. Summary in plain English¶
Training. Before reviewers screen or extract for real, a project can ask them to practise on a small set of Studies whose expected answers an admin has written down. SyRF scores each attempt with rules the admin chose, or an assessor marks it by hand, and the reviewer sees feedback. If they pass, the project can add them automatically to a group whose existing permissions let them review for real. If they fail, the project's policy says whether they may try again. Every attempt, score, assessment and admission is kept.
Training is separate from the real review. Practice answers never count as votes, never fill a reviewer target, never become accepted answers and never appear in PRISMA counts. Turning practice answers into real evidence is a separate, explicit action that an authorised admin takes on purpose. Passing training gives only the group membership the policy names, and that group's existing permissions. Nothing else changes.
Training is planned, but it does not hold up the first general release (Chris, 6 October 2026: "No,
training does not gate GA"). Trainees see which items they got right or wrong after assessment, and
the expected answers only after a pass or when no retry remains (decided as recommended on
6 October 2026). It is proposed that a designer can also attach guidance for trainees to read before
they start (§3.10, TI-R37, PROPOSAL).
Example. Tom Liu joins the stroke review. He has access to the "Training" stage only. He screens 20 practice Studies and scores 72%. The policy needs 80% and allows one more attempt after an admin approves it. Aisha approves; Tom tries again and scores 90%, but one free-text answer needs a human judgement. Ewan marks it correct and passes the attempt. SyRF adds Tom to the "Screeners" group, whose existing grant lets him screen in the live title/abstract stage. The record shows which attempt, which assessment, which policy version and which actor caused that.
Inference beta. Some forms record experimental groups (cohorts) and how they relate, for example "the pregnant animals are a subset of the female animals". With project rules such as "Pregnant is contained in Female", SyRF can point out relationships the reviewer did not state, contradictions, and safe totals. This capability starts as a beta, off by default. A project designer has to switch it on, after reading what it does. Ordinary annotation and reconciliation work the same whether or not a project opts in. Writing and publishing project rules, evaluating them and showing inferred results all need the beta; recording groups, classifications, shared concepts and which local term means which concept does not (Chris, 6 October 2026, DQ-09).
The first version understands only four kinds of statement: "all of these together" (conjunction), "this group sits inside that group" (containment), "these groups share no members" (disjointness) and "these groups together cover everything in scope" (exhaustiveness). It works from one reviewer's own answers, shown only to that reviewer, or from a Study's accepted answers. It never mixes two reviewers' unreconciled answers. It never changes a reported answer and never invents an animal count. When the answers or the rules change, SyRF recomputes, marks the old conclusion as replaced, and keeps it in history.
2. Decisions covered¶
| ID | Decision in one line | Status | Section |
|---|---|---|---|
| D4-04 | A separate training step, excluded from live votes, targets and PRISMA; promotion to live evidence only by explicit action | Decided-amended (S4 expansion; the defer-to-hook-only fallback is not sufficient) | §3.1 to §3.6, §5.1 |
| S4 expansion | Admin-defined training reference versions; scoring rubrics and pass criteria; manual evaluation; optional automatic admission to a project group; no hidden privilege | Decided (amendment outside the register count) | §3.2, §3.3, §5.2 to §5.4 |
| S4 manual assessment and retry | Authorised assessors pass or fail attempts; the training policy decides whether a failed reviewer may retry; every attempt and assessment kept | Decided (amendment outside the register count) | §3.4, §3.5, §5.3 |
| Q-18 | The first reasoner supports conjunction, containment, disjointness and exhaustiveness only | Decided (S5) | §5.6 TI-R29 |
| Q-19 | The Design capability publishes project rules; reviewers confirm per-paper applicability through mapping answers; normal reconciliation resolves disagreements | Decided (S5) | §5.6 TI-R30 |
| S5 approved interpretation behaviour | Candidate inference from the reviewer's own snapshot; collective inference from a pinned accepted result; recompute and withdraw with history; reported answers preserved | Decided (amendment outside the register count) | §3.9, §5.6 |
| S5 beta opt-in amendment | Beta, off by default, explicit project-designer opt-in; never activated by baseline conversion; GA promotion separate | Decided (amendment outside the register count) | §3.7, §5.5 |
| D4-15 | Accepted-answer branching between steps | Deferred beyond MVP (owned by SP); inference never routes work | §5.6 TI-R36 |
| Training thresholds, feedback disclosure, retry defaults, rubric changes | Exact values and behaviour | Brief item (6 October 2026: the feedback disclosure default is decided, DQ-31 below) | §12 |
| Inference refresh timing and performance guarantees | Exact invalidation timing and budgets | Brief item | §12 |
| Training does not gate GA (owner answer, 6 October 2026) | "No, training does not gate GA": the training lane (TR1) is delivery scope and GA never waits for it; this answers rollout R-AMB-01 as option (a) | Decided (Chris, 6 October 2026) | §10 |
| DQ-31 (what trainees see after an attempt) | Per-item correctness after the effective assessment; reference answers only after a pass or when no retry remains | Decided as recommended (Chris, 6 October 2026) | §5.3 TI-R10 |
| DQ-32 (a live step that depends on passed training) | Recommended: group admission as the only access gate; TI-R23's dependency out of MVP | Open (Chris is unsure, 6 October 2026) | §5.4 TI-R23, §12 |
| Training guidance before the attempt (Chris's question, 6 October 2026) | Versioned guidance on the training step, shown before the attempt, with an optional acknowledgement | PROPOSAL |
§3.10, TI-R37 |
| DQ-09 (what the inference beta covers) | Option (a): rule authoring and publication, rule evaluation and inferred results require the project's explicit beta opt-in; ordinary annotation and explicit group or classification recording follow their own availability; CPR's option © is not adopted. A class B clarification: it neither accelerates classification delivery nor adds a separate review workflow | Decided, option (a) (Chris, 6 October 2026; the record says he found the recommendation made sense, "only rule-writing and inference require the beta"; decision register §1.18) | §3.7, §3.8, TI-R30, TI-R38, §12 |
3. Concepts, entities and storage¶
3.1 TrainingStep¶
- What it is. A step kind inside a stage (the R3a step model; SP §3.4 "Step kinds" and SP-R91) where reviewers practise one annotation form, one screening profile, or both, against reference answers.
- Identity. A step inside a
StageSettingsVersion, like any other step. - Binds. An exact form version and/or profile version, one
TrainingReferenceVersionand oneTrainingPolicyVersion. 6 October 2026 (PROPOSAL): and at most oneTrainingGuidanceVersion(§3.10). - Versions and pointers. Changing what the step binds publishes a new stage settings version. Attempts pin the versions current when they start.
- Proposed storage (
PROPOSAL). Part ofpmStageSettingsVersion. - What it is not. Training attempts create no live work, no claims and no stage-pool events; the
stage's own pool history still follows SP. It is not a hidden prerequisite for other steps. A live
step can depend on it only if a designer configures that dependency explicitly (an optional
PROPOSAL, §12). 6 October 2026: that dependency is open as DQ-32, and the recommendation is to leave it out of MVP (§12).
3.2 TrainingReferenceVersion¶
- What it is. The immutable set of practice items and their expected answers: the training gold standard.
- Identity.
(projectId, trainingReferenceId, seq). - Content. The practice items (each a project Study), and per item the expected answers or decision and reasons, pinned to exact form, profile and question versions; optional per-item notes for assessors; the author and time of each change.
- Seeding. An admin may seed a new reference version by copying an existing accepted result or
screening outcome for a Study. The copy records its source version and never stays linked
(
PROPOSAL). - Mutability. Immutable once published; edits publish a new version.
- Proposed storage (
PROPOSAL).pmTrainingReferencehead plus append-onlypmTrainingReferenceVersion. - What it is not. It is never part of
StudyGold, never anAcceptedResultVersionand never a screening outcome.
3.3 TrainingPolicyVersion¶
- What it is. The immutable rules for scoring, passing, feedback, retries, admission and promotion.
- Identity.
(projectId, trainingPolicyId, seq). - Content.
- Scoring rubric per supported answer and decision type (§5.2 table) and per-item overrides.
- Pass criteria: an overall score threshold, optional "must be correct" items, optional minimum item count. Numbers are proposed per policy and not approved platform defaults.
- Assessment mode: automatic only; automatic with manual assessment of judgement items; manual only.
Whether a manual decision may override the automatic result (default yes,
PROPOSAL). - Feedback disclosure: what the trainee sees and when (§5.3).
- Retry rules: whether retries are allowed, a maximum number of attempts, whether a retry needs approval, an optional waiting period.
- Admission: none, automatic on pass, or manual approval on pass, and the target project group.
- Whether promotion to live evidence is allowed for attempts under this policy.
- 6 October 2026 (
PROPOSAL, TI-R37): guidance handling: whether the trainee is asked to acknowledge the guidance ("I have read this") before starting, and whether the guidance stays visible during the attempt. The brief proposes the defaults. - Mutability. Immutable once published; edits publish a new version.
- Proposed storage (
PROPOSAL).pmTrainingPolicyhead plus append-onlypmTrainingPolicyVersion.
3.4 TrainingAttempt and its training sessions¶
- What it is. One reviewer's single go at a training step.
- Identity.
(projectId, trainingStepRef, reviewerId, attemptNumber)with a deterministic ID. - Content. The pinned reference, policy, form and profile versions; one training session per
practice item; state (
InProgress,Submitted,AwaitingManualAssessment,Passed,Failed,Abandoned); the current assessment pointer; the admission outcome (Admitted,AlreadyMember,AwaitingApproval,ConvertedToManualwith reason,Refusedwith reason, or none). - Training sessions. Entities of the attempt with author scope
training(attemptId)(PROPOSALextension ofAuthorScope). They reuse the session engine (drafts, Save checkpoints, validation and the AF2 renderer) and are keyed by attempt and item, so they can never collide with the reviewer's liveFormSessionfor the same Study. - Mutability. State changes are append-only transitions; submitted training sessions are immutable.
- Proposed storage (
PROPOSAL).pmTrainingAttempt; training sessions and revisions in the evidence collections under the training author scope, or in dedicated collections if the F1a ADR prefers physical separation. Either way, every live query filters them out by construction. - What it is not. It is never a candidate session, never qualifies, never holds a capacity place and never appears in live statistics.
3.5 TrainingAssessment¶
- What it is. One immutable judgement of an attempt: automatic (by the system under a rubric version) or manual (by a named assessor).
- Identity.
(attemptId, seq). - Content. Kind (
Automatic,Manual); per-item scores and judgements; the overall score; the decision (Pass,Fail, orPartialwhile manual items remain); assessor feedback; the rubric and reference versions used; the assessor or system rule; time. - Mutability. Append-only. A new assessment never overwrites an earlier one; the attempt's pointer moves to the effective one under the policy.
- Proposed storage (
PROPOSAL). Embedded inpmTrainingAttemptor apmTrainingAssessmentcollection.
3.6 Admission and promotion records¶
- Admission. Uses the existing group membership contracts (R1c) and the authorization programme's
authorizationAudit(C10: no new audit store). The membership audit entry carries the causation: attempt, effective assessment, policy version and actor (the system rule for automatic admission, or the approving admin). - Promotion. An explicit operation that creates new live candidate session versions or decisions
from selected training items. Each new live revision carries
promotedFrom(attempt, item, assessment). The training records never change.PROPOSALstorage: an operation record inpmCanonicalOperation.
3.7 The project's inference beta setting¶
- What it is. An audited project setting that says whether the inference beta is on.
- Content. Enabled or disabled; the actor and time of each change; the version of the explanation text the designer acknowledged; the policy version in force.
- Default. Off for every project, including projects created from templates, copied projects and baseline-converted projects.
- Proposed storage (
PROPOSAL). The project's operational settings with history inpmDefinitionSettingsAudit. - What it gates (
OWNERDQ-09, option (a), 6 October 2026). Rule authoring and publication (ClassificationRuleVersion), rule evaluation and inferred results (TI-R38). - What it is not. It does not switch on explicit classification recording (lane C1), which keeps its own enablement (§12, decided; C1's own per-project setting is CPR-R39). Shared concepts and per-paper concept mappings stay in C1.
3.8 ClassificationRuleVersion and mapping answers¶
- What it is. An immutable version of a project rule (the existing
ProjectRulehead) that links shared concepts with one of the four supported relationships, inside a stated scope. - Identity.
(projectId, ruleId, seq). - Content. Kind (
Conjunction,Containment,Disjointness,Exhaustiveness), the concepts or option sets it links, the scope, the expression AST under a versioned grammar, publisher and time. - Mapping answers. Whether a rule applies to a given paper is recorded by reviewers as ordinary answers (C13). Disagreements go through ordinary reconciliation. Concept and mapping authoring as explicit records in C1 are specified in CPR §3.6 and §3.7 (6 October 2026). DQ-09, decided as option (a) (6 October 2026): rule versions are authored and published only in a project with the beta on, so rule-applicability answers arise only there; CPR's option ©, rule authoring in C1 outside the beta, is not adopted (TI-R38; §12, "Scope of the beta").
- Proposed storage (
PROPOSAL).pmProjectRulewith append-only versions.
3.9 InferenceResult¶
- What it is. A derived, recomputable, explainable conclusion about one Study's cohorts.
- Identity.
(projectId, studyId, populationId, basisKind, basisId, ruleVectorDigest, engineVersion).basisKindisCandidateSnapshot(basis = one reviewer's session version) orAcceptedResult(basis = oneAcceptedResultVersion). - Content. Propositions with a proof state (
Proven,Disproven,Unknown,Conflicting,Unsupported,NotEvaluated), proof dependencies, reason codes, inferred groups and their outcome associations, contradictions with their supporting evidence, and count statements only where proven. - Lifecycle.
Current,Superseded(inputs or rules changed; links its successor),Withdrawn(beta disabled or rule retired). Superseded and withdrawn results stay as history while the project exists. - Proposed storage (
PROPOSAL).pmInferenceResult(already planned). Current results are rebuildable and a rebuild gives identical results; history records are kept as written. - What it is not. Never authoritative, never a reported answer, never a vote, never an accepted answer, never a count used by PRISMA.
3.10 TrainingGuidanceVersion (PROPOSAL, 6 October 2026)¶
Chris asked whether trainees will get guidance to read before being tested. The training records above had none, so this record is proposed (TI-R37).
- What it is. Versioned guidance that a trainee reads before an attempt: rich text, links and attached files, for example a screening guide and a link to the protocol.
- Identity.
(projectId, trainingGuidanceId, seq). - Binds and pins. A training step binds at most one guidance version. Attempts pin the version current when they start, like the reference and policy versions. Where the policy asks for an acknowledgement, the attempt records the trainee, the guidance version and the time.
- Shown. Before the attempt starts, always. During the attempt, as the policy version says (§3.3).
- Mutability. Immutable once published; edits publish a new version and never change an attempt in progress.
- Proposed storage (
PROPOSAL).pmTrainingGuidancehead plus append-onlypmTrainingGuidanceVersion, or embedded as the F1a storage ADR decides. Files stay in the project's existing file storage and are referenced by ID. - What it is not. It is not a reference version and never holds expected answers. It is not the form's operational guidance text (A-15), which it may link to.
4. What loads and writes when¶
Writes follow the C18 commit protocol; after-commit work uses C19 durable effects; history uses the C20
HistoryEvent envelope.
4.1 An admin publishes a reference, policy or guidance version¶
6 October 2026: guidance versions (§3.10, PROPOSAL) publish the same way.
| Step | Detail |
|---|---|
| Reads | The draft; the bound form, profile and question versions; for automatic admission, the target group, its grants and the publisher's authority over that group (C10 anti-escalation) |
| Writes | The new immutable version and the head pointer |
| Transaction | One transaction on the reference or policy aggregate |
| Derived after | Attempts already in progress keep their pinned versions; the designer sees how many; HistoryEvent TrainingDefinitionPublished |
4.2 A reviewer starts an attempt¶
| Step | Detail |
|---|---|
| Reads | Review access to the training step's stage; prior attempts and their outcomes; the current policy version's retry rules; the current reference version; 6 October 2026 (PROPOSAL): the bound guidance version, which is shown before the attempt starts |
| Writes | A TrainingAttempt pinned to the current versions, with empty training sessions. 6 October 2026 (PROPOSAL): the attempt also pins the guidance version; where the policy asks for an acknowledgement, it is recorded (trainee, guidance version, time) before the attempt starts |
| Transaction | The attempt aggregate; the deterministic ID makes a double start idempotent |
| Derived after | My work shows "Training · attempt 2 of 3" |
4.3 A reviewer answers practice items¶
Autosave writes the training session draft (no Study write, no claim, no presence in live counts). Save
writes a training session checkpoint. None of this touches live evidence, Study.CanonicalSummary,
capacity or pools.
4.4 A reviewer submits the attempt¶
| Step | Detail |
|---|---|
| Reads | Every training session (all items answered or explicitly left unanswered; 6 October 2026: an answer may be a value or a reason no value is provided, RD §3.16, DQ-46); the rubric |
| Writes | Attempt state Submitted; the automatic TrainingAssessment; the attempt state Passed, Failed or AwaitingManualAssessment |
| Transaction | One transaction on the attempt when scoring fits the budget; otherwise scoring is a durable effect keyed by the attempt and the state shows "Scoring" |
| Derived after | Feedback per the policy; an assessment task in My work for assessors when manual items remain; admission (§4.6) when passed |
4.5 An assessor assesses an attempt¶
| Step | Detail |
|---|---|
| Reads | Assessor eligibility (§6); the attempt, its training sessions, the reference version and the rubric |
| Writes | A manual TrainingAssessment; the attempt's effective assessment pointer and state |
| Transaction | Compare-and-set on the attempt's assessment pointer; a concurrent assessment returns a typed conflict and keeps the assessor's draft |
| Derived after | Feedback per the policy; admission when the effective result is Passed; notice to the trainee (capture only; delivery not authorised) |
4.6 Admission after a pass¶
- Automatic mode. A durable effect keyed by the attempt runs the group membership command through the R1c contracts as the system actor on behalf of the policy.
- Checks at execution. The group still exists; its grants are still within what the policy publisher could administer when they published; the trainee's project membership is active. If any check fails, the effect converts to a manual approval request with the reason.
- Writes. The membership change and its
authorizationAuditentry with causation, and the attempt's admission outcome. An existing membership recordsAlreadyMemberand changes nothing. - Manual mode. A request appears in My work for members who hold EditMemberships and authority over that group's grants. Approval runs the same membership command with the approver as actor.
- Derived after. The trainee's next request sees the new group's grants;
HistoryEventTrainingAdmissionAppliedorTrainingAdmissionConverted.
4.7 A failed reviewer asks to retry¶
The policy version pinned by the failed attempt decides. If retries are allowed without approval, the next attempt can start (subject to any waiting period). If approval is needed, a request goes to authorised admins; approval is recorded on the failed attempt and unlocks one new attempt.
4.8 An admin changes the rubric and re-scores¶
Publishing a new policy or reference version never re-scores anything by itself. An admin may run "Re-score under policy vN" on chosen attempts. That writes new automatic assessments; earlier assessments stay; a changed effective result is shown with both versions. A re-score never removes an admission; removing membership is a separate explicit action.
4.9 An admin promotes training responses to live evidence¶
| Step | Detail |
|---|---|
| Reads | Selected attempts and items; whether the policy allows promotion; for each item, the reviewer's live session on that Study × form or decision on that Study × profile; live admission, eligibility and capacity at this moment; the exposure history for the item |
| Writes | Per item, a new live candidate session version or decision with promotedFrom provenance and an informed marker when feedback or the reference was shown first; an operation record |
| Transaction | One study-scoped transaction per item; the operation record makes retries idempotent |
| Derived after | Live readiness, task input sets and filter reevaluation as for any live Complete (RS, SP) |
4.10 A project designer enables or disables the inference beta¶
Enabling reads the designer's Design capability and records the setting change with the acknowledged
explanation version. Inference panels appear on the next load. Disabling records the change; a durable
effect marks every current InferenceResult Withdrawn with reason BetaDisabled. No answer changes in
either direction.
4.11 A designer publishes a rule version¶
Publishing needs the project's inference beta to be on (DQ-09, TI-R38); with it off the command is
refused and the rule pages are not shown. The preview shows how many Studies and current results
depend on the rule (reverse dependency index). Publication writes the immutable version. A durable effect recomputes the affected results and marks the
old ones Superseded.
4.12 A reviewer saves or completes a session with cohort answers¶
On an opted-in project, the reviewer's panel computes candidate inference on demand from their own committed session version. While they edit, a provisional preview runs on their own draft and is labelled provisional. A committed result is written for the session version, superseding the reviewer's previous one. Nobody else can read it.
4.13 A new accepted result is published¶
A durable effect keyed by (AcceptedResultVersion, ruleVectorDigest, engineVersion) computes the
collective result and supersedes the Study's previous collective result. Members who can see accepted
answers can see it.
5. Rules¶
5.1 Training scope and separation¶
- TI-R01 Training step. Projects can define training steps for annotation forms and screening activities, each binding exact form or profile versions. Owner decision: S4 expansion.
- TI-R02 Reference answers are separate. Training reference versions are distinct from live accepted
evidence; they are never written to
StudyGoldand never become accepted results or screening outcomes. Owner decision: S4 expansion. - TI-R03 Immutable references. Reference versions are immutable; edits publish a new version with actor and time; attempts pin the version they started with. Owner decision: S4 expansion.
- TI-R04 Separation from live evidence. Training creates no live votes, candidate sessions,
qualifying contributions, accepted results, stage-pool events,
WorkFirstReleasedevents, PRISMA counts, FEAT-024 inputs, default agreement figures, claims or capacity use. Owner decision: D4-04; S4 expansion; consolidation §5. - TI-R05 Practice Studies. Practice items are project Studies chosen by the admin. SyRF recommends
Studies that are already sufficiently reviewed or otherwise outside live work. If a trainee later
reviews the same Study live after seeing that item's reference or feedback, the live session version
is informed (exposure kind
trainingReferenceShown).PROPOSAL, following S4's identity and blinding rule.
5.2 Scoring¶
- TI-R06 Rubrics per type. Scoring rubrics and pass criteria are configurable per supported type.
Exact-text equality never grades free text as if it measured meaning. Owner decision: S4 expansion;
the default table is
PROPOSAL.
| Answer or decision type | Default automatic rule (PROPOSAL) |
Rubric options (PROPOSAL) |
|---|---|---|
| Single choice and yes/no | Exact option match | Mark specific wrong options as acceptable |
| Multiple choice | Identical option set (the AG2 rule) | Per-option partial credit |
| Numeric | Within a per-question tolerance (absolute or relative) | Tolerance per item |
| Unknown, Not reported and every other reason no value is provided (6 October 2026: one row, merging the earlier "Unknown and Not reported" row; RD §3.16; RS-R15; term per DQ-46) | Compared as options: the same reason matches; a reason against a value, or against a different reason, is wrong | Accept a named reason, or either of two named ones, for an item |
"Other, please specify" (6 October 2026, DQ-46: a value, never a reason; PROPOSAL row) |
The option is compared like any option; the specified text is free text and goes to manual assessment | Accept the Other option with any text for an item |
| Unanswered where the reference has a value (6 October 2026: was "Blank") | Scored as missed | None |
| Free text | Not scored automatically; manual assessment | Assessor notes per item |
| Repeatable entities and branches | Manual assessment (needs matching) | A match-based rubric later (§12) |
| Screening decision | Exact Include, Exclude or Unsure match | Accept Unsure for an item whose reference is Include |
| Exclusion reasons | Primary-reason match, or set match, as the policy says | Per-item acceptable alternatives |
- TI-R07 Pass criteria. An overall threshold, optional must-be-correct items and an optional minimum item count. Any number is proposed per policy and not approved as a platform default. Owner decision: S4 expansion.
5.3 Assessment, feedback and retries¶
- TI-R08 Manual assessment. An authorised assessor can pass or fail any attempt. Each assessment is
immutable. Under the default policy, the latest manual decision is the effective result; automatic and
manual assessments are both kept. Owner decision: S4 manual assessment; override default
PROPOSAL. - TI-R09 Assessor eligibility. Assessors hold the training assessment capability for the step (
NEW, working name "Assess training") and never assess their own attempt.PROPOSAL. - TI-R10 Feedback disclosure. The policy chooses what the trainee sees (none, score only, per-item
correctness, per-item with reference answers, assessor comments) and when. References are never shown
before submission. Default: per-item correctness after the effective assessment; reference answers
only after a pass or when no retry remains, so a retry cannot simply replay the answers. Owner
decision: S4 asks the brief to specify; default
PROPOSAL. Decided 6 October 2026: Chris answered DQ-31 as recommended, so this default stands: trainees see right or wrong per item after assessment, and the expected answers only after a pass or when no retry remains. The other disclosure options stay available per policy. - TI-R11 Attempts preserved. A failed attempt is never overwritten. Owner decision: S4 manual assessment.
- TI-R12 Retries by policy. The policy version pinned by the failed attempt decides whether a retry is allowed, the maximum attempts, whether approval is needed and any waiting period. Values are proposed, not approved. Owner decision: S4 manual assessment.
- TI-R13 Rubric changes. A changed rubric or reference publishes new versions. Existing assessments
stay. Only an explicit re-score writes new assessments. A re-score never removes an admission. Owner
decision: S4 asks the brief to specify; rule
PROPOSAL. - TI-R14 Calibration feedback. Agreement of trainees with the reference, per item and per
criterion, appears in a separate training report. It never enters the default agreement view. Owner
decision: D4-04; methods per
T-SI-02.
5.4 Admission and promotion¶
- TI-R15 Optional automatic admission. A policy may admit a passed reviewer automatically to one named project group whose configured permissions allow live reviewing. Manual approval is also available. Owner decision: S4 expansion.
- TI-R16 Recorded causation. Every admission records the attempt, effective assessment, policy version and the system or assessor actor that caused it. Owner decision: S4 expansion.
- TI-R17 No hidden privilege. Admission adds only the named group membership, and the group's existing
grants govern. At publication the policy publisher must be able to administer that group (C10
anti-escalation). At execution SyRF rechecks that the group exists and that its grants are still
within what the publisher could administer; otherwise the admission becomes a manual approval request.
Owner decision: S4 expansion; check mechanics
PROPOSAL. - TI-R18 Idempotent. One membership action per attempt at most; an existing membership records
AlreadyMember; retries and duplicate effects create no duplicate grants; concurrent membership or configuration changes resolve through the group contracts. Owner decision: S4 expansion. - TI-R19 Admission only adds. Admission never removes the trainee from another group. Any removal is a
separate membership action.
PROPOSAL. - TI-R20 Explicit promotion. Promotion to live evidence is a separate, audited action. It creates new live versions with provenance and never relabels training records. It never happens as a side effect of scoring or admission. Owner decision: D4-04; S4 expansion.
- TI-R21 Promotion limits. Promotion is refused for an item where the reviewer already has a live
session or decision on that Study and form or profile. It is evaluated against live admission,
eligibility and capacity at that moment. Promoted evidence is informed when the item's reference or
feedback was shown first.
PROPOSAL. - TI-R22 Identity and blinding. Trainees never see other trainees' attempts. Assessors see the
trainee's identity unless the policy sets blinded assessment. Training reports for others are
aggregate. Protected audit applies. Owner decision: S4 expansion (identity, blinding and protected audit
rules); blinded assessment option
PROPOSAL. - TI-R23 Within the step model. The owner-approved gate is group admission. An explicit dependency
from a live step on "passed this training step" is an optional
PROPOSAL(§12). 6 October 2026: open as DQ-32; the recommendation is to leave this dependency out of MVP (§12).
5.5 Inference beta activation¶
- TI-R24 Beta, off by default. The inference capability starts in beta and is off for every project. Only an authorised project designer (Design capability) can enable it. Templates, project copies and baseline conversion never enable it. Owner decision: S5 beta opt-in.
- TI-R25 Informed opt-in. The opt-in screen shows the beta status, what the capability does and its implications. Every enable and disable records actor, time and the policy version. Owner decision: S5 beta opt-in.
- TI-R26 Ordinary work unaffected. Annotation and reconciliation work fully without the beta. Inference is never required for Save, Complete or reconciliation. Owner decision: S5 beta opt-in; S5 behaviour.
- TI-R27 Disabling. Disabling stops new inference, keeps every reported answer and keeps historical results and provenance. Owner decision: S5 beta opt-in.
- TI-R28 GA is separate. Moving out of beta is a separate readiness and approval decision. Owner decision: S5 beta opt-in.
- TI-R38 Scope of the beta. Rule authoring and publication, rule evaluation and inferred results require the project's explicit beta opt-in. Ordinary annotation and explicit group or classification recording follow their own availability and need no beta: lane C1 keeps explicit recording, shared concepts and per-paper concept mappings under its own setting (CPR-R39). With the beta off, the rule pages and inference panels are not shown and rule authoring or publication is refused. This does not accelerate classification delivery or introduce a separate review workflow. Owner decision: DQ-09, option (a) (Chris, 6 October 2026); CPR's option © is not adopted.
5.6 Inference behaviour¶
- TI-R29 First reasoner scope. Conjunction, containment, disjointness and exhaustiveness only. Any
other expression returns
Unsupportedand is never replaced by a weaker one. Owner decision: Q-18. - TI-R30 Rule publication. The Design capability publishes rule versions. Reviewers confirm per-paper applicability through mapping answers. Disagreements go through normal reconciliation. Owner decision: Q-19. 6 October 2026: only in a project with the beta on (TI-R38; DQ-09).
- TI-R31 Inputs. Candidate inference uses only the viewing reviewer's own authorised snapshot: their
committed session version, or their own draft for a preview labelled provisional. Collective inference
uses only a pinned
AcceptedResultVersion. SyRF never combines different reviewers' unreconciled assertions. The beta enables no other authority. Owner decision: S5 behaviour; draft preview and the beta limitPROPOSAL. - TI-R32 Who sees what. A candidate inference is visible only to that reviewer; in the beta it is not
shown to reconcilers, because revealing inferred structure could break blinding. Collective inference
is visible to members who can see the accepted answers. Owner decision: S5 behaviour (blinding
boundaries); detail
PROPOSAL. - TI-R33 Reported answers and counts. Inference never writes or changes answers, votes, accepted
results or counts. A count statement appears only where the four operators prove it (for example, a
disjoint and exhaustive partition sums to its total; a contained group adds nothing to its container).
Bounds reasoning (for example "between 70 and 100") waits for a later approved scope. Owner decision:
S5 behaviour ("do not invent animal counts"); count scope
PROPOSAL. - TI-R34 Explain and flag. Inferred groups, outcome associations and contradictions are shown with
their evidence and rule provenance. A contradiction keeps the evidence, withholds the conclusions that
depend on it, and is flagged. Owner decision: S5 behaviour; contradiction handling from the
classification research §3.2 (
PROPOSAL). - TI-R35 Recompute and retire. When exact inputs or rule versions change, SyRF computes a new result
and marks the old one
Supersededwith its reason and successor. A stale result is never shown as current. Owner decision: S5 behaviour. Timing and budgets are brief items; until measured, results are computed on demand within a budget, and an exhausted budget returnsNotEvaluated. - TI-R36 No workflow of its own. Inference is not a step, a session or a routing rule. It never offers or withholds work. Owner decision: S5 behaviour; D4-15 deferred.
5.7 Training guidance (6 October 2026)¶
- TI-R37 Guidance before the attempt. A training step may bind one
TrainingGuidanceVersion: versioned guidance (rich text, links or files) shown to the trainee before an attempt starts. The policy version chooses whether an "I have read this" acknowledgement is asked for before the attempt starts, and whether the guidance stays visible during the attempt. Attempts pin the guidance version they started with; an acknowledgement records the trainee, the guidance version and the time. Guidance is separate from the reference version and never draws on its expected answers. Source: Chris's question of 6 October 2026 (will trainees get guidance to read before being tested?); rulePROPOSAL.
6. Authorisation, blinding and provenance¶
| Action | Capability (NEW = proposed) |
Further checks | Provenance recorded |
|---|---|---|---|
| Configure a training step | Design on the stage | None | Stage settings version |
Draft and publish reference and policy versions (6 October 2026: and guidance versions, PROPOSAL) |
Design to draft; NEW Publish review definitions to publish (permission matrix) |
Automatic admission needs authority over the target group (C10) | Version, actor, time |
| Take training | Review on the training step's stage | Retry rules | Attempt, versions pinned |
| Assess an attempt | NEW Assess training |
Not the trainee | Assessment, assessor, rubric and reference versions |
| Approve a retry | NEW Assess training or Design |
Policy requires approval | Approval on the failed attempt |
| Approve a manual admission | EditMemberships plus authority over the group's grants | Anti-escalation | authorizationAudit with causation |
| Promote to live evidence | NEW Promote training evidence (project scope) |
Live admission at commit | Operation record; promotedFrom |
| See training results of others | Assess training, or NEW View training results |
Aggregate unless permitted | Read audit where required |
| Enable or disable the inference beta | Design on the project | Acknowledged explanation | Setting history |
| Publish a rule version | Design on the project (Q-19) | The project's beta is on (DQ-09, TI-R38) | Rule version, actor, time |
| See candidate inference | The reviewer whose snapshot it is | None | Basis session version |
| See collective inference | Accepted-answer view access | Disclosure policy | Basis accepted result version |
Capability names are working names for the F1b catalogue. Assignment or notification never grants any of them (C10).
History events emitted (C20 envelope; names PROPOSAL): TrainingDefinitionPublished,
TrainingAttemptStarted, TrainingAttemptSubmitted, TrainingAssessmentRecorded,
TrainingRetryApproved, TrainingAdmissionApplied, TrainingAdmissionConverted,
TrainingEvidencePromoted, InferenceBetaChanged, ClassificationRulePublished,
InferenceResultSuperseded.
7. Active-work impacts¶
| Change | Warned before commit | Recheck at commit | Notified after (capture only; delivery not authorised) | Apply anyway |
|---|---|---|---|---|
Publish a new reference or policy version (6 October 2026: or guidance version, PROPOSAL) |
Attempts in progress and their trainees | Head pointer | Those trainees (their attempts keep the old versions) | Not applicable |
| Change the admission group or mode | Passed attempts with pending admissions | Group authority | Admins handling manual requests | Not applicable |
| Remove a trainee's access to the training stage | Their attempt in progress | Grants at submit | The trainee; drafts kept | Not applicable |
| Delete or restrict the admission group | Policies naming it | Group contracts | Policy owners; pending admissions become manual | Not applicable |
| Promote training responses | Live impact: targets, readiness, tasks whose inputs change, filter reevaluation | Live admission per item | Reconcilers of affected tasks | Not applicable; refused items are listed |
| Enable the inference beta | What reviewers and reconcilers will see | Setting history | None needed | Not applicable |
| Disable the inference beta | Number of current results that will be withdrawn | Setting history | Designers | Not applicable |
| Publish a rule version | Studies and results to recompute | Rule head | None needed | Not applicable |
8. Failure and recovery¶
| Failure | Behaviour | Recovery |
|---|---|---|
| Automatic scoring fails or times out | The attempt shows "Scoring"; it is never failed automatically | The durable effect retries; an assessor can assess manually |
| Two assessors assess at once | Compare-and-set on the assessment pointer; the second gets a typed conflict and keeps a draft | The second assessor reloads and decides whether to add an assessment |
| Admission effect fails | Attempt shows "Passed · admission pending" | Idempotent retries; after repeated failure the effect becomes a manual request |
| Group deleted or its grants enlarged beyond the publisher's authority | Automatic admission refused | Converted to a manual request with the reason |
| Promotion partly fails | Successful items commit; refused items are listed with reasons | Rerunning skips completed items |
| Inference computation fails or exceeds its budget | NotEvaluated; source answers stay readable |
Recompute on demand or on the next input change |
| A recompute races a newer input | Expected-revision check; a stale calculation never overwrites a newer result | The newer input's recompute wins |
| Reverse dependency index lags | Affected results show "Stale" and are not presented as current | The recompute catches up; a rebuild reproduces current results |
9. User flows and examples¶
Example project. "Neuroprotection in focal ischaemia (example)". Stages: "Training" (one training step on the Title/abstract profile v1 and one on the Study design form v3) and "Title/abstract screening". Groups: "Trainees" (Review on "Training") and "Screeners" (Review on "Title/abstract screening"). People: project designer Aisha Rahman, trainee Tom Liu, assessor Ewan Fraser, reviewers Priya Shah and Omar Haddad, owner Daniel Okafor. Practice items: 20 Studies that already have definite title/abstract outcomes, so they will not be offered again for live screening.
9.1 Setting up training¶
Aisha creates reference version 1 by seeding expected decisions from the 20 Studies' current outcomes,
then edits two of them and adds reasons. She publishes policy version 1: per-item correctness feedback
after assessment, references shown after a pass, pass mark 80% (proposed for this project, not a
platform default), one retry with admin approval, automatic admission to "Screeners". Publishing checks
that Aisha can administer "Screeners"; she can. She also binds guidance version 1, a two-page screening
guide with a link to the protocol, asks trainees to confirm they have read it, and keeps it visible
during attempts (TI-R37, PROPOSAL).
9.2 Failing, retrying and passing¶
- Tom reads the screening guide, ticks "I have read this" (wording illustrative;
PROPOSAL), then starts attempt 1. His screen says "Training · attempt 1 of 2". He saves halfway ("Version checkpoint saved"; wording illustrative) and finishes the next day. - He submits. Automatic scoring gives 72%: "Not passed". He sees which items were wrong, but not the reference answers.
- He asks to retry. Aisha approves. The approval is recorded on attempt 1, which stays unchanged.
- Attempt 2 scores 90% on the automatic items. One free-text reason needs a human. The state is "Awaiting assessment" and Ewan sees the task in My work.
- Ewan marks the item correct and records "Pass" with a comment. The effective result is Passed.
- The admission effect adds Tom to "Screeners". The membership audit says "Admitted by training policy v1, attempt 2, assessment 2 (Ewan Fraser)". Tom now sees "Title/abstract screening" in his project menu. His "Trainees" membership stays.
9.3 Edge cases¶
- Already a member. Priya, already in "Screeners", takes the training to calibrate. Her pass records
AlreadyMemberand changes nothing. - Group grants enlarged. Daniel later gives "Screeners" the Reconcile grant, which Aisha cannot administer. Omar's next pass becomes a manual admission request with the reason "The group's permissions now exceed what the policy publisher can grant".
- Policy change. Aisha publishes policy v2 with a pass mark of 85%. Tom's admission stands. If she re-scores Tom's attempt under v2 and it now falls short, the record shows both assessments and Tom stays in "Screeners" unless someone removes him explicitly.
- Promotion. Aisha promotes Tom's training extraction on one Study to live evidence. Tom had already seen that item's reference, so the new live session version is marked informed. On another Study Tom already has a live session, so promotion is refused for that item.
- Separation check. After Tom's two attempts, live screening counts, outcomes, pool history and statistics for the 20 Studies are unchanged.
9.4 Training screen states¶
| State | What the trainee or assessor sees |
|---|---|
| Loading | Skeleton list with "Loading training" |
| Empty | "No training is set up for this stage" (designers see a set-up link) |
| Pending | "Awaiting assessment" or "Scoring" |
| Incomplete | "Attempt 1 · 12 of 20 answered · Version checkpoint saved" |
| Completed | "Passed · admitted to Screeners" or "Not passed · retry available after approval" |
| Unavailable | "No retries remain under policy v1" or "You no longer have access to this stage" |
| Historical | Every attempt with its scores, assessments and policy versions |
| Conflict | "Another assessor recorded an assessment. Your draft is kept." |
| Failure | "Passed · admission pending. We will retry." |
9.5 Inference beta: opting in¶
Aisha opens Project settings, "Classification inference (beta)". The page explains the four supported statements, that results are suggestions, that reported answers never change, and that she can switch it off at any time. She enables it. The setting history records Aisha, the time and the explanation version. Without this step, nobody in the project sees inference.
9.6 Inference beta: one reviewer's own answers¶
Rule "Pregnant ⊆ Female" (version 2) applies to S-0730. Priya records a Female cohort (N=70, complete coverage) and a Pregnant cohort (N=20) and answers the mapping question "Rule applies: yes". Her panel shows "Pregnant is contained in Female (proven by rule v2). Animals across both cohorts: 70." Omar, on the same Study, records Female with partial coverage. His panel shows "Containment: unknown, Female is a partial sample". Neither sees the other's panel, and SyRF never combines Priya's coverage with Omar's mapping answer.
9.7 Inference beta: accepted answers, change and withdrawal¶
- Ewan reconciles S-0730; the accepted answers say Female is complete. The collective panel shows the containment as proven, citing accepted result v1 and rule v2.
- A query later corrects Female's coverage to partial, producing accepted result v2. The collective result for v1 is marked "Replaced: inputs changed" and kept in history. The new result says "unknown".
- Aisha publishes rule v3, which narrows the rule's scope. Results depending on v2 are recomputed and the old ones marked replaced.
- Aisha disables the beta. Current results become "Withdrawn: beta disabled". Every reported answer is unchanged, and the history stays readable to permitted members.
9.8 Inference panel states¶
| State | What the panel shows |
|---|---|
| Loading | "Checking relationships" |
| Empty | "No cohorts or rules to evaluate" |
| Pending | "Provisional: based on your unsaved changes" |
| Completed | Propositions with proof state and provenance |
| Unavailable | Nothing (beta off), or "Not evaluated: too complex for the beta" |
| Historical | Superseded and withdrawn results with reasons |
| Conflict | "Conflicting evidence: dependent conclusions withheld" |
| Failure | "Not evaluated. Your answers are unaffected." |
10. Rollout and adoption¶
| Capability | Release or lane | Entry dependencies | Flag decision (PROPOSAL) |
|---|---|---|---|
| Training steps, references, policies, attempts, automatic scoring | Training lane (the rollout plan names it; TR1, which does not gate GA) | R3a (steps), R2a (forms), R3b (profiles, for screening training), F3 for the step kind | Canonical enrolment scope plus a training kill switch |
Training guidance with optional acknowledgement (PROPOSAL, TI-R37) |
Same lane, with the first increment | As above | As above |
| Manual assessment, feedback, retries | Same lane | Capabilities named at F1b | As above |
| Automatic group admission | Same lane, after pilot of manual admission | R1c (configurable groups and membership contracts), R1d envelope for anti-escalation | Own flag, off per environment until piloted, because it changes membership |
| Promotion to live evidence | Same lane, last increment | R2a/R3a live commands; RS readiness | Own flag |
| Inference beta setting, rules, candidate inference | Lane C2 (beta) | C1 (explicit classification), Q-18, Q-19, F-C | Per-project opt-in record plus an environment kill switch |
| Collective inference | Lane C2 (beta) | R4a (accepted results) | As above |
- Baseline (MVP). Training is requested for delivery. Reserving an admission hook while deferring the rest is not sufficient without a separate owner scope agreement (S4). The training lane is therefore part of the delivery plan, scheduled after its dependencies; no date is implied.
- Training does not gate GA (decided, 6 October 2026). Chris: "No, training does not gate GA". This answers the rollout plan's R-AMB-01 as option (a): the training lane (TR1) is a delivery-scope lane after its dependencies, and GA readiness never waits for it. It stays in the delivery plan; it is not deferred.
- Opt-in beta. The inference capability ships only as a project-designer opt-in beta. Pilots use seeded projects in preview and staging (Q-07). GA promotion is a separate decision.
- Deferred. Match-based scoring of repeatable entities; bounds reasoning and operators beyond the four; any wider collective authority than accepted results; accepted-answer branching (D4-15, SP).
- Baseline conversion (BC). Converted projects keep the beta off and get no training steps by default. Legacy projects have no training data to convert.
- Gates. Brief approval for
T-TI-00, G0 (T-G0) and the hold lift (T-HOLD). None has happened.
11. Acceptance evidence¶
| ID | Evidence | Method | Amends |
|---|---|---|---|
| TI-AE01 | After full training attempts on a form and a profile, counts of live sessions, decisions, claims, pool events, WorkFirstReleased events, accepted results, PRISMA inputs and statistics inputs are unchanged |
Integration | AC-R3c-18 |
| TI-AE02 | Editing a reference publishes a new version; attempts in progress keep their pinned version | Integration | New |
| TI-AE03 | Rubric fixtures pass for every type in the §5.2 table; free text always routes to manual assessment | Unit fixtures | New |
| TI-AE04 | A manual assessor can pass an automatically failed attempt and fail an automatically passed one; both assessments remain; the effective result follows the policy | Integration | New |
| TI-AE05 | An assessor cannot assess their own attempt | Integration | New |
| TI-AE06 | A failed attempt is preserved; retries follow the pinned policy (allowed, maximum, approval, waiting period) | Integration | New |
| TI-AE07 | Automatic admission adds exactly one membership per passed attempt under retries and duplicate effects; AlreadyMember changes nothing |
Integration; concurrency | New |
| TI-AE08 | After admission the trainee's effective permissions differ from before by exactly the group's grants; enlarged grants or a deleted group convert admission to a manual request | Integration; C10 conformance | New |
| TI-AE09 | Manual admission requests are approvable only by members with authority over the group; the audit entry carries attempt, assessment, policy version and actor | Integration | New |
| TI-AE10 | Promotion writes new live versions with promotedFrom, never changes training records, is refused where a live session exists, and marks informed evidence |
Integration | New |
| TI-AE11 | Feedback obeys the policy; reference answers never appear before submission. 6 October 2026 (DQ-31 decided): under the default policy, per-item correctness appears after the effective assessment, and reference answers appear only after a pass or when no retry remains | End-to-end | New |
| TI-AE12 | A live session on a Study whose training reference the reviewer saw is marked informed | Integration; agreement fixture | New |
| TI-AE13 | The training agreement report is separate and never appears in the default agreement view | Integration | New |
| TI-AE14 | Training works by keyboard, touch and screen reader on phones, tablets and desktops | End-to-end; accessibility | New |
| TI-AE15 | A designer from the tester panel sets up a training step with references, policy and admission without help | User test (D3-08 panel) | New |
| TI-AE16 | The beta is off for new, copied, templated and baseline-converted projects; no inference UI or computation runs until a Design holder enables it; changes are audited | Integration | New |
| TI-AE17 | The annotation and reconciliation suites pass with the beta off | Regression suite | New |
| TI-AE18 | The reasoner supports exactly the four operators; anything else returns Unsupported |
Unit | AC-C2-04 |
| TI-AE19 | A fixture with reviewer A's coverage and reviewer B's mapping answer produces no combined conclusion; candidate results are never returned to another user | Unit; API permission test | New |
| TI-AE20 | Collective results reference the exact AcceptedResultVersion they used |
Integration | New |
| TI-AE21 | Reported answers are byte-identical before and after inference and after disabling; the farm (120 animals in two complete partitions, never 240) and Pregnant ⊆ Female (complete, then partial) fixtures pass | Unit fixtures | AC-C2-01, AC-C2-02 |
| TI-AE22 | An input or rule change supersedes the previous result with its reason and keeps it; no stale result is shown as current; a rebuild reproduces current results | Integration | AC-C2-05 |
| TI-AE23 | Disabling withdraws current results into history and changes no answer | Integration | New |
| TI-AE24 | Only Design holders publish rule versions; mapping-answer disagreements go through ordinary reconciliation | Integration | AC-C1-09 |
| TI-AE25 | Inference latency and payload budgets are measured on the agreed host for the largest pilot project before any wider enablement (budgets set in the brief) | Benchmark | New |
| TI-AE26 | A trainee sees the bound guidance version before starting; with the acknowledgement on, an attempt cannot start until it is given, and the attempt records the guidance version and the time; with the guidance set to hidden during the attempt it is not reachable inside the attempt, and with it set to visible it is; editing guidance publishes a new version and attempts in progress keep theirs (TI-R37) | Integration; end-to-end | New |
| TI-AE27 | With the beta off, a Design holder cannot author or publish a rule version (refused; no rule page shown), while C1 recording, shared concepts and concept mapping answers work when C1 is enabled; with the beta on, rule authoring, publication and inference panels appear; templates, copies and conversion never turn the beta on (TI-R38; DQ-09) | Integration; end-to-end | New |
12. Brief items, specialist inputs and unapproved proposals¶
This page owns none of the 22 session-register alignment entries or the three engineering contracts.
It consumes Q-16 and D4-12 (agreement methods, T-SI-02) for training agreement reports.
| Item | Required treatment | Tracker |
|---|---|---|
| Pass thresholds, tolerances, retry counts and waiting periods | Proposed per policy only; no platform defaults approved | T-TI-00 |
| Feedback disclosure default | Decided 6 October 2026: Chris answered DQ-31 as recommended, so TI-R10's default stands (per-item correctness after assessment; reference answers only after a pass or when no retry remains). The brief specifies the per-policy options only. (Was: "Confirm TI-R10 or choose another default", owner-visible) | T-TI-00 |
| Manual override default | Confirm that the latest manual decision is effective by default (TI-R08) | T-TI-00 |
| Rubric change and re-score | Confirm TI-R13 | T-TI-00 |
| Scoring repeatable entities | Design match-based scoring or keep manual assessment | T-TI-00 |
| Practice-Study selection | Methodologist guidance on choosing practice items; whether a "training-only Study" marker is wanted (not proposed for MVP) | T-TI-00 with CAMARADES methodologists |
| Optional step dependency on passed training (DQ-32) | Decide whether to offer TI-R23's optional dependency in addition to group admission. Open (6 October 2026; Chris is unsure). Recommendation: group admission (decided, S4) is the only access gate, applied through the group's existing grants at stage and activity level (the Q-03 permission model); there are no separate step, profile or form gates; re-qualification can come later as a policy option; TI-R23's step dependency is left out of MVP | T-TI-00 (owner-visible) |
| Training guidance (TI-R37) | PROPOSAL answering Chris's question of 6 October 2026: versioned guidance on the training step, shown before the attempt, with an optional "I have read this" acknowledgement and a policy choice on whether it stays visible during the attempt. The brief proposes the defaults |
T-TI-00 (owner-visible) |
| Admission concurrency | Prove TI-R17 and TI-R18 against the R1c group contracts | T-TI-00, T-AC-00 |
| Inference refresh timing and performance | Measure invalidation fan-out, latency and payload; set budgets | T-TI-00 |
| Scope of the beta (DQ-09) | Decided (DQ-09, option (a), Chris, 6 October 2026; decision register §1.18): rule authoring and publication, rule evaluation and inferred results require explicit project beta opt-in; ordinary annotation and explicit group or classification recording follow their own availability and do not require the beta (TI-R38). Option ©, CPR's reading that rule authoring and publication sit in C1 outside the beta, is not adopted, and CPR §3.7 and §12 item 4 now follow (a). A class B clarification: it does not accelerate classification delivery or introduce a separate review workflow, and it is not a prerequisite to sending the design pack. The brief proposes what reviewers see of existing rule-applicability questions after a project switches the beta off (PROPOSAL: not offered for new work; recorded answers kept, TI-R27). (Was: an ambiguity with options (a), (b) and ©, recommendation (a).) |
T-TI-00 |
| "Explicitly permitted authority" for collective inference | The owner allows collective inference from accepted results "or explicitly permitted authority". The beta enables none beyond accepted results; any wider authority needs approval | T-TI-00 |
| Count statements | Confirm TI-R33 (exact totals only where the four operators prove them; bounds deferred) | T-TI-00 |
| GA readiness criteria | Define the evidence for leaving beta. 6 October 2026: this is the inference beta's own promotion (TI-R28); it is unrelated to the programme's GA, which training does not gate (§10) | T-TI-00 |
| Training agreement methods | Specialist method review | T-SI-02 |
Harmonisation notes (5 October 2026).
- §3.1: the training step now cites SP's step kinds (SP §3.4 and SP-R91), which list it beside review steps and the system adjudication step. SP §3.4 records TI-R23's optional dependency on a passed training step as the only way a live step depends on one.
- §3.1: clarified that training attempts write no pool events while the stage's own pool history still follows SP, so TI-R04 and SP's pool history do not conflict.
- No rule, evidence ID or decision status on this page changed.
13. Amendments to existing package documents¶
- methodology-coverage.md → §2 capability map, "Calibration and training" row. Move scored training against references, manual assessment, retries and group admission from "out of scope" into the plan.
- methodology-coverage.md → §3.4. Replace "R3c (or after GA with the C6 admission hook reserved now)" with the training lane; keep "promotion is an explicit admin action that creates new live submissions with provenance"; mark the "default 30 studies" sample as proposed, not approved.
- methodology-coverage.md → §11.6 and §16.4. Training records are not pool or screening events (unchanged); remove "scored training against gold as an admission prerequisite" from the follow-up backlog.
- contracts.md → C6. Add the training step kind; replace the "passed-training admission hook" with group admission (TI-R15 to TI-R19) and the optional explicit dependency.
- contracts.md → C3. Add exposure kind
trainingReferenceShownand thepromotedFromprovenance. - contracts.md → C10. Add the proposed capabilities (Assess training, Promote training evidence, View training results) and the admission anti-escalation check.
- contracts.md → C12. State that training attempts never affect PRISMA, and that promoted evidence counts only from its promotion onwards, like any other live contribution.
- contracts.md → C13. Add the beta opt-in, the input rule (own snapshot or pinned accepted result),
candidate-result visibility,
SupersededandWithdrawnwith history, and the count rule. - domain-model.md → §4.4 and §4.5. Add
TrainingStep,TrainingReferenceVersion,TrainingPolicyVersion; mapProjectRuleversions toClassificationRuleVersion. - domain-model.md → §4.6. Key
InferenceResultby basis, rule vector and engine version; keep history. - domain-model.md → new training context.
TrainingAttempt(with training sessions under author scopetraining) andTrainingAssessment; §7.2AuthorScopegainstraining(attemptId); §12 adds the training lane row. - integrated-plan.md → §5.5 R3c and §5.8 lane rows. Calibration moves to the training lane; the C2 row becomes an opt-in beta with Q-18 and Q-19 decided.
- acceptance-criteria.md. Amend AC-R3c-18, AC-C1-09, AC-C2-01, AC-C2-02, AC-C2-04, AC-C2-05; add the TI-AE rows.
- decision-register.md and open-questions-and-assumptions.md. Record the §2 statuses for D4-04, Q-18 and Q-19.
- unified-annotation-classification-research.md → §7 U4. Note that the first approved scope is the four operators; count bounds wait.
Owner answers, 6 October 2026. This page now carries the §1 paragraph, the §2 rows, §3.1, §3.3, §3.10, §4.1, §4.2, §4.4, the §5.2 row, TI-R10, TI-R23, TI-R37 (§5.7), §6, §7, §9.1, §9.2, §10, TI-AE11, TI-AE26 and the §12 rows. Updated in the same change: the versioning model §2 and §16.4 (training guidance versions) and contracts C6. Other documents should follow:
- rollout-plan.md → §7.6 R-AMB-01. Answered by Chris on 6 October 2026 as option (a): "No, training does not gate GA".
- integrated-plan.md → the TR1 lane row. "The rollout plan recommends that TR1 does not gate GA" becomes "TR1 does not gate GA (Chris, 6 October 2026)".
- product-owner-guide.md → the Training row. The same change; mention the guidance proposal.
- design-handoff-open-questions.md. DQ-31 answered as recommended (Chris, 6 October 2026); DQ-32 stays open with the recommendation in §12; R-AMB-01 answered.
- acceptance-criteria.md. The row that carries TI-AE11 gains the default-policy clause; add a row for TI-AE26 in the training lane section.
- decision-register.md. The section "Owner answers, 6 October 2026" records training does not
gate GA (decided), DQ-31 (decided as recommended), DQ-32 (open) and the guidance
PROPOSAL. - implementation-tracker.md.
T-TI-00: the feedback default is decided; DQ-32 and TI-R37 remain brief items; nothing here changes the hold.
Design-handoff answers, 6 October 2026 (Chris, Codex discussion; record archived at
owner-session-2026-10-05/syrf-design-decisions-2026-10-06.md.txt;
decision register §1.18). DQ-09 is decided as option (a). This page now carries the companion note, the §1
paragraph, the §2 row, §3.7 ("What it gates"), §3.8, §4.11, TI-R30, the new TI-R38, the §6 row,
the new TI-AE27, the §12 row (decided) and the §14 row. DQ-46's terms reach §4.4 and the §5.2 rubric
(a reason no value is provided; "Other, please specify" as a value, with a PROPOSAL rubric row;
"Unanswered" for "Blank"). Updated in the same change: CPR
§1, §2, §3.7, §4.7, §6, §10.1, CPR-R30, CPR-AE21, §12 and §13 (option © not adopted);
contracts C13; the domain model; the
versioning model §16.5. Other documents should follow:
- design-prototype-handoff.md. DH-U21 is decided; DH-C15's rule pages sit behind the beta without a "To be decided" chip; the cohort pages and shared concepts show without the beta.
- acceptance-criteria.md. Add TI-AE27 beside the C2 rows.
- implementation-tracker.md.
T-TI-00andT-CP-00: the beta scope dependency is settled.
Nothing here accelerates classification delivery or authorises implementation.
14. Superseded wording¶
| Old wording | New wording | Where it appears today |
|---|---|---|
| Calibration step "R3c (or after GA with the C6 admission hook reserved now)"; defer to hook only | Training is in delivery scope; hook-only deferral needs a separate owner agreement | methodology-coverage.md §3.4; original D4-04 recommendation |
| "Scored training against gold as an admission prerequisite (after GA; hook reserved)" listed as out of scope or backlog | Scoring, manual assessment, retries and optional automatic group admission are planned | methodology-coverage.md §2 and §16.4 |
| "A passed-training admission hook in C6" as the admission mechanism | Group admission through existing group contracts, recorded with causation | methodology-coverage.md §2 |
| C2 "Inference and counts" with no activation rule | Beta, off by default, explicit project-designer opt-in; never enabled by baseline conversion | integrated-plan.md §5.8 C2 row; contracts.md C13 |
| U4 "conservative inference" including "safe count/bounds checks" | First reasoner limited to the four operators; bounds wait for a later approved scope | unified-annotation-classification-research.md §7 |
| "Calibration rounds run as a step kind that never votes, never qualifies and never counts for PRISMA" (pending D4-04) | Kept, and extended to references, scoring, assessment, retries, admission and promotion | AC-R3c-18 |
| "The recommendation is that TR1 does not gate GA" (rollout R-AMB-01, open) | Decided: training does not gate GA (Chris, 6 October 2026) | rollout-plan.md §7.6; integrated-plan.md TR1 row; product-owner-guide.md; design-handoff-open-questions.md |
| Trainee feedback default "open" or "owner-visible" (DQ-31) | Decided as recommended: TI-R10's default (Chris, 6 October 2026) | design-handoff-open-questions.md DQ-31; status page CI-15 |
| "Scope of the beta" open as DQ-09, with CPR's option © (rule authoring in C1, outside the beta) as an alternative | Decided, option (a): rule authoring and publication, rule evaluation and inferred results need the beta; C1's explicit recording, shared concepts and concept mappings do not (Chris, 6 October 2026; TI-R38) | CPR §1, §2, §3.7, §10.1, §12 item 4; contracts C13; design-handoff-open-questions.md DQ-09; handoff DH-U21, DH-C15 |