Skip to content

ADR-025: Runtime feature releases and the environment-state service

Status

Proposed, 2026-10-15. Not accepted and not implemented. The front matter says In-Review, as for the other ADRs awaiting Chris, because the docs validator has no Proposed value. This ADR records the change of authority that the runtime feature releases plan asks Chris to approve. Until he accepts it, the current rule stands: GitOps values are the production source of truth and runtime overrides are non-production only (feature-flag rule).

Context

Chris wants feature release and rollback in production through runtime administration, independently of code deployments, followed by project, user and combined targeting and gradual rollouts (functional proposal of 15 October 2026).

Production does not yet run any of the mechanism below: its API and PM run image 104eca9, which has no runtime-flag store, controller, shared reader or migration entry point, and the ADR-023 migration Job is enabled in no environment. This decision takes effect only after the continuous delivery plan's catch-up promotion puts production on main's code in Legacy mode. On main today:

  • production refuses policy mutation (RuntimeFeatureFlagsController.cs:109-110, RuntimeFeatureFlagStore.cs:467-487), ignores stored overrides when it reads (RuntimeFeatureFlagStore.cs:352), and the shared reader refuses to start there (SharedRuntimeFeatureFlagRegistration.cs:31-36);
  • whether an environment may mutate is decided by its environment name alone;
  • the policy document has no environment identity, lineage, digests or mutation ID, and no host records which policy it applied (current-code matrix);
  • the continuous delivery plan's D1 proposed a vendored repository copy of production deployment inputs for deployment checks. Chris refined D1 on 2026-10-15 to an authenticated environment-state and feature-policy service.

Decision (proposed)

  1. Production runtime policy becomes an authority for flags the catalog marks production-eligible, once the catch-up promotion has landed and Increment C ships and a deployment-only setting RuntimeFeatureFlags:ProductionPolicyMode is changed from Legacy (default, today's behaviour) to Runtime by a Chris-approved cluster-gitops change. GitOps remains the authority for binaries, deployment-bound settings and the proposed baseline; the accepted baseline is activated through a reviewed import, never on pod start-up.
  2. Environment identity is a binding record, not a name. Each application database (syrftest, syrf_staging, syrf_pr_{n}, local) holds one binding written by an explicit, reviewed command. Every migration that needs it and every policy or release write checks the configured environment, database name, cluster fingerprint and binding before writing.
  3. Release records are separate from policy. A release (limited, general, public preview publication) has its own ID, append-only events and outbox in the same database as the policy. Confirmation times are derived by the server. Emergency stop never depends on docs, releases or announcements.
  4. One environment-state and feature-policy service, built by extending the API and flag infrastructure, serves the application and guide, flag administration, deployment automation and internal collectors through scoped views over one catalog, one evaluator and one authoritative policy. Deployment checks read a versioned compatibility report with a revision vector; reports are bounded run artifacts, never a second authority. Staging and production use the same contract and check workflow with separate state and credentials. This supersedes CD plan D1 option A.
  5. Concurrent rollouts are protected by a deployment compatibility constraint, recorded by the trusted coordinator and enforced inside the policy transaction. Compatible changes and emergency stops stay allowed.

Consequences

Positive. Release and rollback stop waiting for promotions. The safety properties already designed (revision checks, atomic audit, immutable snapshots, baseline acceptance, recovery, domain readiness, operation boundaries, compatible rollback) extend to production instead of being bypassed. Deployment checks see the real environment without a token in fork lanes or a hand-maintained mirror.

Negative. Production gains a runtime control plane that must itself be available, audited and recoverable. The CD plan's render comparison is valid only in Legacy mode and needs replacing after cutover. Production automation needs a machine identity that does not exist today (production is on Auth0; SyRF Identity is staging only), which means an Auth0 tenant change or waiting for the Identity migration.

Risks and mitigations. A permissive fallback re-enabling a stopped feature: fail-closed stale limits per flag. A restored backup posing as production: binding epoch, rebind, delivery disabled. A rollout escaping a preflight: the deployment constraint. A binary rollback that cannot honour policy: readiness and the CD plan's G4 compatibility lanes.

Alternatives considered

  • Keep GitOps-only production values (status quo). Rejected by Chris's requirement; every release and rollback would be a promotion.
  • A managed feature-flag control plane (OpenFeature with Unleash or similar). Deferred to before targeting (README §11 decision 9); it supplies targeting UI but not SyRF's dependencies, durable domain modes or audit guarantees, and would still need this binding and release model.
  • A vendored repository copy of production deployment inputs (CD plan D1 option A). Superseded by Chris on 2026-10-15 in favour of the service in decision 4.