Version-matrix golden generation: derive minimal goldens and assert mutation coverage

[sourcehawk]

Problem

Consumers render version-spanning resources by registering version-gated mutations and letting each feature.Gate decide whether it fires for the concrete spec.version. Today they verify this with hand-picked whole-resource goldens at a few hand-chosen versions. A real consumer audit (camunda-operator zeebe) found two failure modes this leaves open: a key that was correct for 8.9 but a silent no-op for 8.8 went uncaught because the goldens only covered 8.9 (regime coverage is guessed, not derived); and a registered mutation can sit dead, or fire for the wrong fixture or version, with every existing test green (no proof a mutation is exercised where assumed). We want an optional, test-only helper, driven by one declarative matrix, that derives the minimal goldens covering every behaviorally-distinct version regime and asserts, intentionally, that every mutation fires for the fixtures and versions the author assumed, and that every registered mutation is consciously either covered or excluded.

In scope

• Read-only framework introspection of the registered mutation set and which mutations fire at a built version, plus an exported serializer so a generated golden is byte-identical to a hand-written one.
• A test-only goldengen package: a declarative matrix (Versions, Exclude, Fixtures with Requires/Forbids/For) that classifies versions by firing-set, derives minimal goldens per regime, asserts per-fixture gating, and emits a reviewable coverage manifest.
• A user-invoked accounting assertion proving every registered mutation is either required by some fixture or explicitly excluded, plus a worked example and public documentation.
• An optional YAML matrix loader so the whole declaration (minus the Go Build closure and Scheme) can live in a matrix.yaml, with fixture CRs supplied inline or by specFile reference.

Out of scope

• Cross-package coverage aggregation: a package-scoped TestMain covers a component whose tests live in one package; spanning packages is not addressed.
• Deriving or parsing version boundaries: the framework never computes or parses gate constraints; the candidate version universe is consumer-supplied.
• Changing how mutations are registered, gated, or applied: this work only reads existing state.
• Encoding Build or Scheme as data: both stay in Go even under the YAML loader.

Risks & mitigations

• Classifying by rendered bytes would explode regimes (any mutation interpolating the version renders every version differently). Mitigated by classifying strictly on the firing-set vector of Gate.Enabled() results, which is immune to version interpolation.
• Representative-version shift on refinement (a new mutation splitting a regime renames goldens). Mitigated by the always-emitted manifest, which renders a split as a readable diff rather than an unexplained rename.
• Empty or duplicate mutation names would break accounting. Mitigated by the accounting assertion rejecting them rather than silently tolerating them.

Design overview

The consumer declares one matrix: a candidate version universe, an exclude list, and a set of fixtures, each carrying a CR spec plus Requires/Forbids expectations (optionally pinned to a concrete version via For). A consumer-supplied Build(version, spec) closure produces a Unit (a built resource or component). The generator sweeps every fixture across every version, reads the Unit's firing-set, groups versions into behaviorally-distinct regimes (two versions share a regime iff their firing-sets are identical), writes one golden per regime at a representative version (the first in supplied order, which lands on the inclusive boundary when versions are listed ascending), and emits a manifest.yaml mapping each (fixture, regime) to its representative version and firing-set. Per-fixture gating assertions check each Requires/Forbids against the swept firing-sets. A separate, user-invoked accounting assertion (called from the consumer's TestMain) proves union(Requires) ∪ Exclude == RegisteredMutations.

The framework stays black-box about gating: the only surface it reads is the existing feature.Gate.Enabled() outcome, exposed read-only through a new MutationInspector interface (RegisteredMutations() / FiringSet()) implemented once on generic.BaseResource and as a union on *component.Component.

flowchart TD
    M[matrix: Versions, Exclude, Fixtures] --> G[goldengen.Generator]
    B["Build(version, spec) -> Unit"] --> G
    G -->|sweep version x fixture| U[Unit: RegisteredMutations / FiringSet / RenderYAML]
    U -->|firing-set per version| C[classify into regimes]
    C --> GO[minimal goldens per regime]
    C --> MAN[manifest.yaml]
    U -->|gate expectations| GATE[per-fixture Requires/Forbids assertions]
    U -->|all registered names| ACC["AssertComplete: union(Requires) + Exclude == Registered"]

Loading

Sub-issues

Sub-issues will be linked below as they're filed.