feat(conformance): cut contract-v2 with the 18-scenario corpus

CHANGELOG.md

@@ -8,6 +8,17 @@ reference are all pre-1.0, minor (0.x) releases may include breaking changes.
 
 ## [Unreleased]
 
+### Changed
+
+- The conformance contract is now **`contract-v2`**: 18 shared scenarios, up from
+  15, regenerated against toolkit `0.4.5`. `studio ci` and the build-time harness
+  gate on 18/18. The three added scenarios (`firmware-update-success`,
+  `firmware-update-failure`, `refused-authorization`) cover the firmware statuses
+  and the non-`Invalid` authorization refusals the frozen corpus never reached,
+  which is how the rule defects fixed in 0.5.4 stayed green at 15/15. `contract-v2`
+  is a strict superset of v1: the 15 existing fixtures and goldens are unchanged,
+  byte for byte. See [ADR-0013](docs/adr/0013-contract-v2-corpus.md).
+
 ## [0.5.4] - 2026-07-30
 
 ### Fixed

CURRENT_STATE.md

@@ -42,6 +42,35 @@ workflow, which builds and attaches the macOS + Linux packages. Next themes
 
 ## What's done
 
+### contract-v2: the 18-scenario corpus (Issue #96)
+
+The vendored corpus was frozen at 15 scenarios for 0.5.0 (ADR-0012) while the
+toolkit's grew to 18. The three missing ones were exactly the blind spots that let
+#92 hide: no pinned fixture exercised a `FirmwareStatusNotification` status, a
+non-`Invalid` `idTagInfo`, or the connector-status transitions the old matrix got
+wrong, so the harness sat at 15/15 while three rules disagreed with the spec.
+
+- ✅ **`contract-v2`** cut per ADR-0012's rule, recorded in
+  [ADR-0013](docs/adr/0013-contract-v2-corpus.md): 18 fixtures + goldens
+  regenerated from the published toolkit `0.4.5`, harness gate now **18/18** under
+  both `native test` and `studio ci`
+- ✅ Added `firmware-update-success` (`[]`), `firmware-update-failure`
+  (`["FIRMWARE_UPDATE_FAILURE"]`, covering the `InstallationFailed` the old rule
+  missed) and `refused-authorization` (`["FAILED_AUTHORIZATION"]`, covering
+  `Blocked` / `Expired` / `ConcurrentTx`)
+- ✅ **Strict superset of v1:** regenerating all 18 reproduces v1's 15 fixtures and
+  goldens byte for byte, so nothing previously asserted changed. `contract-v1`
+  stays reachable through the `v0.5.4` tag's tree rather than being vendored twice
+- ✅ `harness.zig` derives everything from `scenario_names`, so the three names are
+  the only code change; `docs/CONTRACT.md`, the conformance README, the top-level
+  README and `docs/cli-parity.md` updated. ADRs and CHANGELOG history left as
+  written
+- The contract docs now say plainly that the corpus is a sample rather than a
+  proof: 18 scenarios still do not cover the 81 ordered connector-status pairs or
+  most of the status cross-products, so a green harness is evidence, not proof
+- Does not settle #94 (whether a rule fix that moves no golden needs a bump); "a
+  new scenario" is named explicitly by ADR-0012, so this cut is the clear case
+
 ### OCPP 1.6 spec-conformance parity (Issue #92)
 
 Three rules read the 1.6 spec wrongly in the same way the toolkit's did, fixed
@@ -240,4 +269,4 @@ live inspector surface with notifications (#54–#60) — is landed.
 | `ui` (native views) | ✅ S3 inspector (#27–#32) + replay transport (#44) + live-capture view (#59) + live notifications (#60) |
 | `capture` (live proxy) | ✅ S5: WS transport (#54) + frame decode (#55) + MITM proxy (#56) + live notifications (#60) |
 | `cli` (headless) | ✅ inspect/report/diff/anonymize/ci/scenario (#45) + live `capture` (#57) |
-| `conformance` | ✅ S2 (15/15) + frozen `contract-v1` (#72, docs/CONTRACT.md) |
+| `conformance` | ✅ S2 (15/15) + frozen `contract-v1` (#72, docs/CONTRACT.md); widened to `contract-v2` (18/18) in #96 |

README.md

@@ -20,7 +20,7 @@ Studio is one of two independent products under the OCPP DebugKit umbrella:
 | [`@ocpp-debugkit/toolkit`](https://github.com/ocpp-debugkit/toolkit) | TypeScript | npm library · CLI · web app | The library and CI brain — parse, analyze, and report OCPP traces anywhere |
 | **`ocpp-debugkit/studio`** (this repo) | Zig | native desktop app | The instrument on the bench — live capture, native performance, OS integration |
 
-The two share **no code**. They meet only at a *conformance contract*: the same trace format, the same normalized event model, the same failure taxonomy, and the same scenario fixtures. A trace captured in Studio opens in the toolkit's web inspector, and vice versa — two independent implementations, one format, checked in CI on every change. See **[the conformance contract](docs/CONTRACT.md)** (`contract-v1`) and [ADR-0001](docs/adr/0001-independent-implementation.md).
+The two share **no code**. They meet only at a *conformance contract*: the same trace format, the same normalized event model, the same failure taxonomy, and the same scenario fixtures. A trace captured in Studio opens in the toolkit's web inspector, and vice versa — two independent implementations, one format, checked in CI on every change. See **[the conformance contract](docs/CONTRACT.md)** (`contract-v2`) and [ADR-0001](docs/adr/0001-independent-implementation.md).
 
 ## What it does
 

docs/CONTRACT.md

@@ -1,4 +1,4 @@
-# The conformance contract (`contract-v1`)
+# The conformance contract (`contract-v2`)
 
 OCPP DebugKit Studio and the TypeScript [toolkit](https://github.com/ocpp-debugkit/toolkit)
 are **two independent implementations** of the same OCPP analysis behavior — one
@@ -28,35 +28,56 @@ and it is enforced in CI on every change, not asserted in prose.
   2 info**), and the same thresholds as the reference (heartbeat 60 s, slow
   response 10 s, ±50 % heartbeat deviation, session bounds, the 5-minute boot
   window). The rules live in [`src/ocpp/detection.zig`](../src/ocpp/detection.zig).
-- **Scenarios.** **15 shared scenario traces** exercising the rules end to end,
+- **Scenarios.** **18 shared scenario traces** exercising the rules end to end,
   each pinned to the exact de-duplicated, sorted `FailureCode` set the reference
   implementation detects.
 
 ## How conformance is enforced
 
-The 15 scenarios and their expected failure-code sets are **vendored from the
+The 18 scenarios and their expected failure-code sets are **vendored from the
 toolkit — the source of truth — never hand-authored**, so a golden can't drift
 from what the reference actually detects. On every change, `native test` runs
 Studio's full engine (`parseTrace → buildSessionTimeline → detectFailures`) over
 each fixture and asserts its detected code set equals the golden — the same
-comparison the toolkit's `evaluateScenario` makes. **15/15 must match; a release
+comparison the toolkit's `evaluateScenario` makes. **18/18 must match; a release
 is blocked otherwise.** `studio ci` runs the same check from the command line.
 
+The corpus is a **sample, not a proof**: two implementations can both match every
+golden and still disagree on inputs no fixture reaches. That is not theoretical.
+Three rules disagreed with the specification here for as long as they did in the
+toolkit while the harness stayed green, because no fixture exercised a firmware
+status, a non-`Invalid` `idTagInfo`, or the connector-status transitions involved
+(#93). When a rule changes, widen the corpus rather than trusting the count.
+
 The vendored fixtures, goldens, the harness, and the exact regeneration recipe
 live in [`src/ocpp/conformance/`](../src/ocpp/conformance/README.md).
 
 ## Freeze & versioning
 
-For the 0.5.0 release the contract is **frozen as `contract-v1`** (OCPP 1.6J):
-the vendored fixtures and goldens are pinned, and the harness gates every build
-against them. The freeze rationale is recorded in
-[ADR-0012](adr/0012-freeze-contract-v1.md).
+The current contract is **`contract-v2`** (OCPP 1.6J), 18 scenarios, regenerated
+against toolkit `0.4.5`. The vendored fixtures and goldens are pinned, and the
+harness gates every build against them. The cut is recorded in
+[ADR-0013](adr/0013-contract-v2-corpus.md); `contract-v1` (15 scenarios, frozen
+for 0.5.0) is [ADR-0012](adr/0012-freeze-contract-v1.md).
+
+`contract-v2` is a **strict superset** of v1: regenerating all 18 against toolkit
+0.4.5 reproduces v1's 15 fixtures and goldens byte for byte, so every scenario v1
+pinned, v2 pins identically.
 
 The contract is **immutable within a version.** A change that alters detected
 output — a new or changed rule, a new scenario, a threshold change, or OCPP
-2.0.1 — is a **new contract version** (`contract-v2`, …), regenerated against the
-matching toolkit release and re-tagged, never an in-place edit of `contract-v1`.
-This keeps "two implementations, one format" a checkable claim rather than a hope.
+2.0.1 — is a **new contract version** (`contract-v3`, …), regenerated against the
+matching toolkit release and re-tagged, never an in-place edit of a released
+version. Earlier versions stay reachable through the release tags: `v0.5.4`'s tree
+carries `contract-v1`, so every released Studio remains pinned to a reproducible
+reference. This keeps "two implementations, one format" a checkable claim rather
+than a hope.
+
+One question this policy leaves open, tracked in #94: whether a rule correction
+that leaves **every** golden matching needs a version bump at all, given that the
+contract's artifacts are the fixtures and the goldens. The corpus expansion above
+is not that case, since "a new scenario" is named explicitly, so it does not
+settle it.
 
 See also [ADR-0001](adr/0001-independent-implementation.md) (independent
 implementation + shared contract) and [docs/cli-parity.md](cli-parity.md)

docs/adr/0013-contract-v2-corpus.md

@@ -0,0 +1,62 @@
+# ADR-0013 — Cut `contract-v2`: the 18-scenario corpus
+
+- **Status:** Accepted
+- **Date:** 2026-07-30
+
+## Context
+
+[ADR-0012](0012-freeze-contract-v1.md) froze the vendored conformance corpus as
+`contract-v1` (15 scenarios) for the 0.5.0 release, and set the rule that a new
+scenario means a new contract version rather than an in-place edit.
+
+The toolkit's corpus has since grown to 18. `firmware-update-success` and
+`firmware-update-failure` landed in toolkit 0.4.3/0.4.4, and
+`refused-authorization` in toolkit 0.4.5. None of the three was in the contract.
+
+That gap turned out to matter. Three detection rules disagreed with the OCPP 1.6
+specification here for as long as they did in the toolkit (#92, fixed in #93,
+released in 0.5.4), and the harness stayed at 15/15 the whole time, because no
+pinned fixture exercised a `FirmwareStatusNotification` status, a non-`Invalid`
+`idTagInfo`, or any of the 22 connector-status transitions the old matrix flagged
+wrongly. The contract was green while both implementations were wrong in the same
+way, and later while they were wrong in *different* ways. Verifying #93 needed an
+ad-hoc probe of 97 traces across both engines, which proves a point in time but
+guards nothing afterwards.
+
+The three missing scenarios land precisely on those blind spots.
+
+## Decision
+
+**Cut `contract-v2` (OCPP 1.6J): 18 scenarios, regenerated against toolkit
+`0.4.5`.** The harness gate becomes 18/18, and `studio ci` reports 18 scenarios.
+
+`contract-v2` is a **strict superset** of `contract-v1`. Regenerating all 18 from
+the published toolkit 0.4.5 reproduces v1's 15 fixtures and goldens byte for byte;
+every scenario v1 pinned, v2 pins identically, with three added. So this is an
+additive widening of coverage, not a revision of anything previously asserted.
+
+`contract-v1` is not kept side by side in the tree. ADR-0012's "never edited in
+place" is satisfied by history: the `v0.5.4` tag's tree carries the v1 corpus, so
+every released Studio stays pinned to a reproducible reference, which is the
+consequence ADR-0012 named. Vendoring both versions at once would double the
+embedded test data for no reachable consumer.
+
+## Consequences
+
+- The gate now covers the firmware statuses and the non-`Invalid` authorization
+  refusals it previously never reached, so the class of divergence #92 describes
+  is caught by CI rather than by someone reading the spec against two codebases.
+- The corpus is still a **sample**, and the contract docs now say so. 18 scenarios
+  do not cover the 81 ordered connector-status pairs, or most of the
+  `AuthorizationStatus` and `FirmwareStatus` cross-products. The lesson from #92 is
+  that a green harness is evidence, not proof; when a rule changes, widen the
+  corpus.
+- Regeneration is now expected to be a byte-identical no-op for existing files,
+  which makes an unintended fixture change a visible signal rather than noise.
+- This ADR refines ADR-0012's versioning half by cutting the next version under
+  its rule; it does not supersede it, and ADR-0012 stands as the record of the v1
+  freeze.
+- It does not answer #94, which asks whether a rule correction that leaves every
+  golden matching needs a version bump at all. "A new scenario" is named
+  explicitly by ADR-0012, so this cut is the unambiguous case; the ambiguous one
+  stays open.

docs/adr/README.md

@@ -22,3 +22,4 @@ supersedes the old one rather than editing history.
 | [0010](0010-live-proxy-transport-concurrency.md) | Live-proxy transport & concurrency | Accepted |
 | [0011](0011-live-notification-services.md) | Live-capture notifications via the effects-bound platform services | Accepted |
 | [0012](0012-freeze-contract-v1.md) | Freeze `contract-v1` for the 0.5.0 release | Accepted |
+| [0013](0013-contract-v2-corpus.md) | Cut `contract-v2`: the 18-scenario corpus | Accepted |

docs/cli-parity.md

@@ -13,7 +13,7 @@ intentional difference.
 | `report <file> -f <fmt>` | `studio report <file> [-f markdown\|html]` | ✅ Full — Markdown + self-contained HTML. |
 | `diff <a> <b> --format <fmt>` | `studio diff <a> <b> [--format text\|json]` | ✅ Full — text and JSON. |
 | `anonymize <file>` | `studio anonymize <file>` | ✅ Full — same field rules (see the anonymize module). |
-| `ci [dir]` | `studio ci` | ⚠️ Runs the **vendored** `contract-v1` scenarios (Studio embeds the contract); the `[dir]` argument is not accepted. Exits `0` (all pass) / `1` (any fail). |
+| `ci [dir]` | `studio ci` | ⚠️ Runs the **vendored** `contract-v2` scenarios (Studio embeds the contract); the `[dir]` argument is not accepted. Exits `0` (all pass) / `1` (any fail). |
 | `scenario list` | `studio scenario list` | ✅ Full. |
 | `scenario run <name>` | `studio scenario run <name>` | ✅ Built-in scenarios. |
 | `scenario run --file <path>` | — | ⚠️ Deferred — Studio runs only the built-in contract scenarios. |
@@ -26,7 +26,7 @@ intentional difference.
   convenience is a deferred follow-up (it needs the sandboxed `init.io` write
   path); redirection covers the same need today.
 - **`ci` runs the embedded contract**, not a user directory. Studio vendors the
-  15 `contract-v1` fixtures + goldens under `src/ocpp/conformance/` (ADR-0004),
+  18 `contract-v2` fixtures + goldens under `src/ocpp/conformance/` (ADR-0004),
   so `ci` is a self-contained conformance gate — the same logic the
   `native test` conformance harness runs. See [the conformance
   contract](CONTRACT.md).

src/ocpp/conformance/README.md

@@ -1,10 +1,10 @@
-# Conformance contract (`contract-v1`)
+# Conformance contract (`contract-v2`)
 
 Studio and the TypeScript toolkit are **independent implementations** of the same
 OCPP analysis behavior (ADR-0001). This directory is the machine-checked contract
 between them: the shared scenarios and the failure codes each must detect.
 
-- **`fixtures/<name>.json`** — the 15 shared scenario **traces** (the input).
+- **`fixtures/<name>.json`** — the 18 shared scenario **traces** (the input).
 - **`goldens/<name>.json`** — each scenario's expected **de-duplicated, sorted
   `FailureCode` set** (the output to match), as a JSON array of wire codes.
 - **`harness.zig`** — runs, under `native test`, the full engine
@@ -39,6 +39,16 @@ lets the harness embed it directly — no runtime file I/O, fully hermetic.
 
 ## Versioning
 
-Tagged **`contract-v1`** (OCPP 1.6J). When the contract changes — new rules,
-new scenarios, or OCPP 2.0.1 — regenerate against the matching toolkit release
-and bump the tag.
+Tagged **`contract-v2`** (OCPP 1.6J), 18 scenarios, regenerated against toolkit
+`0.4.5`. When the contract changes — new rules, new scenarios, or OCPP 2.0.1 —
+regenerate against the matching toolkit release and bump the tag. Earlier versions
+stay reachable through the release tags rather than being kept side by side here:
+`v0.5.4`'s tree carries `contract-v1` (15 scenarios).
+
+Regenerating is expected to reproduce the existing files byte for byte, and that
+is worth checking when you do it: if a fixture you did not mean to touch changes,
+either the toolkit version is wrong or the change is bigger than you thought.
+`contract-v2` was cut that way, so it is a strict superset of v1.
+
+See [ADR-0013](../../../docs/adr/0013-contract-v2-corpus.md) for the v2 cut and
+[docs/CONTRACT.md](../../../docs/CONTRACT.md) for the public contract.

src/ocpp/conformance/fixtures/firmware-update-failure.json

@@ -0,0 +1,102 @@
+{
+  "traceId": "scenario-firmware-update-failure",
+  "metadata": {
+    "stationId": "CS-SYNTHETIC-019",
+    "ocppVersion": "1.6",
+    "source": "synthetic-scenario",
+    "description": "Station boots, reports Downloading firmware, then InstallationFailed within seconds, followed by a heartbeat to confirm station is still online."
+  },
+  "events": [
+    {
+      "timestamp": "2026-01-15T07:00:00.000Z",
+      "direction": "CS_TO_CSMS",
+      "message": [
+        2,
+        "msg-001",
+        "BootNotification",
+        {
+          "chargePointVendor": "SyntheticVendor",
+          "chargePointModel": "SM-100",
+          "chargePointSerialNumber": "CS-SYNTHETIC-019",
+          "firmwareVersion": "1.0.0"
+        }
+      ]
+    },
+    {
+      "timestamp": "2026-01-15T07:00:00.500Z",
+      "direction": "CSMS_TO_CS",
+      "message": [
+        3,
+        "msg-001",
+        {
+          "currentTime": "2026-01-15T07:00:00.500Z",
+          "interval": 300,
+          "status": "Accepted"
+        }
+      ]
+    },
+    {
+      "timestamp": "2026-01-15T07:00:30.000Z",
+      "direction": "CS_TO_CSMS",
+      "message": [
+        2,
+        "msg-002",
+        "FirmwareStatusNotification",
+        {
+          "status": "Downloading"
+        }
+      ]
+    },
+    {
+      "timestamp": "2026-01-15T07:00:30.500Z",
+      "direction": "CSMS_TO_CS",
+      "message": [
+        3,
+        "msg-002",
+        {}
+      ]
+    },
+    {
+      "timestamp": "2026-01-15T07:01:00.000Z",
+      "direction": "CS_TO_CSMS",
+      "message": [
+        2,
+        "msg-003",
+        "FirmwareStatusNotification",
+        {
+          "status": "InstallationFailed"
+        }
+      ]
+    },
+    {
+      "timestamp": "2026-01-15T07:01:00.500Z",
+      "direction": "CSMS_TO_CS",
+      "message": [
+        3,
+        "msg-003",
+        {}
+      ]
+    },
+    {
+      "timestamp": "2026-01-15T07:02:00.000Z",
+      "direction": "CS_TO_CSMS",
+      "message": [
+        2,
+        "msg-hb-1",
+        "Heartbeat",
+        {}
+      ]
+    },
+    {
+      "timestamp": "2026-01-15T07:02:00.500Z",
+      "direction": "CSMS_TO_CS",
+      "message": [
+        3,
+        "msg-hb-1",
+        {
+          "currentTime": "2026-01-15T07:02:00.500Z"
+        }
+      ]
+    }
+  ]
+}