From aab575796d321957fe38721fb75da2b29e1bf73a Mon Sep 17 00:00:00 2001 From: sepehr-safari Date: Thu, 30 Jul 2026 13:42:22 +0300 Subject: [PATCH 1/2] feat(conformance): cut contract-v2 with the 18-scenario corpus The vendored corpus has been frozen at 15 scenarios since 0.5.0 (ADR-0012) while the toolkit's grew to 18: firmware-update-success and firmware-update-failure in toolkit 0.4.3/0.4.4, refused-authorization in toolkit 0.4.5. Those three are exactly the blind spots that let #92 hide. No pinned fixture exercised a FirmwareStatusNotification status, a non-Invalid idTagInfo, or any of the 22 connector-status transitions the old matrix flagged wrongly, so the harness sat at 15/15 while three rules disagreed with the specification, first in the same way as the toolkit and then in a different way once the toolkit was fixed. Verifying #93 took an ad-hoc probe of 97 traces across both engines, which proves a point in time and guards nothing afterwards. Widening the corpus is the part that keeps holding. ADR-0012 names a new scenario as a trigger for a new contract version, so this is contract-v2: 18 fixtures and goldens regenerated from the published toolkit 0.4.5, gate now 18/18 under both native test and studio ci. Recorded in ADR-0013, which refines ADR-0012's versioning half rather than superseding it. contract-v2 is a strict superset of v1. Regenerating all 18 reproduces v1's 15 fixtures and goldens byte for byte, so every scenario v1 pinned, v2 pins identically, with three added. v1 is not vendored a second time: the v0.5.4 tag's tree carries it, so every released Studio stays pinned to a reproducible reference, which is the consequence ADR-0012 named. harness.zig derives everything from scenario_names and @embedFile, so three names are the only code change. docs/CONTRACT.md, the conformance README, the top-level README and docs/cli-parity.md are updated; ADRs and CHANGELOG history are left as written, since they record what was true when written. The contract docs now also state that the corpus is a sample rather than a proof. 18 scenarios still do not cover the 81 ordered connector-status pairs or most of the status cross-products, and #92 is the standing evidence that a green harness is evidence rather than proof. This does not settle #94, which asks whether a rule correction that leaves every golden matching needs a bump at all. That case stays open; this one was never ambiguous. --- CURRENT_STATE.md | 31 ++- README.md | 2 +- docs/CONTRACT.md | 43 ++-- docs/adr/0013-contract-v2-corpus.md | 62 ++++++ docs/adr/README.md | 1 + docs/cli-parity.md | 4 +- src/ocpp/conformance/README.md | 20 +- .../fixtures/firmware-update-failure.json | 102 ++++++++++ .../fixtures/firmware-update-success.json | 102 ++++++++++ .../fixtures/refused-authorization.json | 183 ++++++++++++++++++ .../goldens/firmware-update-failure.json | 1 + .../goldens/firmware-update-success.json | 1 + .../goldens/refused-authorization.json | 1 + src/ocpp/conformance/harness.zig | 9 +- 14 files changed, 539 insertions(+), 23 deletions(-) create mode 100644 docs/adr/0013-contract-v2-corpus.md create mode 100644 src/ocpp/conformance/fixtures/firmware-update-failure.json create mode 100644 src/ocpp/conformance/fixtures/firmware-update-success.json create mode 100644 src/ocpp/conformance/fixtures/refused-authorization.json create mode 100644 src/ocpp/conformance/goldens/firmware-update-failure.json create mode 100644 src/ocpp/conformance/goldens/firmware-update-success.json create mode 100644 src/ocpp/conformance/goldens/refused-authorization.json diff --git a/CURRENT_STATE.md b/CURRENT_STATE.md index 288edea..c771f06 100644 --- a/CURRENT_STATE.md +++ b/CURRENT_STATE.md @@ -42,6 +42,35 @@ workflow, which builds and attaches the macOS + Linux packages. Next themes ## What's done +### contract-v2: the 18-scenario corpus (Issue #96) + +The vendored corpus was frozen at 15 scenarios for 0.5.0 (ADR-0012) while the +toolkit's grew to 18. The three missing ones were exactly the blind spots that let +#92 hide: no pinned fixture exercised a `FirmwareStatusNotification` status, a +non-`Invalid` `idTagInfo`, or the connector-status transitions the old matrix got +wrong, so the harness sat at 15/15 while three rules disagreed with the spec. + +- ✅ **`contract-v2`** cut per ADR-0012's rule, recorded in + [ADR-0013](docs/adr/0013-contract-v2-corpus.md): 18 fixtures + goldens + regenerated from the published toolkit `0.4.5`, harness gate now **18/18** under + both `native test` and `studio ci` +- ✅ Added `firmware-update-success` (`[]`), `firmware-update-failure` + (`["FIRMWARE_UPDATE_FAILURE"]`, covering the `InstallationFailed` the old rule + missed) and `refused-authorization` (`["FAILED_AUTHORIZATION"]`, covering + `Blocked` / `Expired` / `ConcurrentTx`) +- ✅ **Strict superset of v1:** regenerating all 18 reproduces v1's 15 fixtures and + goldens byte for byte, so nothing previously asserted changed. `contract-v1` + stays reachable through the `v0.5.4` tag's tree rather than being vendored twice +- ✅ `harness.zig` derives everything from `scenario_names`, so the three names are + the only code change; `docs/CONTRACT.md`, the conformance README, the top-level + README and `docs/cli-parity.md` updated. ADRs and CHANGELOG history left as + written +- The contract docs now say plainly that the corpus is a sample rather than a + proof: 18 scenarios still do not cover the 81 ordered connector-status pairs or + most of the status cross-products, so a green harness is evidence, not proof +- Does not settle #94 (whether a rule fix that moves no golden needs a bump); "a + new scenario" is named explicitly by ADR-0012, so this cut is the clear case + ### OCPP 1.6 spec-conformance parity (Issue #92) Three rules read the 1.6 spec wrongly in the same way the toolkit's did, fixed @@ -240,4 +269,4 @@ live inspector surface with notifications (#54–#60) — is landed. | `ui` (native views) | ✅ S3 inspector (#27–#32) + replay transport (#44) + live-capture view (#59) + live notifications (#60) | | `capture` (live proxy) | ✅ S5: WS transport (#54) + frame decode (#55) + MITM proxy (#56) + live notifications (#60) | | `cli` (headless) | ✅ inspect/report/diff/anonymize/ci/scenario (#45) + live `capture` (#57) | -| `conformance` | ✅ S2 (15/15) + frozen `contract-v1` (#72, docs/CONTRACT.md) | +| `conformance` | ✅ S2 (15/15) + frozen `contract-v1` (#72, docs/CONTRACT.md); widened to `contract-v2` (18/18) in #96 | diff --git a/README.md b/README.md index 5275aeb..ef1afc5 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,7 @@ Studio is one of two independent products under the OCPP DebugKit umbrella: | [`@ocpp-debugkit/toolkit`](https://github.com/ocpp-debugkit/toolkit) | TypeScript | npm library · CLI · web app | The library and CI brain — parse, analyze, and report OCPP traces anywhere | | **`ocpp-debugkit/studio`** (this repo) | Zig | native desktop app | The instrument on the bench — live capture, native performance, OS integration | -The two share **no code**. They meet only at a *conformance contract*: the same trace format, the same normalized event model, the same failure taxonomy, and the same scenario fixtures. A trace captured in Studio opens in the toolkit's web inspector, and vice versa — two independent implementations, one format, checked in CI on every change. See **[the conformance contract](docs/CONTRACT.md)** (`contract-v1`) and [ADR-0001](docs/adr/0001-independent-implementation.md). +The two share **no code**. They meet only at a *conformance contract*: the same trace format, the same normalized event model, the same failure taxonomy, and the same scenario fixtures. A trace captured in Studio opens in the toolkit's web inspector, and vice versa — two independent implementations, one format, checked in CI on every change. See **[the conformance contract](docs/CONTRACT.md)** (`contract-v2`) and [ADR-0001](docs/adr/0001-independent-implementation.md). ## What it does diff --git a/docs/CONTRACT.md b/docs/CONTRACT.md index 4e31eed..d8ba9c9 100644 --- a/docs/CONTRACT.md +++ b/docs/CONTRACT.md @@ -1,4 +1,4 @@ -# The conformance contract (`contract-v1`) +# The conformance contract (`contract-v2`) OCPP DebugKit Studio and the TypeScript [toolkit](https://github.com/ocpp-debugkit/toolkit) are **two independent implementations** of the same OCPP analysis behavior — one @@ -28,35 +28,56 @@ and it is enforced in CI on every change, not asserted in prose. 2 info**), and the same thresholds as the reference (heartbeat 60 s, slow response 10 s, ±50 % heartbeat deviation, session bounds, the 5-minute boot window). The rules live in [`src/ocpp/detection.zig`](../src/ocpp/detection.zig). -- **Scenarios.** **15 shared scenario traces** exercising the rules end to end, +- **Scenarios.** **18 shared scenario traces** exercising the rules end to end, each pinned to the exact de-duplicated, sorted `FailureCode` set the reference implementation detects. ## How conformance is enforced -The 15 scenarios and their expected failure-code sets are **vendored from the +The 18 scenarios and their expected failure-code sets are **vendored from the toolkit — the source of truth — never hand-authored**, so a golden can't drift from what the reference actually detects. On every change, `native test` runs Studio's full engine (`parseTrace → buildSessionTimeline → detectFailures`) over each fixture and asserts its detected code set equals the golden — the same -comparison the toolkit's `evaluateScenario` makes. **15/15 must match; a release +comparison the toolkit's `evaluateScenario` makes. **18/18 must match; a release is blocked otherwise.** `studio ci` runs the same check from the command line. +The corpus is a **sample, not a proof**: two implementations can both match every +golden and still disagree on inputs no fixture reaches. That is not theoretical. +Three rules disagreed with the specification here for as long as they did in the +toolkit while the harness stayed green, because no fixture exercised a firmware +status, a non-`Invalid` `idTagInfo`, or the connector-status transitions involved +(#93). When a rule changes, widen the corpus rather than trusting the count. + The vendored fixtures, goldens, the harness, and the exact regeneration recipe live in [`src/ocpp/conformance/`](../src/ocpp/conformance/README.md). ## Freeze & versioning -For the 0.5.0 release the contract is **frozen as `contract-v1`** (OCPP 1.6J): -the vendored fixtures and goldens are pinned, and the harness gates every build -against them. The freeze rationale is recorded in -[ADR-0012](adr/0012-freeze-contract-v1.md). +The current contract is **`contract-v2`** (OCPP 1.6J), 18 scenarios, regenerated +against toolkit `0.4.5`. The vendored fixtures and goldens are pinned, and the +harness gates every build against them. The cut is recorded in +[ADR-0013](adr/0013-contract-v2-corpus.md); `contract-v1` (15 scenarios, frozen +for 0.5.0) is [ADR-0012](adr/0012-freeze-contract-v1.md). + +`contract-v2` is a **strict superset** of v1: regenerating all 18 against toolkit +0.4.5 reproduces v1's 15 fixtures and goldens byte for byte, so every scenario v1 +pinned, v2 pins identically. The contract is **immutable within a version.** A change that alters detected output — a new or changed rule, a new scenario, a threshold change, or OCPP -2.0.1 — is a **new contract version** (`contract-v2`, …), regenerated against the -matching toolkit release and re-tagged, never an in-place edit of `contract-v1`. -This keeps "two implementations, one format" a checkable claim rather than a hope. +2.0.1 — is a **new contract version** (`contract-v3`, …), regenerated against the +matching toolkit release and re-tagged, never an in-place edit of a released +version. Earlier versions stay reachable through the release tags: `v0.5.4`'s tree +carries `contract-v1`, so every released Studio remains pinned to a reproducible +reference. This keeps "two implementations, one format" a checkable claim rather +than a hope. + +One question this policy leaves open, tracked in #94: whether a rule correction +that leaves **every** golden matching needs a version bump at all, given that the +contract's artifacts are the fixtures and the goldens. The corpus expansion above +is not that case, since "a new scenario" is named explicitly, so it does not +settle it. See also [ADR-0001](adr/0001-independent-implementation.md) (independent implementation + shared contract) and [docs/cli-parity.md](cli-parity.md) diff --git a/docs/adr/0013-contract-v2-corpus.md b/docs/adr/0013-contract-v2-corpus.md new file mode 100644 index 0000000..fd5baaa --- /dev/null +++ b/docs/adr/0013-contract-v2-corpus.md @@ -0,0 +1,62 @@ +# ADR-0013 — Cut `contract-v2`: the 18-scenario corpus + +- **Status:** Accepted +- **Date:** 2026-07-30 + +## Context + +[ADR-0012](0012-freeze-contract-v1.md) froze the vendored conformance corpus as +`contract-v1` (15 scenarios) for the 0.5.0 release, and set the rule that a new +scenario means a new contract version rather than an in-place edit. + +The toolkit's corpus has since grown to 18. `firmware-update-success` and +`firmware-update-failure` landed in toolkit 0.4.3/0.4.4, and +`refused-authorization` in toolkit 0.4.5. None of the three was in the contract. + +That gap turned out to matter. Three detection rules disagreed with the OCPP 1.6 +specification here for as long as they did in the toolkit (#92, fixed in #93, +released in 0.5.4), and the harness stayed at 15/15 the whole time, because no +pinned fixture exercised a `FirmwareStatusNotification` status, a non-`Invalid` +`idTagInfo`, or any of the 22 connector-status transitions the old matrix flagged +wrongly. The contract was green while both implementations were wrong in the same +way, and later while they were wrong in *different* ways. Verifying #93 needed an +ad-hoc probe of 97 traces across both engines, which proves a point in time but +guards nothing afterwards. + +The three missing scenarios land precisely on those blind spots. + +## Decision + +**Cut `contract-v2` (OCPP 1.6J): 18 scenarios, regenerated against toolkit +`0.4.5`.** The harness gate becomes 18/18, and `studio ci` reports 18 scenarios. + +`contract-v2` is a **strict superset** of `contract-v1`. Regenerating all 18 from +the published toolkit 0.4.5 reproduces v1's 15 fixtures and goldens byte for byte; +every scenario v1 pinned, v2 pins identically, with three added. So this is an +additive widening of coverage, not a revision of anything previously asserted. + +`contract-v1` is not kept side by side in the tree. ADR-0012's "never edited in +place" is satisfied by history: the `v0.5.4` tag's tree carries the v1 corpus, so +every released Studio stays pinned to a reproducible reference, which is the +consequence ADR-0012 named. Vendoring both versions at once would double the +embedded test data for no reachable consumer. + +## Consequences + +- The gate now covers the firmware statuses and the non-`Invalid` authorization + refusals it previously never reached, so the class of divergence #92 describes + is caught by CI rather than by someone reading the spec against two codebases. +- The corpus is still a **sample**, and the contract docs now say so. 18 scenarios + do not cover the 81 ordered connector-status pairs, or most of the + `AuthorizationStatus` and `FirmwareStatus` cross-products. The lesson from #92 is + that a green harness is evidence, not proof; when a rule changes, widen the + corpus. +- Regeneration is now expected to be a byte-identical no-op for existing files, + which makes an unintended fixture change a visible signal rather than noise. +- This ADR refines ADR-0012's versioning half by cutting the next version under + its rule; it does not supersede it, and ADR-0012 stands as the record of the v1 + freeze. +- It does not answer #94, which asks whether a rule correction that leaves every + golden matching needs a version bump at all. "A new scenario" is named + explicitly by ADR-0012, so this cut is the unambiguous case; the ambiguous one + stays open. diff --git a/docs/adr/README.md b/docs/adr/README.md index a3736e0..5d90376 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -22,3 +22,4 @@ supersedes the old one rather than editing history. | [0010](0010-live-proxy-transport-concurrency.md) | Live-proxy transport & concurrency | Accepted | | [0011](0011-live-notification-services.md) | Live-capture notifications via the effects-bound platform services | Accepted | | [0012](0012-freeze-contract-v1.md) | Freeze `contract-v1` for the 0.5.0 release | Accepted | +| [0013](0013-contract-v2-corpus.md) | Cut `contract-v2`: the 18-scenario corpus | Accepted | diff --git a/docs/cli-parity.md b/docs/cli-parity.md index 090176c..b87a29c 100644 --- a/docs/cli-parity.md +++ b/docs/cli-parity.md @@ -13,7 +13,7 @@ intentional difference. | `report -f ` | `studio report [-f markdown\|html]` | ✅ Full — Markdown + self-contained HTML. | | `diff --format ` | `studio diff [--format text\|json]` | ✅ Full — text and JSON. | | `anonymize ` | `studio anonymize ` | ✅ Full — same field rules (see the anonymize module). | -| `ci [dir]` | `studio ci` | ⚠️ Runs the **vendored** `contract-v1` scenarios (Studio embeds the contract); the `[dir]` argument is not accepted. Exits `0` (all pass) / `1` (any fail). | +| `ci [dir]` | `studio ci` | ⚠️ Runs the **vendored** `contract-v2` scenarios (Studio embeds the contract); the `[dir]` argument is not accepted. Exits `0` (all pass) / `1` (any fail). | | `scenario list` | `studio scenario list` | ✅ Full. | | `scenario run ` | `studio scenario run ` | ✅ Built-in scenarios. | | `scenario run --file ` | — | ⚠️ Deferred — Studio runs only the built-in contract scenarios. | @@ -26,7 +26,7 @@ intentional difference. convenience is a deferred follow-up (it needs the sandboxed `init.io` write path); redirection covers the same need today. - **`ci` runs the embedded contract**, not a user directory. Studio vendors the - 15 `contract-v1` fixtures + goldens under `src/ocpp/conformance/` (ADR-0004), + 18 `contract-v2` fixtures + goldens under `src/ocpp/conformance/` (ADR-0004), so `ci` is a self-contained conformance gate — the same logic the `native test` conformance harness runs. See [the conformance contract](CONTRACT.md). diff --git a/src/ocpp/conformance/README.md b/src/ocpp/conformance/README.md index f2627b3..60bdb13 100644 --- a/src/ocpp/conformance/README.md +++ b/src/ocpp/conformance/README.md @@ -1,10 +1,10 @@ -# Conformance contract (`contract-v1`) +# Conformance contract (`contract-v2`) Studio and the TypeScript toolkit are **independent implementations** of the same OCPP analysis behavior (ADR-0001). This directory is the machine-checked contract between them: the shared scenarios and the failure codes each must detect. -- **`fixtures/.json`** — the 15 shared scenario **traces** (the input). +- **`fixtures/.json`** — the 18 shared scenario **traces** (the input). - **`goldens/.json`** — each scenario's expected **de-duplicated, sorted `FailureCode` set** (the output to match), as a JSON array of wire codes. - **`harness.zig`** — runs, under `native test`, the full engine @@ -39,6 +39,16 @@ lets the harness embed it directly — no runtime file I/O, fully hermetic. ## Versioning -Tagged **`contract-v1`** (OCPP 1.6J). When the contract changes — new rules, -new scenarios, or OCPP 2.0.1 — regenerate against the matching toolkit release -and bump the tag. +Tagged **`contract-v2`** (OCPP 1.6J), 18 scenarios, regenerated against toolkit +`0.4.5`. When the contract changes — new rules, new scenarios, or OCPP 2.0.1 — +regenerate against the matching toolkit release and bump the tag. Earlier versions +stay reachable through the release tags rather than being kept side by side here: +`v0.5.4`'s tree carries `contract-v1` (15 scenarios). + +Regenerating is expected to reproduce the existing files byte for byte, and that +is worth checking when you do it: if a fixture you did not mean to touch changes, +either the toolkit version is wrong or the change is bigger than you thought. +`contract-v2` was cut that way, so it is a strict superset of v1. + +See [ADR-0013](../../../docs/adr/0013-contract-v2-corpus.md) for the v2 cut and +[docs/CONTRACT.md](../../../docs/CONTRACT.md) for the public contract. diff --git a/src/ocpp/conformance/fixtures/firmware-update-failure.json b/src/ocpp/conformance/fixtures/firmware-update-failure.json new file mode 100644 index 0000000..41fee58 --- /dev/null +++ b/src/ocpp/conformance/fixtures/firmware-update-failure.json @@ -0,0 +1,102 @@ +{ + "traceId": "scenario-firmware-update-failure", + "metadata": { + "stationId": "CS-SYNTHETIC-019", + "ocppVersion": "1.6", + "source": "synthetic-scenario", + "description": "Station boots, reports Downloading firmware, then InstallationFailed within seconds, followed by a heartbeat to confirm station is still online." + }, + "events": [ + { + "timestamp": "2026-01-15T07:00:00.000Z", + "direction": "CS_TO_CSMS", + "message": [ + 2, + "msg-001", + "BootNotification", + { + "chargePointVendor": "SyntheticVendor", + "chargePointModel": "SM-100", + "chargePointSerialNumber": "CS-SYNTHETIC-019", + "firmwareVersion": "1.0.0" + } + ] + }, + { + "timestamp": "2026-01-15T07:00:00.500Z", + "direction": "CSMS_TO_CS", + "message": [ + 3, + "msg-001", + { + "currentTime": "2026-01-15T07:00:00.500Z", + "interval": 300, + "status": "Accepted" + } + ] + }, + { + "timestamp": "2026-01-15T07:00:30.000Z", + "direction": "CS_TO_CSMS", + "message": [ + 2, + "msg-002", + "FirmwareStatusNotification", + { + "status": "Downloading" + } + ] + }, + { + "timestamp": "2026-01-15T07:00:30.500Z", + "direction": "CSMS_TO_CS", + "message": [ + 3, + "msg-002", + {} + ] + }, + { + "timestamp": "2026-01-15T07:01:00.000Z", + "direction": "CS_TO_CSMS", + "message": [ + 2, + "msg-003", + "FirmwareStatusNotification", + { + "status": "InstallationFailed" + } + ] + }, + { + "timestamp": "2026-01-15T07:01:00.500Z", + "direction": "CSMS_TO_CS", + "message": [ + 3, + "msg-003", + {} + ] + }, + { + "timestamp": "2026-01-15T07:02:00.000Z", + "direction": "CS_TO_CSMS", + "message": [ + 2, + "msg-hb-1", + "Heartbeat", + {} + ] + }, + { + "timestamp": "2026-01-15T07:02:00.500Z", + "direction": "CSMS_TO_CS", + "message": [ + 3, + "msg-hb-1", + { + "currentTime": "2026-01-15T07:02:00.500Z" + } + ] + } + ] +} diff --git a/src/ocpp/conformance/fixtures/firmware-update-success.json b/src/ocpp/conformance/fixtures/firmware-update-success.json new file mode 100644 index 0000000..afd7bbd --- /dev/null +++ b/src/ocpp/conformance/fixtures/firmware-update-success.json @@ -0,0 +1,102 @@ +{ + "traceId": "scenario-firmware-update-success", + "metadata": { + "stationId": "CS-SYNTHETIC-016", + "ocppVersion": "1.6", + "source": "synthetic-scenario", + "description": "Station boots, reports the firmware as downloaded and installed within seconds, then sends a heartbeat." + }, + "events": [ + { + "timestamp": "2024-01-15T06:00:00.000Z", + "direction": "CS_TO_CSMS", + "message": [ + 2, + "msg-001", + "BootNotification", + { + "chargePointVendor": "SyntheticVendor", + "chargePointModel": "SM-100", + "chargePointSerialNumber": "CS-SYNTHETIC-016", + "firmwareVersion": "1.0.0" + } + ] + }, + { + "timestamp": "2024-01-15T06:00:00.500Z", + "direction": "CSMS_TO_CS", + "message": [ + 3, + "msg-001", + { + "currentTime": "2024-01-15T06:00:00.500Z", + "interval": 300, + "status": "Accepted" + } + ] + }, + { + "timestamp": "2024-01-15T06:00:30.000Z", + "direction": "CS_TO_CSMS", + "message": [ + 2, + "msg-002", + "FirmwareStatusNotification", + { + "status": "Downloaded" + } + ] + }, + { + "timestamp": "2024-01-15T06:00:30.500Z", + "direction": "CSMS_TO_CS", + "message": [ + 3, + "msg-002", + {} + ] + }, + { + "timestamp": "2024-01-15T06:01:00.000Z", + "direction": "CS_TO_CSMS", + "message": [ + 2, + "msg-003", + "FirmwareStatusNotification", + { + "status": "Installed" + } + ] + }, + { + "timestamp": "2024-01-15T06:01:00.500Z", + "direction": "CSMS_TO_CS", + "message": [ + 3, + "msg-003", + {} + ] + }, + { + "timestamp": "2024-01-15T06:03:00.000Z", + "direction": "CS_TO_CSMS", + "message": [ + 2, + "msg-hb-1", + "Heartbeat", + {} + ] + }, + { + "timestamp": "2024-01-15T06:03:00.500Z", + "direction": "CSMS_TO_CS", + "message": [ + 3, + "msg-hb-1", + { + "currentTime": "2024-01-15T06:03:00.500Z" + } + ] + } + ] +} diff --git a/src/ocpp/conformance/fixtures/refused-authorization.json b/src/ocpp/conformance/fixtures/refused-authorization.json new file mode 100644 index 0000000..c111b4c --- /dev/null +++ b/src/ocpp/conformance/fixtures/refused-authorization.json @@ -0,0 +1,183 @@ +{ + "traceId": "scenario-refused-authorization", + "metadata": { + "stationId": "CS-SYNTHETIC-017", + "ocppVersion": "1.6", + "source": "synthetic-scenario", + "description": "Station boots, connector prepares, then three idTags are refused in turn: one Blocked, one Expired, one ConcurrentTx. Section 7.2 marks ConcurrentTx as relevant to StartTransaction, so an Authorize response carrying it is irregular; it is included because the rule treats every non-Accepted status as a refusal wherever it appears. The connector returns to Available without a transaction." + }, + "events": [ + { + "timestamp": "2026-01-15T09:00:00.000Z", + "direction": "CS_TO_CSMS", + "message": [ + 2, + "msg-001", + "BootNotification", + { + "chargePointVendor": "SyntheticVendor", + "chargePointModel": "SM-100", + "chargePointSerialNumber": "CS-SYNTHETIC-017", + "firmwareVersion": "1.0.0" + } + ] + }, + { + "timestamp": "2026-01-15T09:00:00.500Z", + "direction": "CSMS_TO_CS", + "message": [ + 3, + "msg-001", + { + "currentTime": "2026-01-15T09:00:00.500Z", + "interval": 300, + "status": "Accepted" + } + ] + }, + { + "timestamp": "2026-01-15T09:00:05.000Z", + "direction": "CS_TO_CSMS", + "message": [ + 2, + "msg-002", + "StatusNotification", + { + "connectorId": 0, + "status": "Available", + "errorCode": "NoError" + } + ] + }, + { + "timestamp": "2026-01-15T09:00:05.500Z", + "direction": "CSMS_TO_CS", + "message": [ + 3, + "msg-002", + {} + ] + }, + { + "timestamp": "2026-01-15T09:00:10.000Z", + "direction": "CS_TO_CSMS", + "message": [ + 2, + "msg-003", + "StatusNotification", + { + "connectorId": 1, + "status": "Preparing", + "errorCode": "NoError" + } + ] + }, + { + "timestamp": "2026-01-15T09:00:10.500Z", + "direction": "CSMS_TO_CS", + "message": [ + 3, + "msg-003", + {} + ] + }, + { + "timestamp": "2026-01-15T09:00:20.000Z", + "direction": "CS_TO_CSMS", + "message": [ + 2, + "msg-004", + "Authorize", + { + "idTag": "SYNTHETIC-TAG-201" + } + ] + }, + { + "timestamp": "2026-01-15T09:00:20.500Z", + "direction": "CSMS_TO_CS", + "message": [ + 3, + "msg-004", + { + "idTagInfo": { + "status": "Blocked" + } + } + ] + }, + { + "timestamp": "2026-01-15T09:00:35.000Z", + "direction": "CS_TO_CSMS", + "message": [ + 2, + "msg-005", + "Authorize", + { + "idTag": "SYNTHETIC-TAG-202" + } + ] + }, + { + "timestamp": "2026-01-15T09:00:35.500Z", + "direction": "CSMS_TO_CS", + "message": [ + 3, + "msg-005", + { + "idTagInfo": { + "status": "Expired" + } + } + ] + }, + { + "timestamp": "2026-01-15T09:00:50.000Z", + "direction": "CS_TO_CSMS", + "message": [ + 2, + "msg-006", + "Authorize", + { + "idTag": "SYNTHETIC-TAG-203" + } + ] + }, + { + "timestamp": "2026-01-15T09:00:50.500Z", + "direction": "CSMS_TO_CS", + "message": [ + 3, + "msg-006", + { + "idTagInfo": { + "status": "ConcurrentTx" + } + } + ] + }, + { + "timestamp": "2026-01-15T09:01:05.000Z", + "direction": "CS_TO_CSMS", + "message": [ + 2, + "msg-007", + "StatusNotification", + { + "connectorId": 1, + "status": "Available", + "errorCode": "NoError" + } + ] + }, + { + "timestamp": "2026-01-15T09:01:05.500Z", + "direction": "CSMS_TO_CS", + "message": [ + 3, + "msg-007", + {} + ] + } + ] +} diff --git a/src/ocpp/conformance/goldens/firmware-update-failure.json b/src/ocpp/conformance/goldens/firmware-update-failure.json new file mode 100644 index 0000000..4427c4f --- /dev/null +++ b/src/ocpp/conformance/goldens/firmware-update-failure.json @@ -0,0 +1 @@ +["FIRMWARE_UPDATE_FAILURE"] diff --git a/src/ocpp/conformance/goldens/firmware-update-success.json b/src/ocpp/conformance/goldens/firmware-update-success.json new file mode 100644 index 0000000..fe51488 --- /dev/null +++ b/src/ocpp/conformance/goldens/firmware-update-success.json @@ -0,0 +1 @@ +[] diff --git a/src/ocpp/conformance/goldens/refused-authorization.json b/src/ocpp/conformance/goldens/refused-authorization.json new file mode 100644 index 0000000..50b085c --- /dev/null +++ b/src/ocpp/conformance/goldens/refused-authorization.json @@ -0,0 +1 @@ +["FAILED_AUTHORIZATION"] diff --git a/src/ocpp/conformance/harness.zig b/src/ocpp/conformance/harness.zig index c2fe451..72f06ba 100644 --- a/src/ocpp/conformance/harness.zig +++ b/src/ocpp/conformance/harness.zig @@ -1,12 +1,12 @@ //! Conformance harness — pins Studio's detection to the toolkit's. //! -//! For each of the 15 shared scenarios, the vendored **trace** runs through the +//! For each of the 18 shared scenarios, the vendored **trace** runs through the //! full engine (`parseTrace → buildSessionTimeline → detectFailures`) and its //! de-duplicated, sorted `FailureCode` set is compared to the locked **golden** //! — the same comparison semantics as the toolkit's `evaluateScenario`. //! //! Fixtures and goldens under `fixtures/` and `goldens/` are generated from the -//! toolkit (the source of truth), tagged `contract-v1`; they are not authored by +//! toolkit (the source of truth), tagged `contract-v2`; they are not authored by //! hand. See `README.md` in this directory. This layout lives under `src/` so the //! zero-config build can `@embedFile` it (ADR-0004). //! @@ -18,7 +18,7 @@ const parser = @import("../parser.zig"); const timeline = @import("../timeline.zig"); const detection = @import("../detection.zig"); -/// The 15 shared scenarios (the toolkit's `scenarioNames`). +/// The 18 shared scenarios (the toolkit's `scenarioNames`), in contract order. const scenario_names = [_][]const u8{ "normal-session", "failed-auth", @@ -35,6 +35,9 @@ const scenario_names = [_][]const u8{ "short-session", "heartbeat-irregular", "unresponsive-csms", + "firmware-update-success", + "firmware-update-failure", + "refused-authorization", }; /// The shared scenario names, in contract order. From 13621743a02d9bdf075d09eb740038996b952bf3 Mon Sep 17 00:00:00 2001 From: sepehr-safari Date: Thu, 30 Jul 2026 13:46:49 +0300 Subject: [PATCH 2/2] docs(changelog): record the contract-v2 corpus under Unreleased --- CHANGELOG.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 738ef2c..ee005d7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,17 @@ reference are all pre-1.0, minor (0.x) releases may include breaking changes. ## [Unreleased] +### Changed + +- The conformance contract is now **`contract-v2`**: 18 shared scenarios, up from + 15, regenerated against toolkit `0.4.5`. `studio ci` and the build-time harness + gate on 18/18. The three added scenarios (`firmware-update-success`, + `firmware-update-failure`, `refused-authorization`) cover the firmware statuses + and the non-`Invalid` authorization refusals the frozen corpus never reached, + which is how the rule defects fixed in 0.5.4 stayed green at 15/15. `contract-v2` + is a strict superset of v1: the 15 existing fixtures and goldens are unchanged, + byte for byte. See [ADR-0013](docs/adr/0013-contract-v2-corpus.md). + ## [0.5.4] - 2026-07-30 ### Fixed