Rectify: canonical JSON producer/consumer registry ratchet - #4413
Merged
Trecek merged 6 commits intoJul 29, 2026
Merged
Conversation
audit-impl and make-plan SKILL.md prose instructed the LLM to hand-assemble byte-exact canonical JSON and hand-compute SHA-256 domain-separated digests for four hash-bound artifacts (authority.json, inventory.json, PlanDispositionReport, plan-association files), all read back with decode_versioned_json_bytes(..., require_canonical=True). No Python function ever constructed these artifacts in production code — write_canonical_versioned_json had zero call sites — so the LLM producer path had no mechanical verification before writing, and ordinary pretty-printed JSON was rejected outright. Adds write_audit_cycle_artifact, a narrow MCP tool (mirroring commit_files' headless-reachable, gate-free registration) that performs construction, digest computation via the existing .create() classmethods, dataclass validation, and canonical serialization entirely server-side for all four artifact kinds. Relocates the plan-association domain/keys/file-count constants to core/types/_type_audit_cycle.py (IL-0) so the new write-side tool and the existing read-side _resolve_plan_disposition share one definition. Adds an exclusive=True write-once guard to atomic_write/ write_canonical_versioned_json to close a TOCTOU window between an existence check and the write. Wires both SKILL.md files to call the new tool by name instead of describing the artifact only by filename. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The new MCP tool's module was never imported in server/__init__.py, so @mcp.tool() registration never ran and the tool was invisible to every session. Import it, add the missing broad-except logger call (ARCH-003), fix an import-sort violation in core/__init__.pyi, and update every downstream ratchet (tool counts, doc counts, cascade maps, display categories, layer-boundary allowlist, path-guard patterns, subpackage file-count exemption, and the make-plan disposition-report contract test) to reflect the tool's existence.
_SERVER_TOOL_MODULES in test_tool_params_matches_mcp_handler_signatures enumerates server tool modules by hand rather than discovering them — the new tools_audit_cycle.py module was missing, so its handler was invisible to the TOOL_REGISTRY parity check.
…t B) Adds tests/infra/test_canonical_json_producer_convention.py, an AST-scan ratchet mirroring test_schema_version_convention.py: every decode_versioned_json_bytes(require_canonical=True) consumer site in src/autoskillit/ must be registered against a verified server-side producer (write_audit_cycle_artifact) and a SKILL.md section naming it, closing the gap that let #4406's canonical/non-canonical mismatch land undetected. Also corrects core/io.py's module docstring, which recommended write_versioned_json unconditionally with no carve-out for artifacts consumed with require_canonical=True.
…failure atomic_write's exclusive=True path claims path via O_CREAT|O_EXCL before the try block. If the subsequent temp-file write/fsync/os.replace fails, the except handler now also unlinks the placeholder at path (in addition to the temp file), preventing a permanently poisoned path that would raise FileExistsError on every future retry.
…der cleanup mkstemp() ran outside the try/except, so a mkstemp failure (ENOSPC, permission error) would leave the O_CREAT|O_EXCL placeholder behind, permanently poisoning the path with FileExistsError on every retry.
Trecek
deleted the
audit-impl-writes-authority-json-pretty-printed-but-the-veri/4406-2
branch
July 29, 2026 15:22
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Part A adds a server-side MCP tool (
write_audit_cycle_artifact) that performs construction, digest computation, dataclass validation, and canonical serialization entirely server-side for the four hash-bound audit-cycle artifacts — closing the gap where LLM-agent skills (audit-impl,make-plan) had no mechanical way to produce the byte-exact canonical JSON that the strict verifier (core/audit_cycle_verifier.py) requires. Part B adds the systemic guard: an AST-scan test ratchet, mirroring the existingwrite_versioned_jsonconvention tests, that failstask test-allimmediately if a futurerequire_canonical=Trueconsumer is added with no registered, verified producer — plus corrects a module docstring that had pointed developers at the wrong JSON-writing helper.Individual Group Plans
Part A — new MCP tool
write_audit_cycle_artifactaudit-implandmake-planare markdown-driven LLM-agent skills. Their SKILL.md proseinstructs the agent to write four JSON artifacts —
authority.json,inventory.json,the
PlanDispositionReport, and plan-association files — that a strict Python verifier(
core/audit_cycle_verifier.pyfor the first three,recipe/_cmd_rpc_guards.py:284-288for the fourth) later reads via
decode_versioned_json_bytes(..., require_canonical=True).That call demands byte-exact canonical JSON (sorted keys, compact separators, no
whitespace). No SKILL.md names a serializer that produces such bytes; the one helper that
does (
write_canonical_versioned_json,core/io.py:392-401) has zero call sitesanywhere in the codebase. The LLM naturally emits ordinary pretty-printed JSON, which is
rejected outright.
The root defect is not "wrong helper named" — it is that these four artifacts are the
only hash-bound, tamper-evident artifacts in the codebase for which no Python function
ever constructs or serializes an instance in production code. Every dataclass in this
family (
AuditCycleAuthority,PlanDispositionReport, their row/ref types) has a.create()classmethod that computes its own content digest from semantic fields, and astrict
__post_init__that re-derives and checks that digest — butgrep -rn "AuditCycleAuthority(\|AuditCycleAuthority\.create(\|PlanDispositionReport(\|PlanDispositionReport\.create(" src/returns zero matches. The only place these types are ever instantiated is.from_dict(), on read. The "producer" is 100% LLM prose asking a language model tohand-assemble byte-exact JSON and correctly compute SHA-256 domain-separated digests by
hand — a task with no mechanical verification available to the model before it writes.
The architectural fix is to give the LLM producer session a single, narrow, MCP-tool call
that performs construction, digest computation, dataclass validation, and canonical
serialization entirely server-side — removing both failure modes (bad JSON formatting
and bad digest arithmetic) from the token-by-token generation path, the same way
commit_files(server/tools/tools_git.py:634-660) already removes "the LLM mustcorrectly stage/commit/run pre-commit hooks" from the git-write path for these same
sessions. A closely related but architecturally invalid candidate — routing the write
through the existing
run_pythonRPC mechanism (recipe/_cmd_rpc.py), as several otherskills already do for precision-sensitive writes — was investigated and rejected:
run_pythonis hard-gated to
SessionType.ORCHESTRATOR/FLEETonly (_require_orchestrator_or_higher,server/_guards.py:59-79), andaudit-impl/make-planrun as childSessionType.SKILLsessions dispatched via
run_skillfrom theimplementation.yamlorchestrator recipe.run_pythonis tag-invisible to aSKILL+HEADLESSsession by default (its tags are{"autoskillit", "kitchen", "kitchen-core"}— noheadlesstag) and, even ifAUTOSKILLIT_HEADLESS_AUTO_GATE=1revealed thekitchen-coretag, the tool would still berejected by its own Python-layer gate because
session_type()resolves toSKILL, notORCHESTRATOR/FLEET. A new MCP tool taggedheadless(mirroringcommit_files, whichcarries no
_require_enabled()/_require_orchestrator_or_higher()gate at all) is the onlymechanism in this codebase's existing tool-gating architecture that a
SKILL+HEADLESSsession can reach directly and unconditionally.
Part B — systemic AST-scan ratchet
The underlying investigation (GitHub #4406) found that a strict Python verifier
(
core/audit_cycle_verifier.py,recipe/_cmd_rpc_guards.py:274-278) requires byte-exactcanonical JSON (
decode_versioned_json_bytes(..., require_canonical=True)) for fourhash-bound artifacts, while the LLM-agent producers of those artifacts (
audit-impl,make-planSKILL.md prose) had no mechanical way to guarantee that byte shape. A prior taskfixed the four known instances by routing the write through a new server-side MCP tool. This
part adds the systemic guard: a structural test ratchet that makes a fifth, future
instance of this same bug class (a new
require_canonical=Trueconsumer with no registered,verified producer) fail
task test-allimmediately, instead of surfacing only inproduction the way #4406 did — plus corrects a module docstring that contributed to the
original gap by pointing developers at the wrong JSON-writing helper.
This mirrors an existing, working pattern already in this codebase:
tests/infra/test_schema_version_convention.pyAST-scanssrc/autoskillit/for everyatomic_write(path, json.dumps({...}))call site and requires it to either usewrite_versioned_jsonor be in an explicit, curated allowlist; a companion file,tests/infra/test_schema_read_convention.py, checks that everywrite_versioned_jsoncaller has matching read-side validation. No equivalent ratchet exists for the canonical
family (
grep -rn "require_canonical" tests/returns zero hits before this part) — thisis the load-bearing gap this part closes, for the canonical family specifically, since that
is the family where drift silently breaks tamper-evidence rather than just schema-version
detection.
Closes #4406
Implementation Plan
Plan files:
/home/talon/projects/autoskillit-runs/remediation-20260728-224547-129022/.autoskillit/temp/rectify/rectify_audit_cycle_canonical_json_producer_2026-07-28_232500_part_a.md/home/talon/projects/autoskillit-runs/remediation-20260728-224547-129022/.autoskillit/temp/rectify/rectify_audit_cycle_canonical_json_producer_2026-07-28_232500_part_b.md🤖 Generated with Claude Code via AutoSkillit
Token Usage Summary
* Step used a non-Anthropic provider; caching behavior may differ.
Token Efficiency
Model Usage Breakdown