feat: representability repair loop + casualty-free generation view (0.4.0) - #57
Merged
Conversation
New src/run/representability.test.ts (ScriptedAdapter + inline mini-kit
contract + a v2 emit profile via dspack-emit loadProfile declaring
'marquee' a cannot-represent casualty) pins the ratified mechanism:
1. an EmitSurfaceError enters the repair loop (refusal -> repair turn ->
corrected surface -> passed; refusal recorded per-attempt);
2. repairs exhausted on refusals stays terminal failed-gate/exit 3 with
the refusal recorded and repairMessages.length === maxRepairs;
3. casualtyFreeView strips casualties from the generation context
(system prompt / schema / few-shot) while S-gates keep the ratified
contract;
4. the Ollama request body carries think: false (existing adapter test).
Premise checks (casualty surface lints S1-S3 clean; CURRENT emitSurface
refuses it with the declared-casualty reason) pass today by design.
casualty-view.js is imported dynamically so this run reports behavioral
failures instead of one file-level module error.
Failing output on current code (fail-first evidence):
> src/run/representability.test.ts (11 tests | 8 failed)
x emitter refusal enters the repair loop > refusal -> repair -> corrected surface -> passed
-> expected 'failed-gate' to be 'passed' // Object.is equality
x emitter refusal enters the repair loop > the refusal is recorded on the refused attempt, not only terminally
-> expected undefined to deeply equal { pass: false, ...(1) }
x emitter refusal enters the repair loop > the repair message carries the refusal verbatim plus the casualty hint
-> expected +0 to be 1 // Object.is equality
x emitter refusal enters the repair loop > the conversation delta is the model's own output + the repair turn (ADR-7 shape)
-> expected 1 to be 2 // Object.is equality
x emitter refusal enters the repair loop > generation never saw the casualty: system prompt and few-shot are casualty-free
-> expected 'You generate user interface surfaces ...' not to contain 'marquee'
x repairs exhausted on refusals > every attempt refused -> terminal failed-gate, exit 3, refusal recorded (unchanged semantics)
-> expected 1 to be 3 // Object.is equality
x casualtyFreeView > compileContext over the view drops the casualty from system prompt, schema, and few-shot
-> Failed to load url ./casualty-view.js ... Does the file exist?
x casualtyFreeView > no profile -> the identical contract object (not a copy)
-> Failed to load url ./casualty-view.js ... Does the file exist?
> src/adapters/adapters.test.ts (13 tests | 1 failed)
x OllamaAdapter > produces a parsed result ...; the generation schema round-trips verbatim
-> expected undefined to be false // Object.is equality
Test Files 2 failed (2)
Tests 9 failed | 15 passed (24)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…eave generation (0.4.0)
Three ratified changes, all run-layer; src/core/ is byte-untouched
(`git diff --stat main -- src/core/` is empty; ds-mcp vendors core).
1. Refusals become repairable (src/run/orchestrator.ts): the a2ui
EmitSurfaceError catch records `attempts[].representability =
{ pass: false, refusal }` and — while `index < maxRepairs` — renders
the refusal as the next repair turn (base message + one refusal-class
hint: declared casualty / donation boundary / join-key), pushes the
assistant raw + repair onto the conversation exactly like the S3
loop, emits the `repair` event, and continues the loop. Budget
exhausted keeps the original terminal semantics (failed-gate, exit 3,
emitted.refusal). The post-emit validations-fail branch stays
terminal, with a comment that emit-side self-validation
(dspack-emit >= 0.7) makes it unreachable from emit output. S1-S3 and
their repair path untouched. The p05 eval cell is that terminal A3
class, so the fake-eval golden is byte-identical (CI gate passed).
2. Casualty-free generation view (src/run/casualty-view.ts):
`casualtyFreeView(contract, profile)` shallow-copies the contract
minus the profile's `casualtyComponents` ids and minus every example
whose surface tree uses one; identity when no profile/casualties.
The orchestrator compiles generation context from the view ONLY —
lintSurface and contractDigest keep the ratified original (S2
vocabulary and report identity are profile-independent).
3. Ollama think:false (src/adapters/ollama.ts): structured-output
generation cannot budget reasoning tokens — thinking models burned
the whole num_predict on reasoning and returned "empty model output";
verified harmless on non-thinking models. ("qwen 3.6" is spelled
with a space because the no-default-model guard scans this file.)
Also: audit-report v1 schema + docs/AUDIT.md gain the additive
`attempts[].representability` field; the legacy pipeline.test.ts
refusal test pins the terminal case via maxRepairs: 0 (the loop is
covered by representability.test.ts); version 0.3.2 -> 0.4.0 with
CHANGELOG.
Fail-first evidence: the previous commit (0ad85cb) carries the full
failing output of these tests against pre-change code — 9 behavioral
failures (failed-gate where passed was asserted, 1 attempt instead of
3, marquee present in the system prompt, think undefined). This commit
changes no assertion in that suite; test files touched here are the
legacy refusal test's options/comment only.
npm test: 16 files, 149/149 passed. tsc build clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… consumers) Extends the OR-range so an app pinning emit 0.7 dedupes to ONE emitter copy — the representability loop must run the same emitSurface the app validates with. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR implements Phase 2 of representability handling in the run pipeline: emitter refusals (EmitSurfaceError) are now treated as repair-loop events (when budget remains) instead of immediate terminal failures, and generation is compiled from a casualty-free view of the contract to avoid steering models into known-unrepresentable components.
Changes:
- Route
EmitSurfaceErrorthrough the bounded repair loop and record per-attempt representability refusals in the audit report (attempts[].representability). - Introduce
casualtyFreeView(contract, profile)and use it only for generation context compilation (system prompt / schema / few-shot), while keeping S-gates and contract digest tied to the original contract. - Add
think: falseto Ollama chat requests to prevent “thinking” models from consuming structured-output token budget.
Reviewed changes
Copilot reviewed 11 out of 12 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| src/run/representability.test.ts | Adds coverage for refusal-as-repair behavior and casualty-free generation context. |
| src/run/pipeline.test.ts | Updates the refusal failure-path test to pin terminal semantics via maxRepairs: 0 and asserts attempt-level representability recording. |
| src/run/orchestrator.ts | Implements representability refusal repair messaging, per-attempt refusal recording, and uses casualty-free contract view for generation-only compilation. |
| src/run/casualty-view.ts | Adds helper to remove profile-declared casualties (and casualty-using examples) from the generation-side contract view. |
| src/audit/report.ts | Extends AttemptRecord with optional representability refusal metadata. |
| src/adapters/ollama.ts | Sends think: false in chat requests to avoid empty structured-output results on thinking models. |
| src/adapters/adapters.test.ts | Asserts Ollama requests include think: false. |
| schemas/audit-report.v1.schema.json | Updates audit-report v1 schema to allow attempts[].representability. |
| package.json | Bumps package version to 0.4.0 and extends @aestheticfunction/dspack-emit range to include ^0.7.0. |
| package-lock.json | Updates lockfile version metadata to 0.4.0 and aligns dependency range. |
| docs/AUDIT.md | Documents new attempts[].representability semantics for repairable refusals. |
| CHANGELOG.md | Adds 0.4.0 release notes covering representability repairs, casualty-free view, and Ollama think: false. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 2 of the ratified P1+P2 milestone (Gateway MCP corpus investigation follow-up).
Refusals become repairable
An
EmitSurfaceError(declared casualty, donation boundary, Collect join violation — contract-legal surfaces the profile cannot represent) previously exited terminalfailed-gateon the FIRST attempt: the repair loop never saw it, though the refusal text is a precise repair instruction. Measured on the 24-cell Gateway corpus: 8 of 12 shadcn cells died this way after burning all model turns on S-repairs. Now the refusal rides the same bounded repair loop — verbatim refusal + one class-targeted hint becomes the repair turn; exhausted budget keeps the exact terminal semantics. New additiveattempts[].representabilityon the audit report (run layer). The post-emit validations branch stays terminal (emit ≥0.7 self-validates its output; branch kept as the guard for older emitters). S1–S3 untouched.Casualty-free generation view
casualtyFreeView(contract, profile)(run layer, pure): generation compiles from the contract minus the profile'scasualtyComponentsand minus examples whose surfaces use them — system prompt, generation schema, and few-shot stop steering the model into guaranteed refusals (measured: 5 of 24 corpus cells were casualty refusals, and 3 shipped shadcn examples themselves cannot emit). ONLYcompileContextsees the view;lintSurfaceandcontractDigestkeep the original contract — S2 vocabulary and report identity unchanged, asserted by test.Ollama thinking models
think: falseon the chat request: structured-output generation cannot budget reasoning tokens — qwen3.6 burned the wholenum_predicton reasoning and returned "empty model output" through the product path (4/4 reproductions). Empirically harmless on non-thinking models.Evidence
Fail-first:
src/run/representability.test.tscommitted first; 9 failures on pre-change code pasted in the commit body. 149/149 green;src/core/byte-untouched (git diff main -- src/core/empty — no ds-mcp re-pin pressure); CI eval golden byte-identical. Emit range extended to admit ^0.7.0 so consumers pinning the new emitter dedupe to one copy.🤖 Generated with Claude Code