feat: P3a generation quality — composition notes + required join ids (0.5.0) - #58
Merged
Conversation
…red schema
Evidence-tied to the Gateway corpus Phase-2 baseline: every field-donation
failure used the un-prescribed field-label/field-content idiom while the
contract's own composition.notes state the correct nesting verbatim and never
reach the prompt; every join failure omitted or prefix-mismatched ids that the
schema leaves optional while the profile's collects join on self.id.
Failing run on pre-change code:
× composition notes reach the generation system prompt (P3a) > carries the noted component's first two sentences on its vocabulary line 6ms
× composition notes reach the generation system prompt (P3a) > hard ceiling: a single giant sentence is word-truncated near 360 chars with an ellipsis 1ms
× join-participating sub-components require id in the generation schema (P3a) > joinIdComponents derives exactly the self.id-keyed participants 4ms
× join-participating sub-components require id in the generation schema (P3a) > requireJoinIds makes id required on every unroll level for participants, and only them 1ms
× join-participating sub-components require id in the generation schema (P3a) > without a profile the schema is returned unchanged by reference 0ms
× join-participating sub-components require id in the generation schema (P3a) > the pipeline hands adapters the tightened schema when an emit profile is active 7ms
⎯⎯⎯⎯⎯⎯⎯ Failed Tests 6 ⎯⎯⎯⎯⎯⎯⎯
Tests 6 failed | 2 passed (8)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ids required (0.5.0)
Both mechanisms feed the model knowledge the stack already holds:
- core/compiler.ts renders each component's own composition.notes (2-sentence/
360-char cap) on its vocabulary line. The Gateway corpus showed every field
donation failure using an idiom the contract's prose explicitly rules out
('inside a Field: FieldLabel, the control, …') and every tab key mismatch
contradicting 'must match exactly one'.
- run/join-id-view.ts derives the sub-components whose profile collect/join
keys on self.id and requires `id` on their generation-schema branches at
every unroll level. Generation-only, same layering as casualtyFreeView.
Stack alignment: emit dep pinned to published 0.7.0; p05 eval scripts extended
through the repair loop and the pipeline emitter-gate test now pins the 0.7
invariant (missing-required refuses at emission, terminal at zero budget, no
validations array); compiler + eval goldens regenerated deliberately.
157/157 green. src/core changes: compiler.ts only (prompt rendering — ds-mcp
inherits at its next deliberate re-pin).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR targets the ratified P3a milestone by improving generation quality using information already present in the contract/profile: it injects per-component composition semantics into the system prompt and tightens the generation schema to require id for profile join-participating sub-components, while aligning tests/goldens to @aestheticfunction/dspack-emit 0.7.0 behavior.
Changes:
- Extend system prompt vocabulary lines to include capped
composition.notes(two sentences / ~360 chars). - Add a run-layer “join-id” schema view that makes
idrequired for components whose profile joins key onself.id, and wire it into the pipeline compile context. - Rebaseline pipeline/eval expectations and dependencies for dspack-emit 0.7.0 refusal-at-emission semantics.
Reviewed changes
Copilot reviewed 11 out of 12 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/run/pipeline.test.ts | Updates emitter-gate failure expectations to match dspack-emit ≥0.7 refusal-at-emission semantics. |
| src/run/orchestrator.ts | Applies requireJoinIds() to the compiled generation schema when an emit profile is active. |
| src/run/join-id-view.ts | New schema view: derives id-keyed join participants from the profile and requires id in the generation schema for them. |
| src/run/join-id-view.test.ts | New fail-first coverage for join-id participant derivation and schema tightening; validates pipeline passes tightened schema to adapters. |
| src/core/composition-notes.test.ts | New fail-first coverage ensuring composition notes reach the system prompt with caps applied. |
| src/core/compiler.ts | Adds capped composition-notes rendering onto each component vocabulary line. |
| package.json | Bumps package version to 0.5.0 and pins @aestheticfunction/dspack-emit to ^0.7.0. |
| package-lock.json | Locks @aestheticfunction/dspack-emit to 0.7.0 and updates lock metadata for 0.5.0. |
| fixtures/golden/eval/results.fake.json | Updates eval golden outputs for new emitter/refusal behavior and attempt counts. |
| fixtures/golden/context/shadcn.destructive-action.json | Updates golden system prompt to include component composition notes. |
| eval/matrix.fake.json | Adjusts eval matrix entries (adds additional fixture runs for p05 gate-fail). |
| CHANGELOG.md | Adds 0.5.0 entry describing P3a generation-quality changes and emitter 0.7 alignment. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+13
to
+15
| import { buildGenerationSchema } from "../core/generation-schema.js"; | ||
| import { compileContext } from "../core/compiler.js"; | ||
| import type { Contract } from "../core/contract.js"; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The ratified P3a milestone: improve generation quality using knowledge the stack already holds — no contract expansion, no new vocabulary, no governance change, no new intents.
Evidence (Gateway corpus, Phase-2 baseline): every remaining shadcn field-donation failure used
field → [field-label, field-content]— an idiom the contract's owncomposition.notesexplicitly rules out ("inside a Field: FieldLabel, the control, FieldDescription, FieldError") — and every join failure either omitted ids or invented a prefix convention (trigger-x/content-x) that the exact-match join must refuse ("Every TabsTrigger value must match exactly one TabsContent value"). Generation never saw either sentence, andidwas schema-optional.Changes:
core/compiler.ts— each component's vocabulary line carries its owncomposition.notes, capped at two sentences / 360 chars. Production shadcn v3 prompt: +7% (casualty view already strips four noted components); field/tabs/radio-group lines verified to carry their prescriptions verbatim.run/join-id-view.ts— sub-components whose profile collect/join keys onself.id(derived: tabs-trigger, tabs-content, radio-group-item on the production profile) getidREQUIRED in the generation schema at every unroll level. Generation-only, same layering ascasualtyFreeView; S-gates untouched.failed-gateoutcomes instead of pre-0.7 emit-invalid-then-A3); compiler + eval goldens regenerated deliberately.Fail-first: 6 behavioral failures on pre-change code pasted in the test commit (8bb7cab). 157/157 green. Few-shot audit outcome: the worked examples already demonstrate the correct idioms consistently — no example edits needed (the gap was prose + schema, not exemplars).
src/coredelta: compiler.ts only (prompt rendering; ds-mcp inherits at its next deliberate re-pin).🤖 Generated with Claude Code