feat(carve S6): auto-decomposition planning and iteration feedback (dormant) - #657
feat(carve S6): auto-decomposition planning and iteration feedback (dormant)#657isadeks wants to merge 3 commits into
Conversation
3ff2f6a to
b51246e
Compare
08bc626 to
d3c250f
Compare
b51246e to
18bcf34
Compare
Automated review — carve S6 (#657)Reviewed as its own diff only ( Slice standalone-ness: PASSTypecheck: PASS. Zero deletions, so there is no revert risk in this slice — confirmed via Dormancy claim: CONFIRMEDWorth stating explicitly, because the premise in the PR description does not hold:
The dormancy claim is accurate. The PR description's reason for it is not. 1. MAJOR — a private commit sha is the stated justification for bypassing a prompt-injection guardrail
This is not an ordinary shorthand leak. It is the single sentence explaining why a 2. MINOR — private shorthand
|
| input | matches |
|---|---|
Build the login form, plus, a signup page |
0 |
Build the login form, plus,a signup page |
1 |
do a; b; c |
0 |
do a;b;c |
2 |
this and also that |
1 (correct) |
So the natural, correctly-punctuated enumerations score zero while the unnatural no-space variants score. Minor because the consequence is a mis-scored decomposition hint, not a wrong outcome — but the two punctuation alternatives are currently dead weight as written.
4. MINOR — HeartbeatTaskView.latestProgressNote is never populated anywhere in the stack
iteration-heartbeat.ts:79 declares it and planHeartbeat renders a progressNote from it; both the module and construct docs advertise "elapsed + an optional progress note". The only producer of a HeartbeatTaskView in the entire stack is toView() in iteration-heartbeat-sweep.ts, and at #657, #659 and #662 that function never sets the field — nor is there a persisted progress_note-shaped attribute to read. So the progress half of the heartbeat is permanently undefined in production and exercised only by its unit test. Either wire it or drop it before publishing; a documented feature that cannot fire is worse than an absent one.
5. MINOR — stale absolute line reference dangles on this PR's own tree
iteration-heartbeat.ts:69 justifies a load-bearing design claim by citing linear-webhook-processor.ts ~1317. At this slice that file is 463 lines — the cited site does not exist until #662. The one pointer a reader would follow to verify the claim goes nowhere. Line-number references across files are fragile generally; name the function instead.
6. MINOR — DEFAULT_REVISE_MODEL_ID hardcodes a model id, bypassing the repo's own resolver
orchestration-plan-revise-interpret.ts:56 hardcodes us.anthropic.claude-sonnet-4-6, while the base slice already establishes bedrock-models.ts (DEFAULT_BEDROCK_MODEL_IDS + resolveBedrockModelIds) as the single source of truth — its own doc says both grant sites derive from that list "so the two backends can never drift". This is a hand-synced third copy of exactly the value that resolver exists to centralise. Given #654 already contains a live model/grant drift bug in this stack, this is worth fixing on principle.
7. MINOR — new IterationHeartbeat construct ships with no construct test
The slice adds 11 test files, none under cdk/test/constructs/. IterationHeartbeat creates a Lambda, an EventBridge schedule, a grantReadData, and two cdk-nag suppressions, and its own comment makes a least-privilege claim ("No write…") that nothing asserts. The repo convention is a synth assertion per construct (26 such files exist, including closely-analogous scheduled Lambdas). This is the one place in the slice where source and test coverage part company.
Reviewed with Claude Code. Verification: tsc --noEmit on this slice's own base; dormancy checked by import-graph and stack-instantiation grep at this tip rather than via the env-var gate; the looksMultiPart regex and the bfc57c5 ancestry claim both checked by execution.
d3c250f to
7af69f3
Compare
18bcf34 to
4581e88
Compare
Rendered user-facing copy — output read, not sourcePer the "render it, don't read it" rule, I compiled Two notes first, because my initial pass got them wrong and rendering is what corrected it:
So the classifier arc is sound. What rendering did surface: 1. The error excerpt runs into the next sentence — every agent-failure replyThere is no terminating punctuation between the excerpt and
This is exactly the class of defect that is invisible in source — the template reads fine as 2. A double space and dangling em-dash when the excerpt is empty but presentReproduced from 3. A classified reason echoes a secret ARN with the account id into a user-visible tracker commentThe 4. Stuttering when the classified title and the excerpt say the same thingCosmetic, but it makes the reply look machine-assembled. When the excerpt is a case-insensitive superset of the title, dropping the excerpt would read better. Verified goodThe panel renderer ( It correctly returns Why the test suite does not catch #1–#4
expect(body).toMatch(/^❌/);
expect(body).toMatch(/build\/tests didn't pass/i);
expect(body).toMatch(/build log in CloudWatch for task `t1`/);
expect(body).toMatch(/reply with guidance/i);Nothing asserts how the pieces join, which is where all four defects live. A single full-string snapshot per representative case would have caught every one of them. Verification: compiled at the s8 tip in a scratch worktree with the stack's own |
7af69f3 to
83344ef
Compare
4581e88 to
8b85d8e
Compare
Review findings addressedThank you for checking dormancy by import graph and stack instantiation rather than taking the PR description's word for it — you are right that the description's stated reason was wrong. MAJOR #1 — a private commit sha justifying a MINOR #2 — MINOR #3 — MINOR #4 — MINOR #5 — stale absolute line reference. Fixed, replaced with the function name. Agreed on the general point; cross-file line numbers were used in several places in this arc and I have removed the ones I found. MINOR #6 — MINOR #7 — |
83344ef to
cc75f8c
Compare
8b85d8e to
32e969a
Compare
cc75f8c to
fedea84
Compare
32e969a to
f5afe7e
Compare
fedea84 to
99f1007
Compare
f5afe7e to
0fb2cc1
Compare
Second-round review: the regex over-corrected, and my guardrail comment overclaimedTwo findings on my own fixes, both confirmed by execution. The conjunction regex now false-positived on pasted code. Dropping the trailing
Each would have nagged the user to decompose a plain bug report. My own previous commit message argued for avoiding exactly that direction of error, and the new test set couldn't catch it — it only asserted positives, and its one negative case was 73 chars, below the 80-char length floor, so it returned false without ever reaching the conjunction count. The bare My guardrail comment asserted a safety property the code did not have — which is worse than the private commit sha it replaced, because a reviewer will trust prose. Three claims failed, all shown by execution:
Rather than soften the prose, the first two are now true: the delimiter is neutralized in reviewer text and both the instruction and the returned fields are bounded. The comment now names the real backstop — the downstream guardrail screen at task creation, which the reviewer correctly identified as what actually catches content — and says plainly that the diff is a reviewer-facing summary, not a security control. All three new guards are mutation-verified. Verified and not changed: |
99f1007 to
3c02a74
Compare
0fb2cc1 to
b1410ea
Compare
scottschreckengaust
left a comment
There was a problem hiding this comment.
Verdict: Approve (with non-blocking notes)
Reviewing as a Principal AWS Solutions Architect. This is Slice 6 of the linear-vercel carve, stacked on #656 (carve/s5-dag-core-channel) — I reviewed only the incremental diff vs that base, not code introduced by earlier slices. Every file is net-new and provably dormant: I verified there are no production (non-test) importers of any new module, IterationHeartbeat is instantiated in no stack, and iteration-heartbeat-sweep.ts is referenced by no stack. Synth + unit tests only, no deployed-resource or behaviour change. Given the strong test coverage, the exceptional inline documentation of the design rationale, and the safe-by-construction dormancy, this is a clear approve.
Vision alignment
Strongly aligned with the VISION tenets. Fire-and-forget, escalate by policy: :decompose gates spend behind an explicit @bgagent approve (spend-safe default) while :auto opts out — the precedence rule in parseDecompositionMode deterministically makes the spend-safe choice win on ambiguous label sets. Bounded blast radius & cost: per-project caps (applyPlanCaps) reject-never-trim over-cap plans, budget is derived from S/M/L sizing rather than model-asserted (a stable, explainable ceiling), and the sweep is capped at MAX_EDITS_PER_SWEEP. Reviewable outcomes: the proposal comment, the computed before→after diff, and the idempotent Linear write-back keep the graph inspectable and human-editable. No tenet is traded; the work lands part of #299/#247 (both approved P0) and is tracked by epic #668.
Governance
PASS. Backing issues #299 and #247 both carry approved + P0. The carve/s6-decompose-iteration branch name does not follow (feat|fix|...)/<issue>-..., but per the review contract a carve/* branch is a de-facto-waived nit for the staged carve, not a blocker.
Blocking issues
None.
Non-blocking suggestions / nits
-
Latent wiring gap for single-kind approval via
runPlanVerdict(cdk/src/handlers/shared/orchestration-decomposition-flow.ts:106,353). TheconsumePendingPlaneffect is typed as returning only{ nodes }, discarding the row'spending_kindandsingle_task_description. The approve branch then unconditionally callsfinalizeWriteBack(..., { nodes: taken.nodes }). For apending_kind:'single'pending plan (nodes is[]), that reacheswriteBackPlanwhich returns{kind:'error', message:'No sub-issues to create.'}— the single-task approval would never run the coding task. This is inert today (dormant, and the reconciler wiring lands in a later slice), so it is not blocking, but the S8 wiring slice must route a single-kind approval down a distinct path (run the persistedsingle_task_description) rather than throughfinalizeWriteBack. Consider widening the effect's return type now so the gap can't be missed at wire-up time. -
looksMultiPartconjunction heuristic (cdk/src/handlers/shared/orchestration-decomposition-mode.ts:213). The regex has been iterated twice in this PR's own commits and is now conservative (word alternatives + boundarylessplus,). It is a pure hint that never changes what runs, so correctness risk is nil; I only flag that a heuristic this fiddly may be worth a small table-driven test matrix of realistic issue bodies (code pastes, stack traces, prose lists) to lock the false-positive boundary — the current tests cover the fixed cases but this class of regex tends to regress. -
extractJsonObjectdivergence between the two parsers. The planner's extractor (orchestration-decomposition-planner.ts:221) scans all top-level objects and prefers the last plan-shaped one (robust against prose that quotes CSS/JSON); the interpret extractor (orchestration-plan-revise-interpret.ts:348) takes the first balanced object. The interpret prompt demands "ONE JSON object, no prose", so first-match is acceptable, but the two "same idea, slightly different" implementations are a mild coherence smell — a shared helper would prevent them drifting.
Documentation
Acceptable for a dormant slice. No docs/guides/ or docs/design/ prose changed and no Starlight mirror is touched, so the "Fail build on mutation" step is not at risk. The user-facing label grammar (:decompose/:auto/:help), the caps, and the new env vars will need a guide update when the wiring slice activates them — I'd expect that documentation to land with S8 (activation), not here, and #668 already carries the slice/review guidance. No missing ADR: the design decisions (derived budget, reject-never-trim, deterministic diff, claim protocol) are extensively documented inline and none constitutes a new architectural tenet trade requiring an ADR.
Tests & CI
Strong. Every source module has a matching */test/ file; the PR adds ~4,900 lines of tests against ~4,600 lines of source. Failure paths are covered, not just happy paths (DAG-invalid plans, over-cap rejection, redelivery no-ops, race-losing consume, partial write-back resume, throttle/backoff, claim exhaustion, build-gate vs agent-crash classification). The new IterationHeartbeat construct test asserts the least-privilege claim its own comment makes (read-only on TaskTable; explicitly not PutItem/UpdateItem/DeleteItem/BatchWriteItem) and the schedule/rate — good discipline. CI is green (build (agentcore) pass, secrets/deps/workflow scan pass, dead-code advisory pass). Test-performance guidance (#366): the construct test uses per-test synth() helpers but the suite is tiny (1 construct) and bundling is globally disabled, so no concern.
Bootstrap synth-coverage: not applicable. The diff introduces no new CloudFormation resource types — the heartbeat construct uses Lambda + AWS::Events::Rule (already in resource-action-map.ts:77 → events:PutRule) + a DynamoDB read grant, and the construct is not instantiated in any deployed stack. No cdk/src/bootstrap/* change is required for this PR; the bootstrap bundle must be revisited in the slice that actually instantiates the construct.
Review agents run
The pr-review-toolkit agent plugins were not available as invocable subagents in this environment, so I applied their review lenses directly over the diff and state which lens covered what:
- code-reviewer (guidelines/style) — applied: MIT headers present, L2 constructs, cdk-nag suppressions justified, Node 24 runtime matches the repo norm (44/46 constructs), pure/effect separation is consistent.
- silent-failure-hunter (error handling) — applied: this is the highest-risk surface and it is handled well.
writeBackPlan/linearGraphqlFnnever throw and surface resumable errors;interpretRevisereturns{kind:'error'}on model/parse failure (caller falls back, no silent no-op); the sweep is best-effort and never wedges;releaseReplyClaimbounds retries to prevent the observed 900-iteration spin. No swallowed failures that lose user intent found. - type-design-analyzer (new types) — applied: discriminated unions (
DecompositionResult,PlanCapResult,PlanEdit,ReviseInterpretation,DecompositionFlowResult) are well-modelled and make illegal states unrepresentable; the one nit is the over-narrowconsumePendingPlanreturn (item 1). - comment-analyzer (comment accuracy) — applied: comments are unusually load-bearing and accurate — the PR's own follow-up commits fixed two over-claiming comments (the guardrail-exception note and a dead regex). The
latestProgressNotefield is honestly documented as RESERVED/no-producer rather than over-sold. - pr-test-analyzer (coverage gaps) — applied: see Tests section; coverage is thorough. Only gap is the multi-part heuristic boundary (item 2).
- /security-review — applied (prompt-injection + IAM + secrets surface): the
interpretprompt is the sensitive path. It sanitizes the"""delimiter in reviewer text, bounds instruction + returned-field length, validates the model output against a closed edit vocabulary before applying, and correctly names the real backstop (the downstreamcreateTaskCoreguardrail screen at task creation) rather than claiming the diff is a security control. IAM is least-privilege (read-only TaskTable, per-workspace secret grant deferred to the stack). No secrets logged. Fail-closed posture throughout.
Human heuristics
- Proportionality — Pass. The module split (mode / planner / caps / render / store / writeback / flow / revise / revise-interpret / plan-commands) maps to genuinely distinct responsibilities with injected I/O, not speculative abstraction; each is independently unit-testable.
orchestration-decomposition-render.tsis the one large file (674 lines) but it is a flat catalog of ~30 small render functions, not accreted complexity. - Coherence — Pass (minor). Same concepts use the same terms across modules (
PlannedSubIssue,depends_onindices,SIZE_DEFAULT_BUDGET_USD). The twoextractJsonObjectvariants (item 3) are the only mild parallel-but-not-shared smell. - Clarity — Pass. Names communicate intent; error handling surfaces failures rather than hiding them behind plausible defaults (the
renderPlannerErrorNotepath deliberately refuses to falsely claim "single cohesive change"). Magic values are named constants with rationale. - Appropriateness — Pass. The Linear GraphQL integration encodes real API behaviour verified against
linear-feedback.ts(e.g. theIssueRelationTypeenum-vs-String note at writeback.ts:138 documents a real 400-causing footgun), not just self-written mocks. Tests assert what the code should do (idempotent reuse-by-title, DAG validity, spend-safe precedence), not merely what it does.
🤖 Reviewed by Claude (Principal AWS Solutions Architect persona) via the ABCA review_pr workflow.
3c02a74 to
e777e0d
Compare
b1410ea to
d10be52
Compare
…ormant)
Turns "one plain issue" into a reviewable sub-issue graph, and makes a long
iteration legible while it runs. Like S5, none of this is wired into a stack
yet — no handler calls it and no construct is instantiated, so this slice is
unit-test-only and changes no deployed behaviour.
Auto-decomposition — plan, review, revise, then run:
- decomposition-mode: pure label parsing. The plain trigger label runs an issue
as one task (or runs its existing sub-issue graph); `…:decompose` plans first
and waits for approval; `…:auto` plans and starts immediately. A decompose
suffix on an issue that already has sub-issues is a no-op.
- decomposition-planner: parse and validate the plan a `coding/decompose-v1`
agent emits as an artifact. Budget is DERIVED from each piece's S/M/L size
rather than asked of the model, so the Σ is a stable, explainable ceiling.
Edges are indices into the plan's own list, validated as a DAG before anything
is created.
- decomposition-caps: per-project limits. Over-cap REJECTS with a message and
never trims — trimming can silently drop a node others depend on.
- decomposition-render: the proposal comment, and every note around it. Plain
English throughout: no "critical path" or "cost ceiling" jargon, the spend
figure is framed as the safety limit it is, and each decline says what actually
happened rather than a plausible-sounding stand-in.
- decomposition-store: the pending plan survives between the propose webhook and
the approve one. Create-once so a redelivery is a no-op; consume is a
conditional delete-and-return so two racing approvals can't both write back.
- decomposition-writeback: create the real sub-issues and their `blockedBy`
relations. Idempotent and resumable — a planned node whose title already
exists is reused, so a retry after a partial write-back doesn't double-create.
- decomposition-flow: ties the above into caps → propose-or-seed, plus the
approve/reject verdict path. All I/O injected, so the control flow is testable
without Linear or DynamoDB.
- plan-commands / plan-revise / plan-revise-interpret: a reviewer can edit the
plan directly ("drop 3", "merge 1 and 2") or in prose. Edits apply
deterministically to the CURRENT plan and the "What changed" line is a COMPUTED
before→after diff — never a model self-report, which used to let a dropped
piece quietly reappear with a fabricated justification.
Iteration feedback:
- iteration-heartbeat (+ construct and scheduled sweep): a long iteration used to
show "starting on this" and then nothing until it finished. The sweep edits the
SAME maturing reply in place to show elapsed time and the latest progress note
— no new comments. Eligibility keys on the reply-routing fields, so standalone
iterations are covered too, not just orchestrated ones.
- iteration-reply-claim: claim a reply so two writers converge on one comment.
- failure-reply: failure is answerable. A red build points at the build log by
task id (the agent runs the build itself, and the target repo may have no CI);
an agent crash gives the classified reason, a truncated excerpt, and whether to
retry or escalate. Never the raw build output — that's untrusted repo code.
- clarify-resume: resume a task that stopped to ask a question before spending.
… a dead regex **A private commit sha was the whole justification for skipping a PROMPT_ATTACK screen.** That is the one sentence a security reviewer needs to be able to follow, and it pointed at a commit no public reader can resolve. Replaced with the actual structural argument: the reviewer's instruction is embedded as delimited data inside a prompt whose only job is to classify it into a fixed edit vocabulary, the output is validated against that closed set before anything is applied, so a jailbreak cannot widen what the caller does — and a note not to reuse this shape where the output is executed rather than matched. **The multi-part conjunction regex could not match its own intended phrasings.** A trailing `\b` after the `plus,` and `;` alternatives requires a word character to follow, which inverts the intent: "the form, plus, a signup page" scored zero while "plus,a signup page" scored one, and "do a; b; c" scored zero while "do a;b;c" scored two. So the correctly-punctuated prose this is meant to catch was the case it missed, and both punctuation alternatives were dead weight. Word alternatives keep their boundaries; the punctuation ones no longer require one. **`latestProgressNote` was documented as a shipped capability but has no producer** anywhere in the stack — the sweep never sets it and there is no persisted attribute to read, so the heartbeat shows elapsed time only. The render path is written and tested, so rather than delete it or half-wire it, the field and the module/construct docs now say plainly that it is reserved and what wiring it needs. **`IterationHeartbeat` shipped without a construct test**, the one place in this slice where source and test coverage parted company. Added, following the repo's per-construct synth-assertion convention — including an assertion for the least-privilege claim the construct's own comment makes but nothing checked. Also: a cross-file line-number reference that pointed at a line which does not exist until a later slice (named the function instead), and the last of the private work-item shorthand.
…t overclaimed Two findings from a second review pass on my own fixes. **The conjunction regex now false-positived on pasted code.** Removing the trailing `\b` fixed the punctuation alternatives, but a bare `;` with no boundary matches EVERY semicolon — and an issue description written for a coding agent routinely contains a snippet, a stack trace or CSS. Four realistic single-task bug reports flipped to "multi-part" and would have nagged the user to decompose them. That is a false positive in the direction that costs attention, and my previous commit message argued for avoiding exactly that. The bare `;` is dropped. It never earned its place in either form: with the boundary it only matched unnatural no-space input, and without one it matched everything. `plus,` keeps the boundaryless form and the word alternatives keep theirs, which is enough for both real multi-part prose shapes; a numbered list is caught by the separate list-item path. A test now pins that a code paste stays single-task, which is what the previous test set never asserted. **The guardrail comment asserted a safety property the code did not have** — which is worse than the private commit sha it replaced, because a reviewer will trust prose. Three claims failed: the instruction was interpolated raw, so reviewer text containing the `"""` fence could close the data block and continue at prompt level; the free-text fields were type-checked but unbounded; and the computed diff is title-keyed, so an edit reusing an existing title reports as "modified" rather than "added" and cannot be relied on to surface a malicious edit. Rather than soften the prose, the first two are now true: the delimiter is neutralized in reviewer text and the instruction and returned fields are bounded. The comment names the real backstop — the downstream guardrail screen at task creation — and says plainly that the diff is a reviewer-facing summary, not a security control. All three new guards are mutation-verified.
e777e0d to
a112305
Compare
d10be52 to
9a4a043
Compare
Slice 6 of the staged carve. Stacked on #656 (
carve/s5-dag-core-channel) — review that first; this PR's diff is only its own commit.What this adds
Two capabilities, both inert until a later slice wires them up.
Auto-decomposition — plan, review, revise, then run
Turns one plain issue into a reviewable sub-issue graph instead of one giant task.
decomposition-mode— pure label parsing. The plain trigger label runs an issue as one task (or runs its existing sub-issue graph);…:decomposeplans first and waits for approval;…:autoplans and starts immediately. A decompose suffix on an issue that already has sub-issues is a no-op — you can't decompose what's already a graph.decomposition-planner— parse and validate the plan acoding/decompose-v1agent emits as an artifact. Two design choices worth calling out: budget is derived from each piece's S/M/L size rather than asked of the model (so the Σ is a stable, explainable ceiling), and dependency edges are indices into the plan's own list, validated as a DAG before anything is created.decomposition-caps— per-project limits. Over-cap rejects with a message and never trims: trimming a graph can silently drop a node others depend on, producing a broken partial result.decomposition-render— the proposal comment and every note around it. Deliberately plain English: no "critical path" or "cost ceiling" jargon, the spend figure is framed as the safety limit it is (not a forecast that anchors the reviewer at the ceiling), and each decline path says what actually happened rather than a plausible-sounding stand-in.decomposition-store— the pending plan has to survive between the propose webhook and the approve one, so it's one TTL'd row.putis create-once (a redelivery is a no-op);consumeis a conditional delete-and-return, so two racing approvals can't both write back.decomposition-writeback— create the real sub-issues and theirblockedByrelations. Idempotent and resumable: a planned node whose title already exists is reused, so a retry after a partial write-back doesn't double-create.decomposition-flow— ties the above into caps → propose-or-seed, plus the approve/reject verdict path. All I/O is injected, so the control flow is testable without Linear or DynamoDB.plan-commands/plan-revise/plan-revise-interpret— a reviewer can edit the plan directly (`drop 3`, `merge 1 and 2`) or in prose. Edits apply deterministically to the current plan, and the "What changed" line is a computed before→after diff — never a model self-report, which previously let a dropped piece quietly reappear with a fabricated justification for it.Iteration feedback
iteration-heartbeat(+ construct and scheduled sweep) — a long iteration used to show "starting on this" and then nothing at all until it finished. The sweep edits the same maturing reply in place with elapsed time and the latest progress note; it never posts new comments. Eligibility keys on the reply-routing fields, so standalone iterations are covered too, not just orchestrated ones.iteration-reply-claim— claim a reply so two concurrent writers converge on one comment instead of overwriting each other.failure-reply— failure is answerable. A red build points at the build log by task id (the agent runs the build itself, and the target repo may have no CI at all); an agent crash gives the classified reason, a truncated excerpt, and whether to retry or escalate. Never the raw build output — that's untrusted repo code.clarify-resume— resume a task that stopped to ask a question before spending.Why this is safe to merge early
Every file here is net-new. No handler calls into any of it, the heartbeat construct is not instantiated in any stack, and no schedule is created — the wiring lands in a later slice behind an env-var gate. Synth and unit tests only; no deployed resource or behaviour changes.
Verification
tsc --noEmitcleaneslintclean🤖 Generated with Claude Code
Tracking
Slice S6 of the linear-vercel → main carve. Tracking issue: #668 (slice table, why it is sliced rather than merged, and review guidance).
Lands part of #247 (parent/sub-issue orchestration) and #299 (auto-decomposition). Deliberately not
Closes— no single slice closes either; they close when S8 activates the arc.Stacked on #656 — review that first; this PR's diff is against its branch.