feat(carve S5): orchestration DAG core, row store, and the channel abstraction (dormant) - #656
feat(carve S5): orchestration DAG core, row store, and the channel abstraction (dormant)#656isadeks wants to merge 3 commits into
Conversation
30aa161 to
0df8c2c
Compare
08bc626 to
d3c250f
Compare
0df8c2c to
c58d51b
Compare
d3c250f to
7af69f3
Compare
Automated review — carve S5 (#656)Reviewed as its own diff only ( Slice standalone-ness: PASS
All 29 deletions checked — no Nothing is introduced-but-unused. I checked all 16 new source modules for a consumer in Dormancy claim: CONFIRMED — but not for the stated reasonThe PR description's premise does not hold:
One thing worth flagging, because it is the only place the dormancy argument is non-obvious: this slice modifies 1. MAJOR — this slice turns off four lint rules for all 152 test files, to accommodate 13 new ones
'@typescript-eslint/no-magic-numbers': 'off',
+ '@typescript-eslint/no-shadow': 'off',
+ 'no-shadow': 'off',
+ '@stylistic/max-len': 'off',
+ 'max-len': 'off',The stated reason is this slice's own tests: "long fixture/assertion lines, and reuse small helper names ( But the override is repo-wide. At this slice there are 152 test files, 139 of which pre-existed this PR. So This is also precisely the "source file and the lint config that permits it, split across a boundary" case — except here they are in the same slice (good) with a blast radius far wider than the code that needed it (not good). Prefer scoping the override to the new files' paths, or fixing the ~13 files, or at minimum keeping 2. MINOR —
|
c58d51b to
efd1765
Compare
7af69f3 to
83344ef
Compare
efd1765 to
430576f
Compare
83344ef to
cc75f8c
Compare
430576f to
227058d
Compare
cc75f8c to
fedea84
Compare
Review findings addressedI had missed this review initially — apologies. MAJOR #1 — four lint rules switched off across 152 test files to land 13. Fixed, taking your minimum ask. This was my change and your blast-radius point is right: So
Any of those three could have had a test asserting against the wrong fixture and still passing. Worth noting the rule only surfaced the third one once the relaxation came off at the tip of the stack, which is itself an argument for your position. MINOR #2 — MINOR #3 — raw internal UUID on the label fallback path. Confirmed the same Gates: 2920 CDK tests, |
2a8e7b2 to
dadf00f
Compare
fedea84 to
99f1007
Compare
Second-round review: the shadow renames hold upA second adversarial pass attacked this fix specifically. Nothing broke, which is worth recording as explicitly as a defect would be. The regex-driven renames are clean. The whole delta is the three intended call sites plus the definition and the deleted duplicate helper — no property access, no string literal, and no The renamed tests still assert the real thing. The reviewer mutation-tested them three ways — removing the legacy-name fallback, breaking
Worth noting the rule earned its keep immediately: re-enabling it surfaced a third shadow in #659 (a local The two MINOR findings stand as noted, unchanged: |
dadf00f to
1a3f6bf
Compare
99f1007 to
3c02a74
Compare
scottschreckengaust
left a comment
There was a problem hiding this comment.
Review — carve S5: orchestration DAG core, row store, channel abstraction (dormant)
Reviewed as a Principal AWS Solutions Architect, as this PR's own incremental diff only (carve/s4-linear-surface..carve/s5-dag-core-channel, 35 files, +8748/−29). This is a stacked PR; its base (#655) is reviewed separately and its code is out of scope here.
1. Verdict — Approve (with two non-blocking comment-accuracy nits)
This is exceptionally well-engineered, purely-additive foundation code: pure graph logic (Kahn's-algorithm DAG validation + layering), a DynamoDB row store, and a surface-agnostic Channel abstraction with Linear/Jira/Slack adapters. Every module is fail-closed where it matters, best-effort where feedback is advisory, and carries a rationale comment for each non-obvious decision. It is genuinely dormant and deploy-safe. The two findings are stale doc comments on dormant code — not worth blocking a merge.
2. Vision alignment
Strongly aligned. Bounded blast radius: fetchSubIssueGraph fails loud on an over-100 epic rather than silently truncating children (and dropping their dependency edges → out-of-order release) — the correct choice for the orchestration tenet. Fire-and-forget preserved: the comment-trigger parser has a robust self-loop guard (bot-authored-prefix detection) that prevents the agent talking to itself, and a deliberately-narrow near-miss allowlist so a mistyped handle nudges rather than silently drops. Reviewable outcomes: orchestration-log-events.ts centralizes greppable event names as an explicit test contract for an event-driven plane with no synchronous API. Backing issues #247 and #299 both carry approved + P0; tracking epic #668 documents the slicing.
3. Blocking issues
None.
4. Non-blocking suggestions / nits
orchestration-integration-node.ts:79— thewithIntegrationNodedoc says the synthetic id is`<orchestrationId>#integration`, but the code (line 53/92) produces`<orchestrationId>__integration`viaINTEGRATION_NODE_SUFFIX. Worse, the#form is the exact idempotency-key-invalid shape that lines 46-52 explicitly warn against (createTaskCorevalidates${orch}_${sub}against/^[a-zA-Z0-9_-]{1,128}$/, and a#would 400 the child). The code is correct; the comment misdirects toward a broken value. Fix the comment to__integration.orchestration-store.ts:836—loadOrchestration's doc says it "mirrorsfindOrchestrationIds's Scan pagination," but nofindOrchestrationIdsexists at this slice (onlyfindOrchestrationChildByBranch). It's a forward-reference to a later slice; a reader who greps for it finds nothing. Either drop the reference or note it lands later.- (pre-noted by author, agree)
extractLinearIdentifierFromBranch(linear-issue-lookup.ts) ships three slices before its only consumer (#662). Clean, but its motivating rationale references behaviour not yet present. Boundary-placement judgement; fine to leave. - (pre-noted by author, agree) the
display_id ?? sub_issue_idlabel fallback can surface a raw internal UUID to a user. On dormant panel code here; a friendlier fallback would arguably mask a data defect, so recording rather than fixing is defensible.
5. Documentation
No user-facing docs, ADRs, or Starlight mirror needed — this slice deploys no resource and changes no runtime behaviour or contract. The changes to live files (linear-issue-lookup.ts, linear-feedback.ts, slack-api.ts) are additive exports plus a docs-only ABCA- → ENG- example rename (the right direction for a public repo). No doc drift.
6. Tests & CI
Strong. 13 new test files, table-driven, covering failure paths (cycles, dangling edges, duplicate ids, truncation-fail-loud, idempotent replay, tenant-collision refusal, diamond/linear/root base selection, self-trigger loop guard, reaction-replace incompleteness). CI build (agentcore) — the full CDK suite + eslint — passed; author reports 2917 tests / 150 suites. no-shadow is correctly kept enforced in tests (the final commit reverted the earlier over-broad relaxation and fixed the 3 real shadows it surfaced).
Bootstrap synth-coverage: N/A. OrchestrationTable is a DynamoDB table (AWS::DynamoDB::Table already in resource-action-map.ts) and is not instantiated in any stack at this slice (verified: no new OrchestrationTable( in cdk/src). No new CFN resource type, no IAM grant takes effect, so no bootstrap-policy / BOOTSTRAP_VERSION bump is required in this PR. Correct to defer to the wiring slice (#662).
7. Review agents run
- code-reviewer /
/code-review(high effort,HEAD~3..HEAD) — RAN. Surfaced the two comment-accuracy nits above; no correctness or style defects. - comment-analyzer — RAN (folded into above); the two findings are its output.
- silent-failure-hunter — RAN (in scope: heavy error handling). The channel adapters and store are best-effort-with-structured-logging by design (feedback must never gate orchestration);
fetchSubIssueGraphandlinearGraphSourcedeliberately fail loud rather than degrade. No silent swallowing found. - type-design-analyzer — RAN (in scope: many new types).
DagValidationResult,FetchSubIssueGraphResult,OrchestrationGraphResult, andChannelare well-formed discriminated unions with fail-closed defaults;ChannelKind/registry are deliberately open to satisfy the extensibility tenet. No concerns. - pr-test-analyzer — RAN. Coverage is comprehensive and asserts failure paths, not just happy paths; CI full suite green.
- /security-review — RAN (in scope: DynamoDB construct, GraphQL Bearer auth, comment input-gateway). No deployed IAM/Cedar/network change (construct un-instantiated); Bearer-token handling and 8s-timeout+AbortController mirror the proven
linear-feedbackpattern; the comment-trigger self-loop guard and near-miss allowlist are defensive. Orchestration id derivation is documented as non-tenant-scoped but safe for workspace-unique UUIDs, with an explicitOrchestrationIdCollisionErrorfail-closed at seed for non-unique refs. No secrets exposure. - Omitted: none in scope.
8. Human heuristics
- Proportionality — PASS. The registry-over-switch and three-tier graph-source seam are justified by the extensibility tenet, not speculative; the Slack adapter exists specifically to prove capability-gating against a non-tracker surface.
orchestration-store.tsis large (954 lines) but cohesive (one table's surface) rather than accreted. - Coherence — PASS. Consistent vocabulary (
orchestration_id/sub_issue_id/depends_on) across construct, store, DAG, and channel. The three adapters are parallel with real substance, not copy-paste boilerplate (Jira genuinely omits optional capabilities; Slack genuinely maps thread→issue). - Clarity — CONCERN (minor). Two stale doc comments misdescribe real values (
integration-node.ts:79,store.ts:836); see nits. Naming and error surfacing are otherwise excellent. - Appropriateness — PASS. Integration code is written against the real Linear GraphQL shape and the documented DynamoDB
BatchWriteItem25-item / 1MB-page limits (paginated Query, chunked BatchWrite, meta-row-last partial-failure ordering, TTL-aliased reserved keyword). Tests assert intended behaviour (mutation-tested by the author's second pass), not merely current behaviour.
Reviewed with Claude Code as scottschreckengaust. Dormancy independently verified via import-graph + new OrchestrationTable( grep across cdk/src; the only caller of the new linear-feedback exports is the new dormant orchestration-channel-slack.ts; linear-issue-context-probe.ts references fetchRecentComments only in a doc comment, not a call.
| * all leaves) when there is more than one leaf. | ||
| * | ||
| * ``orchestrationId`` namespaces the synthetic node's id so it is unique + | ||
| * recognizable (``<orchestrationId>#integration``). The node carries no |
There was a problem hiding this comment.
Nit (comment accuracy): this says the synthetic id is `<orchestrationId>#integration`, but the code builds `<orchestrationId>__integration` (line 53/92 via INTEGRATION_NODE_SUFFIX). The # form is precisely the idempotency-key-invalid shape lines 46-52 warn against — createTaskCore validates ${orch}_${sub} against /^[a-zA-Z0-9_-]{1,128}$/ and a # would 400 the child. Please update the comment to __integration so it doesn't misdirect.
| * (many children + accumulated ``ack#`` marker rows) would otherwise silently | ||
| * truncate — dropping children from the completion check / panel and stranding | ||
| * the epic. Follows ``LastEvaluatedKey`` to read all rows (mirrors | ||
| * ``findOrchestrationIds``'s Scan pagination). |
There was a problem hiding this comment.
Nit (comment accuracy): references findOrchestrationIds's Scan pagination as a sibling to mirror, but no findOrchestrationIds exists at this slice (only findOrchestrationChildByBranch). It's a forward-reference to a later slice — a reader who greps for it finds nothing. Consider dropping the reference or noting it lands later.
Test code already has an exemption from the magic-numbers rule, on the grounds that inline literals ARE the fixture. The same reasoning covers two more rules that only produce noise in tests: - no-shadow: sibling describe blocks reuse small helper names (`row`, `makeDdb`, `child`) for their own local fixture. Renaming each to a unique name makes the tests harder to read, not safer — nothing closes over the outer binding. - max-len: an assertion or a fixture URL is one expression; wrapping it mid-string to satisfy a column limit hurts readability. Correctness rules stay on everywhere. Scoped to `test/**` only, so production code is unaffected.
…straction
The dormant foundation for parent/sub-issue orchestration: pure graph logic,
the DynamoDB row store, and a surface-agnostic feedback layer. Nothing here is
instantiated in a stack yet — the executor and the stack wiring land later, so
this slice is synth-and-unit-test only and changes no deployed behaviour.
Graph core (pure, no I/O):
- orchestration-dag: validate a `depends_on` graph (duplicate id, dangling
edge, cycle) and return its topological layering via Kahn's algorithm. The
layers are what a reconciler releases children from, in dependency order.
- orchestration-graph-source: the seam between "where the graph came from" and
the executor. Three tiers — a tracker that has native sub-issues (read it), a
caller that supplies the DAG declaratively (CLI/API, or a planner), or a
structureless trigger (single task).
- orchestration-integration-node: when a graph fans out to several leaves,
append a synthetic node depending on all of them so there is one combined
artifact instead of N unrelated PRs.
- orchestration-epic-tip: where a newly-added, dependency-less node stacks — the
leaf frontier, so it inherits the epic's accumulated unmerged work rather than
branching off the default branch.
- orchestration-base-branch: pick a child's base branch from its predecessors.
Row store and table:
- OrchestrationTable construct: orchestration_id (PK) + sub_issue_id (SK), with
sparse GSIs to resolve a child back from its task id or its head branch.
- orchestration-store: the read/write surface over those rows, including the
idempotency markers that keep duplicate webhook deliveries from double-acting.
- orchestration-log-events: structured events for a run, plus the scheduled
sweep's backstop.
- linear-subissue-fetch: read a parent's children and their `blocks` relations
into a DAG. Fails loud rather than silently truncating an over-size epic.
Channel abstraction:
- orchestration-channel: the surface-agnostic interface — required operations
every channel must implement, optional ones gated by declared capability, so
the engine never branches on which tracker it is talking to.
- orchestration-channel-{linear,jira,slack} + factory: per-surface adapters
selected from the stored row.
- orchestration-comment-trigger: parse a mention comment into a command,
ignoring the bot's own comments.
- iteration-reply: one maturing threaded reply per iteration rather than a
stream of new comments; the two async writers of a reply converge instead of
overwriting each other.
…it repo-wide The test-scope lint relaxation switched off four rules across all 152 test files to accommodate 13 new ones. `max-len` and `no-magic-numbers` are fair there — literal fixtures and long expected strings are the point of a test. `no-shadow` is not: it is correctness-adjacent in test code, where a shadowed `row` or `mock` inside a nested describe is a common way to assert against the wrong fixture and still pass. So `no-shadow` stays on, and the two shadows it found are fixed rather than silenced. One was a redundant local `makeDdb` identical to the file-level helper (deleted). The other was a local `child` that builds a persisted ROW shadowing a file-level `child` that builds a graph node — genuinely two different shapes under one name, which is exactly the hazard. Renamed to say which it is.
1a3f6bf to
0a2610a
Compare
3c02a74 to
e777e0d
Compare
Slice 5 of the staged carve. Stacked on #655 (
carve/s4-linear-surface) — review that first; this PR's diff is only its own commits.What this adds
The dormant foundation for parent/sub-issue orchestration. Three groups, all inert:
Pure graph logic — no I/O, so it is unit-testable in isolation.
orchestration-dag— validate adepends_ongraph (duplicate id, dangling edge, cycle) and return its topological layering (Kahn's algorithm). A reconciler releases children layer by layer.orchestration-graph-source— the seam between where a graph came from and the executor that runs it. Three tiers: a tracker with native sub-issues (read them), a caller that supplies the DAG (CLI/API, or a planner), or a structureless trigger (single task).orchestration-integration-node— when a graph fans out to several leaves, append a synthetic node depending on all of them, so there's one combined artifact instead of N unrelated PRs. Reuses the existing multi-predecessor merge path; no new merge code.orchestration-epic-tip— where a newly-added, dependency-less node stacks: the leaf frontier, so it inherits the epic's accumulated unmerged work instead of branching off the default branch.orchestration-base-branch— pick a child's base branch from its predecessors.Row store and table
OrchestrationTableconstruct —orchestration_id(PK) +sub_issue_id(SK), with sparse GSIs to resolve a child back from its task id or its head branch.orchestration-store— the read/write surface over those rows, including the idempotency markers that stop a duplicate webhook delivery from double-acting.orchestration-log-events— structured per-run events, plus the scheduled sweep's backstop.linear-subissue-fetch— read a parent's children and their `blocks` relations into a DAG. Fails loud rather than silently truncating an over-size epic (dropping children also drops their dependency edges, so survivors would release out of order).Channel abstraction — the engine never branches on which tracker it's talking to.
orchestration-channel— the surface-agnostic interface: operations every channel must implement, plus optional ones gated by declared capability.orchestration-channel-{linear,jira,slack}+ factory — per-surface adapters, selected from the stored row.orchestration-comment-trigger— parse a mention comment into a command; ignore the bot's own comments.iteration-reply— one maturing threaded reply per iteration instead of a stream of new comments; the two async writers of a reply converge rather than overwrite each other.Why this is safe to merge early
OrchestrationTableis not instantiated in any stack in this slice, and no handler calls into the store or the executor seam — the wiring lands in a later slice behind an env-var gate. So this is synth-and-unit-test only: it changes no deployed resource and no runtime behaviour.Also here
One separate commit relaxes two stylistic ESLint rules (
no-magic-numbers,no-shadow,max-len) insidetest/**only. Table-driven tests with literal fixtures fight those rules constantly, and a lint-policy change shouldn't hide inside a feature commit — hence its own commit.Verification
tsc --noEmitcleaneslintclean oncdkandcliruff check/ruff format --check/ty checkall cleanTracking
Slice S5 of the linear-vercel → main carve. Tracking issue: #668 (slice table, why it is sliced rather than merged, and review guidance).
Lands part of #247 (parent/sub-issue orchestration) and #299 (auto-decomposition). Deliberately not
Closes— no single slice closes either; they close when S8 activates the arc.Stacked on #655 — review that first; this PR's diff is against its branch.