feat(carve S2): configurable ECS Fargate task sizing + read-only planning task - #653
feat(carve S2): configurable ECS Fargate task sizing + read-only planning task#653isadeks wants to merge 4 commits into
Conversation
|
The red The same job has been failing on Fixed in #658 (lockfile + pin only, against One note for #658's reviewer: the usual re-resolve isn't sufficient this time. The |
b07b9cb to
76e1eb1
Compare
65ec18d to
b4c5d1d
Compare
b4c5d1d to
5b75520
Compare
Automated review — carve S2 (#653)Reviewed as its own diff only ( Slice standalone-ness: PASS
Reverts: CLEANI checked the 296 deletions rather than assuming they were rewrites. Every
The deletions are genuine restructuring — most notably collapsing the two Fargate task definitions onto one shared This PR also deletes the stray 1. MAJOR — four build-tool env vars are hardcoded to this project's own monorepo and are not overridableThe build task definition hardcodes into its container environment: (plus a pre-push-hook skip) The comments are careful and clearly measured — but every measurement is against this repo: " A public adopter's repo may use none of Crucially, unlike the CPU/memory sizing, these are not configurable: Suggest an 2. MAJOR — the default build task is the Fargate maximum, ~5× the cost of a modest size
To be fair to the PR: the docstring is honest about this ("deliberately generous because they're tuned for a worst case: a large TypeScript + Python monorepo"), it explains that 120 GB is the Fargate ceiling at 16 vCPU, and it points the reader at The concern is the default, since that is what an adopter who changes nothing gets. Rough us-east-1 on-demand Fargate:
≈ 5.1× per build. For a public sample, a modest default with a documented "raise this for a large monorepo" note inverts the surprise in the safer direction — an adopter who under-provisions gets a slow build, whereas one who over-provisions gets a bill. The planning default (2 vCPU / 8 GB) is well judged and I would not change it. 3. MINOR — "read-only planning task" describes the workflow, not the IAMThe naming and comments consistently call this the read-only task def ("PLANNING task: 2 vCPU / 8 GB … For read-only workflows that clone and read the repo to produce a plan but never build"). But both task definitions are created by the same That is a deliberate, well-argued choice — the comment says so explicitly, that splitting the roles is how a grant silently drifts between the two defs — and I am not asking for it to change. But "read-only" invites a reader (or a future security reviewer) to assume a reduced-privilege role that does not exist. Worth one clarifying clause: "read-only refers to the workflow's behaviour; both task defs deliberately share one role so grants cannot drift." Verified clean
Reviewed with Claude Code. Verification: Note: this review was written directly rather than by the review agent assigned to it — that agent terminated mid-run without producing a result, so I reviewed the diff myself. Same for #656. |
Note: force-push during review — plus a BLOCKING finding that moved into this PRMy review above was written against NEW BLOCKING — this PR now admits two workflows whose agent YAML does not exist until #654The
Failure: merge #653 alone and deploy. A user submits The fix is the same as before, one slice over: move the two Status of my original findings at
|
| finding | status |
|---|---|
1. MAJOR — BUILD_VERIFY_TIMEOUT_S/JEST_MAX_WORKERS/MISE_JOBS hardcoded to this project's monorepo, not overridable |
still present |
| 2. MAJOR — default build task is the Fargate maximum (16 vCPU / 120 GB), ≈5× the cost of a modest size | still present |
| 3. MINOR — "read-only planning task" shares the build task's role and grants | still present |
| Reverts / lost exports | still clean — re-checked, no main export lost |
Deletes #647's stray cdk/node_modules symlink |
still true, still welcome |
5b75520 to
f2b5774
Compare
Review findings addressedI had missed this review in my first pass — apologies. All three findings are now fixed, and the blocking one from your addendum was already resolved by then. BLOCKING (addendum) — this PR admitted two workflows whose YAML arrives a slice later. Already fixed before I read your addendum, by moving the MAJOR #1 — four build-tool env vars hardcoded to this project's monorepo, not overridable. Fixed. You were right that these are this deployment's opinion rather than a universal default, and that MAJOR #2 — default build task was the Fargate maximum. Fixed. Your cost table is the argument: ~5.1× per build for the default an adopter gets by changing nothing. Now 4 vCPU / 16 GB on Fargate's own 20 GiB disk, with the ceiling reachable through Two of the tests that pinned the old defaults have been rewritten rather than deleted, so the heavy-monorepo size is still asserted — just as an override rather than as everyone's baseline. MINOR #3 — "read-only planning task" describes the workflow, not the IAM. Fixed as you suggested, with the clarifying clause. You are right not to ask for the roles to be split: the shared role is what stops a grant drifting between the two defs, and that reasoning is now stated where a future security reviewer will read it. Thank you also for checking all 296 deletions for lost exports rather than assuming they were rewrites, and for flagging the stray Gates: 2544 CDK tests, |
Second-round review: my de-opinionation was unreachable, and the env override was a footgunTwo findings on my own fix, both confirmed. The prop had no caller, so the ceiling was unreachable. Lowering the default to 4 vCPU / 16 GB is only defensible if a heavy monorepo can raise it without editing the construct — but nothing passed Sizing and the build-tool env are now read from deploy context, the same shape as the A malformed value throws at synth rather than silently defaulting — "I set the flag and the build still OOM'd" is a worse afternoon than a failed synth. Reachability is now asserted end to end (context → resolver → construct → template); unwiring the call site fails that test, where before it left the whole suite green.
Stale docstrings fixed. They still advertised 16 vCPU / 120 GB / 100 GiB two lines above code saying 4096 / 16384 / 21, and the per-field Verified and not changed: 4096/16384 is a legal Fargate pair and 21 GiB is the legal ephemeral-storage floor (20 throws) — confirmed by a full |
scottschreckengaust
left a comment
There was a problem hiding this comment.
Verdict: Request changes
The sizing half of this PR is excellent — modest defaults, deploy-context knobs, reserved-key guard, and an end-to-end reachability test (context → resolver → construct → template). The clientToken=taskId RunTask idempotency and the family-not-revision dispatch (toTaskDefinitionFamily) are both real robustness wins. The shared-role refactor (makeTaskDef, one taskRole/executionRole) is a genuine least-privilege improvement over two independently-granted defs.
But the PR's other headline capability — routing read-only workflows to the smaller planning task def — is inert at runtime: the ARN that selects it is never wired into the orchestrator Lambda, so readOnly planning tasks always fall back to the build def. That is half the PR's stated value shipping as a no-op, and no integration test would catch it. Blocking on that + a test that proves the wiring.
Vision alignment
Strongly aligned with bounded blast radius & cost (VISION.md). Rightsizing planning to 2 vCPU / 8 GB and de-opinionating the build default from 16 vCPU / 120 GB to a modest 4 vCPU / 16 GB inverts the cost surprise in the safe direction (a slow build is diagnosable and one prop from fixed; a bill is not). The shared-role design keeps the read-only planner from silently diverging in privilege. No tenet is traded; backing issues #247 and #299 are both approved / P0, tracked under epic #668.
Blocking issues
1. The planning task def is created and exposed but never wired to the orchestrator — read-only routing is a no-op. EcsComputeStrategy reads process.env.ECS_PLANNING_TASK_DEFINITION_ARN (cdk/src/handlers/shared/strategies/ecs-strategy.ts:51,155), but nothing sets it: TaskOrchestrator's ecsConfig (cdk/src/constructs/task-orchestrator.ts:152-160) has no planning field and its env block (:269-275) only emits ECS_TASK_DEFINITION_ARN; AgentStack (cdk/src/stacks/agent.ts:683-690) never references ecsCluster.planningTaskDefinition. Result: ECS_PLANNING_TASK_DEFINITION_ARN is always undefined, so readOnly && ECS_PLANNING_TASK_DEFINITION_ARN is always falsy and every task — including coding/decompose-v1 — runs on the build def. The planning def is synthesized (and registered) but never selected. The strategy unit tests only pass because they set the env var by hand via isolateModules; agent.test.ts asserts two task defs exist but never that the planning ARN reaches the Lambda, so the gap is invisible to the suite.
Fix: add planningTaskDefinitionArn: string to ecsConfig, pass ecsCluster.planningTaskDefinition.taskDefinitionArn from agent.ts, emit ECS_PLANNING_TASK_DEFINITION_ARN in the orchestrator env, and add an agent.test.ts assertion that the orchestrator Lambda's environment carries it (mirroring the existing ECS_TASK_DEFINITION_ARN wiring). If the wiring is deliberately deferred to a later carve slice, say so in the PR body and drop the strategy-side consumption from this slice so the two land together — shipping the consumer without the producer leaves dead config in the tree.
Non-blocking suggestions / nits
2. Stale sizing in docstrings after the de-opinionation. The class-field JSDoc still says The 64 GB / 16 vCPU BUILD task def (ecs-agent-cluster.ts:278) even though this PR lowered the build default to 4 vCPU / 16 GB. Same stale figure in compute-strategy.ts:55 ("the 64 GB build def") and orchestrator.ts:276 ("the context-gated 64GB ECS tier"). The prior review round explicitly flagged and claimed to fix stale docstrings; these three survived. Reword to "the larger BUILD task def (default 4 vCPU / 16 GB, raisable to the Fargate ceiling)".
3. MAX_TURNS fallback silently diverges (100 vs 200). ecs-strategy.ts:209 changed the inline default from 100 to 200, while DEFAULT_MAX_TURNS = 100 (validation.ts:34) still governs the hydrate path (orchestrator.ts:566). The two only disagree when payload.max_turns is absent, but there's no comment explaining the bump and it's out of band with this PR's stated scope. Either route both through DEFAULT_MAX_TURNS or add a one-line rationale.
Documentation
No docs/guides or docs/design change is included. The new deploy-context knobs (ecsBuildTaskCpu, ecsBuildTaskMemoryMiB, ecsBuildTaskEphemeralStorageGiB, ecsPlanningTaskCpu, ecsPlanningTaskMemoryMiB, ecsExtraBuildEnv) are documented only in an inline comment in agent.ts. For a public sample where sizing is now the operator's lever, these belong in the ECS/compute deployment guide alongside the compute_type=ecs gate. Non-blocking for this slice but worth a follow-up. No Starlight mirror edits needed (no docs/ sources touched). Backing issues #247/#299 exist and are approved; #668 tracks the epic.
Tests & CI
Sizing/override paths are well covered (resolver rejects malformed numbers, non-string env values, and reserved platform keys; end-to-end reachability asserted in agent.test.ts). The parity test (both defs share one task+execution role) is a strong regression guard. Gap: no test asserts the planning ARN is wired into the orchestrator env (Blocking #1) — the isolated-module tests set it manually, so they cannot catch the missing glue. CI: 4/4 checks green; the author documented (issue comment) that the deps-scan red is a pre-existing main advisory unrelated to this diff — verified plausible (no lockfile/manifest touched).
Bootstrap policy coverage: N/A. This PR adds a second AWS::ECS::TaskDefinition and two AWS::IAM::Roles — both existing CFN resource types already covered; no new resource type is introduced, so no bootstrap policy / action-map / BOOTSTRAP_VERSION bump is required.
Review agents run
- code-reviewer — ran (manual principal-architect pass; Semgrep MCP invocation failed on a
.semgrep.ymlconfig-deprecation error, so I fell back to targeted grep + manual review of the three core source files). - silent-failure-hunter — ran.
parseMergeBranchesswallows malformed JSON but logs awarnand returns[](documented best-effort) — acceptable, not a silent swallow. The planning-def fallback (Blocking #1) is a safe silent degradation, but it degrades the feature to a no-op invisibly — hence flagged. - type-design-analyzer — ran. New
EcsTaskSizinginterface: all-optional fields with clear per-field defaults;readOnly?added to theComputeStrategy.startSessioncontract with a documented default. Coherent, no over-abstraction. - comment-analyzer — ran. Surfaced the three stale
64 GBreferences (Nit #2). - pr-test-analyzer — ran. Surfaced the missing orchestrator-env wiring assertion (Blocking #1).
- security-review — ran (IAM/env change). Shared-role refactor is a least-privilege improvement;
RESERVED_BUILD_ENV_KEYScorrectly fails closed on attempts to overrideAGENT_SESSION_ROLE_ARN/table names viaecsExtraBuildEnv;clientTokenprevents double-RunTask. No new wildcard actions.
Human heuristics
- Proportionality — Pass.
makeTaskDeffactory is justified (forces build/planning parity); no gratuitous abstraction. - Coherence — Concern. "read-only planning" naming is now well-clarified in comments, but the feature it names does not actually route yet (
ecs-strategy.ts:51), so the concept is present in code without a working path. - Clarity — Concern. Stale
64 GBdocstrings (ecs-agent-cluster.ts:278) contradict the code two lines from the new 4096/16384 defaults. - Appropriateness — Concern. Tests assert what the isolated strategy does (env preset by hand) rather than what the deployed system should do (stack wires the ARN) — the AI005 failure mode.
| * (e.g. a deployment that hasn't provisioned the planning task) — never worse | ||
| * than running everything on the build task. | ||
| */ | ||
| const ECS_PLANNING_TASK_DEFINITION_ARN = process.env.ECS_PLANNING_TASK_DEFINITION_ARN; |
There was a problem hiding this comment.
Blocking: nothing sets ECS_PLANNING_TASK_DEFINITION_ARN. TaskOrchestrator.ecsConfig (task-orchestrator.ts:152-160) has no planning field and its env block (:269-275) emits only ECS_TASK_DEFINITION_ARN; agent.ts:683-690 never references ecsCluster.planningTaskDefinition. So this constant is always undefined, the readOnly && ECS_PLANNING_TASK_DEFINITION_ARN guard at line 155 is always falsy, and read-only workflows never reach the planning def — the feature is inert. Wire the ARN through ecsConfig → orchestrator env and add an agent.test.ts assertion that the Lambda environment carries it (the isolated-module unit tests set it by hand, so they can't catch this).
|
|
||
| export class EcsAgentCluster extends Construct { | ||
| public readonly cluster: ecs.Cluster; | ||
| /** The 64 GB / 16 vCPU BUILD task def — for coding workflows that run a full |
There was a problem hiding this comment.
Stale: this PR lowered the build default to 4 vCPU / 16 GB, but the docstring still says "64 GB / 16 vCPU". Same stale figure in compute-strategy.ts:55 and orchestrator.ts:276. Reword to "the larger BUILD task def (default 4 vCPU / 16 GB, raisable to the Fargate ceiling via taskSizing)."
| ...(payload.prompt ? [{ name: 'TASK_DESCRIPTION', value: String(payload.prompt) }] : []), | ||
| ...(payload.issue_number ? [{ name: 'ISSUE_NUMBER', value: String(payload.issue_number) }] : []), | ||
| { name: 'MAX_TURNS', value: String(payload.max_turns ?? 100) }, | ||
| { name: 'MAX_TURNS', value: String(payload.max_turns ?? 200) }, |
There was a problem hiding this comment.
Nit: the inline MAX_TURNS fallback changed 100 → 200, but DEFAULT_MAX_TURNS is still 100 (validation.ts:34) and governs the hydrate path (orchestrator.ts:566). The two disagree only when payload.max_turns is absent, but there's no comment explaining the bump. Either route both through DEFAULT_MAX_TURNS or add a one-line rationale.
…sizing Slice 2 of the linear-vercel→main carve (stacked on S1 foundation contracts). Adds a second, smaller read-only "planning" Fargate task definition alongside the existing build task def, and routes read-only workflows to it so a clone-and-read task (which never builds) doesn't allocate the full build task's CPU/memory. - ecs-agent-cluster.ts: add the planning task def (2 vCPU / 8 GB) next to the build def; make BOTH task sizes overridable via a new optional `taskSizing` prop, with the current values as defaults. The defaults are generous (tuned for a large monorepo build); a consumer with a lighter repo can shrink the build task to cut Fargate cost, so sizing is configuration rather than a fixed value. - ecs-strategy.ts / compute-strategy.ts / orchestrate-task.ts: select the planning task def for read-only workflows on the ECS substrate (ignored by AgentCore); the read-only flag comes from the workflow registry (S1). - orchestrator.ts: thread the per-repo build/lint commands + read-only routing. Also rewrote the comments in these files to plain what/why so they're understandable without internal tracking-ticket context (this is public-sample code). Comment-only edits — verified the non-comment source is identical to the source branch. Gates: cdk compile + eslint + full jest (135 suites / 2522 tests) green.
…able env Two findings from the automated review, both instances of one-deployment's measurements becoming everyone's default. **The default build task was the Fargate maximum** — 16 vCPU / 120 GB plus a 100 GiB root filesystem. The docstring was honest about why (tuned for a large TypeScript + Python monorepo) and pointed at the sizing prop, but a default is what an adopter who changes nothing pays for, and that is roughly 5x the per-build cost of a modest size in us-east-1 on-demand. Now 4 vCPU / 16 GB on Fargate's own 20 GiB disk, with the ceiling reachable through `taskSizing` and a test showing that path. Under-provisioning is a slow or OOM-ing build, which is diagnosable and one prop from fixed; over-provisioning is a silent bill. The planning default (2 vCPU / 8 GB) was already well judged and is unchanged. **Four build-tool env vars were hardcoded with no override.** A verify timeout and parallelism caps for a mise/jest/pytest-shaped build are meaningful only for the toolchain a given repo uses — inert for an adopter who uses none of them, and actively wrong for one who uses mise with plenty of memory, since the cap serialises their build to solve a memory problem they may not have. Added `extraBuildEnvironment`, merged over the defaults so a caller's key wins. The measured reasoning stays in the repo as the comment it always was; it is no longer welded into the construct. Also clarifies that "read-only planning task" describes the workflow's behaviour rather than a reduced-privilege role — both task defs deliberately share one role so grants cannot drift between them, and the name should not be mistaken for a privilege boundary.
…latform env keys
A second review pass found my de-opinionation had swapped one problem for its
mirror image, plus a footgun.
**The prop had no caller, so the ceiling was unreachable.** Lowering the default
to 4 vCPU / 16 GB is only defensible if a large monorepo can raise it without
editing the construct — but nothing passed `taskSizing`, `AgentStack` takes no
custom props, and no context key read it. So the fix traded "everyone pays for
16 vCPU" for "nobody can reach 16 vCPU", and the second is worse for exactly the
repo the old default existed to serve. Sizing and the build-tool env are now read
from deploy context, the same shape as the `compute_type` gate beside them:
cdk deploy -c compute_type=ecs -c ecsBuildTaskCpu=16384 \
-c ecsBuildTaskMemoryMiB=122880 -c ecsBuildTaskEphemeralStorageGiB=100
A malformed value throws at synth rather than silently falling back — "I set the
flag and the build still OOM'd" is a worse afternoon than a failed synth.
**`extraBuildEnvironment` could clobber platform wiring.** It spreads over the
whole container env, not just the four build-tool vars it was added for, so a
mistyped key could reach table names, bucket names, or `AGENT_SESSION_ROLE_ARN` —
whose absence makes the agent fall back to ambient credentials with per-tenant
scoping silently OFF, which the agent's own session module calls its most
dangerous failure mode. Reserved keys are now rejected at synth.
Also corrects the docstrings, which still advertised 16 vCPU / 120 GB / 100 GiB
two lines above code saying 4096 / 16384 / 21 — whichever an operator read first
was wrong.
Reachability is asserted end to end (context → resolver → construct → template):
unwiring the call site fails that test, where before it left the whole suite green.
Stress-tested the modest default on the workload the old 120 GB default existed
for: a full parallel `mise run build` of this monorepo (agent + CDK + CLI + docs)
on ECS, with the build gate actually running.
memory 3132 MiB peak of 16384 → ~5x headroom
disk 14.68 GiB peak of ~21 → ~1.4x headroom
result build_passed, ECS exit 0, no OOM, no ENOSPC
So CPU and memory are comfortably right, and for a reason worth recording:
`MISE_JOBS=1` serialises the packages, so peak memory is max-single-package rather
than sum-of-all. The old ceiling was never the binding constraint.
Disk is the outlier. 70% of the requested figure consumed by a single build with
a warm clone means a heavier dependency cache, or a second build sharing the task,
plausibly runs it out of space — and running out of space surfaces as a spurious
build FAILURE rather than an obvious resource error, which is the worst way for
this to present. 50 GiB restores real margin, and ephemeral storage is a small
fraction of the per-task cost next to vCPU and RAM, so the cost win that motivated
the modest default is intact.
Sized on measurement rather than on the assumption that the old numbers were all
equally over-provisioned. The docstring records the figures, and a test pins the
default so it cannot drift back to the floor unnoticed.
a837ad2 to
7ecfba5
Compare
Stacked on #647 (
carve/s1-foundation-contracts). Review that first; this PR's diff is only its own slice.What this adds
A read-only planning task definition for the ECS Fargate compute backend, and makes every Fargate task size configurable instead of hardcoded.
Planning-type work (reading a repo, producing a plan) needs no write access and far less compute than a build. Routing it to its own smaller, read-only task definition means it no longer borrows the build tier's footprint.
Sizing is configuration, not a baked-in opinion
The build tier's defaults are generous because a build of a large monorepo can genuinely need them, but nothing here is fixed: a new
taskSizingprop overrides any of the CPU / memory / ephemeral-storage values independently, and omitting it keeps the defaults. Operators whose workloads are smaller (or larger) set what fits rather than editing the construct.Defaults: build 16 vCPU / 120 GiB / 100 GiB ephemeral; planning 2 vCPU / 8 GiB.
Note for reviewers
Updating the sizing replaces the task definition in place (a new revision) rather than adding one — there's a test asserting exactly that, so an override can't silently leave the old definition behind.
Verification
Full CDK suite green. The override path was deploy-verified: setting
taskSizingproduced a smaller task definition, and the default path was unchanged.Tracking
Slice S2 of the linear-vercel → main carve. Tracking issue: #668 (slice table, why it is sliced rather than merged, and review guidance).
Lands part of #247 (parent/sub-issue orchestration) and #299 (auto-decomposition). Deliberately not
Closes— no single slice closes either; they close when S8 activates the arc.Stacked on #647 — review that first; this PR's diff is against its branch.