Skip to content

fix(project): validate runtimeVersion on CodeZip runtimes - #1972

Open
notgitika wants to merge 1 commit into
refactorfrom
fix/codezip-runtime-version
Open

fix(project): validate runtimeVersion on CodeZip runtimes#1972
notgitika wants to merge 1 commit into
refactorfrom
fix/codezip-runtime-version

Conversation

@notgitika

Copy link
Copy Markdown
Contributor

Stacked on #1970 (itself on #1969) — review those first. Follows up on the second commit of #1970, which set runtimeVersion on the python template; this makes the CLI able to catch the same class of error itself.

The gap

The CDK construct library's AgentEnvSpecSchema refines:

if (data.build !== 'Container' && !data.runtimeVersion) {
  // "runtimeVersion is required for CodeZip builds"
}

Our copy of that schema marked the field plain .optional(), with no such rule. So the CLI accepted a spec that synthesis would refuse, and the user only discovered it after a full tsc + cdk synth — from a stack trace inside the generated app rather than from the CLI.

Our superRefine already mirrors the library's checks almost one for one, including the identical container-only-fields loop directly beneath. This adds the one that was missing, in the same position the library has it:

if (data.build !== "Container" && !data.runtimeVersion) {
  ctx.addIssue({
    code: "custom",
    message: "runtimeVersion is required for CodeZip builds",
    path: ["runtimeVersion"],
  });
}

Container builds take their version from the image and stay exempt, consistent with the check below it.

Reporting the rule earlier is only useful if it says what to fix

Moving the error earlier initially made it worse, which is worth calling out because it's the larger half of this diff. FsProjectManager.resolve was collapsing every validation failure into one line:

Error: invalid project configuration at /path/to/agentcore/agentcore.json

The zod issues were attached as cause, which nothing renders — they were reachable only from the debug log. So the user traded CDK's precise runtimes[0].runtimeVersion: runtimeVersion is required for CodeZip builds for a message that doesn't name a field.

resolve() now renders the failing paths:

Error: invalid project configuration at /path/to/agentcore/agentcore.json
  - runtimes[0].runtimeVersion: runtimeVersion is required for CodeZip builds

That's every validation failure of agentcore.json, not just this one — any spec error now names its field. A JSON syntax error carries no zod issues, so that path is unchanged and still covered by its existing test.

Blast radius

ProjectSpecSchema has exactly one consumer: resolve() at manager.tsx:56. So the only configs this newly rejects are ones the CDK app would also reject at synth — no workflow that previously succeeded starts failing. It does mean the whole CLI now refuses to load such a config rather than failing at build time, which is the intent.

Two test fixtures were CodeZip runtimes without runtimeVersion and are now invalid by this rule, so they gain one (schema/runtime.test.ts, schema/project.test.ts). Worth a look during review that the surrounding assertions still test what they claim — the security-group-cap test at runtime.test.ts asserts a CodeZip spec succeeds, so it would have silently started passing for the wrong reason had the fixture not been fixed.

Verification

Against the same real scaffolded project used to verify #1970, with runtimeVersion removed from its agentcore.json:

  • Before: tsc ran, cdk synth ran, then the construct library's zod error.
  • After: the CLI stops at config load with runtimes[0].runtimeVersion: runtimeVersion is required for CodeZip builds — no compile, no synth, no node_modules needed.

New unit tests: the rule rejects a CodeZip runtime with no runtimeVersion and reports it at path: ["runtimeVersion"]; a container runtime without one still validates; and resolve() surfaces the field name in its message.

  • bun test — 1063 pass, 0 fail
  • tsc --noEmit clean, oxlint clean

@github-actions github-actions Bot added the size/s PR size: S label Aug 11, 2026
@github-actions github-actions Bot added agentcore-harness-reviewing AgentCore Harness review in progress and removed agentcore-harness-reviewing AgentCore Harness review in progress labels Aug 11, 2026
@notgitika
notgitika force-pushed the feat/project-build-synth branch from 305bf56 to 639227f Compare August 11, 2026 19:03
@notgitika
notgitika force-pushed the fix/codezip-runtime-version branch from 4c2ac79 to 1ed88a7 Compare August 12, 2026 19:54
@github-actions github-actions Bot added size/s PR size: S and removed size/s PR size: S labels Aug 12, 2026
@tejaskash
tejaskash force-pushed the feat/project-build-synth branch from 78097a8 to 0b2da99 Compare August 12, 2026 20:24
@notgitika
notgitika force-pushed the feat/project-build-synth branch from 0b2da99 to 05c244e Compare August 13, 2026 15:25
@notgitika
notgitika force-pushed the fix/codezip-runtime-version branch from 1ed88a7 to 1ac4641 Compare August 13, 2026 15:29
@github-actions github-actions Bot added size/s PR size: S and removed size/s PR size: S labels Aug 13, 2026
Base automatically changed from feat/project-build-synth to refactor August 13, 2026 16:26
The CDK construct library rejects a CodeZip runtime that declares no
runtimeVersion, but our own AgentEnvSpecSchema marked the field plain
`.optional()`. The CLI therefore accepted a spec that synthesis would refuse,
and the user only found out after a compile and a synth.

Mirror the library's rule so the CLI reports it against agentcore.json instead.
Container builds take their version from the image and stay exempt, matching the
adjacent container-only-fields check.

Surfacing the rule earlier is only useful if it says what to fix, and
`resolve()` was collapsing every validation failure into "invalid project
configuration at <path>" with the reasons reachable only from the debug log. It
now renders the failing paths, so the message reads:

  invalid project configuration at .../agentcore/agentcore.json
    - runtimes[0].runtimeVersion: runtimeVersion is required for CodeZip builds

A JSON syntax error has no zod issues to render and is unchanged.
@notgitika
notgitika force-pushed the fix/codezip-runtime-version branch from 1ac4641 to 86d59df Compare August 13, 2026 18:06
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.91%. Comparing base (2d6e44e) to head (86d59df).

Additional details and impacted files
@@            Coverage Diff            @@
##           refactor    #1972   +/-   ##
=========================================
  Coverage     96.91%   96.91%           
=========================================
  Files           356      356           
  Lines         20136    20159   +23     
=========================================
+ Hits          19515    19538   +23     
  Misses          621      621           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@notgitika

Copy link
Copy Markdown
Contributor Author

typecheck failing on refactor head and therefore also on this PR

* Without this the cause is only reachable from the debug log, leaving the user with
* "invalid project configuration" and no indication of which field to fix.
*/
function describeValidationFailure(cause: unknown): string {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would z.prettifyError help us here? https://zod.dev/error-formatting

@@ -63,9 +86,10 @@ export class FsProjectManager implements ProjectManager {
} catch (error) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would it be easier to drop this try/catch and update the error in

throw new DeserializationError(filePath, { cause: parseResult.error });
to include a better message? (I think deserialization could also be changed to a user error since all cases up to this point are user controlled).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/s PR size: S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants