Summary
The publish_layer job in make-release can fail because it starts installing the just-published packages from npm before the registry actually serves them. needs: publish-npm guarantees that npm publish returned — not that registry.npmjs.org resolves the new versions. The layer's CDK bundling does a real npm i @aws-lambda-powertools/<pkg>@<version>, so it is fully exposed to registry propagation delay.
This broke the v2.35.0 release (run 32124487679). publish_layer / build-layer failed in the CDK build step and blocked every downstream job — beta and prod layer deploys, GovCloud, China, the SSM update, and the layer ARN docs PR:
npm error code ETARGET
npm error notarget No matching version found for @aws-lambda-powertools/commons@2.35.0.
«FailedToBundleAsset» Failed to bundle asset LayerPublisherStack/LambdaPowertoolsLayer/Code/Stage
Why is this needed?
A release that publishes cleanly to npm can still leave the layers unpublished and the docs un-updated, requiring manual intervention to notice and re-run. The failure is also misleading: ETARGET reads as "we asked for a version that doesn't exist", i.e. a bug in our own version wiring, rather than "the registry hasn't caught up yet".
Timeline for v2.35.0 (UTC, comparing the publish job log against each package's time.<version> field in its packument):
| event |
time |
npm publish prints + @aws-lambda-powertools/commons@2.35.0 |
10:20:09 |
npm publish prints + @aws-lambda-powertools/logger@2.35.0 |
10:20:16 |
last package published, publish-npm job succeeds |
10:21:01 |
build-layer starts npm i ...@2.35.0 |
10:21:58 |
npm i fails with ETARGET on commons@2.35.0 |
10:22:04 |
commons@2.35.0 becomes resolvable on the registry |
10:22:14 |
logger@2.35.0 becomes resolvable on the registry |
10:27:23 |
commons missed by 10 seconds. logger took 7m07s after its publish call returned. Twelve of the fourteen packages propagated within ~1s, so this is intermittent and unpredictable — a fixed sleep would not reliably cover it.
Worth recording for whoever hits this next: during the lag window the version 404s on every registry endpoint (version doc, tarball, packument entry, npm view) while npm's attestations endpoint already serves the signed publish and SLSA provenance attestations for that exact version:
GET /-/npm/v1/attestations/@aws-lambda-powertools/logger@2.35.0 → 200
subject: pkg:npm/@aws-lambda-powertools/logger@2.35.0
GET /@aws-lambda-powertools/logger/2.35.0 → 404 (same moment)
That asymmetry looks exactly like npm dropped or partially wrote the publish, which makes republishing tempting. It isn't, and republishing over a version npm already holds a record for risks a 409 that would need npm support to unpick. The correct response is to wait, re-verify, and re-run.
Which area does this relate to?
Automation
Solution
Add a registry-availability gate so the release either waits or fails loudly with an accurate reason:
- Verify step at the end of
publish-npm — for every non-private workspace, poll https://registry.npmjs.org/<pkg>/<version> until it returns 200 (and HEAD the dist.tarball URL), with a timeout of ~10 minutes given the 7-minute observation. This keeps the failure inside the job that owns publishing and gives an unambiguous message.
- Belt and braces in the layer build — retry the bundling
npm i a few times with backoff, so a slow CDN edge doesn't take out a release that is otherwise fine.
Option 1 alone addresses the reported failure. Option 2 additionally covers the case where the packument is served but the tarball hasn't propagated.
Relevant code:
.github/workflows/make-release.yml — publish_layer gated only on needs: publish-npm.github/workflows/publish_layer.yml — build-layer / CDK build steplayers/src/layer-publisher-stack.ts — bundling shells out to npm i for the published versions
Workaround in the meantime: wait for the versions to appear, confirm with npm view <pkg>@<version> version, then gh run rerun <run-id> --failed.
Acknowledgment
Future readers
Please react with 👍 and your use case to help us understand customer demand.
Summary
The
publish_layerjob inmake-releasecan fail because it starts installing the just-published packages from npm before the registry actually serves them.needs: publish-npmguarantees thatnpm publishreturned — not thatregistry.npmjs.orgresolves the new versions. The layer's CDK bundling does a realnpm i @aws-lambda-powertools/<pkg>@<version>, so it is fully exposed to registry propagation delay.This broke the v2.35.0 release (run 32124487679).
publish_layer / build-layerfailed in theCDK buildstep and blocked every downstream job — beta and prod layer deploys, GovCloud, China, the SSM update, and the layer ARN docs PR:Why is this needed?
A release that publishes cleanly to npm can still leave the layers unpublished and the docs un-updated, requiring manual intervention to notice and re-run. The failure is also misleading:
ETARGETreads as "we asked for a version that doesn't exist", i.e. a bug in our own version wiring, rather than "the registry hasn't caught up yet".Timeline for v2.35.0 (UTC, comparing the publish job log against each package's
time.<version>field in its packument):npm publishprints+ @aws-lambda-powertools/commons@2.35.0npm publishprints+ @aws-lambda-powertools/logger@2.35.0publish-npmjob succeedsbuild-layerstartsnpm i ...@2.35.0npm ifails with ETARGET oncommons@2.35.0commons@2.35.0becomes resolvable on the registrylogger@2.35.0becomes resolvable on the registrycommonsmissed by 10 seconds.loggertook 7m07s after its publish call returned. Twelve of the fourteen packages propagated within ~1s, so this is intermittent and unpredictable — a fixedsleepwould not reliably cover it.Worth recording for whoever hits this next: during the lag window the version 404s on every registry endpoint (version doc, tarball, packument entry,
npm view) while npm's attestations endpoint already serves the signed publish and SLSA provenance attestations for that exact version:That asymmetry looks exactly like npm dropped or partially wrote the publish, which makes republishing tempting. It isn't, and republishing over a version npm already holds a record for risks a 409 that would need npm support to unpick. The correct response is to wait, re-verify, and re-run.
Which area does this relate to?
Automation
Solution
Add a registry-availability gate so the release either waits or fails loudly with an accurate reason:
publish-npm— for every non-private workspace, pollhttps://registry.npmjs.org/<pkg>/<version>until it returns 200 (and HEAD thedist.tarballURL), with a timeout of ~10 minutes given the 7-minute observation. This keeps the failure inside the job that owns publishing and gives an unambiguous message.npm ia few times with backoff, so a slow CDN edge doesn't take out a release that is otherwise fine.Option 1 alone addresses the reported failure. Option 2 additionally covers the case where the packument is served but the tarball hasn't propagated.
Relevant code:
.github/workflows/make-release.yml—publish_layergated only onneeds: publish-npm.github/workflows/publish_layer.yml—build-layer/CDK buildsteplayers/src/layer-publisher-stack.ts— bundling shells out tonpm ifor the published versionsWorkaround in the meantime: wait for the versions to appear, confirm with
npm view <pkg>@<version> version, thengh run rerun <run-id> --failed.Acknowledgment
Future readers
Please react with 👍 and your use case to help us understand customer demand.