feat(setup-node-with-cache): add lockfile-only-cache-key input - #258
Draft
sh-waqar wants to merge 1 commit into
Draft
feat(setup-node-with-cache): add lockfile-only-cache-key input#258sh-waqar wants to merge 1 commit into
sh-waqar wants to merge 1 commit into
Conversation
The dependency cache key hashes the lockfile plus every package.json. In a workspace repo that makes the cache far more fragile than it needs to be: editing a description, bumping a package version, or adding a script invalidates the whole cache even though none of those change what gets installed. Measured on Typeform/bob-the-builder (17 workspaces), replaying both key variants over the last 60 commits on main: lockfile + all package.json (today) 30/59 = 51% of commits miss lockfile only 16/59 = 27% of commits miss That repo's cache is ~2.8GB, and a miss there costs roughly 10 extra minutes of wall-clock: every job re-downloads the cache, runs a full install, and then all of them compress and upload ~2.9GB while only one wins the save race. Safety: the lockfile already pins the fully resolved tree, and yarn/pnpm rewrite it whenever a manifest edit actually changes resolution, so a package.json change that matters always comes with a lockfile change. --frozen-lockfile turns the remaining case into a build failure rather than a silent wrong install. All 14 commits in the sample that touched a package.json without touching yarn.lock were checked by hand: 12 were non-dependency edits, 2 added packages already resolved in the lockfile. Defaults to false, so existing callers are unaffected and keep their current cache keys. hashFiles() cannot take a variable glob, so both candidate hashes are computed in a new dep-hash step and the requested one is exported for the cache steps to consume. The two places that echo the key for logging now read the same output, so the logged key always matches the real one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an opt-in
lockfile-only-cache-keyinput tosetup-node-with-cache. Defaults tofalse, so no existing caller changes behaviour.The dependency cache key currently hashes the lockfile plus every
package.json. In a workspace repo that makes the cache far more fragile than it needs to be — editing a description, bumping a package version, or adding a script invalidates the entire cache, even though none of those change what gets installed.Measured impact
Replaying both key variants over the last 60 commits on
maininbob-the-builder(17 workspaces):package.json(today)Roughly half the misses disappear.
That repo's cache is ~2.8 GB, and a miss is expensive: every job re-downloads it, runs a full
yarn install, and then all four jobs compress and upload ~2.9 GB while only one wins the save race — the other three discard ~10 minutes of work each. Measured cost of a miss there is ~10-12 min of extra wall-clock per run.Why it's safe
The lockfile already pins the fully resolved dependency tree. Yarn and pnpm rewrite it whenever a manifest edit actually changes resolution, so a
package.jsonchange that matters is always accompanied by a lockfile change. A manifest change that doesn't touch the lockfile cannot alter what gets installed.--frozen-lockfilecloses the remaining gap: a manifest edit that would need new resolutions fails the build rather than silently installing something different.I verified this rather than assuming it. All 14 commits in the sample that touched a
package.jsonwithout touchingyarn.lock:resolutions, one already present as a transitive dep. In both cases the lockfile was unchanged precisely because the package was already installed.Implementation note
hashFiles()can't take a variable glob, so both candidate hashes are computed in a newdep-hashstep and the requested one is exported for the cache steps to consume. The two places thatechothe key for logging now read the same output, so the logged key always matches the real one (previously they'd have drifted apart under any key change).Covers both the yarn and pnpm paths.
Testing
dep-hashruns before both cache steps)Two unrelated observations from the same investigation
Not fixed here — flagging in case they're useful:
enable-yarn-cacheappears to be a dead input. It's declared ininputs:but never referenced anywhere in the action body —cache-modeis the only control. Anyone settingenable-yarn-cache: falsetoday is silently ignored.Two steps show a consistent ~38s gap before their script runs. On
bob-the-builder,Debug cache contentsandCheck if install needed on cache hiteach cost ~37-38s across all four jobs. The delay is before the first line executes:The scripts themselves are only
echo/ls -1/find -maxdepth 0, which can't account for it, and sibling steps in the same composite start in milliseconds. I tried to test whether deleting the step recovers the time, but the probe was invalidated by cache branch-scoping. Raising it as an observation rather than a diagnosis — you may recognise the cause immediately.