Skip to content

--incremental: after a pnpm dependency version change, the cached run is slower than a cold run and most of the time is unattributed #63731

Description

@mushan0x0

--incremental: after a pnpm dependency version change, the cached run is slower than a cold run, and most of the time is not attributed to any reported phase

Summary

When .tsbuildinfo is invalidated by a dependency path change (pnpm moves node_modules/x to a new .pnpm/x@<version>/… target on every version bump), tsc --noEmit does more work than a cold build with no cache at all — and the extra time is not attributed to parse, bind, or check.

Minimal repro (80 files, zod): the post-change run takes 2.89s vs 1.63s cold, with Check time: 0.07s. Roughly 2.5s of a 2.89s run is unaccounted.

On a real 8773-file application the same shape becomes severe: 114.6M instantiations vs 7.7M cold (14.9x a full build), 202.4s vs 24.7s (8.2x slower than no cache), 5.45 GB peak RSS vs 2.54 GB, and it OOMs at node's default heap.

Editing source files does not trigger this — invalidating a module that 75 files transitively import is cheaper than a cold build, as expected. Only the dependency-path case regresses.

Minimal reproduction

The trigger is pnpm's content-addressed layout, not "a dependency changed" as such. With npm, npm i zod@<other> replaces node_modules/zod in place and tsc does not invalidate at all (it takes the fast path, Instantiations: 0). With pnpm, node_modules/zod is a symlink into .pnpm/zod@<version>/node_modules/zod, so a version change moves every resolved path — and that is what perturbs the cache.

mkdir repro && cd repro
printf '{ "name": "repro", "private": true, "type": "module" }\n' > package.json
pnpm add zod@3.25.76 typescript@5.9.3
# tsconfig.json: { "compilerOptions": { "strict": true, "noEmit": true,
#   "incremental": true, "skipLibCheck": true, "moduleResolution": "bundler",
#   "module": "esnext", "target": "ESNext" }, "include": ["src/**/*.ts"] }
# src/shared.ts re-exports zod + a generic `make<B, R>({ body, response, handler })`
# src/route0..79.ts each build a nested z.object and call make() (see gist/attachment)

./node_modules/.bin/tsc --noEmit --extendedDiagnostics -p tsconfig.json   # A: cold
./node_modules/.bin/tsc --noEmit --extendedDiagnostics -p tsconfig.json   # B: fresh cache
pnpm add zod@3.24.4                                                       # symlink moves
./node_modules/.bin/tsc --noEmit --extendedDiagnostics -p tsconfig.json   # D
run Types Instantiations Check time Total time
A cold 84,190 368,421 1.63s
B fresh cache 85 0 0.23s
D after pnpm version change 84,190 378,157 0.07s 2.89s

Two things to note in D, both of which also hold in the large project below:

  1. The incremental run is slower than the cold run (2.89s vs 1.63s). The cache is not merely useless here, it is a net cost.
  2. Check time is 0.07s while Total time is 2.89s. Parse and bind are ~0.2s. Roughly 2.5s — the large majority of the run — is not attributed to any phase --extendedDiagnostics reports.

The same signature at scale

In a real Next.js application (8773 files, ~136k lines of TS, ~672k lines of definitions; zod + drizzle-orm + next-rest-framework generic chains), the same trigger produces a far more severe version of the same shape:

run state Types Instantiations Check time Total time peak RSS
A no .tsbuildinfo (cold) 964,787 7,687,382 18.93s 24.72s 2.54 GB
B fresh .tsbuildinfo 86 0 5.18s 0.99 GB
C fresh, then one widely-imported source file edited (75 direct dependents) 627,108 5,095,840 8.52s 13.66s 1.80 GB
D fresh, then pnpm dependency version changed 3,965,709 114,589,573 9.35s 202.39s 5.45 GB
  • D does 14.9x the instantiations of a full cold build, and is 8.2x slower than having no cache at all.
  • Check time in D (9.35s) is lower than in the cold build (18.93s), yet total time is 202s. ~193s is unaccounted.
  • At node's default heap D does not complete: it OOMs after 254s at 4.4 GB. The figures above required --max-old-space-size=8192.
  • Run C is the control: invalidating a module that 75 files import transitively is cheaper than a cold build (13.66s / 1.80 GB). Incremental behaves correctly for source edits. Only the dependency-path case regresses.

I could not reproduce the instantiation blow-up at minimal scale — it appears to need the real project's type complexity. The minimal repro does reproduce the two structural facts: the cached run is slower than cold, and the time is not attributed to parse/bind/check.

Why this matters

The failure mode is silent and counter-intuitive: enabling incremental makes the common CI / post-npm install case an order of magnitude worse than disabling it, and can push a project that type-checks fine in 2.5 GB over the default heap limit. Teams hit it as "our type-check randomly OOMs" and, not suspecting the cache, raise --max-old-space-size — which masks it, since the run then merely takes 200s instead of 25s.

There appears to be no heuristic along the lines of "the cached state is mostly invalid, discard it and do a cold check", which is what the numbers say should happen here.

Questions

  1. What phase accounts for the ~193s that is neither parse, bind, nor check?
  2. Why do Types/Instantiations exceed a full cold build by 4x/15x when the amount of code to check is, at most, all of it?
  3. Would it be reasonable for tsc to drop the cached program when the module-resolution inputs (installed package versions / resolved paths) have changed, rather than reconciling it?

Environment

  • TypeScript: 5.9.3
  • Node: v22.16.0
  • OS: macOS (darwin arm64)
  • tsconfig: incremental: true, skipLibCheck: true, strict: true, moduleResolution: "bundler", noEmit: true

Metadata

Metadata

Labels

Fix AvailableA PR has been opened for this issueNeeds InvestigationThis issue needs a team member to investigate its status.

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions