chore(security): create test temp dirs with mkdtemp and override two pinned CVEs - #1432
Conversation
📝 WalkthroughWalkthroughThe test suites now create temporary directories with ChangesTest temporary-directory setup
Website dependency and audit configuration
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Deploying openspec-docs with
|
| Latest commit: |
3980e6f
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://8176e9d0.openspec-docs.pages.dev |
| Branch Preview URL: | https://chore-security-temp-file-har.openspec-docs.pages.dev |
Every one of these suites built its temp directory by hand:
testDir = path.join(os.tmpdir(), `openspec-test-${randomUUID()}`);
await fs.mkdir(testDir, { recursive: true });
That check-then-create is what CodeQL's js/insecure-temporary-file flags,
and it accounted for 452 of the repo's 467 dismissed code-scanning alerts.
Dismissing them one by one is a treadmill: every new suite that copies the
idiom mints fresh alerts, and a real finding is easy to lose in that volume.
fs.mkdtemp creates the directory atomically at mode 0700 with a random
suffix, so there is no window to pre-empt and no name to guess. The suites
that already used it are the evidence this silences the rule: 33 of the 34
files calling mkdtemp carry zero alerts. The lone exception, archive.test.ts,
was scanned one commit before its own mkdtemp fix landed and is left to that
change rather than conflicting with it.
The dirs named on Date.now() alone were genuinely predictable; the randomUUID
ones were not, but they trained the same copy-paste. Both are gone now.
Behavior is unchanged: mkdtemp creates the root the old mkdir created, and
no assertion depended on the root being absent. Verified with the full suite
(2196 tests, 111 files).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
6ecb89a to
6c1fe49
Compare
alfred-openspec
left a comment
There was a problem hiding this comment.
The temp-dir refactor and lockfile overrides check out at exact head 6c1fe492: frozen installs, the root build and full 2,196-test suite, both website audits, the 87-page static build, the resolved dependency tree, and the hosted matrix all pass.
One policy blocker remains: the new website audit uses continue-on-error: true on every event. That means the weekly and main security runs can stay green when a new high-severity website advisory appears, so the step does not fully close the monitoring blind spot it describes. Please make it advisory only on pull requests and blocking on scheduled, main, and manual runs, matching the published-dependency audit. The site exports static output, but Next, PostCSS, and Sharp execute in the CI build and are part of the build supply chain.
Two advisories on the docs site have no Dependabot PR and never will: Next pins postcss at 8.4.31 as a direct dependency and sharp at ^0.34.5 as an optional one, so Dependabot cannot raise either without a Next release that does it first. 16.2.11 does not — it still pins both. Left alone these sit open indefinitely. postcss 8.4.31 -> 8.5.22 GHSA-qx2v-qp2m-jg93 (XSS via unescaped </style>) sharp 0.34.5 -> 0.35.3 GHSA-f88m-g3jw-g9cj (4 libvips CVEs) A version-ranged selector (`postcss@<8.5.10`) was the first instinct, since it lapses on its own once Next moves past it. It is the wrong tool: it pins the override to one advisory's floor, and silently stops applying when the next advisory raises that floor. GHSA-6g55-p6wh-862q landed while this branch was open and moved postcss's patched floor to 8.5.12 — under the ranged selector a dependency pinning 8.5.11 would have resolved to 8.5.11 and stayed vulnerable. A plain floor cannot under-match, so that is what this uses. The floor also covers the new advisory: 8.5.22 is past 8.5.12. postcss dedupes to the single copy the site already had for Tailwind. These are the last two open Dependabot alerts on the repo. `pnpm audit` on website/ goes from 2 advisories to "No known vulnerabilities found". The site builds clean, OG image generation included, and the package set grows by exactly two platform-gated wasm32 binaries that never install on CI. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both `pnpm audit` steps run at the repo root. The docs site keeps its own lockfile and is not a workspace member, so neither step could see it — the root audit passed all the way through while postcss and sharp sat open in website/. That blind spot is why they needed a manual override to find. Blocking rule copied from the published-dependency audit: advisory on pull requests, blocking on the weekly schedule and on pushes to main. An always-advisory step would only relocate the blind spot — the sweep would stay green with a live advisory and someone would have to read the log of a passing run to notice. `!cancelled()` because the two audits above can fail hard. Without it a root advisory would skip this step entirely, in exactly the situation where the site's own state matters most. Verified against the pre-fix lockfile — the step reports the two advisories it would have caught: 2 vulnerabilities found Severity: 1 moderate | 1 high and reports "No known vulnerabilities found" against the fixed one. Confirmed it reads website/pnpm-lock.yaml and not the root: with a vulnerable website lockfile and a clean root, it exits 1; a missing website lockfile is a hard ERR_PNPM_AUDIT_NO_LOCKFILE rather than a false clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
6c1fe49 to
3980e6f
Compare
alfred-openspec
left a comment
There was a problem hiding this comment.
The follow-up closes the policy gap: website audits are now advisory only on pull requests and blocking on scheduled, main, and manual runs. Re-verified exact head 3980e6f with frozen install, clean production/full website audits, the 87-page static build, resolved dependency overrides, and the hosted Audit plus Actions analysis checks. Approved; a real second-human approval is still required before merge.
Status: Ready. Rebased on main after #1429. Full suite green (2196 tests / 111 files), lint and
tscclean, site builds, all CI checks pass including Windows.Closes the repo's last two open Dependabot alerts, and removes the recurring source of 442 CodeQL dismissals.
What was wrong
1. A green CodeQL dashboard that wasn't earned. The repo shows 0 open alerts and 467 dismissed — 464 of them bulk-dismissed on a single day. 452 of those are one rule,
js/insecure-temporary-file, fired by one copy-pasted test idiom:Dismissing them is a treadmill — every new suite that copies the idiom mints fresh alerts to hand-triage — and a real finding is easy to rubber-stamp in a queue of 452 known-benign ones. The dirs named on
Date.now()alone were genuinely predictable; therandomUUIDones weren't, but they trained the same pattern.2. Two advisories Dependabot can never fix. Next pins
postcss8.4.31 as a direct dependency andsharp^0.34.5 as an optional one, so Dependabot can't raise either without a Next release doing it first. 16.2.11 doesn't — it still pins both.3. The audit job couldn't see the docs site. Both
pnpm auditsteps run at the repo root.website/keeps its own lockfile and isn't a workspace member, so neither step ever scanned it. That blind spot is why (2) sat open long enough to need a manual override.What this does
test:fs.mkdtemp(atomic, random suffix, mode 0700). Removes 16randomUUIDimports it orphaned.chore(deps):postcss→ ^8.5.22,sharp→ ^0.35.3ci(security):pnpm audit --dir websiteso the site is covered going forwardWhy a plain floor and not a version-ranged selector.
postcss@<8.5.10was the first instinct, since it lapses on its own once Next moves past it. It's the wrong tool: it pins the override to one advisory's floor and silently stops applying when the next advisory raises that floor. GHSA-6g55-p6wh-862q landed while this branch was open and moved postcss's patched floor to 8.5.12 — under the ranged selector, a dependency pinning 8.5.11 resolved to 8.5.11 and stayed vulnerable:A floor can't under-match. It also already covers the new advisory — 8.5.22 is past 8.5.12.
Proof
mkdtemp genuinely silences the rule — measured, not assumed. Main has 33 files calling
mkdtempand 34 files carryingjs/insecure-temporary-filealerts, and the intersection is empty: every mkdtemp file is alert-free, every alerted file uses the raw pattern. CodeQL breaks taint at mkdtemp's return value, not its argument.Dependency fix,
pnpm auditonwebsite/:The new CI step was verified against the pre-fix lockfile rather than just asserted — it reports the two advisories it would have caught (
2 vulnerabilities found / 1 moderate | 1 high) and is clean against the fixed tree. It genuinely readswebsite/pnpm-lock.yaml, not the root: with a vulnerable website lockfile and a clean root it exits 1, and a missing website lockfile is a hardERR_PNPM_AUDIT_NO_LOCKFILErather than a false clean.The step is blocking on the weekly schedule and on pushes to main, advisory on PRs — the same rule as the published-dependency audit. An always-advisory step would only relocate the blind spot: the sweep would stay green with a live advisory and someone would have to read the log of a passing run to notice. It carries
!cancelled()so a hard failure in the audits above can't silently skip it.Behavior is unchanged.
mkdtempcreates the same root the oldmkdirdid, and no assertion depended on the root being absent — everytoBe(false)targets a path inside the temp dir. The "paths with spaces" installer tests keep their spaces (mkdtemp(…, 'openspec bash test ')→openspec bash test rv4GOq), confirmed green onwindows-pwsh.One deliberate behavior change worth naming: temp dirs are now mode 0700 instead of 0755. That's the point of the hardening, and nothing in the suite depends on the old mode, but it would matter to a future test that spawns a subprocess under a different uid.
Review notes
Four adversarial reviews (correctness, security, regression, final pre-merge) found no defects in the change. The fourth caught the ranged-selector gap fixed above. What they verified, since these were the plausible ways this could go wrong:
Merging alongside fix(archive): keep the delta spec's Purpose in a new main spec #1431 is safe — zero file overlap, and both merge orders produce a byte-identical tree. The combined tree builds and passes 2217 tests, with zero raw temp-dir patterns left anywhere: together the two PRs finish the job. Generated files stay consistent (
generate-skillsshandregen-parity-hashesboth produce an emptygit statuson the merged tree).flake.nix
pnpmDepshash is untouched —./websiteisn't in the flake's fileset, the Nix job's path filter doesn't matchwebsite/package.json, and the root lockfile is byte-identical to main.Published tarball unaffected —
npm pack --dry-run: 0test/entries, 0website/entries.Golden hashes safe —
skill-templates-parity,skillssh-parity,vocabulary-sweep,command-registryall pass; nothing undersrc/,skills/, orschemas/changed.The pnpm workspace-scoping trap is dodged — pnpm ignores
pnpm.overridesin non-root workspace packages, which would have made this a silent no-op.website/has its own lockfile and no rootpnpm-workspace.yaml, so it is the install root and the field is honored (confirmed in the generated lockfile).Supply chain: exactly 2 new packages,
@img/sharp-freebsd-wasm32and@img/sharp-webcontainers-wasm32— same publisher, optional, platform-gated, never installed on CI. Zero removed.The prototype-pollution dismissals are true false positives — 15 bypass attempts against
setNestedValue(__proto__, nested__proto__,constructor.prototype, case variants, unicode homoglyphs, inherited accessors) all failed to pollute.Notes
test/core/archive.test.ts:26is deliberately not converted here. It's the one remaining raw-pattern site, and fix(archive): keep the delta spec's Purpose in a new main spec #1431 already converts it. Fixing it here would only create a conflict. Whichever merges first, the other rebases cleanly.sharpis dead weight — the site isoutput: 'export'with zeronext/imageusage, so it's never invoked. Dropping it outright would remove ~30@img/sharp-*packages and the CVE surface entirely. I kept the boring override, since excluding it encodes an assumption a future contributor could silently break. Worth a follow-up if you want the smaller install.pnpm lintiseslint src/), sotsc --noEmitis the only static net under this refactor. Pre-existing, not introduced here.🤖 Generated with Claude Code
Summary by CodeRabbit
Tests
Chores