Skip to content

chore(security): create test temp dirs with mkdtemp and override two pinned CVEs - #1432

Merged
clay-good merged 3 commits into
mainfrom
chore/security-temp-file-hardening
Jul 23, 2026
Merged

chore(security): create test temp dirs with mkdtemp and override two pinned CVEs#1432
clay-good merged 3 commits into
mainfrom
chore/security-temp-file-hardening

Conversation

@clay-good

@clay-good clay-good commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

Status: Ready. Rebased on main after #1429. Full suite green (2196 tests / 111 files), lint and tsc clean, site builds, all CI checks pass including Windows.

Closes the repo's last two open Dependabot alerts, and removes the recurring source of 442 CodeQL dismissals.

What was wrong

1. A green CodeQL dashboard that wasn't earned. The repo shows 0 open alerts and 467 dismissed — 464 of them bulk-dismissed on a single day. 452 of those are one rule, js/insecure-temporary-file, fired by one copy-pasted test idiom:

testDir = path.join(os.tmpdir(), `openspec-test-${randomUUID()}`);
await fs.mkdir(testDir, { recursive: true });

Dismissing them is a treadmill — every new suite that copies the idiom mints fresh alerts to hand-triage — and a real finding is easy to rubber-stamp in a queue of 452 known-benign ones. The dirs named on Date.now() alone were genuinely predictable; the randomUUID ones weren't, but they trained the same pattern.

2. Two advisories Dependabot can never fix. Next pins postcss 8.4.31 as a direct dependency and sharp ^0.34.5 as an optional one, so Dependabot can't raise either without a Next release doing it first. 16.2.11 doesn't — it still pins both.

3. The audit job couldn't see the docs site. Both pnpm audit steps run at the repo root. website/ keeps its own lockfile and isn't a workspace member, so neither step ever scanned it. That blind spot is why (2) sat open long enough to need a manual override.

What this does

Commit Change
test: 41 temp-dir call sites across 30 files → fs.mkdtemp (atomic, random suffix, mode 0700). Removes 16 randomUUID imports it orphaned.
chore(deps): pnpm override floors: postcss → ^8.5.22, sharp → ^0.35.3
ci(security): Adds pnpm audit --dir website so the site is covered going forward

Why a plain floor and not a version-ranged selector. postcss@<8.5.10 was the first instinct, since it lapses on its own once Next moves past it. It's the wrong tool: it pins the override to one advisory's floor and silently stops applying when the next advisory raises that floor. GHSA-6g55-p6wh-862q landed while this branch was open and moved postcss's patched floor to 8.5.12 — under the ranged selector, a dependency pinning 8.5.11 resolved to 8.5.11 and stayed vulnerable:

ranged selector (postcss@<8.5.10) + dep pinning 8.5.11  ->  postcss@8.5.11   (vulnerable)
plain floor     (postcss: ^8.5.22) + dep pinning 8.5.11  ->  postcss@8.5.22   (safe)

A floor can't under-match. It also already covers the new advisory — 8.5.22 is past 8.5.12.

Proof

mkdtemp genuinely silences the rule — measured, not assumed. Main has 33 files calling mkdtemp and 34 files carrying js/insecure-temporary-file alerts, and the intersection is empty: every mkdtemp file is alert-free, every alerted file uses the raw pattern. CodeQL breaks taint at mkdtemp's return value, not its argument.

Dependency fix, pnpm audit on website/:

before:  2 vulnerabilities (1 moderate | 1 high)
after:   No known vulnerabilities found

The new CI step was verified against the pre-fix lockfile rather than just asserted — it reports the two advisories it would have caught (2 vulnerabilities found / 1 moderate | 1 high) and is clean against the fixed tree. It genuinely reads website/pnpm-lock.yaml, not the root: with a vulnerable website lockfile and a clean root it exits 1, and a missing website lockfile is a hard ERR_PNPM_AUDIT_NO_LOCKFILE rather than a false clean.

The step is blocking on the weekly schedule and on pushes to main, advisory on PRs — the same rule as the published-dependency audit. An always-advisory step would only relocate the blind spot: the sweep would stay green with a live advisory and someone would have to read the log of a passing run to notice. It carries !cancelled() so a hard failure in the audits above can't silently skip it.

Behavior is unchanged. mkdtemp creates the same root the old mkdir did, and no assertion depended on the root being absent — every toBe(false) targets a path inside the temp dir. The "paths with spaces" installer tests keep their spaces (mkdtemp(…, 'openspec bash test ')openspec bash test rv4GOq), confirmed green on windows-pwsh.

One deliberate behavior change worth naming: temp dirs are now mode 0700 instead of 0755. That's the point of the hardening, and nothing in the suite depends on the old mode, but it would matter to a future test that spawns a subprocess under a different uid.

Review notes

Four adversarial reviews (correctness, security, regression, final pre-merge) found no defects in the change. The fourth caught the ranged-selector gap fixed above. What they verified, since these were the plausible ways this could go wrong:

  • Merging alongside fix(archive): keep the delta spec's Purpose in a new main spec #1431 is safe — zero file overlap, and both merge orders produce a byte-identical tree. The combined tree builds and passes 2217 tests, with zero raw temp-dir patterns left anywhere: together the two PRs finish the job. Generated files stay consistent (generate-skillssh and regen-parity-hashes both produce an empty git status on the merged tree).

  • flake.nix pnpmDeps hash is untouched./website isn't in the flake's fileset, the Nix job's path filter doesn't match website/package.json, and the root lockfile is byte-identical to main.

  • Published tarball unaffectednpm pack --dry-run: 0 test/ entries, 0 website/ entries.

  • Golden hashes safeskill-templates-parity, skillssh-parity, vocabulary-sweep, command-registry all pass; nothing under src/, skills/, or schemas/ changed.

  • The pnpm workspace-scoping trap is dodged — pnpm ignores pnpm.overrides in non-root workspace packages, which would have made this a silent no-op. website/ has its own lockfile and no root pnpm-workspace.yaml, so it is the install root and the field is honored (confirmed in the generated lockfile).

  • Supply chain: exactly 2 new packages, @img/sharp-freebsd-wasm32 and @img/sharp-webcontainers-wasm32 — same publisher, optional, platform-gated, never installed on CI. Zero removed.

  • The prototype-pollution dismissals are true false positives — 15 bypass attempts against setNestedValue (__proto__, nested __proto__, constructor.prototype, case variants, unicode homoglyphs, inherited accessors) all failed to pollute.

Notes

  • test/core/archive.test.ts:26 is deliberately not converted here. It's the one remaining raw-pattern site, and fix(archive): keep the delta spec's Purpose in a new main spec #1431 already converts it. Fixing it here would only create a conflict. Whichever merges first, the other rebases cleanly.
  • sharp is dead weight — the site is output: 'export' with zero next/image usage, so it's never invoked. Dropping it outright would remove ~30 @img/sharp-* packages and the CVE surface entirely. I kept the boring override, since excluding it encodes an assumption a future contributor could silently break. Worth a follow-up if you want the smaller install.
  • Test files get no lint coverage (pnpm lint is eslint src/), so tsc --noEmit is the only static net under this refactor. Pre-existing, not introduced here.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Tests

    • Improved temporary workspace setup across the test suite for more reliable, isolated test execution.
  • Chores

    • Added dependency version overrides for the documentation site.
    • Expanded security auditing to include documentation site dependencies, with appropriate pull request handling.

@clay-good
clay-good requested a review from TabishB as a code owner July 23, 2026 14:32
@coderabbitai

coderabbitai Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The test suites now create temporary directories with fs.mkdtemp or fs.mkdtempSync instead of manually generated paths and explicit directory creation. Website package configuration adds pnpm overrides for transitive postcss and sharp versions, and CI audits the website dependencies.

Changes

Test temporary-directory setup

Layer / File(s) Summary
Command and artifact test workspaces
test/commands/*, test/core/artifact-graph/*, test/core/commands/*
Command, schema, and artifact tests use mkdtemp APIs for temporary workspaces.
Completion and installer test homes
test/core/completions/*
Completion and shell installer tests use mkdtemp for standard and space-containing paths.
Core command and detection workspaces
test/core/{global-config,init,legacy-cleanup,list,migration,profile-sync-drift,shared,update,view}.test.ts
Core tests replace UUID- and timestamp-based setup with mkdtemp calls.
Telemetry and utility test workspaces
test/telemetry/*, test/utils/*
Telemetry and utility tests adopt mkdtemp while retaining existing cleanup and assertions.

Website dependency and audit configuration

Layer / File(s) Summary
Website package and security audit configuration
website/package.json, .github/workflows/security.yml
Website package overrides update postcss and sharp ranges, while the security workflow runs a high-severity audit for website, non-blocking on pull requests.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

Suggested reviewers: tabishb

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the two main changes: switching test temp dirs to mkdtemp and addressing two website CVEs via dependency overrides.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch chore/security-temp-file-hardening

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 23, 2026

Copy link
Copy Markdown

Deploying openspec-docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: 3980e6f
Status: ✅  Deploy successful!
Preview URL: https://8176e9d0.openspec-docs.pages.dev
Branch Preview URL: https://chore-security-temp-file-har.openspec-docs.pages.dev

View logs

Every one of these suites built its temp directory by hand:

    testDir = path.join(os.tmpdir(), `openspec-test-${randomUUID()}`);
    await fs.mkdir(testDir, { recursive: true });

That check-then-create is what CodeQL's js/insecure-temporary-file flags,
and it accounted for 452 of the repo's 467 dismissed code-scanning alerts.
Dismissing them one by one is a treadmill: every new suite that copies the
idiom mints fresh alerts, and a real finding is easy to lose in that volume.

fs.mkdtemp creates the directory atomically at mode 0700 with a random
suffix, so there is no window to pre-empt and no name to guess. The suites
that already used it are the evidence this silences the rule: 33 of the 34
files calling mkdtemp carry zero alerts. The lone exception, archive.test.ts,
was scanned one commit before its own mkdtemp fix landed and is left to that
change rather than conflicting with it.

The dirs named on Date.now() alone were genuinely predictable; the randomUUID
ones were not, but they trained the same copy-paste. Both are gone now.

Behavior is unchanged: mkdtemp creates the root the old mkdir created, and
no assertion depended on the root being absent. Verified with the full suite
(2196 tests, 111 files).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@clay-good
clay-good force-pushed the chore/security-temp-file-hardening branch 2 times, most recently from 6ecb89a to 6c1fe49 Compare July 23, 2026 15:14

@alfred-openspec alfred-openspec left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The temp-dir refactor and lockfile overrides check out at exact head 6c1fe492: frozen installs, the root build and full 2,196-test suite, both website audits, the 87-page static build, the resolved dependency tree, and the hosted matrix all pass.

One policy blocker remains: the new website audit uses continue-on-error: true on every event. That means the weekly and main security runs can stay green when a new high-severity website advisory appears, so the step does not fully close the monitoring blind spot it describes. Please make it advisory only on pull requests and blocking on scheduled, main, and manual runs, matching the published-dependency audit. The site exports static output, but Next, PostCSS, and Sharp execute in the CI build and are part of the build supply chain.

clay-good and others added 2 commits July 23, 2026 10:50
Two advisories on the docs site have no Dependabot PR and never will:
Next pins postcss at 8.4.31 as a direct dependency and sharp at ^0.34.5 as
an optional one, so Dependabot cannot raise either without a Next release
that does it first. 16.2.11 does not — it still pins both. Left alone these
sit open indefinitely.

  postcss  8.4.31 -> 8.5.22   GHSA-qx2v-qp2m-jg93  (XSS via unescaped </style>)
  sharp    0.34.5 -> 0.35.3   GHSA-f88m-g3jw-g9cj  (4 libvips CVEs)

A version-ranged selector (`postcss@<8.5.10`) was the first instinct, since it
lapses on its own once Next moves past it. It is the wrong tool: it pins the
override to one advisory's floor, and silently stops applying when the next
advisory raises that floor. GHSA-6g55-p6wh-862q landed while this branch was
open and moved postcss's patched floor to 8.5.12 — under the ranged selector a
dependency pinning 8.5.11 would have resolved to 8.5.11 and stayed vulnerable.
A plain floor cannot under-match, so that is what this uses.

The floor also covers the new advisory: 8.5.22 is past 8.5.12. postcss dedupes
to the single copy the site already had for Tailwind.

These are the last two open Dependabot alerts on the repo. `pnpm audit` on
website/ goes from 2 advisories to "No known vulnerabilities found". The site
builds clean, OG image generation included, and the package set grows by
exactly two platform-gated wasm32 binaries that never install on CI.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both `pnpm audit` steps run at the repo root. The docs site keeps its own
lockfile and is not a workspace member, so neither step could see it — the
root audit passed all the way through while postcss and sharp sat open in
website/. That blind spot is why they needed a manual override to find.

Blocking rule copied from the published-dependency audit: advisory on pull
requests, blocking on the weekly schedule and on pushes to main. An
always-advisory step would only relocate the blind spot — the sweep would stay
green with a live advisory and someone would have to read the log of a passing
run to notice.

`!cancelled()` because the two audits above can fail hard. Without it a root
advisory would skip this step entirely, in exactly the situation where the
site's own state matters most.

Verified against the pre-fix lockfile — the step reports the two advisories it
would have caught:

  2 vulnerabilities found
  Severity: 1 moderate | 1 high

and reports "No known vulnerabilities found" against the fixed one. Confirmed
it reads website/pnpm-lock.yaml and not the root: with a vulnerable website
lockfile and a clean root, it exits 1; a missing website lockfile is a hard
ERR_PNPM_AUDIT_NO_LOCKFILE rather than a false clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@clay-good
clay-good force-pushed the chore/security-temp-file-hardening branch from 6c1fe49 to 3980e6f Compare July 23, 2026 15:51

@alfred-openspec alfred-openspec left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The follow-up closes the policy gap: website audits are now advisory only on pull requests and blocking on scheduled, main, and manual runs. Re-verified exact head 3980e6f with frozen install, clean production/full website audits, the 87-page static build, resolved dependency overrides, and the hosted Audit plus Actions analysis checks. Approved; a real second-human approval is still required before merge.

@clay-good
clay-good added this pull request to the merge queue Jul 23, 2026
Merged via the queue into main with commit a874d1d Jul 23, 2026
17 checks passed
@clay-good
clay-good deleted the chore/security-temp-file-hardening branch July 23, 2026 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants