Skip to content

Fix security-scanner and duplicate-detector false positives#63

Open
abk1969 wants to merge 13 commits into
braedonsaunders:mainfrom
abk1969:security-scanner-precision
Open

Fix security-scanner and duplicate-detector false positives#63
abk1969 wants to merge 13 commits into
braedonsaunders:mainfrom
abk1969:security-scanner-precision

Conversation

@abk1969

@abk1969 abk1969 commented Jul 11, 2026

Copy link
Copy Markdown

Summary

The Security Scanner and duplicate-function detector in detectSecurity/Parser.detectDuplicates (embedded in index.html) were producing false positives when run against a real external project: no awareness of file role (test/fixture/tooling vs. shipped code), no word-boundary precision on several regexes, and a commonNames allowlist missing Next.js App Router conventions.

This PR adds a shared isNonProductionPath() classifier (composed from the existing isArchitectureTestFile/isArchitectureFixtureFile helpers, previously only wired into architecture/dead-code analysis) and applies seven targeted precision fixes:

  • Hardcoded Secret — excluded on test/fixture/tooling/docs paths.
  • Command/Shell Execution (JS, VBA, Python) — same path exclusion, plus .exec( narrowed to child_process/cp receivers so it no longer matches unrelated regex.exec().
  • SQL Injection Risk — anchored to a real query/execute/raw call receiver instead of matching bare English words; this also fixes a false negative where the old regex was order-dependent and missed SELECT ... ${id}.
  • XSS Vulnerability — skips dangerouslySetInnerHTML when every occurrence in a file resolves to a pure string literal (e.g. static i18n copy), while still flagging variable/expression interpolation; also gated on non-production paths, consistent with the other rules.
  • Function Constructor — removed an unanchored includes('Function(') substring check that matched things like a comment mentioning getFunction(x); kept only the anchored \bnew\s+Function\s*\( regex.
  • Debug Statements — severity downgraded to info for server-only code (via the existing isArchitectureBackendPath classifier), since a server-side console.log never reaches the browser.
  • Duplicate function names — added Next.js route-handler/metadata export names (GET, POST, generateMetadata, etc.) to commonNames, so framework-mandated exports repeated across route files are no longer flagged as a naming conflict. The separate structural type:'code' duplicate detector is intentionally left untouched — genuinely identical bodies are still worth flagging regardless of name.

Each fix is backed by a new fixture pair (false-positive case + true-positive case) under tests/fixtures/security-precision-world/, with corresponding tests in the new tests/security-precision.test.mjs, following the same VM-harness pattern as tests/codeflow-golden.test.mjs.

Design doc: docs/superpowers/specs/2026-07-11-security-scanner-precision-design.md
Implementation plan: docs/superpowers/plans/2026-07-11-security-scanner-precision.md

Known limitations (out of scope for this PR)

  • No full data-flow/taint analysis — the fixes stay heuristic/regex-based, matching the rest of the analyzer's approach.
  • commonNames additions cover only Next.js App Router conventions; other frameworks aren't addressed.
  • isNonProductionPath matches directory segments anywhere in the path (scripts/, tools/, tooling/, docs/), so a real secret in a shipped src/scripts/ folder would be suppressed — this is the same tradeoff the design doc explicitly accepted.
  • The new 'info' severity (Debug Statements) isn't yet tallied in the UI's High/Medium/Low badge counts, and isn't in the severity sort map at index.html:1606 — display-layer follow-ups, no UI work was in scope here.

Test plan

  • node --test from repo root — 47/47 pass (all pre-existing tests unchanged + 9 new tests in tests/security-precision.test.mjs)
  • tests/codeflow-golden.test.mjs stays green, unmodified
  • Verified in a clean git worktree (not just the working directory) to rule out local-state false positives

abk1969 and others added 13 commits July 11, 2026 23:47
Diagnosed via an external run against AIROI where all 10 reported
security issues and the duplicate POST/GET finding were false
positives, each traced to a specific regex/allowlist gap in the
analyzer embedded in index.html.
…on fixes

Task-by-task plan executing the approved design spec, each fix verified
against the real analyzer (via card/lib/analyzer.js) before being written
down, so every "run the test, expect it to fail/pass" step is grounded in
actual measured behavior rather than assumption.
… literal-safety, not just the first

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…g the other rules and the documented behavior

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ated architecture-diagram grouping issue

The previous fixture (tests/fixtures/security-precision-world/test/preview.test.tsx)
was correct in intent but its placement — a second file added to an
already-populated fixture subdirectory — triggered a pre-existing, unrelated
fragility in the architecture-diagram block-grouping logic (out of scope for
this branch). Moved to a root-level markdown fixture, which exercises the
same isNonProductionPath code path via its file extension instead of its
directory, without perturbing directory topology elsewhere in the repo.
@vercel

vercel Bot commented Jul 11, 2026

Copy link
Copy Markdown

@abk1969 is attempting to deploy a commit to the braedonsaunders' projects Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant