diff --git a/CHANGELOG.md b/CHANGELOG.md index 88e4d1b..911df4c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,16 @@ All notable changes to FixMap are documented here. +## 0.7.3 - 2026-07-26 + +### Added + +- `fixmap verify --report --diff ` closes the loop after an edit. It compares a saved plan against the diff that followed and reports five things: edits in generated or retired locations that the next build discards, files the change needed that the plan never ranked, an untouched leading file, source moving with no test moving, and risk areas the plan never flagged. Both inputs are things the user already has, so nothing is executed and no repository code runs. Only a discarded edit exits non-zero — that one is wrong regardless of the task, while everything else is advisory, because a plan can be wrong and a change can still be right. Available as `verifyPlan` from the core package and with `--format json`. + +### Fixed + +- Test routes list only the tests each command can actually run. `findRelatedTests` ran once and its result was assigned to every route, so a report claimed `npm --prefix packages/core run test` would exercise `packages/action/test/runner.test.ts`, which that command never reaches. On this repository all three routes carried an identical eight files spanning three packages. Related files are now scoped to the route's package directory; a repository-root script keeps everything. + ## 0.7.2 - 2026-07-26 ### Added diff --git a/README.md b/README.md index 19a60fb..5e107fb 100644 --- a/README.md +++ b/README.md @@ -97,6 +97,36 @@ or path from this file in the task to raise it. It distinguishes the cases that actually differ: the file was ranked, it scored below the cutoff, it was deliberately excluded (a test, a lockfile, generated output whose source was ranked instead), or the scan never saw it. When a scan hits its file limit, it says so rather than implying the path does not exist. Add `--format json` for the machine-readable form. +### Verify the change afterwards + +`plan` answers where to start. `verify` answers whether the change that followed matches the plan — by comparing the saved report against a real git diff: + +```bash +npx -y @aryam/fixmap@latest plan --issue "password reset emails fail" \ + --format json --output fixmap-report.json + +# ...make the change... + +npx -y @aryam/fixmap@latest verify --report fixmap-report.json --diff main...HEAD +``` + +```text +FixMap verified 3 changed files against the plan and raised 1 error and 2 warnings. + +- **error** A file was edited in a generated or retired location. A build regenerates + these, so the change will be lost. Edit the source they are produced from. + - `dist/auth/reset-password.js` +- **warning** One file changed that the plan did not rank. Either the task grew beyond + the original description, or the ranking missed them — worth checking which. + - `src/billing/charge.ts` +- **warning** Code changed but no test did. The plan routed this test as most related. + - `test/reset-password.test.ts` +``` + +It checks five things: edits in generated or retired locations, files the change needed that the plan never ranked, an untouched leading file, source moving with no test moving, and risk areas the plan never flagged. Nothing is executed — both inputs are things you already have. + +Only a discarded edit exits non-zero, because that one is wrong regardless of the task. Everything else is advisory: a plan can be wrong and a change can still be right, and FixMap reports the gap rather than judging it. `verify` is CLI-only for now; MCP and Action support follow. + ### MCP server FixMap exposes one stdio tool, `fixmap_plan`, so an agent can request the same report directly. @@ -167,7 +197,7 @@ jobs: with: fetch-depth: 0 - id: fixmap - uses: aryamthecodebreaker/FixMap@v0.7.2 + uses: aryamthecodebreaker/FixMap@v0.7.3 with: github-token: ${{ secrets.GITHUB_TOKEN }} ``` @@ -247,11 +277,11 @@ Read the full [benchmark methodology and scanner measurements](docs/BENCHMARKS.m npm run evaluate:heldout ``` -## What changed in v0.7.2 +## What changed in v0.7.3 -`--explain ` answers the question a ranked list cannot: why the file you expected is missing. It separates being ranked, scoring below the cutoff, being deliberately excluded, and never being scanned. +`fixmap verify` closes the loop: it compares a saved plan against the diff that followed, flagging edits a build will discard, files the change needed that the plan never ranked, source moving without tests, and risk the plan never mentioned. -Every published rate now carries a confidence interval, because at twelve cases one result flipping moves Top-3 by eight points. And both evaluations report how often a wrong file ranks *first* while the right one sits lower — the regression suite's 100% Top-3 was concealing a 40% misleading rate, which is what an agent actually pays for. +Test routes now list only the tests each command can actually run. Every route previously carried the same repository-wide list, so a report claimed `npm --prefix packages/core run test` would exercise another package's tests. [Inspect the changelog](CHANGELOG.md) · [See the held-out results](benchmarks/heldout/README.md) · [See every regression ranking](benchmarks/external/README.md) · [Audit the efficiency assumptions](docs/BENCHMARKS.md) diff --git a/apps/web/package.json b/apps/web/package.json index 92c4fa8..44fa08b 100644 --- a/apps/web/package.json +++ b/apps/web/package.json @@ -15,7 +15,7 @@ "typecheck": "tsc --noEmit" }, "dependencies": { - "@aryam/fixmap-core": "0.7.2", + "@aryam/fixmap-core": "0.7.3", "next": "16.2.11", "react": "19.2.7", "react-dom": "19.2.7" diff --git a/docs/LAUNCH_KIT.md b/docs/LAUNCH_KIT.md index 3284e96..c7a79eb 100644 --- a/docs/LAUNCH_KIT.md +++ b/docs/LAUNCH_KIT.md @@ -207,7 +207,7 @@ Marketplace: https://github.com/marketplace/actions/fixmap npm: https://www.npmjs.com/package/@aryam/fixmap -Release: https://github.com/aryamthecodebreaker/FixMap/releases/tag/v0.7.2 +Release: https://github.com/aryamthecodebreaker/FixMap/releases/tag/v0.7.3 I would especially value feedback on the ranking explanations and which repository signals would make FixMap more useful before an agent starts editing. diff --git a/package-lock.json b/package-lock.json index 1888301..a2ee32a 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "fixmap-workspace", - "version": "0.7.2", + "version": "0.7.3", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "fixmap-workspace", - "version": "0.7.2", + "version": "0.7.3", "license": "MIT", "workspaces": [ "packages/*", @@ -30,7 +30,7 @@ "name": "@fixmap/web", "version": "0.0.0", "dependencies": { - "@aryam/fixmap-core": "0.7.2", + "@aryam/fixmap-core": "0.7.3", "next": "16.2.11", "react": "19.2.7", "react-dom": "19.2.7" @@ -8529,18 +8529,18 @@ }, "packages/action": { "name": "@fixmap/action", - "version": "0.7.2", + "version": "0.7.3", "license": "MIT", "dependencies": { - "@aryam/fixmap-core": "0.7.2" + "@aryam/fixmap-core": "0.7.3" } }, "packages/cli": { "name": "@aryam/fixmap", - "version": "0.7.2", + "version": "0.7.3", "license": "MIT", "dependencies": { - "@aryam/fixmap-core": "0.7.2", + "@aryam/fixmap-core": "0.7.3", "@modelcontextprotocol/sdk": "1.29.0" }, "bin": { @@ -8552,7 +8552,7 @@ }, "packages/core": { "name": "@aryam/fixmap-core", - "version": "0.7.2", + "version": "0.7.3", "license": "MIT", "devDependencies": {} } diff --git a/package.json b/package.json index 848e84f..52708c8 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "fixmap-workspace", - "version": "0.7.2", + "version": "0.7.3", "private": true, "description": "Local-first repo context for coding agents: paste a GitHub issue URL to get ranked files, test routes, and risks.", "license": "MIT", diff --git a/packages/action/package.json b/packages/action/package.json index 8d2f99b..0770b7e 100644 --- a/packages/action/package.json +++ b/packages/action/package.json @@ -1,6 +1,6 @@ { "name": "@fixmap/action", - "version": "0.7.2", + "version": "0.7.3", "description": "GitHub Action wrapper for FixMap pull request reports.", "private": true, "license": "MIT", @@ -11,6 +11,6 @@ "typecheck": "tsc -p tsconfig.json --noEmit" }, "dependencies": { - "@aryam/fixmap-core": "0.7.2" + "@aryam/fixmap-core": "0.7.3" } } diff --git a/packages/cli/package.json b/packages/cli/package.json index 268ea5b..e8835dd 100644 --- a/packages/cli/package.json +++ b/packages/cli/package.json @@ -1,6 +1,6 @@ { "name": "@aryam/fixmap", - "version": "0.7.2", + "version": "0.7.3", "mcpName": "io.github.aryamthecodebreaker/fixmap", "description": "Local-first CLI and MCP server mapping GitHub issue URLs, tasks, and diffs to ranked files, tests, and risks.", "license": "MIT", @@ -46,7 +46,7 @@ "typecheck": "tsc -p tsconfig.json --noEmit" }, "dependencies": { - "@aryam/fixmap-core": "0.7.2", + "@aryam/fixmap-core": "0.7.3", "@modelcontextprotocol/sdk": "1.29.0" }, "engines": { diff --git a/packages/cli/src/cli-runner.ts b/packages/cli/src/cli-runner.ts index 463d8e7..92476e4 100644 --- a/packages/cli/src/cli-runner.ts +++ b/packages/cli/src/cli-runner.ts @@ -5,6 +5,8 @@ import { fileURLToPath } from "node:url"; import { explainFile, renderJsonReport, + renderVerifyMarkdown, + verifyPlan, renderMarkdownReport, scanRepo, type FileExplanation, @@ -22,6 +24,7 @@ export type CliOptions = { format: "markdown" | "json"; output?: string | undefined; explainPath?: string | undefined; + reportPath?: string | undefined; unknownArgs: string[]; invalidValues: string[]; }; @@ -43,10 +46,12 @@ Usage: fixmap plan --issue "Fix login" --repo https://github.com/owner/repository fixmap plan --diff main...HEAD fixmap plan --base main --head HEAD --format json + fixmap verify --report fixmap-report.json --diff main...HEAD fixmap mcp Commands: plan Generate a FixMap report for a task or diff + verify Compare a saved report against the diff that followed it mcp Run FixMap as an MCP server over stdio for AI coding agents Options: @@ -58,6 +63,7 @@ Options: --format Output format: markdown (default) or json --output Write the report to a file instead of stdout --explain Explain why one file was ranked where it was, or left out + --report Verify command only: the JSON report the change was planned from --help, -h Show this help --version, -v Show the FixMap version `; @@ -88,7 +94,7 @@ export async function runCli(args: string[], dependencies: CliDependencies = {}) } const options = parseArgs(args); - if (options.command !== "plan") { + if (options.command !== "plan" && options.command !== "verify") { stderr(`Unknown command: ${options.command || "(none)"}\n\n${USAGE}`); return 1; } @@ -104,6 +110,10 @@ export async function runCli(args: string[], dependencies: CliDependencies = {}) stderr(`${sections.join("\n")}\n\n${USAGE}`); return 1; } + if (options.command === "verify") { + return runVerify(options, { stdout, stderr }); + } + if (!options.issueText && !options.diffSpec && !options.baseRef) { stderr("Provide --issue, --diff, or --base/--head so FixMap has a task signal.\n"); return 1; @@ -185,6 +195,7 @@ export function parseArgs(args: string[]): CliOptions { let format: "markdown" | "json" = "markdown"; let output: string | undefined; let explainPath: string | undefined; + let reportPath: string | undefined; const unknownArgs: string[] = []; const invalidValues: string[] = []; @@ -236,6 +247,10 @@ export function parseArgs(args: string[]): CliOptions { } else { invalidValues.push(`--format received ${JSON.stringify(value ?? "(missing)")}; expected "markdown" or "json"`); } + } else if (arg === "--report") { + consumeValue(); + if (value?.trim()) reportPath = value.trim(); + else invalidValues.push("--report requires a path to a FixMap JSON report"); } else if (arg === "--explain") { consumeValue(); if (value?.trim()) explainPath = value.trim(); @@ -259,6 +274,7 @@ export function parseArgs(args: string[]): CliOptions { format, output, explainPath, + reportPath, unknownArgs, invalidValues }; @@ -271,6 +287,70 @@ function readVersion(): string { return packageJson.version; } +/** + * Compares a saved plan against the diff that followed it. Both inputs are things the + * user already has, so nothing is executed and no repository code runs. + */ +async function runVerify( + options: CliOptions, + io: { stdout: (text: string) => void; stderr: (text: string) => void } +): Promise { + if (!options.reportPath) { + io.stderr("Provide --report with the JSON report this change was planned from.\n"); + return 1; + } + if (!options.diffSpec && !options.baseRef) { + io.stderr("Provide --diff or --base/--head so FixMap can see what changed.\n"); + return 1; + } + if (/^https?:\/\//i.test(options.repo ?? "")) { + io.stderr("verify needs a local checkout; remote mode cannot resolve a diff.\n"); + return 1; + } + + let report: FixMapReport; + try { + report = JSON.parse(readFileSync(options.reportPath, "utf8")) as FixMapReport; + } catch (error) { + io.stderr( + `Could not read "${options.reportPath}": ${error instanceof Error ? error.message : String(error)}\n` + + "Generate one with: fixmap plan --issue \"...\" --format json --output fixmap-report.json\n" + ); + return 1; + } + if (!Array.isArray(report.contextFiles)) { + io.stderr(`"${options.reportPath}" is not a FixMap JSON report: no contextFiles array.\n`); + return 1; + } + + try { + const repo = await scanRepo({ + repoRoot: options.repo ?? process.cwd(), + diffSpec: options.diffSpec, + baseRef: options.baseRef, + headRef: options.headRef + }); + const unresolvedDiff = repo.diagnostics.find((diagnostic) => diagnostic.code === "diff-unavailable"); + if (unresolvedDiff) { + io.stderr(`${unresolvedDiff.message}\nVerification needs a resolvable diff to compare against.\n`); + return 1; + } + + const result = verifyPlan(report, repo); + io.stdout( + options.format === "json" + ? `${JSON.stringify(result, null, 2)}\n` + : renderVerifyMarkdown(result) + ); + // A generated-location edit is discarded by the next build, so it fails the command + // rather than being reported and ignored. Everything else is advisory. + return result.findings.some((finding) => finding.severity === "error") ? 1 : 0; + } catch (error) { + io.stderr(`${error instanceof Error ? error.message : String(error)}\n`); + return 1; + } +} + /** Renders one file's explanation. The summary carries the answer; reasons show the working. */ function renderExplanation(explanation: FileExplanation): string { const lines = [`# Why ${explanation.path}`, "", explanation.summary]; diff --git a/packages/core/package.json b/packages/core/package.json index f83f38c..2dba7db 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -1,6 +1,6 @@ { "name": "@aryam/fixmap-core", - "version": "0.7.2", + "version": "0.7.3", "description": "Deterministic local-first repository scanner, context ranker, and report renderer for coding agents.", "license": "MIT", "repository": { diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index af71ddc..9dfdff6 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -1,4 +1,5 @@ export { buildFixMapReport } from "./plan.js"; +export { renderVerifyMarkdown, verifyPlan } from "./verify.js"; export { explainFile } from "./explain.js"; export type { FileExplanation } from "./explain.js"; export { @@ -23,5 +24,7 @@ export type { RiskNote, ScanDiagnostic, TaskAnalysis, - TestRoute + TestRoute, + VerifyFinding, + VerifyResult } from "./types.js"; diff --git a/packages/core/src/types.ts b/packages/core/src/types.ts index 7defda6..b206c88 100644 --- a/packages/core/src/types.ts +++ b/packages/core/src/types.ts @@ -112,3 +112,21 @@ export type FixMapReport = { diagnostics: ScanDiagnostic[]; analysis?: TaskAnalysis; }; + +export type VerifyFinding = { + code: + | "edit-in-generated-location" + | "unmapped-change" + | "leading-file-untouched" + | "no-test-changed" + | "new-risk-area"; + severity: "info" | "warning" | "error"; + paths: string[]; + message: string; +}; + +export type VerifyResult = { + summary: string; + changedFiles: string[]; + findings: VerifyFinding[]; +}; diff --git a/packages/core/src/verify.ts b/packages/core/src/verify.ts new file mode 100644 index 0000000..c4de5d9 --- /dev/null +++ b/packages/core/src/verify.ts @@ -0,0 +1,149 @@ +// The second half of the job. `plan` answers "where should I start"; `verify` answers +// "did I finish safely" by comparing the plan against the diff that followed it. +// +// Every check compares two things FixMap already has — a recorded report and a real git +// diff — so nothing here executes repository code, installs anything, or calls a model. +// Findings are evidence about the gap between plan and change, not a correctness verdict: +// a plan can be wrong, a change can be right, and only a human or agent reading both can +// say which. The output says what differs and leaves that judgement alone. + +import { isBackupPath, isGeneratedPath, moduleStem } from "./paths.js"; +import { buildRiskNotes } from "./report.js"; +import type { FixMapReport, RepoMap, VerifyFinding, VerifyResult } from "./types.js"; + +export function verifyPlan(report: FixMapReport, repo: RepoMap): VerifyResult { + const changed = repo.changedFiles; + const findings: VerifyFinding[] = []; + + if (changed.length === 0) { + return { + summary: "No changes to verify: the diff resolved to zero files.", + changedFiles: [], + findings + }; + } + + const planned = new Set(report.contextFiles.map((file) => file.path)); + const fileByPath = new Map(repo.files.map((file) => [file.path, file])); + const isTest = (path: string) => fileByPath.get(path)?.isTest === true; + + // 1. Edits somewhere the next build discards. This is the only finding that is + // wrong regardless of what the task was. + const maintainedStems = new Set( + repo.files + .filter((file) => file.isSource && !isGeneratedPath(file.path) && !isBackupPath(file.path)) + .map((file) => moduleStem(file.path)) + ); + const discardedEdits = changed.filter((path) => + isBackupPath(path) || (isGeneratedPath(path) && maintainedStems.has(moduleStem(path))) + ); + if (discardedEdits.length > 0) { + findings.push({ + code: "edit-in-generated-location", + severity: "error", + paths: discardedEdits, + message: + `${discardedEdits.length === 1 ? "A file was" : `${discardedEdits.length} files were`} edited in a generated or retired location. ` + + "A build regenerates these, so the change will be lost. Edit the source they are produced from." + }); + } + + // 2. Files the change needed that the map never offered. This is the honest measure + // of whether the plan was complete, and it is the finding worth acting on. + const unmapped = changed.filter((path) => + !planned.has(path) && + !isTest(path) && + !discardedEdits.includes(path) && + fileByPath.get(path)?.isSource !== false + ); + if (unmapped.length > 0) { + findings.push({ + code: "unmapped-change", + severity: "warning", + paths: unmapped, + message: + `${unmapped.length === 1 ? "One file" : `${unmapped.length} files`} changed that the plan did not rank. ` + + "Either the task grew beyond the original description, or the ranking missed them — worth checking which." + }); + } + + // 3. The plan's best guess going untouched is worth surfacing, not condemning. It + // happens legitimately when a file is read for context and needs no edit. + const leading = report.contextFiles[0]; + if (leading && !changed.includes(leading.path)) { + findings.push({ + code: "leading-file-untouched", + severity: leading.confidence === "high" ? "warning" : "info", + paths: [leading.path], + message: + `The highest-ranked file was not changed (${leading.confidence} confidence). ` + + "That is expected if it was only read for context, and worth a second look if it was not opened at all." + }); + } + + // 4. Source moved without any test moving. Test routes name what would exercise it. + const changedSource = changed.filter((path) => !isTest(path) && fileByPath.get(path)?.kind === "code"); + const changedTests = changed.filter(isTest); + if (changedSource.length > 0 && changedTests.length === 0) { + const suggested = [...new Set(report.testRoutes.flatMap((route) => route.relatedFiles))].filter(isTest); + findings.push({ + code: "no-test-changed", + severity: "warning", + paths: suggested, + message: + suggested.length > 0 + ? `Code changed but no test did. The plan routed ${suggested.length === 1 ? "this test" : "these tests"} as most related.` + : "Code changed but no test did, and the plan found no related test to point at." + }); + } + + // 5. Risk the plan never mentioned, because the change reached further than the map did. + const plannedAreas = new Set(report.risks.map((risk) => risk.area)); + const newRisks = buildRiskNotes(changed, changed).filter((risk) => !plannedAreas.has(risk.area)); + for (const risk of newRisks) { + findings.push({ + code: "new-risk-area", + severity: risk.severity === "high" ? "warning" : "info", + paths: changed.filter((path) => path.toLowerCase().includes(risk.area.split("-")[0] ?? risk.area)), + message: `The change touches ${risk.area}, which the original plan did not flag: ${risk.reason}.` + }); + } + + return { + summary: buildVerifySummary(changed.length, findings), + changedFiles: changed, + findings + }; +} + +function buildVerifySummary(changedCount: number, findings: VerifyFinding[]): string { + const files = `${changedCount} changed ${changedCount === 1 ? "file" : "files"}`; + if (findings.length === 0) { + return `FixMap verified ${files} against the plan and found nothing to flag.`; + } + const errors = findings.filter((finding) => finding.severity === "error").length; + const warnings = findings.filter((finding) => finding.severity === "warning").length; + const counts = [ + errors > 0 ? `${errors} ${errors === 1 ? "error" : "errors"}` : "", + warnings > 0 ? `${warnings} ${warnings === 1 ? "warning" : "warnings"}` : "" + ].filter(Boolean); + const tail = counts.length > 0 ? counts.join(" and ") : `${findings.length} note${findings.length === 1 ? "" : "s"}`; + return `FixMap verified ${files} against the plan and raised ${tail}.`; +} + +export function renderVerifyMarkdown(result: VerifyResult): string { + const lines = ["# FixMap Verification", "", result.summary, "", "## Findings", ""]; + if (result.findings.length === 0) { + lines.push("- None found"); + } else { + for (const finding of result.findings) { + lines.push(`- **${finding.severity}** ${finding.message}`); + for (const path of finding.paths.slice(0, 8)) { + lines.push(` - \`${path}\``); + } + } + } + lines.push("", "## Changed Files", ""); + lines.push(...(result.changedFiles.length > 0 ? result.changedFiles.map((path) => `- \`${path}\``) : ["- None found"])); + return `${lines.join("\n")}\n`; +} diff --git a/packages/core/test/verify.test.ts b/packages/core/test/verify.test.ts new file mode 100644 index 0000000..f1e25eb --- /dev/null +++ b/packages/core/test/verify.test.ts @@ -0,0 +1,147 @@ +import { describe, expect, it } from "vitest"; +import { renderVerifyMarkdown, verifyPlan } from "../src/verify.js"; +import type { FixMapReport, RepoFile, RepoMap } from "../src/types.js"; + +function source(path: string, overrides: Partial = {}): RepoFile { + return { + path, + extension: `.${path.split(".").pop() ?? "ts"}`, + sizeBytes: 40, + isSource: true, + isTest: false, + kind: "code", + textSample: "export const value = 1;", + ...overrides + }; +} + +const files: RepoFile[] = [ + source("src/auth/reset-password.ts"), + source("src/billing/charge.ts"), + source("dist/auth/reset-password.js"), + source("test/reset-password.test.ts", { isTest: true }) +]; + +function repoWith(changedFiles: string[]): RepoMap { + return { + root: "/repo", + files, + packageScripts: [], + changedFiles, + diffText: "", + packageManager: "npm", + diagnostics: [] + }; +} + +function planFor(topPath: string): FixMapReport { + return { + summary: "", + contextFiles: [{ path: topPath, score: 20, confidence: "high", reasons: ["path matches task terms: reset"] }], + testRoutes: [{ command: "npm run test", reason: "root script", relatedFiles: ["test/reset-password.test.ts"] }], + risks: [], + changedFiles: [], + diagnostics: [] + }; +} + +describe("verifyPlan", () => { + it("says there is nothing to verify when the diff is empty", () => { + const result = verifyPlan(planFor("src/auth/reset-password.ts"), repoWith([])); + + expect(result.findings).toEqual([]); + expect(result.summary).toContain("zero files"); + }); + + it("raises an error for an edit a build will discard", () => { + const result = verifyPlan( + planFor("src/auth/reset-password.ts"), + repoWith(["dist/auth/reset-password.js"]) + ); + const finding = result.findings.find((entry) => entry.code === "edit-in-generated-location"); + + expect(finding?.severity).toBe("error"); + expect(finding?.paths).toEqual(["dist/auth/reset-password.js"]); + expect(finding?.message).toContain("will be lost"); + }); + + it("names files the change needed that the plan never ranked", () => { + const result = verifyPlan( + planFor("src/auth/reset-password.ts"), + repoWith(["src/auth/reset-password.ts", "src/billing/charge.ts"]) + ); + const finding = result.findings.find((entry) => entry.code === "unmapped-change"); + + expect(finding?.paths).toEqual(["src/billing/charge.ts"]); + expect(finding?.severity).toBe("warning"); + }); + + it("does not count a new test as an unmapped change", () => { + const result = verifyPlan( + planFor("src/auth/reset-password.ts"), + repoWith(["src/auth/reset-password.ts", "test/reset-password.test.ts"]) + ); + + expect(result.findings.map((entry) => entry.code)).not.toContain("unmapped-change"); + expect(result.findings.map((entry) => entry.code)).not.toContain("no-test-changed"); + }); + + it("flags source moving with no test moving, and points at the routed test", () => { + const result = verifyPlan( + planFor("src/auth/reset-password.ts"), + repoWith(["src/auth/reset-password.ts"]) + ); + const finding = result.findings.find((entry) => entry.code === "no-test-changed"); + + expect(finding?.paths).toEqual(["test/reset-password.test.ts"]); + }); + + it("notes an untouched leading file without calling it wrong", () => { + const result = verifyPlan( + planFor("src/auth/reset-password.ts"), + repoWith(["src/billing/charge.ts", "test/reset-password.test.ts"]) + ); + const finding = result.findings.find((entry) => entry.code === "leading-file-untouched"); + + expect(finding?.severity).toBe("warning"); + expect(finding?.message).toContain("expected if it was only read for context"); + }); + + it("reports risk the plan never mentioned", () => { + const plan = planFor("src/billing/charge.ts"); + const result = verifyPlan(plan, repoWith(["src/auth/reset-password.ts", "test/reset-password.test.ts"])); + const finding = result.findings.find((entry) => entry.code === "new-risk-area"); + + expect(finding?.message).toContain("authentication"); + }); + + it("stays quiet about risk the plan already flagged", () => { + const plan = planFor("src/auth/reset-password.ts"); + plan.risks = [{ area: "authentication", severity: "high", reason: "authentication-related files are affected" }]; + + const result = verifyPlan(plan, repoWith(["src/auth/reset-password.ts", "test/reset-password.test.ts"])); + + expect(result.findings.map((entry) => entry.code)).not.toContain("new-risk-area"); + }); + + it("reports a clean change as clean", () => { + // A real plan that ranks an auth file also carries the auth risk, so nothing + // about this change is new to it. + const plan = planFor("src/auth/reset-password.ts"); + plan.risks = [{ area: "authentication", severity: "high", reason: "authentication-related files are affected" }]; + + const result = verifyPlan(plan, repoWith(["src/auth/reset-password.ts", "test/reset-password.test.ts"])); + + expect(result.findings).toEqual([]); + expect(result.summary).toContain("nothing to flag"); + }); + + it("renders findings and changed files as markdown", () => { + const result = verifyPlan(planFor("src/auth/reset-password.ts"), repoWith(["dist/auth/reset-password.js"])); + const markdown = renderVerifyMarkdown(result); + + expect(markdown).toContain("# FixMap Verification"); + expect(markdown).toContain("**error**"); + expect(markdown).toContain("`dist/auth/reset-password.js`"); + }); +}); diff --git a/server.json b/server.json index 578de37..094d910 100644 --- a/server.json +++ b/server.json @@ -6,12 +6,12 @@ "url": "https://github.com/aryamthecodebreaker/FixMap", "source": "github" }, - "version": "0.7.2", + "version": "0.7.3", "packages": [ { "registryType": "npm", "identifier": "@aryam/fixmap", - "version": "0.7.2", + "version": "0.7.3", "transport": { "type": "stdio" },