Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions docs/knowledge/investigations.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,26 @@ PR #72 で `--fallback-env` を導入し、PR #74 で stale trailer retry と bo

## Resolved Investigations

### PR #76 deletion-only AI Ratio が 0% になる

- 対象 PR: `#76`
- 対象 commit: `0bc0e77 fix(dashboard): remove preview badge`
- 観測結果: Prompt selection と file attribution は正しく、`packages/dashboard/src/pages/index.astro` は `by_ai: true` でした。一方、commit は `Preview` badge の 1 行削除のみだったため `attribution.lines` は `{ "ai_added": 0, "total_added": 0, "deleted": 1 }` になり、PR Report では `AI Ratio: 0%` と表示されました。
- 原因: `calcAiRatio()` には `totalAddedLines === 0` のとき file-level ratio に fallback する unit test がありました。しかし `buildEntry()` の `resolveMethod()` が先に `method: "none"` を選び、`calcAiRatio()` を呼ぶ前に `ai_ratio: 0` を固定していました。helper 単体の test は存在していたが、実際の entry assembly path を固定していなかったことが漏れの本質です。
- 修正: `lineCounts` があっても `totalAddedLines === 0` の場合は、eligible file が残っていれば `method: "file"` に fallback します。削除のみ、rename のみ、mode change のように added-line denominator がない commit でも、AI が触った file evidence は AI Ratio に反映します。すべての file が generated artifact / `.agentnoteignore` で AI Ratio 対象外なら、従来通り `method: "none"` です。
- Regression coverage: `packages/cli/src/core/entry.test.ts` で、`buildEntry()` が `0 added / N deleted / AI file` を `method: "file"` として保存することを確認します。同じ test file で、eligible file が 0 件の場合だけ `method: "none"` に落ちることも確認します。これは `calcAiRatio()` helper だけでなく、git note に保存される final entry contract を固定するための regression です。

#### Test strengthening policy from PR #76

今回の漏れは「低レベル helper の test はあるが、上位 caller がその helper を迂回する」形でした。今後 attribution / prompt selection / report rendering を触るときは、以下を regression 方針として扱います。

- Helper test だけで完了扱いにしません。`calcAiRatio()`、prompt scoring、fallback predicate のような helper を変更した場合は、少なくとも 1 つ上の public contract (`buildEntry()`、`recordCommitEntry()`、`renderMarkdown()`、CLI output) でも同じ境界を固定します。
- Ratio / attribution の test matrix には、`added > 0` だけでなく `added = 0, deleted > 0`、`added = 0, deleted = 0`、AI file あり、AI file なし、eligible file なし、generated / `.agentnoteignore` 除外を含めます。特に denominator が 0 になる path は、`0%` と `—` と file-level fallback のどれが正しいかを明示します。
- PR Report / Dashboard / CLI 表示は、stored note の `method` と `lines` の組み合わせを fixture で確認します。`method: "line"` だけでなく、`method: "file"` + `lines` あり、`method: "file"` + `lines` なし、`method: "none"` を分けて見る必要があります。
- Generated bundle を持つ package の shared core を触った場合は、source test と bundle sync を同じ PR で確認します。`packages/pr-report` は `packages/cli/src/core/entry.ts` を bundle に含むため、CLI 側の attribution bug は Action output にも影響します。
- Regression test 名には、実ユーザーの症状を入れます。例: `falls back to method=file when line counts have no added lines` のように、内部条件だけでなく「何が守られるか」が分かる名前にします。
- 「100 ケース」などの数を増やすだけでは十分ではありません。今回のような漏れは scenario count ではなく axis の欠落で起きるため、matrix の各 axis が user-visible contract に接続しているかを review checklist に含めます。

### CLI dist tracking policy

- 決定: `packages/pr-report/dist/index.js` は tracked artifact として維持します。`action.yml` が GitHub Action runtime でこの bundle を直接実行するため、repository に存在しないと published Action が動きません。
Expand Down
9 changes: 5 additions & 4 deletions packages/cli/dist/cli.js
Original file line number Diff line number Diff line change
Expand Up @@ -2447,10 +2447,11 @@ function calcAiRatio(files, lineCounts) {
if (eligible.total === 0) return 0;
return Math.round(eligible.ai / eligible.total * PERCENT_DENOMINATOR);
}
function resolveMethod(lineCounts) {
function resolveMethod(files, lineCounts) {
if (!lineCounts) return "file";
if (lineCounts.totalAddedLines === 0) return "none";
return "line";
if (lineCounts.totalAddedLines > 0) return "line";
const eligible = countAiRatioEligibleFiles(files);
return eligible.total > 0 ? "file" : "none";
}
function buildEntry(opts) {
const generatedFiles = new Set(opts.generatedFiles ?? []);
Expand All @@ -2461,7 +2462,7 @@ function buildEntry(opts) {
...generatedFiles.has(path) ? { generated: true } : {},
...aiRatioExcludedFiles.has(path) ? { ai_ratio_excluded: true } : {}
}));
const method = resolveMethod(opts.lineCounts);
const method = resolveMethod(files, opts.lineCounts);
const aiRatio = method === "none" ? 0 : calcAiRatio(files, opts.lineCounts);
const attribution = { ai_ratio: aiRatio, method };
if (opts.lineCounts) {
Expand Down
21 changes: 20 additions & 1 deletion packages/cli/src/core/entry.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -517,7 +517,7 @@ describe("buildEntry", () => {
assert.equal(entry.attribution.lines, undefined);
});

it("sets method=none and ai_ratio=0 when totalAddedLines is 0", () => {
it("falls back to method=file when line counts have no added lines", () => {
const lineCounts: LineCounts = { aiAddedLines: 0, totalAddedLines: 0, deletedLines: 5 };
const entry = buildEntry({
sessionId,
Expand All @@ -526,6 +526,25 @@ describe("buildEntry", () => {
aiFiles: ["a.ts"],
lineCounts,
});
assert.equal(entry.attribution.method, "file");
assert.equal(entry.attribution.ai_ratio, 100);
assert.deepEqual(entry.attribution.lines, {
ai_added: 0,
total_added: 0,
deleted: 5,
});
});

it("sets method=none when no added-line or file-level denominator exists", () => {
const lineCounts: LineCounts = { aiAddedLines: 0, totalAddedLines: 0, deletedLines: 5 };
const entry = buildEntry({
sessionId,
interactions: [],
commitFiles: ["packages/cli/dist/cli.js"],
aiFiles: ["packages/cli/dist/cli.js"],
aiRatioExcludedFiles: ["packages/cli/dist/cli.js"],
lineCounts,
});
assert.equal(entry.attribution.method, "none");
assert.equal(entry.attribution.ai_ratio, 0);
});
Expand Down
14 changes: 9 additions & 5 deletions packages/cli/src/core/entry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -536,11 +536,15 @@ export function calcAiRatio(files: FileEntry[], lineCounts?: LineCounts): number
return Math.round((eligible.ai / eligible.total) * PERCENT_DENOMINATOR);
}

/** Determine attribution method from available data. */
function resolveMethod(lineCounts?: LineCounts): "line" | "file" | "none" {
/** Determine the strongest safe attribution method for the current commit evidence. */
function resolveMethod(files: FileEntry[], lineCounts?: LineCounts): "line" | "file" | "none" {
if (!lineCounts) return "file";
if (lineCounts.totalAddedLines === 0) return "none";
return "line";
if (lineCounts.totalAddedLines > 0) return "line";

// A deletion-only or rename-only commit has no added-line denominator.
// Keep AI involvement visible by falling back to file-level attribution.
const eligible = countAiRatioEligibleFiles(files);
return eligible.total > 0 ? "file" : "none";
}

/** Build the final git-note entry from collected session, attribution, and prompt data. */
Expand All @@ -566,7 +570,7 @@ export function buildEntry(opts: {
...(aiRatioExcludedFiles.has(path) ? { ai_ratio_excluded: true } : {}),
}));

const method = resolveMethod(opts.lineCounts);
const method = resolveMethod(files, opts.lineCounts);
const aiRatio = method === "none" ? 0 : calcAiRatio(files, opts.lineCounts);

const attribution: Attribution = { ai_ratio: aiRatio, method };
Expand Down
1 change: 0 additions & 1 deletion packages/dashboard/src/pages/index.astro
Original file line number Diff line number Diff line change
Expand Up @@ -1599,7 +1599,6 @@ const dashboardIndex = buildDashboardIndex();
<aside class="sidebar" id="sidebar">
<div class="sidebar-header">
<a href={homePath} class="logo">Agent Note</a>
<span class="badge accent">Preview</span>
</div>
<div class="tab-row" id="tab-row">
<button data-tab="prs" class="active">PRs</button>
Expand Down
29 changes: 21 additions & 8 deletions packages/pr-report/dist/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -36471,6 +36471,10 @@ const SESSION_FILE = "session";
const SESSION_AGENT_FILE = "agent";
/** Gemini pending commit state file used between BeforeTool and AfterTool. */
const PENDING_COMMIT_FILE = "pending_commit.json";
/** One-shot marker allowing post-commit to recover a stale-heartbeat session. */
const POST_COMMIT_FALLBACK_FILE = "post_commit_fallback";
/** Marker value indicating that post-commit may try strict HEAD recovery. */
const POST_COMMIT_FALLBACK_HEAD = "head";
// ─── Display limits ───
/** Maximum commits scanned by commands that need bounded history traversal. */
const MAX_COMMITS = 500;
Expand Down Expand Up @@ -36508,6 +36512,8 @@ const PRE_BLOBS_FILE = "pre_blobs.jsonl";
const COMMITTED_PAIRS_FILE = "committed_pairs.jsonl";
/** Session files that prove a commit can produce a non-empty git note. */
const RECORDABLE_SESSION_FILES = [PROMPTS_FILE, CHANGES_FILE, PRE_BLOBS_FILE];
/** Session files that let plain git hooks safely attach a session trailer. */
const TRAILER_SESSION_FILES = [CHANGES_FILE, PRE_BLOBS_FILE];
// ─── Git ───
/** SHA-1 hash of a git blob with empty content (canonical git empty blob). */
const EMPTY_BLOB = "e69de29bb2d1d6434b8b29ae775ad8c2e48c5391";
Expand Down Expand Up @@ -36898,13 +36904,16 @@ function calcAiRatio(files, lineCounts) {
return 0;
return Math.round((eligible.ai / eligible.total) * PERCENT_DENOMINATOR);
}
/** Determine attribution method from available data. */
function resolveMethod(lineCounts) {
/** Determine the strongest safe attribution method for the current commit evidence. */
function resolveMethod(files, lineCounts) {
if (!lineCounts)
return "file";
if (lineCounts.totalAddedLines === 0)
return "none";
return "line";
if (lineCounts.totalAddedLines > 0)
return "line";
// A deletion-only or rename-only commit has no added-line denominator.
// Keep AI involvement visible by falling back to file-level attribution.
const eligible = countAiRatioEligibleFiles(files);
return eligible.total > 0 ? "file" : "none";
}
/** Build the final git-note entry from collected session, attribution, and prompt data. */
function buildEntry(opts) {
Expand All @@ -36916,7 +36925,7 @@ function buildEntry(opts) {
...(generatedFiles.has(path) ? { generated: true } : {}),
...(aiRatioExcludedFiles.has(path) ? { ai_ratio_excluded: true } : {}),
}));
const method = resolveMethod(opts.lineCounts);
const method = resolveMethod(files, opts.lineCounts);
const aiRatio = method === "none" ? 0 : calcAiRatio(files, opts.lineCounts);
const attribution = { ai_ratio: aiRatio, method };
if (opts.lineCounts) {
Expand Down Expand Up @@ -37003,12 +37012,13 @@ async function git(args, options) {
async function git_gitSafe(args, options) {
try {
const stdout = await git(args, options);
return { stdout, exitCode: 0 };
return { stdout, stderr: "", exitCode: 0 };
}
catch (err) {
const e = err;
return {
stdout: typeof e.stdout === "string" ? e.stdout.trim() : "",
stderr: typeof e.stderr === "string" ? e.stderr.trim() : "",
exitCode: typeof e.code === "number" ? e.code : 1,
};
}
Expand Down Expand Up @@ -37211,7 +37221,10 @@ function injectGitCommitTrailer(command, trailer) {
/** Write an Agent Note entry as a git note on a commit. */
async function writeNote(commitSha, data) {
const body = JSON.stringify(data, null, 2);
await gitSafe(["notes", `--ref=${NOTES_REF}`, "add", "-f", "-m", body, commitSha]);
const result = await gitSafe(["notes", `--ref=${NOTES_REF}`, "add", "-f", "-m", body, commitSha]);
if (result.exitCode !== 0) {
throw new Error(result.stderr || "failed to write Agent Note git note");
}
}
/** Read an Agent Note entry from a git note, returning null when none exists. */
async function readNote(commitSha) {
Expand Down
Loading