Skip to content

[auto] sync agent CLI harnesses with upstream docs#485

Merged
chhhee10 merged 5 commits into
mainfrom
auto/sync-cli-harnesses-20260709T095400Z
Jul 9, 2026
Merged

[auto] sync agent CLI harnesses with upstream docs#485
chhhee10 merged 5 commits into
mainfrom
auto/sync-cli-harnesses-20260709T095400Z

Conversation

@NiveditJain

@NiveditJain NiveditJain commented Jul 9, 2026

Copy link
Copy Markdown
Member

Automated sync of failproofai's per-CLI hook harnesses against upstream documentation. One subagent per CLI fetched the primary docs; every actioned claim below was then re-verified by fetching the primary source directly from the main run.

Summary

CLI scope-1 (events) scope-2 (tools/payload) scope-3 (settings shape) status
Claude Code ✅ appended Setup, MessageDisplay ✅ fixed timeout unit (s) drift
OpenAI Codex ✅ appended 4 (map deferred → intentional tsc fail) drift
GitHub Copilot ✅ appended 4; 📋 2 deferred 📋 4 tool ids deferred drift
Cursor Agent — (deliberate subset; 📋 observations) ✅ fixed timeout unit (s) drift
OpenCode 📋 new bus events deferred (observation-only) 📋 registration note drift (report only)
Pi unverified
Gemini CLI 📋 save_memory note up to date

✅ applied · 📋 deferred to reviewer checklist · — no drift · ❔ could not verify

Per-CLI detail

Claude Code — drift (applied)

  • scope-1: upstream lifecycle table now documents Setup (fires on --init/--maintenance) and MessageDisplay (fires while assistant text is displayed; docs lower its timeout default to 10s). Both appended to HOOK_EVENT_TYPES (no event map → build stays green). Test counts updated 28 → 30 in manager.test.ts.
  • scope-3: docs state timeout is seconds ("Seconds before canceling. Defaults: 600 for command …; 60 for agent"). The repo wrote 60000 (documented as ms) → ~16.7h. Fixed writer + .claude/settings.json fixture + unit tests to 60.

OpenAI Codex — drift (applied, map deferred)

  • scope-1: docs now document SubagentStart, PreCompact, PostCompact, SubagentStop beyond the existing 6. Appended their snake_case forms (subagent_start, pre_compact, post_compact, subagent_stop) to CODEX_HOOK_EVENT_TYPES, following the repo's established snake_case→Pascal convention. CODEX_EVENT_MAP entries are intentionally omitted so the exhaustive Record<CodexHookEventType, HookEventType> fails tsc until a reviewer picks the canonical mapping (see checklist).
  • scope-2 / scope-3: none. timeout (seconds) and the no-version shape remain correct (already fixed in fix(codex): drop invalid top-level version field + correct timeout unit (seconds) #482).
  • Casing note: the current Codex docs render config keys in PascalCase; the repo models Codex as snake_case identifiers mapped to Pascal keys via CODEX_EVENT_MAP. The appended events use the same snake_case derivation as the existing 6 — flagged for reviewer awareness, not changed.

GitHub Copilot — drift (applied + deferred)

  • scope-1 (applied): appended PostToolUseFailure, ErrorOccurred, PreCompact, PermissionRequest — the docs show explicit PascalCase "VS Code compatible" variants for each. No event map → build stays green.
  • scope-1 (deferred): notification and subagentStart are newly documented but appear camelCase-only (no PascalCase variant shown) — deferred rather than guessing a casing.
  • scope-2 (deferred): the tool reference lists web_search, ask_user, update_todo, task, which are absent from COPILOT_TOOL_MAP. Not auto-added (canonical mapping is a human decision).

Cursor Agent — drift (applied)

  • scope-3: docs state timeout is seconds ("Execution timeout in seconds"; examples 30, 10). Repo wrote 60000 (documented as ms). Fixed writer + .cursor/hooks.json fixture + unit test to 60.
  • scope-1 (observation only): the repo subscribes to a deliberate 7-event parity subset. Upstream documents many more (postToolUseFailure, subagentStart, beforeShellExecution, afterShellExecution, beforeMCPExecution, afterMCPExecution, beforeReadFile, afterFileEdit, preCompact, stop, afterAgentResponse, afterAgentThought, plus Tab/App-lifecycle families). These are by-design omissions — deferred to the reviewer.

OpenCode — drift (report only, no code change)

  • scope-1: the docs Events section lists ~29 bus/plugin events; the repo's 6 bus events all still appear verbatim and permission.ask is still a real plugin function (none renamed/removed). The bulk of the extras are observation-only with no canonical mapping; the only actionable new ones are permission.asked, permission.replied, and shell.env — deferred.
  • scope-3: current docs claim .opencode/plugins/ files auto-load without a plugin-array entry, contradicting the repo's live-verified (opencode v1.14.33) "must register" gotcha. The explicit registration is a safe superset and is kept; re-verify before removing.

Pi — unverified

  • The npm docs URL returned HTTP 403 on repeated fetches, so no parseable enumeration of events/tools/schema was available (expected — Pi's surface lives in package source / d.ts). No changes; existing PI_* tables untouched.

Gemini CLI — up to date

  • scope-1: the hooks page enumerates exactly the 11 PascalCase events already in GEMINI_HOOK_EVENT_TYPES.
  • scope-3: timeout is genuinely milliseconds (docs: "Execution timeout in milliseconds (default: 60000)"), so the repo's 60000 is correct — left unchanged (this is why the Claude/Cursor fix does not touch Gemini).
  • scope-2 (note): save_memory (→ Memory) was not in the current tools-reference snapshot — deferred for verification.

Reviewer checklist

  • Add subagent_start: "???" to CODEX_EVENT_MAP in src/hooks/types.ts (canonical Claude HookEventType chosen by reviewer — likely SubagentStart).
  • Add pre_compact: "???" to CODEX_EVENT_MAP in src/hooks/types.ts (likely PreCompact).
  • Add post_compact: "???" to CODEX_EVENT_MAP in src/hooks/types.ts (likely PostCompact).
  • Add subagent_stop: "???" to CODEX_EVENT_MAP in src/hooks/types.ts (likely SubagentStop).
  • Confirm whether Codex now emits PascalCase --hook event args (docs show PascalCase config keys) vs the repo's snake_case model; adjust the snake_case appends if needed.
  • Copilot: confirm the PascalCase "VS Code compatible" variants for notification / subagentStart and append Notification / SubagentStart to COPILOT_HOOK_EVENT_TYPES if valid.
  • Copilot: decide canonical mappings for web_search, ask_user, update_todo, task and add to COPILOT_TOOL_MAP in src/hooks/types.ts.
  • Cursor: decide whether to widen the deliberate 7-event parity subset to any newly-documented events (postToolUseFailure, subagentStart, preCompact, afterAgentResponse, …).
  • OpenCode: decide whether to subscribe to permission.asked / permission.replied / shell.env (the only actionable new events; rest are observation-only).
  • OpenCode: re-verify against the current opencode release whether .opencode/plugins/ auto-loads before touching the explicit plugin-array registration (kept as a safe superset).
  • Gemini: verify whether save_memory was renamed/removed (absent from the current tools-reference snapshot) before changing GEMINI_TOOL_MAP.
  • Pi: re-run drift check when the npm page (or an authoritative d.ts enumeration) is fetchable — it returned HTTP 403 this run.

Sources

Unverified notes

  • Pi: the authoritative npm docs URL returned HTTP 403 Forbidden on repeated fetches, yielding no parseable enumeration of hook events, tool names, or settings schema. Pi's surface is documented in package source / d.ts rather than a clean README enumeration, so this was expected. No non-authoritative third-party sources were substituted.

CI is expected to fail on this PR if a map-bearing CLI gained new events — a reviewer must add the missing *EVENT_MAP entries (replacing "???") before merging. For drift in Claude or Copilot only (no event map), CI should pass on this commit alone. CI must pass and this PR must be reviewed before merging.

Summary by CodeRabbit

  • New Features

    • Expanded hook event coverage across Claude, Cursor, Codex, and Copilot integrations, including additional setup and lifecycle events.
  • Bug Fixes

    • Corrected integration hook timeout handling to use 60 seconds (instead of 60,000 milliseconds) for improved consistency with upstream expectations.
  • Tests

    • Updated hook integration and manager tests to reflect the revised timeout units and expanded expected hook event counts.
  • Chores

    • Added a changelog entry for version 0.0.13-beta.0 documenting the release notes.

Append newly-documented hook events and fix two timeout-unit bugs found
by comparing each integrated CLI's upstream docs to this repo.

Scope 1 (event names):
- Claude: append `Setup`, `MessageDisplay` (no event map — build stays green)
- Codex: append `subagent_start`, `pre_compact`, `post_compact`,
  `subagent_stop`. CODEX_EVENT_MAP entries are intentionally NOT added so
  `tsc` fails until a reviewer picks the canonical HookEventType (per the
  map-bearing-CLI rule) — see PR checklist.
- Copilot: append `PostToolUseFailure`, `ErrorOccurred`, `PreCompact`,
  `PermissionRequest` (docs show explicit PascalCase variants; no event
  map — build stays green). `notification` / `subagentStart` are
  camelCase-only upstream and deferred to the reviewer.

Scope 3 (settings-file shape, #482-class timeout-unit fix):
- Claude and Cursor read `timeout` in SECONDS per their docs (Claude
  "60 for agent"; Cursor "Execution timeout in seconds"), so the old
  `60000` meant ~16.7h. Corrected writer + dogfood fixtures + unit tests
  to `60`. Gemini's `timeout` is genuinely milliseconds (default 60000)
  and is left unchanged.

Verified each claim by fetching the primary docs directly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 24a520f4-2346-41ca-86e1-2ae42493350c

📥 Commits

Reviewing files that changed from the base of the PR and between 4b8df5f and 2eaadd0.

📒 Files selected for processing (4)
  • CHANGELOG.md
  • __tests__/hooks/manager.test.ts
  • src/hooks/integrations.ts
  • src/hooks/types.ts
✅ Files skipped from review due to trivial changes (1)
  • CHANGELOG.md
🚧 Files skipped from review as they are similar to previous changes (2)
  • tests/hooks/manager.test.ts
  • src/hooks/integrations.ts

📝 Walkthrough

Walkthrough

Hook timeout values for Claude Code and Cursor integrations were corrected from milliseconds to seconds across configs and builders. Codex, Copilot, and Claude hook event type unions were expanded with new documented events. Tests and changelog were updated to match.

Changes

Hook Timeout Fix and Event Type Expansion

Layer / File(s) Summary
Expanded hook event type unions
src/hooks/types.ts
CODEX_HOOK_EVENT_TYPES, CODEX_EVENT_MAP, COPILOT_HOOK_EVENT_TYPES, and HOOK_EVENT_TYPES were expanded with newly documented event names and mappings.
Integration hook builders: timeout unit correction
src/hooks/integrations.ts
claudeCode and cursor builders changed timeout from 60_000 to 60; codex builder comments were updated to describe seconds without changing the value.
Claude and Cursor config timeout updates
.claude/settings.json, .cursor/hooks.json
All hook entries' timeout values were updated from 60000 to 60.
Test assertions and changelog updates
__tests__/hooks/integrations.test.ts, __tests__/hooks/manager.test.ts, CHANGELOG.md
Tests now expect timeout: 60, 29 hook entries, and 10 documented Codex events; changelog adds a 0.0.13-beta.0 entry documenting the timeout fix and new events.

Estimated code review effort: 2 (Simple) | ~10 minutes

Poem

I hopped through hooks with twitchy ears,
Swapped milliseconds for seconds clear.
New events joined the bunny line,
Codex, Claude, and Cursor shine.
A tidy thump, a happy trail 🐇

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the PR’s main goal of syncing agent CLI harnesses with upstream documentation.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Warning

Tools execution failed with the following error:

Failed to run tools: 13 INTERNAL: Received RST_STREAM with code 2 (Internal server error)


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@NiveditJain NiveditJain requested a review from chhhee10 July 9, 2026 10:27

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/hooks/integrations.ts`:
- Around line 598-604: The timeout change is fine, but the Cursor hook event set
is inconsistent: `CURSOR_HOOK_EVENT_TYPES` includes `subagentStop`, while the
`.cursor/hooks.json` configuration only wires up six events and omits it. Update
the hook setup in `src/hooks/integrations.ts` and the generated Cursor config so
the event count matches the intended “7-event subset,” or explicitly keep
`subagentStop` excluded if that is intentional and make the mapping/comments
consistent.

In `@src/hooks/types.ts`:
- Around line 21-30: Add the missing Codex hook event mappings so the exhaustive
Record<CodexHookEventType, HookEventType> in src/hooks/types.ts is complete.
Update CODEX_EVENT_MAP to include entries for subagent_start, pre_compact,
post_compact, and subagent_stop, matching the existing HookEventType naming used
by the other Codex events, and keep CODEX_HOOK_EVENT_TYPES and CODEX_EVENT_MAP
in sync so tsc no longer fails on the missing keys.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: bdf81f39-d5fe-43ce-8577-42722e4ebcee

📥 Commits

Reviewing files that changed from the base of the PR and between 00cddcb and 4b8df5f.

📒 Files selected for processing (7)
  • .claude/settings.json
  • .cursor/hooks.json
  • CHANGELOG.md
  • __tests__/hooks/integrations.test.ts
  • __tests__/hooks/manager.test.ts
  • src/hooks/integrations.ts
  • src/hooks/types.ts

Comment thread src/hooks/integrations.ts
Comment on lines +598 to +604
// `timeout` is documented in SECONDS in Cursor's schema per
// https://cursor.com/docs/hooks ("Execution timeout in seconds"; doc examples
// use 30 and 10), NOT milliseconds. 60 = 60s; the old 60000 meant ~16.7h.
return {
type: "command",
command,
timeout: 60_000,
timeout: 60,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Count events in .cursor/hooks.json and check Cursor event types in types.ts
cat .cursor/hooks.json | jq '.hooks | keys | length'
rg -n 'cursor' src/hooks/types.ts -i

Repository: FailproofAI/failproofai

Length of output: 2224


🏁 Script executed:

#!/bin/bash
# View the CURSOR_HOOK_EVENT_TYPES array definition
sed -n '187,196p' src/hooks/types.ts

echo "---"
# View the .cursor/hooks.json to see exactly which events are configured
cat .cursor/hooks.json

Repository: FailproofAI/failproofai

Length of output: 1473


🏁 Script executed:

# Search for any mention of "7" in the file or "7-event" in PR context
rg '7\s*event|7-event' src/hooks/integrations.ts -i -A 2 -B 2
rg '7' src/hooks/types.ts -A 1 -B 1

Repository: FailproofAI/failproofai

Length of output: 1205


Cursor timeout fix is correct; verify event count discrepancy.

The timeout correction from 60_000 to 60 with seconds-based comments is consistent with the Cursor builder, .cursor/hooks.json, and integrations.test.ts:417. The PR objectives correctly reference a "7-event subset"; src/hooks/integrations.ts:173 confirms this is "6 events that align with Claude's canonical types plus subagentStop". However, .cursor/hooks.json configures only 6 events (sessionStart, sessionEnd, beforeSubmitPrompt, preToolUse, postToolUse, stop), omitting subagentStop despite CURSOR_HOOK_EVENT_TYPES in types.ts:187 including it. Clarify whether the subagentStop hook configuration is intentionally excluded or should be added to the .cursor/hooks.json file.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/hooks/integrations.ts` around lines 598 - 604, The timeout change is
fine, but the Cursor hook event set is inconsistent: `CURSOR_HOOK_EVENT_TYPES`
includes `subagentStop`, while the `.cursor/hooks.json` configuration only wires
up six events and omits it. Update the hook setup in `src/hooks/integrations.ts`
and the generated Cursor config so the event count matches the intended “7-event
subset,” or explicitly keep `subagentStop` excluded if that is intentional and
make the mapping/comments consistent.

Comment thread src/hooks/types.ts
Comment on lines +21 to +30
// Newly documented upstream (https://developers.openai.com/codex/hooks) —
// snake_case forms of the documented SubagentStart / PreCompact / PostCompact /
// SubagentStop events. Their CODEX_EVENT_MAP entries are intentionally NOT added
// here: the canonical HookEventType is a reviewer decision, and the missing keys
// make the exhaustive `Record<CodexHookEventType, HookEventType>` below fail tsc
// on purpose so this drift is not silently half-applied. See PR body checklist.
"subagent_start",
"pre_compact",
"post_compact",
"subagent_stop",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify CODEX_EVENT_MAP exists and uses Record<CodexHookEventType, HookEventType>
rg -n 'CODEX_EVENT_MAP' src/hooks/types.ts -A 15

Repository: FailproofAI/failproofai

Length of output: 1422


🏁 Script executed:

sed -n '1,120p' src/hooks/types.ts

Repository: FailproofAI/failproofai

Length of output: 5714


Add the missing Codex event mappings
CODEX_HOOK_EVENT_TYPES now includes subagent_start, pre_compact, post_compact, and subagent_stop, but CODEX_EVENT_MAP still omits those keys. That leaves the exhaustive Record<CodexHookEventType, HookEventType> incomplete and keeps tsc failing until the four mappings are added.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/hooks/types.ts` around lines 21 - 30, Add the missing Codex hook event
mappings so the exhaustive Record<CodexHookEventType, HookEventType> in
src/hooks/types.ts is complete. Update CODEX_EVENT_MAP to include entries for
subagent_start, pre_compact, post_compact, and subagent_stop, matching the
existing HookEventType naming used by the other Codex events, and keep
CODEX_HOOK_EVENT_TYPES and CODEX_EVENT_MAP in sync so tsc no longer fails on the
missing keys.

@hermes-exosphere

Copy link
Copy Markdown

🔍 Automated code review started — analyzing [{"additions":26,"deletions":26,"path":".claude/settings.json"},{"additions":6,"deletions":6,"path":".cursor/hooks.json"},{"additions":8,"deletions":0,"path":"CHANGELOG.md"},{"additions":3,"deletions":3,"path":"tests/hooks/integrations.test.ts"},{"additions":4,"deletions":4,"path":"tests/hooks/manager.test.ts"},{"additions":10,"deletions":4,"path":"src/hooks/integrations.ts"},{"additions":26,"deletions":0,"path":"src/hooks/types.ts"}] files, +83/-43...

⏱️ This may take a few minutes. Results will be posted here when complete.

Comment thread src/hooks/types.ts
Comment on lines +27 to +30
"subagent_start",
"pre_compact",
"post_compact",
"subagent_stop",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 BLOCKER — this intentionally fails tsc, and that failure cascades into all 7 CI jobs.

These 4 events are added to CODEX_HOOK_EVENT_TYPES but not to CODEX_EVENT_MAP (line 34), which is typed Record<CodexHookEventType, HookEventType>. That is a hard compile error:

./src/hooks/types.ts:34:14
Type error: Type '{ session_start … stop: "Stop"; }' is missing the following
properties from type 'Record<…>': subagent_start, pre_compact, post_compact, subagent_stop

Why it's worse than "just a tsc gate": package.json"prepare": "bun run build"next build → type-check. Every CI job begins with bun install --frozen-lockfile, which runs prepare, so the type error aborts the install step itself:

error: script "build" exited with code 1
error: prepare script from "failproofai" exited with 1
##[error]Final attempt failed. Child_process exited with error code 1

Result: quality, test (×3), build, docs, and test-e2e all go red at Install dependencies — including jobs that have nothing to do with this change. (Confirmed in the quality job log and reproduced locally.)

Runtime bug, independent of tsc: writeHookEntries (integrations.ts:307-310) does const pascalKey = CODEX_EVENT_MAP[eventType]undefined for these 4, then s.hooks[pascalKey] writes under the literal string key "undefined". All 4 collapse into one bogus "undefined" entry, so a real Codex install emits a malformed hooks.json and only 7 of 10 hooks land. This is exactly why the runtime test fails:

FAIL __tests__/hooks/integrations.test.ts > OpenAI Codex integration >
     removeHooksFromFile clears all failproofai entries (returns count)
AssertionError: expected 7 to be 10

The shipped dist/index.js is esbuild-bundled and never type-checked, so the tsc "gate" does not protect end users from this — it only reddens CI.

The fix is trivial and unambiguous — every canonical target already exists in HOOK_EVENT_TYPES (lines 560, 561, 573, 574). Add these 4 lines to CODEX_EVENT_MAP:

subagent_start: "SubagentStart",
pre_compact:    "PreCompact",
post_compact:   "PostCompact",
subagent_stop:  "SubagentStop",

CI then goes green and the removeHooksFromFile test passes. The "canonical mapping is a reviewer decision" framing overstates the difficulty: this is a 1:1 mapping onto events that already exist, and the PR body itself already names each target ("likely SubagentStart", …). Recommend resolving it in this PR rather than shipping a knowingly-red branch that also blocks the (correct) timeout fix from landing.

});

it("eventTypes are exactly the 6 documented Codex events (snake_case)", () => {
it("eventTypes are exactly the 10 documented Codex events (snake_case)", () => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Minor (test quality): the description now says "exactly the 10 documented Codex events", but the assertion below (expect(codex.eventTypes).toEqual(CODEX_HOOK_EVENT_TYPES)) is tautological — codex.eventTypes is CODEX_HOOK_EVENT_TYPES (see integrations.ts:257), so it passes for any array of any length and never actually pins the count at 10.

If the intent is to guard the count, assert it directly:

expect(codex.eventTypes).toHaveLength(10);

Not blocking, but worth tightening while you're touching this line — otherwise a future accidental add/remove of a Codex event slips through this test silently.

@hermes-exosphere

Copy link
Copy Markdown

🔍 Automated Code Review

📋 Executive Summary

This PR does two independent things: (1) a genuine, well-verified bug fix — correcting the Claude & Cursor hook timeout unit from 60000 to 60 (both CLIs read seconds, so the old value meant ~16.7h ≈ no timeout); and (2) a drift-sync that appends newly-documented hook events for Claude, Copilot, and Codex. The Codex additions are deliberately left half-applied (added to CODEX_HOOK_EVENT_TYPES but not to CODEX_EVENT_MAP) to force a tsc failure. That single omission turns all 7 CI jobs red and causes a runtime test failure. The timeout fix and the Claude/Copilot event additions are correct and low-risk; the Codex omission is the sole blocker and its fix is trivial.


📊 Change Architecture

graph TD
    A["types.ts — CODEX_HOOK_EVENT_TYPES<br/>+4 events (subagent_start …)"] -->|"NOT added to"| B["CODEX_EVENT_MAP<br/>Record&lt;CodexHookEventType, HookEventType&gt;<br/>(still only 6 keys)"]
    B -->|"missing 4 keys → TS error"| C["tsc / next build FAILS"]
    C -->|"package.json prepare → bun run build"| D["bun install --frozen-lockfile FAILS"]
    D --> E["🔴 ALL 7 CI JOBS RED<br/>quality · test×3 · build · docs · test-e2e"]
    A -->|"runtime: pascalKey = undefined"| F["writeHookEntries → s.hooks['undefined']<br/>🔴 removeHooksFromFile: expected 7 to be 10"]

    G["integrations.ts + .claude/settings.json<br/>+ .cursor/hooks.json — timeout 60000 → 60"] -->|"verified vs upstream docs"| H["🟢 correct fix (seconds, not ms)"]
    I["Claude +Setup +MessageDisplay<br/>Copilot +4 events"] -->|"no event map"| J["🟢 build-safe additions"]

    style A fill:#FFD700
    style B fill:#FFD700
    style C fill:#FF6B6B
    style D fill:#FF6B6B
    style E fill:#FF6B6B
    style F fill:#FF6B6B
    style G fill:#87CEEB
    style H fill:#90EE90
    style I fill:#90EE90
    style J fill:#90EE90
Loading

Legend: 🟢 New / correct · 🔵 Modified · 🟡 Breaking-change risk · 🔴 Failing


🔴 Breaking Changes

  1. 🔴 CI is fully red by designCODEX_EVENT_MAP (src/hooks/types.ts:34) is Record<CodexHookEventType, HookEventType> but only holds 6 of the now-10 required keys → tsc error. Because package.json's prepare script runs bun run build (→ next build → type-check), and every CI job starts with bun install --frozen-lockfile, the failure aborts the install step of all 7 jobs. This also means a git-based install of the package at this commit (prepare runs) would fail. The PR is not mergeable as-is.

  2. 🟡 Behavioral change (intended & correct): hooks previously never effectively timed out (60000s); they now time out at 60s. For the dogfood bun …/failproofai.mjs hooks this is fine. Production users on npx -y failproofai incur a first-run package fetch — 60s is the documented default and is a reasonable ceiling, but worth being aware this is a real runtime change, not just a comment edit.


⚠️ Issues Found

  1. 🔴 Blockersrc/hooks/types.ts:27-34 — 4 Codex events added to the union but omitted from CODEX_EVENT_MAPtsc fails → cascades to all CI jobs. (inline comment posted)
  2. 🔴 Correctness (runtime)src/hooks/integrations.ts:307-310 — for the unmapped events CODEX_EVENT_MAP[eventType] is undefined, so s.hooks[pascalKey] writes under the literal key "undefined"; the 4 events collapse into one bogus entry. A real Codex install produces a malformed hooks.json and only 7/10 hooks. Reproduced: removeHooksFromFile … expected 7 to be 10.
  3. 🔵 Info (test quality)__tests__/hooks/integrations.test.ts:194-195 — the "exactly the 10 …" assertion is tautological and never pins the count. (inline comment posted)
  4. 🔵 Info (consistency).claude/settings.json still lists 26 hook entries and lacks Setup/MessageDisplay even though they were added to HOOK_EVENT_TYPES. This is pre-existing subset drift (this PR only changed timeouts there), not a regression — flagging for awareness.

🔬 Logical / Bug Analysis

  • Root cause is a single design choice: the "fail tsc on purpose so drift isn't silently half-applied" gate. It works, but it (a) reddens unrelated jobs (docs, test-e2e), (b) blocks the correct timeout fix from landing, and (c) does not protect end users — dist/index.js is esbuild-bundled and never type-checked, so the runtime "undefined"-key bug ships anyway if merged without the map.
  • The fix is trivial and unambiguous — all 4 canonical targets already exist in HOOK_EVENT_TYPES (SubagentStart:560, SubagentStop:561, PreCompact:573, PostCompact:574):
    subagent_start: "SubagentStart",
    pre_compact:    "PreCompact",
    post_compact:   "PostCompact",
    subagent_stop:  "SubagentStop",
    Adding these 4 lines turns CI green and fixes the runtime test. The "reviewer must decide the mapping" framing overstates the difficulty (the PR body itself names each target).
  • Timeout fix verified against primary docs (see Evidence). Claude & Cursor timeout are both seconds; the 60000 → 60 correction is right. Gemini's timeout is genuinely ms and correctly left unchanged. Codex/Claude/Copilot/Cursor event-name additions are plausible and, except for Codex, map-free and build-safe.

🧪 Evidence — Build & Test Results

Type check (bunx tsc --noEmit / next build) — FAIL
./src/hooks/types.ts:34:14
Type error: Type '{ session_start: "SessionStart"; pre_tool_use: "PreToolUse";
permission_request: "PermissionRequest"; post_tool_use: "PostToolUse";
user_prompt_submit: "UserPromptSubmit"; stop: "Stop"; }' is missing the following
properties from type 'Record<"session_start" | … | "subagent_stop", …>':
subagent_start, pre_compact, post_compact, subagent_stop
CI job log — quality fails at the *Install dependencies* step (cascade)
quality  Install dependencies  ✓ Compiled successfully in 5.7s
quality  Install dependencies    Running TypeScript ...
quality  Install dependencies  Failed to type check.
quality  Install dependencies  ./src/hooks/types.ts:34:14
quality  Install dependencies  Type error: … missing … subagent_start, pre_compact, post_compact, subagent_stop
quality  Install dependencies  error: script "build" exited with code 1
quality  Install dependencies  error: prepare script from "failproofai" exited with 1
quality  Install dependencies  ##[error]Final attempt failed. Child_process exited with error code 1

All 7 CI jobs (quality, test (default), test (log-debug, debug), test (hook-log-file, 1), build, docs, test-e2e) report FAILURE.

Unit tests (bun run test:run) — 1 failed / 1906 passed
❯ __tests__/hooks/integrations.test.ts (118 tests | 1 failed)
   × removeHooksFromFile clears all failproofai entries (returns count)

FAIL __tests__/hooks/integrations.test.ts > OpenAI Codex integration >
     removeHooksFromFile clears all failproofai entries (returns count)
AssertionError: expected 7 to be 10 // Object.is equality

Test Files  1 failed | 105 passed (106)
     Tests  1 failed | 1906 passed (1907)

(The manager.test.ts 28→30 and Claude/Cursor timeout 60 assertions all pass — those changes are internally consistent.)

Upstream-doc verification of the timeout unit — fix is CORRECT
  • Claudehttps://code.claude.com/docs/en/hooks: "timeout … Seconds before canceling. Defaults: 600 for command, http, and mcp_tool; 30 for prompt; 60 for agent. … MessageDisplay lowers it to 10." → seconds, and confirms the new Setup/MessageDisplay events + the 10s note.
  • Cursorhttps://cursor.com/docs/hooks: "Execution timeout in seconds", example "timeout": 10. → seconds.

So 60000 was indeed a unit bug (~16.7h); 60 is correct.


🔗 Issue Linkage

⚠️ No issue linked (closingIssuesReferences is empty). This is an automated drift-sync PR; a tracking issue for the deferred reviewer-checklist items (Codex map, Copilot/Cursor event widening, Pi re-verify) would improve traceability.


💡 Suggestions

  • Resolve the Codex map in this PR (4 lines above) so CI is green and the timeout fix can land together. If the intent is genuinely "alert a human, don't auto-apply," prefer splitting: land the timeout fix + map-free events on a green branch, and track the Codex-map decision separately — rather than committing a knowingly-red branch that also breaks bun install from git.
  • Consider --ignore-scripts implications: because prepare runs the full next build, any type error anywhere blocks install for every job. A lighter prepare (or gating the heavy build behind an explicit script) would make failures localize to the relevant job instead of masquerading as install failures across docs/test-e2e.
  • Minor: the Claude comment cites "60 for agent", but failproofai hooks are type: "command" (documented default 600s). 60 is a fine, stricter choice — just note the cited default doesn't match the hook type.
  • Tighten the tautological Codex count test (inline).
  • CodeRabbit also flags the PR description is missing the repo's required template sections/checklist — worth adding.

🏆 Verdict

🔴 CHANGES REQUESTED — 1 blocking issue (+1 runtime bug sharing its root cause). Add the 4 CODEX_EVENT_MAP entries to src/hooks/types.ts; that turns all CI green and fixes the removeHooksFromFile failure. The timeout-unit fix and the Claude/Copilot event additions are correct and can merge once the Codex map is completed.


Automated code review · 2026-07-09 · evidence reproduced locally (tsc, bun run test:run) and cross-checked against the live CI logs + upstream docs.

@hermes-exosphere hermes-exosphere left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Changes requested — 1 blocker.

All 7 CI jobs are red and one unit test fails, both from a single root cause: 4 new Codex events were added to CODEX_HOOK_EVENT_TYPES but not to CODEX_EVENT_MAP (src/hooks/types.ts:34), so tsc/next build fails — and because package.json's prepare runs the build, that aborts bun install --frozen-lockfile in every job (including docs/test-e2e). At runtime it also makes writeHookEntries emit a bogus "undefined" hook key (removeHooksFromFile … expected 7 to be 10).

Fix (4 lines, targets already exist in HOOK_EVENT_TYPES):

subagent_start: "SubagentStart",
pre_compact:    "PreCompact",
post_compact:   "PostCompact",
subagent_stop:  "SubagentStop",

The rest is good: the Claude/Cursor timeout 60000 → 60 fix is verified correct against the upstream docs (both read seconds; old value ≈ 16.7h), and the Claude/Copilot map-free event additions are build-safe. See the full summary comment + inline notes for evidence.

chhhee10 and others added 4 commits July 9, 2026 16:37
The prior commit added subagent_start/pre_compact/post_compact/subagent_stop
to CODEX_HOOK_EVENT_TYPES but intentionally left their CODEX_EVENT_MAP entries
out to fail tsc as a reviewer flag. Each has an exact 1:1 canonical
HookEventType (SubagentStart/PreCompact/PostCompact/SubagentStop, all already
in HOOK_EVENT_TYPES), so fill them in directly.

Besides unblocking tsc, this fixes a real runtime bug: with no mapping,
writeHookEntries resolved CODEX_EVENT_MAP[event] to undefined and collapsed
all four events into a single literal "undefined" key in users'
.codex/hooks.json (the source of the removeHooksFromFile "7 vs 10" test
failure). The map is exhaustive, so tsc still guards against a future
event-added-mapping-missing drift.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The prior commit appended both Setup and MessageDisplay to HOOK_EVENT_TYPES.
writeHookEntries installs a failproofai hook for every entry in that array, so
appending MessageDisplay would spawn a subprocess on every assistant message
render — and the docs mark MessageDisplay observe-only (cannot block or modify,
no matcher support), so that cost buys zero enforcement value.

Keep Setup (fires only on --init/--maintenance; low-frequency, harmless to
install) and defer MessageDisplay to the reviewer checklist with a documented
NOTE, mirroring how the sync bot defers uncertain items. Net event count is
29, so update the two manager.test count assertions and the description.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The prior commit's comment claimed both notification and subagentStart are
camelCase-only upstream. Re-checking the cli-hooks-reference: Notification DOES
have a documented PascalCase "VS Code compatible" variant, so append it to
COPILOT_HOOK_EVENT_TYPES (no event map, build stays green). Only subagentStart
is genuinely camelCase-only, so it stays deferred to the reviewer checklist.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- The Codex timeout comment said the field is `timeout_sec`; docs confirm it
  is literally `timeout` (unit seconds, default 600). Fix the comment.
- Rewrite the 0.0.13-beta.0 changelog entry to describe the reviewed result:
  Codex map completed, Copilot Notification added, MessageDisplay deferred.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@hermes-exosphere

Copy link
Copy Markdown

🔍 Automated code review started — analyzing [{"additions":26,"deletions":26,"path":".claude/settings.json"},{"additions":6,"deletions":6,"path":".cursor/hooks.json"},{"additions":8,"deletions":0,"path":"CHANGELOG.md"},{"additions":3,"deletions":3,"path":"tests/hooks/integrations.test.ts"},{"additions":4,"deletions":4,"path":"tests/hooks/manager.test.ts"},{"additions":12,"deletions":5,"path":"src/hooks/integrations.ts"},{"additions":40,"deletions":0,"path":"src/hooks/types.ts"}] files, +99/-44...

⏱️ This may take a few minutes. Results will be posted here when complete.


it("eventTypes are exactly the 6 documented Codex events (snake_case)", () => {
it("eventTypes are exactly the 10 documented Codex events (snake_case)", () => {
expect(codex.eventTypes).toEqual(CODEX_HOOK_EVENT_TYPES);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Nit (non-blocking, test hygiene) — this assertion is tautological. codex.eventTypes is CODEX_HOOK_EVENT_TYPES (src/hooks/integrations.ts:427), so toEqual(CODEX_HOOK_EVENT_TYPES) compares the array to itself and can never fail. The title now claims "exactly the 10", but nothing pins the count — if an 11th Codex event were appended, this test stays green.

Fix — assert the length so the title's invariant is actually enforced:

expect(codex.eventTypes).toHaveLength(10);
expect(codex.eventTypes).toContain("permission_request");

Separately, the comment two tests below (line ~208, buildHookEntry sets timeout in SECONDS) still reads "its timeout_sec field". Per the docs this PR cites (developers.openai.com/codex/hooks), the field is literally timeout — the source comment in integrations.ts was corrected in 2eaadd0, but this parallel test comment was missed. Both are cosmetic; neither blocks merge.

@hermes-exosphere

Copy link
Copy Markdown

🔍 Automated Code Review (re-review — blocker resolved ✅)

📋 Executive Summary

This re-review covers the 4 new commits (2cd5eda, 5830f7c, 2272768, 2eaadd0) pushed since my prior review, which had requested changes for a single blocker: CODEX_EVENT_MAP was missing the 4 newly-added Codex events, so the exhaustive Record<CodexHookEventType, HookEventType> failed tsc (cascading to all CI jobs) and produced a runtime undefined-key bug (removeHooksFromFile … expected 7 to be 10).

That blocker is fully resolved. Commit 2cd5eda adds all 4 canonical mappings; tsc --noEmit now passes, the previously-failing test passes, all 1907 unit tests are green, and all 7 CI jobs pass on HEAD (2eaadd0). Every substantive claim in the new commits was re-verified against the primary docs the PR cites — all correct. Only two cosmetic, non-blocking test-hygiene nits remain.


📊 What changed since the last review

graph TD
    A["Prior review: 🔴 CHANGES_REQUESTED<br/>CODEX_EVENT_MAP missing 4 keys → tsc fail<br/>+ runtime undefined-key bug"] -->|"2cd5eda"| B["CODEX_EVENT_MAP +4 entries<br/>subagent_start→SubagentStart · pre_compact→PreCompact<br/>post_compact→PostCompact · subagent_stop→SubagentStop"]
    B --> C["tsc --noEmit ✅ PASS"]
    B --> D["removeHooksFromFile now 10/10 ✅<br/>(was 7/10)"]
    C --> E["🟢 All 7 CI jobs GREEN on HEAD<br/>quality · test×3 · build · docs · test-e2e"]
    D --> E
    F["5830f7c — drop MessageDisplay<br/>(observe-only, per-message, no matcher)"] -->|"30 → 29 events"| G["manager tests updated ✅"]
    H["2272768 — Copilot +Notification<br/>(PascalCase variant confirmed in docs)"] --> I["build-safe (no COPILOT_EVENT_MAP)"]
    J["2eaadd0 — comment: field is 'timeout'<br/>not 'timeout_sec' + changelog"] --> K["✅ matches docs · one stale test<br/>comment (L208) left behind 🔵"]
    style B fill:#90EE90
    style C fill:#90EE90
    style D fill:#90EE90
    style E fill:#90EE90
    style G fill:#90EE90
    style I fill:#90EE90
    style K fill:#FFD700
Loading

Legend: 🟢 Fixed / correct · 🟡 Minor cosmetic nit


🔴 Breaking Changes

None remaining. The prior "CI red by design" state is gone — CODEX_EVENT_MAP is now a complete exhaustive record, so tsc, the prepare → bun run build install step, and all CI jobs pass. A git-based install at this commit now succeeds.


✅ Blocker resolution — verified

Prior finding Status Evidence
CODEX_EVENT_MAP missing 4 keys → tsc fail → all CI red Fixed (2cd5eda) bunx tsc --noEmit exits clean; all 7 CI jobs SUCCESS on 2eaadd0
Runtime undefined-key bug → 7/10 Codex hooks written Fixed removeHooksFromFile … (returns count) now asserts CODEX_HOOK_EVENT_TYPES.length (10) and passes
Claude count 30 vs .claude/settings.json subset Addressed (5830f7c) MessageDisplay dropped → 29 events; manager.test.ts updated 30→29; deliberate dogfood subset unchanged

🔬 Correctness verification vs. upstream docs

  • Codex events — the map's 10 targets (SessionStart, PreToolUse, PermissionRequest, PostToolUse, UserPromptSubmit, Stop, SubagentStart, PreCompact, PostCompact, SubagentStop) match the Codex hooks docs exactly. All 4 new mappings are the obvious 1:1 targets, and all 4 already exist in HOOK_EVENT_TYPES.
  • Codex timeout field — docs: "If timeout is omitted, Codex uses 600 seconds." The field is literally timeout, in seconds2eaadd0's corrected source comment is accurate (and the emitted timeout: 60 key is right).
  • Copilot Notification — the CLI hooks reference documents notification with a PascalCase Notification variant, so appending it is correct. subagentStart really is camelCase-only and was correctly left deferred.
  • Claude MessageDisplay deferral — sound: it's observe-only, fires on every assistant-message render with no matcher, so installing it would spawn a subprocess per render for zero enforcement value.

⚠️ Remaining issues (both non-blocking, 🔵 Info)

  1. 🔵 __tests__/hooks/integrations.test.ts:194 — the "exactly the 10 documented Codex events" test is tautological: expect(codex.eventTypes).toEqual(CODEX_HOOK_EVENT_TYPES) compares the array to itself and never pins the count. An accidental 11th event would pass green. Add expect(codex.eventTypes).toHaveLength(10). (inline comment posted)
  2. 🔵 __tests__/hooks/integrations.test.ts:208 — stale comment still says "its timeout_sec field"; the field is timeout (source comment was corrected in 2eaadd0, this one was missed).

🧪 Evidence — reproduced locally on 2eaadd0

Type check — bunx tsc --noEmit → PASS (the prior blocker)
(no output — exits 0)
Full unit suite — bun run test:run → 1907 passed
 Test Files  106 passed (106)
      Tests  1907 passed (1907)
   Duration  48.19s

(Includes the previously-failing removeHooksFromFile clears all failproofai entries (returns count), now green. The 429 Too Many Requests / policy-threw lines are expected error-path test noise.)

CI on HEAD (run 29014983237) — all jobs SUCCESS
quality                    SUCCESS
test (default)             SUCCESS
test (log-debug, debug)    SUCCESS
test (hook-log-file, 1)    SUCCESS
build                      SUCCESS
docs                       SUCCESS
test-e2e                   SUCCESS
OSV-Scanner / Socket       SUCCESS

(The prior run 29011509039 on the original commit was FAILURE — matching the last review.)


🔗 Issue Linkage

⚠️ No linked issue (automated drift-sync PR). The PR body still carries a "reviewer checklist" of deferred items (Copilot/Cursor event widening, OpenCode/Pi/Gemini re-verification); a tracking issue for those would improve traceability, but none are blocking for this PR.


🏆 Verdict

APPROVED — the sole blocking issue from the previous review is resolved: CODEX_EVENT_MAP is complete, tsc and all 1907 tests pass, and all 7 CI jobs are green on HEAD. The Codex map, Copilot Notification, Claude Setup/MessageDisplay-deferral, and the timeout unit/field-name changes are all verified correct against upstream docs. The two remaining items are cosmetic test-comment/assertion nits and do not block merge.


Automated re-review · 2026-07-09 · evidence reproduced locally (tsc, bun run test:run) and cross-checked against live CI + upstream docs.

@hermes-exosphere hermes-exosphere left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated re-review — blocker resolved. ✅

The previous review requested changes for one blocker: CODEX_EVENT_MAP was missing the 4 newly-added Codex events, failing tsc (all CI red) and writing an undefined hook key at runtime. Commit 2cd5eda completes the map with the correct 1:1 canonical mappings.

Verified on HEAD (2eaadd0):

  • bunx tsc --noEmit passes.
  • bun run test:run → 1907/1907 pass (incl. the previously-failing removeHooksFromFile, now 10/10).
  • All 7 CI jobs green.
  • Codex events/mappings, Copilot Notification, and the Codex timeout field name all match the upstream docs the PR cites.

Two cosmetic test-hygiene nits remain (tautological "exactly the 10" assertion; a stale timeout_sec comment) — non-blocking, noted inline. Good to merge.

@chhhee10 chhhee10 merged commit 57dc8a4 into main Jul 9, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants