Skip to content

ci: install Ladon PR reviewer, replacing the Argus workflow - #983

Merged
bokelley merged 4 commits into
mainfrom
install-ladon
Aug 16, 2026
Merged

ci: install Ladon PR reviewer, replacing the Argus workflow#983
bokelley merged 4 commits into
mainfrom
install-ladon

Conversation

@andybevan-scope3

@andybevan-scope3 andybevan-scope3 commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Why

Ladon is the shared AdCP PR reviewer, maintained in adcontextprotocol/actions and consumed by floating major tag. This repo carried its own reviewer instead — a claude-code-action invocation driving a 245-line vendored prompt at .github/ai-review/expert-adcp-reviewer.md — so every engine fix had to be re-ported here by hand. Installed per ladon/INSTALL.md, following the "replacing an existing reviewer" path.

What changed

  • .github/workflows/ai-review.yml — replaced with the canonical consumer workflow from INSTALL.md, copied verbatim. Pins adcontextprotocol/actions/ladon/review@ladon/review/v1 (tag resolves to 2b4b3aa), keeps model: claude-opus-4-8, and preserves the pull_request_target posture: base-SHA-only checkout, PR head never fetched or executed, every github.event.* value reaching a shell via env:. Adds the workflow-modification gate the old workflow lacked.
  • Deleted .github/ai-review/expert-adcp-reviewer.md — voice, severity model, largest-file rule, test-plan honesty, operational-readiness audit, and inline-comment format now come from the baseline rules in the actions repo (ladon/reviewer/rules/*.md). Nothing else referenced this file.
  • Added LADON.md — the repo-specific half of the retired prompt, restated as parsed config. ## Repo Context carries the mandatory rules that are only true here: semver signal on the adcp.* surface (release-please cuts a minor from feat:, so a breaking diff under it ships without a major), discriminated-union forward compat, the types import-layering allowlist, generated-code-is-not-source including the codegen renumbering-churn rule from CLAUDE.md, the ctx_metadata credential prohibition, and CI-gate regressions. Plus ## High-Risk Paths, ## Trivial Paths, ## Escalation Reviewers, and ## Release Stack Branches.

Job name code_review is unchanged, so the required-status-check context is preserved.

Secrets

The reviewer moves from IPR_APP_ID / IPR_APP_PRIVATE_KEY to SECRETARIAT_APP_ID / SECRETARIAT_APP_PRIVATE_KEY. All of SECRETARIAT_APP_ID, SECRETARIAT_APP_PRIVATE_KEY, and ANTHROPIC_API_KEY already resolve to this repo as org-level secrets (verified via repos/.../actions/organization-secrets) — no new secrets to add.

One prerequisite I could not verify: whether the AAO Secretariat GitHub App is installed on this repo (reading that requires org-admin scope). Per INSTALL.md, secrets existing is not the same as the App being installed — if the token-mint step fails on the first run without a secret-not-found error, that is the cause.

Notes for the reviewer

Local security hardening applied during the refresh onto current main:

  1. GitHub primitives use this repo’s current SHA pins (checkout v6.1.0, App-token v3.2.0, and github-script v8).
  2. The workflow-modification App token is explicitly limited to pull-requests: write.
  3. Ladon itself is SHA-pinned because that step receives the Secretariat private key and Anthropic API key; updates from ladon/review/v1 are deliberate. Checkout also restores persist-credentials: false.

Also: ladon/review's app-id input is marked deprecated in favour of client-id, but INSTALL.md's template still uses app-id and the action still accepts it. Left as-is to stay on the documented template.

No ## Gated Paths section — this repo has no CODEOWNERS (removed in #968), and per AUTHORING.md a gated path without a matching CODEOWNERS + branch-protection rule becomes a permanently un-liftable block.

I dropped a ## Skip Bot Authors section I had initially written: the action's own default is dependabot[bot],renovate[bot],github-actions[bot], and because LADON.md overrides rather than extends that default, listing only dependabot[bot] would have narrowed coverage. ## Release Stack Branches handles the release-please PRs (branch release-please--branches--main--components--adcp, authored by app/aao-ipr-bot) deterministically instead.

Test plan

  • Workflow YAML parses; single job code_review; trigger, paths-ignore, permissions, and all five steps verified.
  • LADON.md verified against the real parser (ladon/setup/src/ladon-md.ts) — all six H2 headings resolve, 17 high-risk globs / 3 trivial globs / 1 reviewer / 1 release branch parse, ## Repo Context captures all 7 H3 rule blocks.
  • ladon/review/v1 tag exists; passed inputs match the action's declared inputs at that tag.
  • SECRETARIAT_APP_ID / SECRETARIAT_APP_PRIVATE_KEY / ANTHROPIC_API_KEY confirmed available to this repo.
  • No remaining references to .github/ai-review/ anywhere in the tree.
  • PR title passes scripts/check-pr-title.cjs.
  • End-to-end run — not exercisable from this PR. pull_request_target registers from the default branch and runs from the base SHA, and the workflow-mod gate makes Ladon decline to self-review. This PR needs a human approver; Ladon starts on the next PR. After merge, confirm registration with:
    gh api "repos/adcontextprotocol/adcp-client-python/actions/workflows" --jq '.workflows[] | "\(.path) \(.state)"'
    
    .github/workflows/ai-review.yml must be listed and active.

🤖 Generated with Claude Code

Ladon is the shared AdCP PR reviewer, maintained in
adcontextprotocol/actions (ladon/) and consumed by floating major tag.
This repo carried its own reviewer — a claude-code-action invocation
driving a 245-line vendored prompt — so every engine fix had to be
re-ported by hand.

- Replace .github/workflows/ai-review.yml with the canonical consumer
  workflow from ladon/INSTALL.md, copied verbatim. It pins
  adcontextprotocol/actions/ladon/review@ladon/review/v1 and keeps the
  pull_request_target posture: base-SHA-only checkout, PR head never
  fetched or executed, every github.event.* value reaching a shell via
  env:. Job name code_review is unchanged, so the status-check context
  is preserved.
- Delete .github/ai-review/expert-adcp-reviewer.md. Voice, severity
  model, largest-file rule, test-plan honesty, and inline-comment format
  now come from the baseline rules in the actions repo.
- Add LADON.md carrying the repo-specific half of the retired prompt as
  parsed config: semver signal on the adcp.* surface, discriminated-union
  forward compat, types import layering, generated-code-is-not-source
  (including the codegen renumbering-churn rule from CLAUDE.md),
  ctx_metadata credential prohibition, CI-gate regressions, and the
  high-risk / trivial path globs.

Secrets move from IPR_APP_ID / IPR_APP_PRIVATE_KEY to
SECRETARIAT_APP_ID / SECRETARIAT_APP_PRIVATE_KEY; both pairs plus
ANTHROPIC_API_KEY already resolve to this repo as org-level secrets.

Co-Authored-By: Claude <noreply@anthropic.com>
@aao-ipr-bot

aao-ipr-bot Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

⚠️ Argus review could not complete

The automated review encountered an issue (possibly reached max turns, timed out, or failed to post the final gh pr review). A human reviewer should take this PR.

View workflow run

This is an automated message from the Argus AI review workflow.

bokelley
bokelley previously approved these changes Aug 16, 2026

@bokelley bokelley left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refreshed onto v7.0.1-era main and revalidated the canonical Ladon consumer workflow, base-SHA-only security posture, preserved code_review status context, Opus 4.8 model pin, and LADON.md parser structure. Approving the reviewer migration.

@bokelley
bokelley enabled auto-merge (squash) August 16, 2026 21:55
aao-ipr-bot[bot]
aao-ipr-bot Bot previously approved these changes Aug 16, 2026

@aao-ipr-bot aao-ipr-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving — the pull_request_target posture is intact and the workflow-mod gate is fail-closed. Two hardening regressions ride along; neither blocks, but do them before this sees fork traffic. Retires 245 lines of vendored prompt for a floating-tag consumer of the shared engine — the right call, since every Argus fix otherwise had to be hand-ported.

Things I checked

  • Base-SHA isolation preserved. Checkout pins ref: ${{ github.event.pull_request.base.sha }} (ai-review.yml); PR head is never fetched, checked out, or executed. Ladon reads head content via the GitHub API only. Config-poisoning is dead on arrival — LADON.md is read from the trusted base workspace, so a fork rewriting it in its head has no effect on the run.
  • Gate is fail-closed, not fail-open. The while read / case loop matches only the two literals .github/workflows/ai-review.yml|LADON.md from the authoritative API file list; a crashed gate (set -euo pipefail) fails the job and the implicit success() on Run Ladon's if: skips the review. No path reviews a PR that edits the reviewer.
  • No command injection. No github.event.* free-text reaches a run: block; the shell consumes only PR_NUMBER (int), REPO, and the token via quoted env:. The github-script step reads process.env.MODIFIED_FILES, not ${{ }} interpolation. code-reviewer and security-reviewer both cleared this independently.
  • Required-status-check context preserved. Job key stays jobs.code_review.
  • paths-ignore partition is correct. .github/ai-review/** → LADON.md; pure-review-file PRs are suppressed at the trigger, mixed PRs fall through to the gate.
  • LADON.md is internally consistent. No path in both High-Risk and Trivial — generated surfaces (generated_poc/**, _generated.py) are Trivial, hand-maintained type-layer files (_eager.py, _ergonomic.py, _forward_compat.py, aliases.py) are High-Risk. All 17 high-risk / 3 trivial globs resolve to real paths. No dangling .github/ai-review/ reference survives.

Follow-ups (non-blocking — file as issues)

  • Raw org App private key now crosses into a mutable-tag action. Run Ladon (ai-review.yml:501) passes app-private-key: ${{ secrets.SECRETARIAT_APP_PRIVATE_KEY }} straight to adcontextprotocol/actions/ladon/review@ladon/review/v1. Argus minted a short-lived scoped installation token in the app-token step and handed the action only that token — never the private key. The App key's blast radius is org-wide, not this-PR-scoped. security-reviewer: Medium — a single repoint of ladon/review/v1 exfiltrates the private key + ANTHROPIC_API_KEY. Not disclosed in the PR body; this is the sharpest delta from Argus. Mitigation, in order: SHA-pin the action (PR body cites 2b4b3aa) and let Dependabot bump the pin; or, if ladon/review accepts a pre-minted token input, feed it steps.app-token.outputs.token and keep the raw key out of the action.
  • Action pins downgraded to floating tags. actions/checkout@v5, create-github-app-token@v1, github-script@v7, and the org action are mutable refs on a job holding pull-requests: write + id-token: write + secrets. The actions/* are GitHub-owned (low), the org action is the load-bearing one (folds into the finding above). You disclosed this as a verbatim INSTALL.md copy — acceptable, but the org action is worth a SHA pin regardless of the template.
  • persist-credentials: false dropped from checkout (ai-review.yml:356). Low impact — nothing pushes, head never executes — but it was real hardening you flagged yourself. Add it back; it's free.

Test-plan note

The one unchecked box — "End-to-end run — not exercisable from this PR" — is structurally honest, not a dodge: pull_request_target registers from the default branch and the workflow-mod gate makes Ladon decline to self-review, so the new reviewer cannot validate against its own installation PR by design. The mechanics were verified statically by both experts. Confirm registration post-merge with the gh api .../actions/workflows command in the body.

Minor nit (non-blocking)

  1. Test-plan H2 count. The plan says "all six H2 headings resolve," but LADON.md ships five — ## Gated Paths is intentionally omitted (correctly, since there's no CODEOWNERS post-#968). Every other count (17/3/1/1/7) is exact. Reconcile the wording.

Notable that the last review this Argus prompt runs is the one deleting it. Approving.

@bokelley
bokelley dismissed stale reviews from aao-ipr-bot[bot] and themself via 0ba1811 August 16, 2026 22:03
bokelley
bokelley previously approved these changes Aug 16, 2026

@bokelley bokelley left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approved after the focused workflow-security fix: GitHub primitives are SHA-pinned, the gate token is limited to pull-request writes, the intentional Ladon v1 float is documented and narrowly exempted, and local actionlint plus zizmor 1.29.0 report clean.

@bokelley bokelley left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final re-approval: Ladon is now pinned to the reviewed v1 commit because it receives the Secretariat private key, checkout credentials are not persisted, all GitHub primitives are SHA-pinned, App-token scope is least-privilege, and actionlint plus zizmor 1.29.0 are clean.

@aao-ipr-bot aao-ipr-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean CI migration — retires the vendored 245-line Argus prompt for the centrally-maintained Ladon composite. Right call: engine fixes now propagate by tag instead of hand-porting, and the pull_request_target posture is handled correctly rather than re-derived.

The workflow-mod gate means this is the retiring reviewer signing off on its successor — Ladon declines to self-review a PR that touches the review system, so a human (and Argus, once) owns this one.

Things I checked

  • pull_request_target posture is intact. Checkout is github.event.pull_request.base.sha with persist-credentials: false (.github/workflows/ai-review.yml:33-38); PR head is never fetched, checked out, or executed — setup reads head content via the GitHub API as data. security-reviewer traced the full orchestrator → setup → reviewer chain and confirmed untrusted PR content reaches the LLM only as review input, never a run:.
  • The action is SHA-pinned, not floating. adcontextprotocol/actions/ladon/review@2b4b3aa89fee5c91ccfc1ba9fc93a6bea3907d44 # ladon/review/v1security-reviewer dereferenced refs/tags/ladon/review/v1 and it resolves to exactly 2b4b3aa8. The hardening commit 0ba1811 earned its message. (The PR body's floating-v1 framing describes an earlier revision of this branch.)
  • Job name code_review is unchanged — required-status-check context preserved. No branch-protection break.
  • Gate fail-closes. paths-ignore suppresses pure-review-file PRs; the case \"$f\" in .github/workflows/ai-review.yml|LADON.md) match handles mixed PRs; Run Ladon is gated if: steps.workflow-mod.outputs.modified != 'true', and the fallback posts event: 'COMMENT' — never APPROVE. The modified_files heredoc is injection-safe: only the two matched literals can ever enter GITHUB_OUTPUT.
  • No github.event.* reaches a shell as codePR_NUMBER (int) and REPO (github.repository) via env:; the github-script step reads MODIFIED_FILES via process.env. No PR title/body/branch interpolated into run:.
  • Token scoping tightened — gate token mints permission-pull-requests: write only, dropping Argus's permission-issues: write (consistent with the switch from issues.createComment to pulls.createReview).
  • Every path in LADON.md existswebhook_auth.py, signing/**, _eager.py, _idempotency.py, migrate/**, compat/**, mypy_plugin.py, ADCP_VERSION, schemas/**. Release-stack branch release-please--branches--main--components--adcp matches the real release PR head. No ## Gated Paths — correct, CODEOWNERS was removed in #968.

Test plan honesty

The final checkbox — "End-to-end run — not exercisable from this PR" — is unchecked, and it covers the primary behavior (does the reviewer actually run). This isn't a skipped check: pull_request_target registers from the default branch and the workflow-mod gate makes Ladon decline to self-review, so it is structurally impossible to exercise until this lands on main. The post-merge gh api .../actions/workflows verification in the PR body is therefore load-bearing — run it once merged and confirm .github/workflows/ai-review.yml shows active.

Follow-ups (non-blocking — file as issues)

  • LADON.md import-layering allowlist understates the enforced set. The ### Mandatory: type-system import layering block lists seven modules; the enforced allowlist in tests/test_import_layering.py:32-68 has ten — it also includes legacy.py, canonical_creative.py, and canonical_decl.py. Because this is reviewer config, Ladon will read a legitimate generated-layer import in any of those three as a false-positive high layering breach. Fix: add the three, or reword "Only … may import" to defer to tests/test_import_layering.py as source of truth. (## High-Risk Paths omits the same three — worth adding for consistency.)
  • Supply-chain depth (upstream — adcontextprotocol/actions). security-reviewer: the top-level SHA pin stops at the orchestrator's own tree; its frozen action.yml pulls ladon/setup@v1, ladon/reviewer@v1, ladon/arbiter@v1, and create-github-app-token@v3 by floating tag. Anyone able to move those tags runs code with SECRETARIAT_APP_PRIVATE_KEY + ANTHROPIC_API_KEY. Same-org softens it, but the top-level pin is partly cosmetic. Raise upstream.
  • LLM-facing token minted unscoped (upstream). The orchestrator's internal token carries the App's full installation permissions — the only bound on blast radius is the CLI allowedTools read-only allowlist. Confirm the Secretariat App installation is itself minimally scoped.

Minor nits (non-blocking)

  1. id-token: write appears unused. .github/workflows/ai-review.yml:26 grants it, but no step consumes OIDC — app tokens are minted from app-id/private-key. Drop unless a downstream Ladon step needs it.
  2. Silent no-review if the gate step's gh api fails. Check for review-workflow modifications runs set -euo pipefail; a gh api --paginate failure fails the step, and both downstream steps carry custom if: (implicitly ANDed with success()), so the PR gets no review and no failure comment — the old Comment on PR if Argus review failed fallback is gone. Acceptable per the PR body's delegation note; flagging as the one adopter-visible capability actually dropped.
  3. app-id input is deprecated in favor of client-id. PR body already acknowledges staying on the documented template. Fine.

Approving on the strength of the verified base-SHA posture plus the fail-closed workflow-mod gate. Post-merge, run the workflow-registration check.

@bokelley
bokelley merged commit 054c106 into main Aug 16, 2026
28 checks passed
@bokelley
bokelley deleted the install-ladon branch August 16, 2026 22:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants