ci: sync Antigravity reviewer to the fixed comment-selection version - #341
Conversation
Bring the agy PR reviewer up to the version already on RustySNES main (via its #270 template-hardening sync + #273 comment-selection fix), superseding the stale sync PR, which carried an earlier version with a real self-deletion bug that agy flagged. What this version adds over the prior sync: - The just-posted review comment's id now comes from the POST itself (`gh pr comment` prints the new comment's URL; its trailing `#issuecomment-<id>` is authoritative), NOT from re-querying the comment list. The re-query raced GitHub's read replication -- right after posting, the list could still omit the new comment, so the "delete all but the newest" exclusion matched nothing and the run deleted the review it had just published (publish-before-delete turning into publish-then-destroy). - SELECT_STALE_JQ: the delete-selection jq filter is now a named, readonly constant (author + id-exclusion selects), exercised directly by a new offline self-test. - scripts/agy-review-selftest.sh: a network-free, gh-free test of that filter against fixtures (the filter has been wrong twice, both times invisibly -- the review still posted, so nothing observed it). Six checks, all passing. - Fail-closed flock (exit rather than run two agy processes unserialized) plus an empty-AGY_LOCK guard. - Keeps the URL-only OAuth-leak guard (oauth_url_present) unchanged. Byte-identical with RustySNES main and the shared reviewer template. bash -n clean; the self-test passes all six checks. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
📝 WalkthroughWalkthroughThe review script now fails closed for lock and OAuth checks, posts the new comment before cleanup, filters stale comments with a named jq expression, and preserves comments when cleanup data is unavailable. A self-test validates selection, API arguments, and failure behavior. ChangesReview execution hardening
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant agy-review.sh
participant GitHub API
participant jq
agy-review.sh->>GitHub API: Post new review comment
GitHub API-->>agy-review.sh: Return comment ID
agy-review.sh->>GitHub API: List issue comments
GitHub API-->>agy-review.sh: Return comments
agy-review.sh->>jq: Select stale comment IDs
jq-->>agy-review.sh: Return IDs excluding new comment
agy-review.sh->>GitHub API: Delete stale comments
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 8 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (8 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Antigravity review (Gemini via Ultra)This PR updates Blocking issuesNone found. Suggestions
Nitpicks
Automated first-pass review by |
There was a problem hiding this comment.
Pull request overview
This PR updates the Antigravity (agy) PR-review automation to a safer, tested comment-posting/de-duplication implementation (matching the already-fixed upstream template), preventing races that could delete the freshly-posted review and adding an offline guardrail for the deletion-selection jq filter.
Changes:
- Make stale-comment deletion deterministic by extracting the newly-posted comment ID from
gh pr commentoutput (avoids read-replication races) and deleting only prior matching bot comments. - Introduce a named,
readonlySELECT_STALE_JQfilter for stale-comment selection and apply it viajq --arg/--argjson(instead ofgh api --jqmisuse). - Add
scripts/agy-review-selftest.shto validate the selection filter offline against fixtures and prevent regressions.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| scripts/agy-review.sh | Fixes comment post/delete ordering and stale-comment selection (uses POST-returned comment id + named jq filter) and tightens serialization via fail-closed flock. |
| scripts/agy-review-selftest.sh | Adds an offline, fixture-based test to validate the stale-comment jq selection logic and guard against prior regressions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/agy-review.sh`:
- Around line 557-618: Replace the `gh pr comment` publication in the
review-posting flow with `gh api` creating the issue comment, and parse the
response JSON `id` directly into `new_comment_id`. Preserve the existing
post-failure handling and stale-comment deletion logic, using the API response
as the authoritative identifier instead of parsing undocumented CLI output.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 096a2e92-8155-4b20-afeb-f372c7733b7a
📒 Files selected for processing (2)
scripts/agy-review-selftest.shscripts/agy-review.sh
| # --- post fresh, THEN replace any prior review comment -------------------------- | ||
| # Publish-before-delete, deliberately: if this ordering were reversed and posting failed | ||
| # afterward (a transient gh/API error), the PR would be left with NO review comment at all | ||
| # instead of the still-valid prior one. Posting first means a failure here can only ever | ||
| # leave a harmless duplicate, never a silent loss of the last review. | ||
| # The posted comment's id comes from the POST itself, not from a read-back. `gh pr comment` | ||
| # prints the new comment's URL, whose trailing `#issuecomment-<id>` is authoritative the instant | ||
| # it returns. Re-querying the comment list to find "the newest one with our marker" raced with | ||
| # GitHub's own read replication: right after posting, the list can still omit it, and then the | ||
| # exclusion below matched nothing and the script deleted the comment it had just published -- | ||
| # turning publish-before-delete into publish-then-destroy, the exact failure the ordering exists | ||
| # to prevent. | ||
| if ! post_output="$(gh pr comment "$PR" --repo "$REPO" --body-file "$body_file" 2>&1)"; then | ||
| # Nothing is deleted when the post fails: the prior review comment is still the best | ||
| # information the PR has, and removing it would leave no review at all. | ||
| log "failed to post review to ${REPO}#${PR}: ${post_output}" | ||
| exit 1 | ||
| fi | ||
| log "posted review to ${REPO}#${PR}" | ||
| new_comment_id="$(printf '%s\n' "$post_output" | sed -n 's/.*#issuecomment-\([0-9][0-9]*\).*/\1/p' | tail -n 1)" | ||
|
|
||
| # A failed delete is logged, not swallowed: silently ignoring it would let a transient API/perms | ||
| # error leave the old comment in place alongside the new one, so runs accumulate duplicates. | ||
| # The author filter is load-bearing, not cosmetic: without it, ANY user could put the | ||
| # marker (an HTML comment) in a PR comment and have this bot delete arbitrary comments on | ||
| # the next run. Only ever delete OUR OWN bot's prior review comments -- and only ones from | ||
| # BEFORE this run (the just-posted comment's own id is excluded so it can never delete itself). | ||
| if [ -z "$new_comment_id" ]; then | ||
| # FAIL CLOSED. Without a known id there is no way to tell the new comment from the old ones, | ||
| # and the safe direction is unambiguous: a leftover duplicate is noise, deleting the review | ||
| # that was just posted is data loss. | ||
| log "warning: could not determine the posted comment id; leaving prior review comments in place" | ||
| else | ||
| # `--arg`/`--argjson` rather than shell interpolation into the filter: the marker is an HTML | ||
| # comment today, but a quote or a backslash in it would otherwise break the jq program itself | ||
| # rather than simply not matching. | ||
| # | ||
| # Those are JQ flags, so the JSON is fetched raw and piped into a real `jq` — `gh api` has no | ||
| # `--arg`/`--argjson` of its own and rejects them. Handing them to `gh api --jq` made it exit | ||
| # non-zero on every run; with the old `2>/dev/null` swallowing the message and `set -o pipefail` | ||
| # in force, the script then died *after* posting, so the stale comments were never deleted and | ||
| # the job went red for a reason nothing printed. stderr is kept this time for exactly that | ||
| # reason. (`--paginate` without `--jq` emits one JSON array per page; `jq` reads that stream | ||
| # fine, applying `.[]` to each.) | ||
| stale_ids="$( | ||
| gh api "repos/${REPO}/issues/${PR}/comments" --paginate \ | ||
| | jq -r --arg marker "$MARKER" --argjson new_id "$new_comment_id" "$SELECT_STALE_JQ" | ||
| )" || { | ||
| log "warning: could not list prior review comments; leaving them in place" | ||
| stale_ids="" | ||
| } | ||
| while read -r cid; do | ||
| [ -n "$cid" ] || continue | ||
| if ! gh api -X DELETE "repos/${REPO}/issues/comments/${cid}" >/dev/null 2>&1; then | ||
| log "warning: could not delete prior review comment ${cid}; a duplicate may result" | ||
| fi | ||
| done <<< "$stale_ids" | ||
| fi | ||
|
|
||
| # The delete loop above is the last real work; end on a defined status so a stray non-zero from | ||
| # it can never be mistaken for "the review failed" once the comment is already published. | ||
| exit 0 |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🌐 Web query:
Does the gh CLI gh pr comment command print the created comment's URL to stdout on success, and is this documented/stable behavior?
💡 Result:
The gh pr comment command in the GitHub CLI does not print the created comment's URL to stdout upon success [1][2]. The command is designed for interaction with the GitHub PR system, and its standard behavior is to perform the requested action (posting the comment) without returning the URL or other machine-readable metadata about the created resource to standard output [1][2]. Because this behavior is not documented as a feature of the command, it is not considered a stable or supported way to retrieve the comment URL [1]. Users who need the URL or further details about the created comment should instead use the gh api command, which allows for direct interaction with the GitHub REST API and can return the full JSON response containing the resource URL [2][3]. For example, to obtain the comment URL or ID, you can use the REST API endpoint for creating a comment via gh api [2][4]: gh api --method POST /repos/{owner}/{repo}/issues/{issue_number}/comments -f body="your comment body" This approach provides a stable, machine-readable output that includes the full details of the created resource [2][3].
Citations:
- 1: https://manpages.opensuse.org/Tumbleweed/gh/gh-pr-comment.1.en.html
- 2: https://stackoverflow.com/questions/78064829/use-cli-to-write-a-comment-pinned-to-a-file-and-line-number-on-a-github-pull-req
- 3: Direct gh api call for PR review comments returned 422; review payload works cli/cli#13358
- 4: https://docs.github.com/en/rest/guides/working-with-comments
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- script references ---'
rg -n -C 4 'gh pr comment|new_comment_id|gh version|GH_PAGER|gh-cli|github-cli' scripts .github 2>/dev/null || true
printf '%s\n' '--- workflow/action files ---'
git ls-files '.github/*' 'scripts/agy-review.sh' | sed -n '1,160p'Repository: doublegate/RustyNES
Length of output: 6944
🌐 Web query:
site:cli.github.com/manual/gh_pr_comment gh pr comment output URL stdout
💡 Result:
The gh pr comment command does not explicitly output the URL of the created comment to standard output (stdout) [1]. The official documentation for gh pr comment lists options for specifying the body of the comment, such as using --body or --body-file, but it does not document an option to return the URL of the resulting comment upon successful execution [1]. Because this information is not provided by the command's standard output, it is not possible to capture the comment URL directly from the command execution in the manner requested [1].
Citations:
🌐 Web query:
site:github.com/cli/cli "pr comment" "issuecomment-" output
💡 Result:
The term "issuecomment-" refers to the prefix used in GitHub's REST and GraphQL APIs to identify individual comments on issues or pull requests [1][2]. In the context of GitHub Actions and the GitHub CLI (gh), this prefix is used when referencing specific comments for operations like updates or deletions [2][3]. Key points regarding "issuecomment-" and GitHub operations: 1. Identification: GitHub identifies comments using a unique numeric ID [1]. When interacting with the API or using tools that interface with it, these IDs are often prefixed with "issuecomment-" (e.g., issuecomment-123456789) to explicitly denote the object type [2]. 2. Usage in GitHub CLI: While the GitHub CLI (gh) provides high-level commands like gh pr comment --edit-last to simplify common tasks [4][2], it does not natively provide a direct command like gh comment update that takes an "issuecomment-" ID as a primary argument [1][2]. Instead, users often interact with these IDs via the GitHub API directly using gh api [2] or by piping JSON output from commands like gh issue view --json comments into tools like jq [1][3]. 3. PR vs. Issue Comments: In GitHub's data model, a pull request is a type of issue, meaning that comments on pull requests are technically issue comments [5][6]. This is why the same API endpoint (/repos/{owner}/{repo}/issues/comments/{comment_id}) is used to manage comments for both issues and pull requests [1]. 4. Automation and Permissions: When attempting to post or manage comments within a GitHub Actions workflow, ensure your job has the appropriate permissions (typically pull-requests: write or issues: write) [7][6]. If you encounter errors, verify that you are using the correct secret (secrets.GITHUB_TOKEN) and that the workflow event (e.g., pull_request vs. pull_request_target) provides the necessary access to the repository [6]. If you are trying to automate the updating or deletion of a comment, the recommended pattern is to retrieve the list of comments via the GitHub CLI in JSON format, use jq to filter for the specific comment ID, and then call the REST API via gh api [2][3].
Citations:
- 1: [Feature] Add a new command for handling comments. cli/cli#10210
- 2: Support updating comments/reviews cli/cli#3613
- 3:
gh pr/issue commentallow editing selected comments cli/cli#10865 - 4: gh pr comment --edit-last now creates if no previous comment cli/cli#10580
- 5: Putting a comment on an issue doesnt work if that issue is a PR cli/cli#4127
- 6:
gh pr commentdoes not work even withpermissions: write-allin workflow cli/cli#8374 - 7:
gh pr commentdoes not work even withpermissions: write-allin workflow cli/cli#8374
🏁 Script executed:
#!/bin/bash
set -eu
python3 - <<'PY'
import json, urllib.request
urls = [
"https://raw.githubusercontent.com/cli/cli/trunk/pkg/cmd/pr/comment/comment.go",
"https://raw.githubusercontent.com/cli/cli/master/pkg/cmd/pr/comment/comment.go",
]
for url in urls:
try:
data = urllib.request.urlopen(url, timeout=10).read().decode()
except Exception as exc:
print(f"{url}: {exc}")
continue
print(f"--- {url} ---")
for i, line in enumerate(data.splitlines(), 1):
if "URL" in line or "url" in line or "Fprintln" in line or "Println" in line:
print(f"{i}:{line}")
PYRepository: doublegate/RustyNES
Length of output: 586
🌐 Web query:
GitHub CLI source pkg/cmd/pr/comment/comment.go prints comment URL Fprintln
💡 Result:
In the GitHub CLI (gh) source code, the logic for adding a comment to a pull request is not entirely contained within pkg/cmd/pr/comment/comment.go, which serves primarily as the command definition [1]. Instead, the operational logic—including the printing of the resulting comment URL—is largely delegated to shared functionality found in pkg/cmd/pr/shared/commentable.go [2]. Specifically, when a comment is successfully created via the GitHub API, the commentable.go file uses fmt.Fprintln to output the URL of the new comment to the standard output stream (unless the quiet flag is set) [2]. The relevant code block typically looks like this: if!opts.Quiet { fmt.Fprintln(opts.IO.Out, url) } This behavior ensures that after executing gh pr comment, the user is provided with a direct link to the created resource [3][2]. Other formatting and printing of existing comments (e.g., when viewing them) are handled separately in files such as pkg/cmd/pr/shared/comments.go, which also utilize fmt.Fprintln to format and display comment bodies and their associated metadata in the terminal [4].
Citations:
- 1: https://fossies.org/linux/gh-cli/pkg/cmd/pr/comment/comment.go
- 2: https://fossies.org/linux/gh-cli/pkg/cmd/pr/shared/commentable.go
- 3: https://cli.github.com/manual/gh_pr_comment
- 4: https://fossies.org/linux/gh-cli/pkg/cmd/pr/shared/comments.go
🏁 Script executed:
#!/bin/bash
set -eu
for url in \
https://raw.githubusercontent.com/cli/cli/trunk/pkg/cmd/pr/shared/commentable.go \
https://raw.githubusercontent.com/cli/cli/trunk/pkg/cmd/pr/comment/comment.go
do
echo "--- $url ---"
curl -ksSfL "$url" | nl -ba | grep -E -C 5 'Fprintln|Quiet|url|comment' | head -n 100
doneRepository: doublegate/RustyNES
Length of output: 530
🌐 Web query:
site:cli.github.com/manual/gh_pr_comment "URL" "standard output"
💡 Result:
There is no technical concept known as URL standard output. The phrase appears to be a misunderstanding or a conflation of terms. The WHATWG URL Standard defines how URLs are parsed and serialized [1], but it does not define an output format under that specific name. In command-line interface tools, such as the GitHub CLI, standard output is a common concept where commands return information to the terminal or allow data to be piped from a file [1].
Citations:
🌐 Web query:
site:cli.github.com/manual/gh_pr_comment "quiet" "comment"
💡 Result:
No results were found for:
site:cli.github.com/manual/gh_pr_comment "quiet" "comment"
The GitHub CLI manual page may not contain a quiet option for gh pr comment.
🏁 Script executed:
#!/bin/bash
set -eu
python3 - <<'PY'
import ssl, urllib.request
ctx = ssl._create_unverified_context()
urls = [
"https://raw.githubusercontent.com/cli/cli/trunk/pkg/cmd/pr/shared/commentable.go",
"https://raw.githubusercontent.com/cli/cli/trunk/pkg/cmd/pr/comment/comment.go",
"https://cli.github.com/manual/gh_pr_comment",
]
for url in urls:
print(f"--- {url} ---")
try:
text = urllib.request.urlopen(url, context=ctx, timeout=15).read().decode("utf-8", "replace")
except Exception as exc:
print(exc)
continue
lines = text.splitlines()
for i, line in enumerate(lines, 1):
if any(term in line for term in ("Fprintln", "Quiet", "quiet", "URL", "url", "body-file")):
lo, hi = max(1, i - 4), min(len(lines), i + 4)
for j in range(lo, hi + 1):
print(f"{j}:{lines[j-1]}")
print()
PYRepository: doublegate/RustyNES
Length of output: 8849
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- workflow setup and gh invocations ---'
rg -n -C 5 'setup.*gh|install.*gh|gh version|gh pr comment|apt.*gh|brew.*gh|cli/cli' .github scripts README.md 2>/dev/null || trueRepository: doublegate/RustyNES
Length of output: 23612
🌐 Web query:
GitHub CLI gh pr comment manual created comment URL printed stdout
💡 Result:
The GitHub CLI gh pr comment command does not natively print the URL of the created comment to standard output (stdout) [1][2]. The command primarily functions to add a comment and provides user feedback (if any) via standard error (stderr) or the terminal display when run interactively, rather than returning the new comment's URL as machine-readable output. If you need to obtain the URL of a newly created comment within a script, you can use the gh api command instead of gh pr comment. By using the GitHub REST API, you can capture the JSON response, which contains the html_url of the comment [3][4]. Example approach using gh api: # Example script to create a comment and capture its URL comment_url=$(gh api \ --method POST \ -H "Accept: application/vnd.github+json" \ /repos/:owner/:repo/issues/:number/comments \ -f body="Your comment text here" \ --jq.html_url) echo "Comment created at: $comment_url" This method gives you direct access to the created comment's URL, which you can then store or process as needed [5].
Citations:
- 1: https://cli.github.com/manual/gh_pr_comment
- 2: https://manpages.opensuse.org/Tumbleweed/gh/gh-pr-comment.1.en.html
- 3: https://docs.github.com/en/rest/pulls/comments
- 4: https://pygithub.readthedocs.io/en/v1.57/github%5Fobjects/PullRequestComment.html
- 5: https://stackoverflow.com/questions/78064829/use-cli-to-write-a-comment-pinned-to-a-file-and-line-number-on-a-github-pull-req
Pin gh or use a machine-readable comment response
The current gh pr comment implementation prints the created URL when --quiet is absent, and this invocation does not use --quiet. However, the manual does not document this stdout contract, and the workflow does not pin a gh version. Pin the CLI version and test the output, or create the comment with gh api and parse its JSON id.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/agy-review.sh` around lines 557 - 618, Replace the `gh pr comment`
publication in the review-posting flow with `gh api` creating the issue comment,
and parse the response JSON `id` directly into `new_comment_id`. Preserve the
existing post-failure handling and stale-comment deletion logic, using the API
response as the authoritative identifier instead of parsing undocumented CLI
output.
Summary
Brings the
agyPR reviewer up to the fixed version already on RustySNES main (via its #270 template-hardening sync + #273 comment-selection fix + a self-test), superseding the stale sync PR in this repo — which carried an earlier version with a real self-deletion bug agy flagged.Fix over the prior sync
gh pr commentprints the new comment's URL (…#issuecomment-<id>); that id is used directly. The old approach re-queried the comment list to find "the newest with our marker", which raced GitHub's read replication — right after posting, the list could still omit the new comment, so the "delete all but the newest" exclusion matched nothing and the run deleted the review it had just published (publish-before-delete turning into publish-then-destroy).SELECT_STALE_JQ— the delete-selection jq filter is now a namedreadonlyconstant (the author + id-exclusionselects), exercised directly by a new offline test.scripts/agy-review-selftest.sh— a network-free,gh-free test of that filter against fixtures (it has been wrong twice, both times invisibly — the review still posted, so nothing observed it). 6 checks, all passing.AGY_LOCKguard; the URL-only OAuth-leak guard (oauth_url_present) is unchanged.Verification
eabd2571…) with RustySNES main and the shared reviewer template.bash -nclean;bash scripts/agy-review-selftest.sh→ all 6 checks pass.🤖 Generated with Claude Code
Summary by CodeRabbit
Bug Fixes
Tests