Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
184 changes: 184 additions & 0 deletions .agents/skills/get-reviews/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,184 @@
---
name: get-reviews
description: >-
This skill is utilized when fetching reviews from GitHub pull requests.
Get the reviews, organize, and help to resolve them.
argument-hint: "Provide the number of the pull request to get the reviews."
---

<!-- deno-fmt-ignore-file -->

Get reviews from GitHub pull requests
=====================================

This skill is utilized when requesting reviews from GitHub pull requests.

1. Get the reviews.
2. Organize the reviews.
3. Help to resolve the reviews.


Get the reviews
---------------

To get the reviews from a GitHub pull request, you can use the GitHub API.
Check the [`gh` CLI tool][gh] is installed and authenticated.
`gh auth status` can be used to check the authentication status.
If `gh` isn't installed, try installing it by `apt install gh`.
If authentication is not set up, tell the contributor to run `gh auth login`
to authenticate with GitHub.

Use the GraphQL API to fetch the reviews for a specific pull request.
Check [fetch\_reviews.sh](./fetch_reviews.sh) to fetch the reviews
and save them in a JSON file:

- Fill the variables in the command and run the command to fetch the reviews:

~~~~ bash
# Omit the new lines
PR_NUMBER= NUMBER_OF_PR_COMMENTS= NUMBER_OF_REVIEWS= \
NUMBER_OF_THREADS= NUMBER_OF_COMMENTS_PER_THREAD= LAST_CURSOR= \
bash .agents/skills/get-reviews/fetch_reviews.sh
~~~~


- `PR_NUMBER`: The number of the pull request to fetch reviews from.
- `NUMBER_OF_PR_COMMENTS`: The number of PR comments to fetch.
- `NUMBER_OF_REVIEWS`: The number of reviews to fetch.
- `NUMBER_OF_THREADS`: The number of review threads to fetch.
- `NUMBER_OF_COMMENTS_PER_THREAD`:
The number of comments per review thread to fetch.
- `LAST_CURSOR`: The cursor for incremental fetches. Optional.

- For incremental fetches, save `pageInfo.endCursor` from the previous
fetch and pass it as `after: $LAST_CURSOR` (a base64 cursor, not a
review thread node ID) so the query returns only new review threads.

- Use `jq` to filter the reviews and information if necessary.
Comment thread
coderabbitai[bot] marked this conversation as resolved.

The fetched JSON files in *plans/{PR\_NUMBER}/fetched/* contain the raw data
of PR comments, reviews, and review threads (with `pullRequestReview`
back-references on thread comments). When more context is needed later
(e.g., to resolve which review a thread belongs to, or to check the original
body of a comment), refer back to these files instead of re-fetching.

[gh]: https://cli.github.com/


Organize the reviews
--------------------

After fetching the PR and its reviews, organize the reviews.
[*plans* directory in the root](../../../plans/) is a good place to store them.
Comment thread
2chanhaeng marked this conversation as resolved.
*plans/* is already ignored in `.gitignore`, so don't check that it is ignored.

- *plans/{PR\_NUMBER}/index.md*: The main file for the PR.
- The body of the PR
- *plans/{PR\_NUMBER}/reviews/{REVIEW\_ID}.md*: The file for each review
which is not resolved.
- After applying or dismissing the review, move the file to
*plans/{PR\_NUMBER}/reviews/resolved/{REVIEW\_ID}.md*.
- If the review file is too long, move the content to
*plans/{PR\_NUMBER}/reviews/{REVIEW\_ID}/index.md*, and separate the
content into multiple files in the same directory. In this case, after
resolving the review, move the whole directory to
*plans/{PR\_NUMBER}/reviews/resolved/{REVIEW\_ID}/*.
Comment thread
2chanhaeng marked this conversation as resolved.
- *plans/{PR\_NUMBER}/reviews/resolved/{REVIEW\_ID}.md*: The file for each
review which is resolved.

**Don't use the first comment of the review thread as the review ID.**
The ID of the review thread starts with “PRRT\_”.
Use the first comment ID of the review thread only on the link.

Review threads aren't the only place that asks for changes. A PR-level
comment (in `comments` of the fetched JSON, whose ID starts with “IC\_”)
or the body of a review (in `reviews` of the fetched JSON, whose ID starts
with “PRR\_”) can also point out things to fix. When such a comment or
review body requests modifications, organize it as its own review file
alongside the review-thread files, using the same directory layout and
naming rules:

- Use the node ID of the PR comment (“IC\_…”) or the review (“PRR\_…”)
as the `{REVIEW\_ID}` in the file path.
- If a PR comment or review body only contains approval, general
impressions, questions without a modification request, or other content
with nothing to fix, skip it and do not create a file for it.
- If only a part of a longer PR comment or review body requests changes,
create a file only for that request and quote the relevant excerpt in
the file so the context is preserved.

The format of review files should be as [review.md](./review.md).
Read the review and draft the format based on the relevant information.
The files should be written in the contributor's language. But the title of
the item in the file (e.g., “Summary”, “Judgement”, “Plans”) should be in
English for consistency.
Comment thread
coderabbitai[bot] marked this conversation as resolved.

Empty the space between the “Title” and the “Summary” sections.

All related information with the review should be stored in
*plans/{PR\_NUMBER}/reviews/{REVIEW\_ID}.md* or the files in
*plans/{PR\_NUMBER}/reviews/{REVIEW\_ID}/*.

After organizing the reviews, show the links to the files to the contributor.


Help to resolve the reviews
---------------------------

The agent's role in this step is to *help* the contributor resolve the reviews,
not to resolve them on the agent's own authority. The final judgement always
belongs to the contributor; the agent's job is to surface the information needed
to decide, and then to carry out whatever the contributor decides.

For each unresolved review, give the contributor what they need to judge it,
rather than judging on their behalf:

- Add relative links to the code or documentation the review points at.
- Summarize the relevant facts, the trade-offs, and any available options.
- Where the facts are clear, the agent may include a *suggested* judgement,
but it must be clearly marked as a suggestion and never substitute for the
contributor's decision.

Let the contributor read the review files, decide the judgement and the plans
for each review, and update the review files if necessary. Do not apply or
dismiss anything until the contributor has decided. After the contributor
decides the judgement and the plans, apply or dismiss the reviews based on the
files.

Categorize the reviews and the plans, and apply them at once by category.
After applying the review, use [`/commit` skill](../commit/SKILL.md) to commit
Comment thread
2chanhaeng marked this conversation as resolved.
the changes. The commit message should include the related review links:

`https://github.com/fedify-dev/fedify/pull/{PR_NUMBER}#discussion_r{REVIEW_THREAD.COMMENTS[0].DATABASE_ID}`

Because these changes are produced with agent assistance, the commit message
must also carry the `Assisted-by: AGENT_NAME:MODEL_VERSION` trailer required by
[*AI_POLICY.md*](../../../AI_POLICY.md), in addition to the review links.

After committing the changes, update the review file to include the commit hash
and the comment section. If the review is dismissed, update the review file to
include the reason for dismissing and the comment section.

If the `Comments` are written only in the contributor's language, provide an
English translation and have the contributor review it. If they are written in
both languages, check for any discrepancies between the two. If differences
exist between the two versions, review them based on the facts and revise
the English version to match the content in the contributor's language.

Post all review comments in English, even if the file written in the
contributor's native language. The comments should be polite and constructive.

Before pushing commits, posting comments, or marking any review as resolved,
the contributor must verify the actual applied changes and the relevant
test/check results (e.g., `mise run check` and the related tests). Per
[*AI_POLICY.md*](../../../AI_POLICY.md), AI-created work must be fully verified
with human use; do not post comments or mark reviews resolved for changes whose
correctness is only hypothetical. The agent prepares everything for this
verification but leaves the verification itself to the contributor.

After resolving the reviews, pushing commits, posting comments, and updating
Comment thread
2chanhaeng marked this conversation as resolved.
the reviews as resolved, move the review files to
*plans/{PR\_NUMBER}/reviews/resolved*. Before moving the files, check status and
comments of the PR from GitHub. Use [fetch\_reviews.sh](./fetch_reviews.sh),
but instead of fetching all reviews and attributes, fetch only the necessary
attributes to check the review status and comments.
Comment thread
2chanhaeng marked this conversation as resolved.
88 changes: 88 additions & 0 deletions .agents/skills/get-reviews/fetch_reviews.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
#!/usr/bin/env bash
set -euo pipefail

: "${PR_NUMBER:?PR_NUMBER is required}"
: "${NUMBER_OF_PR_COMMENTS:?NUMBER_OF_PR_COMMENTS is required}"
: "${NUMBER_OF_REVIEWS:?NUMBER_OF_REVIEWS is required}"
: "${NUMBER_OF_THREADS:?NUMBER_OF_THREADS is required}"
: "${NUMBER_OF_COMMENTS_PER_THREAD:?NUMBER_OF_COMMENTS_PER_THREAD is required}"

PR_PATH="plans/$PR_NUMBER"
FETCHED_PATH="$PR_PATH/fetched"
mkdir -p "$FETCHED_PATH"
TIMESTAMP=$(date +"%m%d%H%M")
FETCHED_FILE="$FETCHED_PATH/$TIMESTAMP.json"
Comment thread
coderabbitai[bot] marked this conversation as resolved.
gh api graphql -f query='query(
$owner: String!,
$repo: String!,
$number: Int!,
$prComments: Int!,
$reviews: Int!,
$threads: Int!,
$comments: Int!,
$after: String
) {
repository(owner: $owner, name: $repo) {
pullRequest(number: $number) {
comments(first: $prComments) {
nodes {
id
databaseId
author { login }
body
url
createdAt
}
}
reviews(first: $reviews) {
nodes {
id
databaseId
author { login }
body
state
url
createdAt
}
}
reviewThreads(first: $threads, after: $after) {
pageInfo {
hasNextPage
endCursor
}
nodes {
id
isResolved
isOutdated
path
line
comments(first: $comments) {
nodes {
id
databaseId
author { login }
body
url
createdAt
pullRequestReview {
id
databaseId
}
}
}
}
}
}
}
}' \
-F owner=fedify-dev \
-F repo=fedify \
-F number="$PR_NUMBER" \
-F prComments="$NUMBER_OF_PR_COMMENTS" \
-F reviews="$NUMBER_OF_REVIEWS" \
-F threads="$NUMBER_OF_THREADS" \
-F comments="$NUMBER_OF_COMMENTS_PER_THREAD" \
${LAST_CURSOR:+-F after="$LAST_CURSOR"} \
| jq . > "$FETCHED_FILE"

echo "Fetched reviews and comments are saved to $FETCHED_FILE"
Loading