diff --git a/.agents/skills/get-reviews/SKILL.md b/.agents/skills/get-reviews/SKILL.md new file mode 100644 index 000000000..0f4491736 --- /dev/null +++ b/.agents/skills/get-reviews/SKILL.md @@ -0,0 +1,184 @@ +--- +name: get-reviews +description: >- + This skill is utilized when fetching reviews from GitHub pull requests. + Get the reviews, organize, and help to resolve them. +argument-hint: "Provide the number of the pull request to get the reviews." +--- + + + +Get reviews from GitHub pull requests +===================================== + +This skill is utilized when requesting reviews from GitHub pull requests. + +1. Get the reviews. +2. Organize the reviews. +3. Help to resolve the reviews. + + +Get the reviews +--------------- + +To get the reviews from a GitHub pull request, you can use the GitHub API. +Check the [`gh` CLI tool][gh] is installed and authenticated. +`gh auth status` can be used to check the authentication status. +If `gh` isn't installed, try installing it by `apt install gh`. +If authentication is not set up, tell the contributor to run `gh auth login` +to authenticate with GitHub. + +Use the GraphQL API to fetch the reviews for a specific pull request. +Check [fetch\_reviews.sh](./fetch_reviews.sh) to fetch the reviews +and save them in a JSON file: + + - Fill the variables in the command and run the command to fetch the reviews: + + ~~~~ bash + # Omit the new lines + PR_NUMBER= NUMBER_OF_PR_COMMENTS= NUMBER_OF_REVIEWS= \ + NUMBER_OF_THREADS= NUMBER_OF_COMMENTS_PER_THREAD= LAST_CURSOR= \ + bash .agents/skills/get-reviews/fetch_reviews.sh + ~~~~ + + + - `PR_NUMBER`: The number of the pull request to fetch reviews from. + - `NUMBER_OF_PR_COMMENTS`: The number of PR comments to fetch. + - `NUMBER_OF_REVIEWS`: The number of reviews to fetch. + - `NUMBER_OF_THREADS`: The number of review threads to fetch. + - `NUMBER_OF_COMMENTS_PER_THREAD`: + The number of comments per review thread to fetch. + - `LAST_CURSOR`: The cursor for incremental fetches. Optional. + + - For incremental fetches, save `pageInfo.endCursor` from the previous + fetch and pass it as `after: $LAST_CURSOR` (a base64 cursor, not a + review thread node ID) so the query returns only new review threads. + + - Use `jq` to filter the reviews and information if necessary. + +The fetched JSON files in *plans/{PR\_NUMBER}/fetched/* contain the raw data +of PR comments, reviews, and review threads (with `pullRequestReview` +back-references on thread comments). When more context is needed later +(e.g., to resolve which review a thread belongs to, or to check the original +body of a comment), refer back to these files instead of re-fetching. + +[gh]: https://cli.github.com/ + + +Organize the reviews +-------------------- + +After fetching the PR and its reviews, organize the reviews. +[*plans* directory in the root](../../../plans/) is a good place to store them. +*plans/* is already ignored in `.gitignore`, so don't check that it is ignored. + + - *plans/{PR\_NUMBER}/index.md*: The main file for the PR. + - The body of the PR + - *plans/{PR\_NUMBER}/reviews/{REVIEW\_ID}.md*: The file for each review + which is not resolved. + - After applying or dismissing the review, move the file to + *plans/{PR\_NUMBER}/reviews/resolved/{REVIEW\_ID}.md*. + - If the review file is too long, move the content to + *plans/{PR\_NUMBER}/reviews/{REVIEW\_ID}/index.md*, and separate the + content into multiple files in the same directory. In this case, after + resolving the review, move the whole directory to + *plans/{PR\_NUMBER}/reviews/resolved/{REVIEW\_ID}/*. + - *plans/{PR\_NUMBER}/reviews/resolved/{REVIEW\_ID}.md*: The file for each + review which is resolved. + +**Don't use the first comment of the review thread as the review ID.** +The ID of the review thread starts with “PRRT\_”. +Use the first comment ID of the review thread only on the link. + +Review threads aren't the only place that asks for changes. A PR-level +comment (in `comments` of the fetched JSON, whose ID starts with “IC\_”) +or the body of a review (in `reviews` of the fetched JSON, whose ID starts +with “PRR\_”) can also point out things to fix. When such a comment or +review body requests modifications, organize it as its own review file +alongside the review-thread files, using the same directory layout and +naming rules: + + - Use the node ID of the PR comment (“IC\_…”) or the review (“PRR\_…”) + as the `{REVIEW\_ID}` in the file path. + - If a PR comment or review body only contains approval, general + impressions, questions without a modification request, or other content + with nothing to fix, skip it and do not create a file for it. + - If only a part of a longer PR comment or review body requests changes, + create a file only for that request and quote the relevant excerpt in + the file so the context is preserved. + +The format of review files should be as [review.md](./review.md). +Read the review and draft the format based on the relevant information. +The files should be written in the contributor's language. But the title of +the item in the file (e.g., “Summary”, “Judgement”, “Plans”) should be in +English for consistency. + +Empty the space between the “Title” and the “Summary” sections. + +All related information with the review should be stored in +*plans/{PR\_NUMBER}/reviews/{REVIEW\_ID}.md* or the files in +*plans/{PR\_NUMBER}/reviews/{REVIEW\_ID}/*. + +After organizing the reviews, show the links to the files to the contributor. + + +Help to resolve the reviews +--------------------------- + +The agent's role in this step is to *help* the contributor resolve the reviews, +not to resolve them on the agent's own authority. The final judgement always +belongs to the contributor; the agent's job is to surface the information needed +to decide, and then to carry out whatever the contributor decides. + +For each unresolved review, give the contributor what they need to judge it, +rather than judging on their behalf: + + - Add relative links to the code or documentation the review points at. + - Summarize the relevant facts, the trade-offs, and any available options. + - Where the facts are clear, the agent may include a *suggested* judgement, + but it must be clearly marked as a suggestion and never substitute for the + contributor's decision. + +Let the contributor read the review files, decide the judgement and the plans +for each review, and update the review files if necessary. Do not apply or +dismiss anything until the contributor has decided. After the contributor +decides the judgement and the plans, apply or dismiss the reviews based on the +files. + +Categorize the reviews and the plans, and apply them at once by category. +After applying the review, use [`/commit` skill](../commit/SKILL.md) to commit +the changes. The commit message should include the related review links: + +`https://github.com/fedify-dev/fedify/pull/{PR_NUMBER}#discussion_r{REVIEW_THREAD.COMMENTS[0].DATABASE_ID}` + +Because these changes are produced with agent assistance, the commit message +must also carry the `Assisted-by: AGENT_NAME:MODEL_VERSION` trailer required by +[*AI_POLICY.md*](../../../AI_POLICY.md), in addition to the review links. + +After committing the changes, update the review file to include the commit hash +and the comment section. If the review is dismissed, update the review file to +include the reason for dismissing and the comment section. + +If the `Comments` are written only in the contributor's language, provide an +English translation and have the contributor review it. If they are written in +both languages, check for any discrepancies between the two. If differences +exist between the two versions, review them based on the facts and revise +the English version to match the content in the contributor's language. + +Post all review comments in English, even if the file written in the +contributor's native language. The comments should be polite and constructive. + +Before pushing commits, posting comments, or marking any review as resolved, +the contributor must verify the actual applied changes and the relevant +test/check results (e.g., `mise run check` and the related tests). Per +[*AI_POLICY.md*](../../../AI_POLICY.md), AI-created work must be fully verified +with human use; do not post comments or mark reviews resolved for changes whose +correctness is only hypothetical. The agent prepares everything for this +verification but leaves the verification itself to the contributor. + +After resolving the reviews, pushing commits, posting comments, and updating +the reviews as resolved, move the review files to +*plans/{PR\_NUMBER}/reviews/resolved*. Before moving the files, check status and +comments of the PR from GitHub. Use [fetch\_reviews.sh](./fetch_reviews.sh), +but instead of fetching all reviews and attributes, fetch only the necessary +attributes to check the review status and comments. diff --git a/.agents/skills/get-reviews/fetch_reviews.sh b/.agents/skills/get-reviews/fetch_reviews.sh new file mode 100644 index 000000000..4a607a1fe --- /dev/null +++ b/.agents/skills/get-reviews/fetch_reviews.sh @@ -0,0 +1,88 @@ +#!/usr/bin/env bash +set -euo pipefail + +: "${PR_NUMBER:?PR_NUMBER is required}" +: "${NUMBER_OF_PR_COMMENTS:?NUMBER_OF_PR_COMMENTS is required}" +: "${NUMBER_OF_REVIEWS:?NUMBER_OF_REVIEWS is required}" +: "${NUMBER_OF_THREADS:?NUMBER_OF_THREADS is required}" +: "${NUMBER_OF_COMMENTS_PER_THREAD:?NUMBER_OF_COMMENTS_PER_THREAD is required}" + +PR_PATH="plans/$PR_NUMBER" +FETCHED_PATH="$PR_PATH/fetched" +mkdir -p "$FETCHED_PATH" +TIMESTAMP=$(date +"%m%d%H%M") +FETCHED_FILE="$FETCHED_PATH/$TIMESTAMP.json" +gh api graphql -f query='query( + $owner: String!, + $repo: String!, + $number: Int!, + $prComments: Int!, + $reviews: Int!, + $threads: Int!, + $comments: Int!, + $after: String +) { + repository(owner: $owner, name: $repo) { + pullRequest(number: $number) { + comments(first: $prComments) { + nodes { + id + databaseId + author { login } + body + url + createdAt + } + } + reviews(first: $reviews) { + nodes { + id + databaseId + author { login } + body + state + url + createdAt + } + } + reviewThreads(first: $threads, after: $after) { + pageInfo { + hasNextPage + endCursor + } + nodes { + id + isResolved + isOutdated + path + line + comments(first: $comments) { + nodes { + id + databaseId + author { login } + body + url + createdAt + pullRequestReview { + id + databaseId + } + } + } + } + } + } + } +}' \ + -F owner=fedify-dev \ + -F repo=fedify \ + -F number="$PR_NUMBER" \ + -F prComments="$NUMBER_OF_PR_COMMENTS" \ + -F reviews="$NUMBER_OF_REVIEWS" \ + -F threads="$NUMBER_OF_THREADS" \ + -F comments="$NUMBER_OF_COMMENTS_PER_THREAD" \ + ${LAST_CURSOR:+-F after="$LAST_CURSOR"} \ + | jq . > "$FETCHED_FILE" + +echo "Fetched reviews and comments are saved to $FETCHED_FILE" diff --git a/.agents/skills/get-reviews/review.md b/.agents/skills/get-reviews/review.md new file mode 100644 index 000000000..8732536d3 --- /dev/null +++ b/.agents/skills/get-reviews/review.md @@ -0,0 +1,158 @@ +--- +id: REVIEW_ID +description: summary of the review +link: The link to the review on GitHub +# If more than one URL is relevant — related PR comments or review-thread +# replies that provide additional context — use `links` instead of `link`. +# Use the `url` field of each comment from the fetched JSON. +# links: +# - https://github.com/.../pull/123#discussion_r4567 +# - https://github.com/.../pull/123#discussion_r4568 +commit: The hash of commit after applying the review to add the comment +# If the review was applied across multiple commits, use `commits` instead +# of `commit`. Both fields are optional and only set after the review is +# applied. +# commits: +# - abc1234 +# - def5678 +--- + + + + + +Title +===== + + +Summary +------- + + + + +Judgement +--------- + + + +Choose your judgement(remain only one item also, delete this line): + - **CORRECT** + If the review is correct and should be applied. + - **WRONG** + If the review is wrong and should be dismissed. + - **PARTIAL** + If the review is partially correct and should be applied with some + modifications. + - **NEEDS EVALUATION** + If the review can be settled by empirical, mechanical verification of a + factual claim rather than human judgement—for example, whether the code + really cannot handle a certain error or whether some package is actually more efficient than the previously used package. + - **NEEDS DISCUSSION** + If the review needs further discussion, such as about direction of the + project or design choices. + + + +Plans +----- + +If the review is judged as **WRONG**, omit this section or write the plans +to add comments explaining the code to prevent similar misunderstandings. + +If the review is judged as **CORRECT** or **PARTIAL**, +write the plans to apply the review. + - How to apply the review? + - If the review is judged as **PARTIAL**, what are the modifications to + apply? + - Why the plans can apply the review correctly? + +If the review is judged as **WRONG**, write plans to why the reviewer +misunderstood the code and how to prevent similar misunderstandings +in the future. + - What are the reasons for the misunderstanding? + - How to prevent similar misunderstandings in the future? + - Consider adding comments to the code, improving documentation, + renaming variables or functions for clarity, + or refactoring the code to make it more understandable. + +If the review is judged as **NEEDS EVALUATION**, +write the plans to evaluate the review, such as testing plans. + - How to test or evaluate? + - What are the criteria for success or failure? + - Why the tests or evaluation can determine the correctness of the review? + - After the evaluation, write the plans to apply or dismiss the review + based on the evaluation results. + - Record the verification method, the pass/fail criteria, and the measured + result as evidence. + +If the review is judged as **NEEDS DISCUSSION**, +write the plans to discuss the review, such as topics to discuss and +potential options. + - What are the topics to discuss? + - What are the potential options and their pros and cons? + - What is the most reasonable option and why? + + +Comments +-------- + +Prepare the response comments in advance after applying the review or +judging the review as wrong. The comments should be polite and constructive. +Do not use conversational fillers (e.g. "You're right.") Avoid using personal +pronouns such as "I," "we," or "you." Focus on describing how the code has +changed rather than stating "I did something." + +If your comfortable language is not English, consider to separate this section into two parts, + - the language you use + - English, ready to be posted as a response +This will be help when posting comments all at once using the `gh` CLI tool. + +If you want to notify about modifications, add the commit hash at the first. +Consider to start with: "Addressed in {COMMIT_HASH}." + +If the review is judged as **CORRECT**, write comments to apply the review +to explain how to apply the review. + +If the review is judged as **WRONG**, write comments to dismiss the review +to explain why the review is wrong and should be dismissed. + +If the review is judged as **PARTIAL**, write comments to partially apply the +review to explain how to apply the review with modifications and why it is +partially correct. Referring to the above, write the incorrect parts based +on the **WRONG** part, and the correct parts based on the **CORRECT** part. + +If the review is judged as **NEEDS EVALUATION**, write the comments to +evaluate the review to explain how to evaluate the review, the test results, +and the resulting application/rejection details. + - If the results of the evaluation are the review is correct, + write comments referring to the **CORRECT** part. + - If the results of the evaluation are the review is wrong, + write comments referring to the **WRONG** part. + +If the review is judged as **NEEDS DISCUSSION**, +write the comments to discuss the review. + - The comments should explain what to discuss + about the review and why it needs discussion. + - The comments should be polite and constructive.