Skip to content

chore: improve GenAI tests in PR descriptions#928

Open
ovitrif wants to merge 4 commits intomasterfrom
chore/fix-ai-pr-tests-gen
Open

chore: improve GenAI tests in PR descriptions#928
ovitrif wants to merge 4 commits intomasterfrom
chore/fix-ai-pr-tests-gen

Conversation

@ovitrif
Copy link
Copy Markdown
Collaborator

@ovitrif ovitrif commented May 4, 2026

This PR improves the shared /pr instructions so AI-generated PR descriptions split QA notes into focused manual validation and visible automated check coverage.

Description

This PR:

  1. Requires generated QA notes to use ### QA Notes with #### Manual Tests and #### Automated Checks.
  2. Adds a concrete manual-test template with unchecked boxes, bold numbering, navigation arrows, regression labels, and variation sub-lists.
  3. Routes automated commands, test suites, and CI coverage notes into the automated-checks section, including the workflow (after merge) caveat.
  4. Moves durable /pr and /release command specs into .agents/commands while keeping .claude/commands as a compatibility symlink.

Preview

N/A

QA Notes

Manual Tests

  • 1. Repo checkout → inspect .claude/commands: pr.md and release.md resolve through the symlink.
  • 2. Agent docs → open AGENTS.md: durable PR command behavior points to .agents/commands/pr.md.
  • 3. /pr --dry in Claude Code → generate a PR body: QA Notes contains separate Manual Tests and Automated Checks sections.

Automated Checks

  • git diff --check

@ovitrif ovitrif force-pushed the chore/fix-ai-pr-tests-gen branch from 7322d27 to 788add8 Compare May 4, 2026 16:48
@ovitrif ovitrif changed the title docs: share agent command specs docs: improve pr qa instructions May 4, 2026
@ovitrif ovitrif changed the title docs: improve pr qa instructions chore: improve ai generated tests in pr descriptions May 4, 2026
@ovitrif ovitrif changed the title chore: improve ai generated tests in pr descriptions chore: improve GenAI tests in PR descriptions May 4, 2026
@ovitrif ovitrif added this to the 2.3.0 milestone May 4, 2026
@ovitrif ovitrif requested review from jvsena42 and piotr-iohk May 4, 2026 17:08
@ovitrif ovitrif self-assigned this May 4, 2026
jvsena42
jvsena42 previously approved these changes May 4, 2026
Copy link
Copy Markdown
Member

@jvsena42 jvsena42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment thread .agents/commands/pr.md Outdated
@claude

This comment has been minimized.

@ovitrif ovitrif enabled auto-merge May 5, 2026 11:30
@ovitrif ovitrif requested a review from jvsena42 May 5, 2026 19:37
piotr-iohk
piotr-iohk previously approved these changes May 6, 2026
Comment thread .agents/commands/pr.md Outdated
#### Automated Checks
- [x] ./gradlew compileDevDebugKotlin
- [x] ./gradlew testDevDebugUnitTest
- [x] ./gradlew detekt
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: could also list what coverage/modified/removed etc. was added, e.g.

  • unit tests covering X added - link to file(s)
  • unit tests covering Y modified - like to file(s)
  • unit tests covering Z removed - state why

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point 🙌, iterating on this

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 0ac9ad4.

Updated #### Automated Checks to prioritize coverage added, modified, or removed with file paths and short explanations, including a reason when automated coverage is removed.

@ovitrif
Copy link
Copy Markdown
Collaborator Author

ovitrif commented May 6, 2026

@piotr-iohk & @jvsena42 this is ready for another pass, added new commit to keep automated checks grouped under their own heading.

Once approved, I will port it to iOS.

@ovitrif ovitrif disabled auto-merge May 6, 2026 09:21
Comment thread .agents/commands/pr.md Outdated
Comment on lines +184 to +186
- [x] ./gradlew compileDevDebugKotlin
- [x] ./gradlew testDevDebugUnitTest
- [x] ./gradlew detekt
Copy link
Copy Markdown
Member

@jvsena42 jvsena42 May 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These commands are always done by the CI or PR bot, IDK how useful it is place them in the description

  • ./gradlew compileDevDebugKotlin
  • ./gradlew testDevDebugUnitTest
  • ./gradlew detekt

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 0ac9ad4.

Updated the guidance to avoid listing standard CI or PR bot commands as checkbox items by default. Standard CI is now summarized in one sentence when useful, and raw commands are reserved for local, non-standard, special-flag, workflow, or gap-explaining checks.

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 6, 2026

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants