From d076398e057f7014fe9d7bb28f6b05d5381dbd69 Mon Sep 17 00:00:00 2001 From: Serguei Balanovich Date: Tue, 24 Mar 2026 20:07:07 -0400 Subject: [PATCH 1/3] feat(ship-pr): add test evidence gate to Screenshot Check section Co-Authored-By: Claude Opus 4.6 (1M context) --- skills/ship-pr/SKILL.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/skills/ship-pr/SKILL.md b/skills/ship-pr/SKILL.md index 7165681..686a28f 100644 --- a/skills/ship-pr/SKILL.md +++ b/skills/ship-pr/SKILL.md @@ -53,6 +53,10 @@ Update VERSION, package.json, or pyproject.toml (whichever exists). Update CHANG - If the diff includes files in UI/frontend packages AND `evidence/screenshots/` is empty or missing: STOP and warn the user. Recommend running /validate or capturing screenshots now. +- If the diff adds new test files (*.test.*, *.spec.*): the PR body MUST include + proof that the new tests pass. At minimum, embed the verbose test runner output. + If the tests are Playwright/E2E tests, also capture and embed screenshots of the + tested behavior. STOP if no evidence exists — run the tests now and capture output. - If screenshots exist: continue and embed them in the PR body using absolute GitHub blob URLs. - Organize screenshots by feature or mode (e.g., "Docker mode" vs "Sprites mode"). From c0145f8936e2c3afe0bda9eb008f805c09bcff73 Mon Sep 17 00:00:00 2001 From: Serguei Balanovich Date: Tue, 24 Mar 2026 20:08:58 -0400 Subject: [PATCH 2/3] chore: bump to v0.8.1 Co-Authored-By: Claude Opus 4.6 (1M context) --- CHANGELOG.md | 5 +++++ VERSION | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c84e899..df01878 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,10 @@ # Changelog +## 0.8.1 + +### Added +- **`/ship-pr` test evidence gate** — Screenshot Check now also gates on new test files (`*.test.*`, `*.spec.*`). The PR body must include verbose test runner output proving the new tests pass. For Playwright/E2E tests, screenshots of tested behavior are also required. + ## 0.8.0 ### Added diff --git a/VERSION b/VERSION index a3df0a6..6f4eebd 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.8.0 +0.8.1 From 2196d80f5c00f70681eae7a8f6d578e15c0b480f Mon Sep 17 00:00:00 2001 From: Serguei Balanovich Date: Wed, 25 Mar 2026 22:27:50 -0400 Subject: [PATCH 3/3] fix(ship-pr): add polyglot test patterns, E2E detection heuristic, and failure handling Addresses review feedback: - Broaden test file patterns beyond JS/TS to cover Python, Go, and Ruby - Add concrete Playwright/E2E detection criteria (imports, config, e2e/ dir) - Explicitly block ship flow on test failures, not just missing evidence Co-Authored-By: Claude Opus 4.6 (1M context) --- skills/ship-pr/SKILL.md | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/skills/ship-pr/SKILL.md b/skills/ship-pr/SKILL.md index 686a28f..af5b30e 100644 --- a/skills/ship-pr/SKILL.md +++ b/skills/ship-pr/SKILL.md @@ -53,10 +53,15 @@ Update VERSION, package.json, or pyproject.toml (whichever exists). Update CHANG - If the diff includes files in UI/frontend packages AND `evidence/screenshots/` is empty or missing: STOP and warn the user. Recommend running /validate or capturing screenshots now. -- If the diff adds new test files (*.test.*, *.spec.*): the PR body MUST include - proof that the new tests pass. At minimum, embed the verbose test runner output. - If the tests are Playwright/E2E tests, also capture and embed screenshots of the - tested behavior. STOP if no evidence exists — run the tests now and capture output. +- If the diff adds new test files (`*.test.*`, `*.spec.*`, `test_*.py`, `*_test.py`, + `*_test.go`, `*_spec.rb`, `test_*.rb`): the PR body MUST include proof that the + new tests pass. At minimum, embed the verbose test runner output. + To detect Playwright/E2E tests: check for `playwright` imports in the test file, + a `playwright.config.*` at the repo root, or test location in an `e2e/` directory. + If E2E tests are detected, also capture and embed screenshots of the tested behavior. + STOP if no evidence exists — run the tests now and capture output. + STOP if the tests fail — do not embed failure output as "evidence." The tests must + be green before the PR can ship. Report the failures and halt. - If screenshots exist: continue and embed them in the PR body using absolute GitHub blob URLs. - Organize screenshots by feature or mode (e.g., "Docker mode" vs "Sprites mode").