feat: add e2e-validated label for automated acceptance proof on PRs#91
Merged
feat: add e2e-validated label for automated acceptance proof on PRs#91
Conversation
Move completed PRDs to done/ folder: - job-registry.md (Job Registry implementation) - analytics-job.md (Analytics job implementation) - fix-executor-streaming-output.md (Terminal streaming fix) - fix-prd-execution-failures.md (Double rate-limit detection) - open-source-readiness.md (OSS community files) - remove-legacy-personas-and-filesystem-prd.md (Dead code cleanup) - prd-provider-schedule-overrides.md (Provider schedule overrides) - provider-agnostic-instructions.md (Instructions directory refactoring) - night-watch/provider-aware-queue.md (Per-bucket concurrency) Delete obsolete PRD: - refactor-interaction-listener.md (packages/slack/ removed in commit 46637a0) Add architecture docs: - docs/architecture/job-registry.md (Job registry architecture) - docs/architecture/analytics-job.md (Analytics job architecture) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implements the E2E Validated Label PRD across 4 phases: Phase 1 — Label Definition & Config: - Add 'e2e-validated' (green #0e8a16) to NIGHT_WATCH_LABELS in labels.ts - Add validatedLabel: string to IQaConfig in types.ts - Add DEFAULT_QA_VALIDATED_LABEL constant and update DEFAULT_QA in constants.ts - Add validatedLabel extra field to QA job definition in job-registry.ts - Config normalization picks it up automatically via normalizeJobConfig Phase 2 — QA Script Integration: - Pass NW_QA_VALIDATED_LABEL env var from qa.ts buildEnvVars() - Show Validated Label in qa --dry-run config table - Read VALIDATED_LABEL in QA bash script with e2e-validated default - Add ensure_validated_label() helper (idempotent gh label create --force) - Apply label on 'passing' outcome, remove on 'issues_found'/'no_tests_needed' - Track VALIDATED_PRS_CSV and include in emit_result and Telegram messages Phase 3 — Init Label Sync: - Add step 11 to night-watch init: sync all NIGHT_WATCH_LABELS to GitHub - Skips gracefully when no GitHub remote or gh not authenticated - Updates totalSteps from 13 to 14 and adds Label Sync to summary table Phase 4 — Dry-Run & Summary Integration: - Validated Label shown in qa --dry-run config table (merged into Phase 2) - emit_result includes validated= field for all success/warning outcomes Tests: - packages/core/src/__tests__/board/labels.test.ts (new): e2e-validated presence - packages/core/src/__tests__/jobs/job-registry.test.ts: validatedLabel extra field - packages/cli/src/__tests__/commands/qa.test.ts: NW_QA_VALIDATED_LABEL env var - packages/cli/src/__tests__/commands/init.test.ts: label sync preconditions Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
DiffGuard AI AnalysisAI Review Summary🏆 Overall Score: 85/100 This PR implements the E2E Validated Label feature with solid documentation, comprehensive unit tests for the TypeScript/Node layer, and proper integration into the existing job registry and QA workflow. The main gaps are in bash script testing and missing integration tests for the new init label sync functionality. ✅ Key Strengths
|
| Issue Type | Issue Name | Affected Components | Description | Impact/Severity |
|---|---|---|---|---|
| Testing | Init label sync execution untested | packages/cli/src/__tests__/commands/init.test.ts |
Tests verify preconditions but don't mock execSync to verify actual gh label create calls when conditions are met |
Medium |
| Testing | Bash script label logic untested | scripts/night-watch-qa-cron.sh |
Label application/removal logic in case statement has no bats tests | Low |
| Maintainability | Silent failure in label creation | packages/cli/src/commands/init.ts:906-911 |
Empty catch block swallows errors from gh label create, making debugging harder |
Low |
🔚 Conclusion
This is a well-designed and thoroughly documented feature implementation. The TypeScript layer has good test coverage, but the bash script changes and init label sync would benefit from additional test coverage. The feature is merge-ready, though adding integration tests for the init label sync would increase confidence.
Analyzed using z-ai/glm-5
jonit-dev
added a commit
that referenced
this pull request
Mar 25, 2026
) * chore: organize PRDs and create architecture docs Move completed PRDs to done/ folder: - job-registry.md (Job Registry implementation) - analytics-job.md (Analytics job implementation) - fix-executor-streaming-output.md (Terminal streaming fix) - fix-prd-execution-failures.md (Double rate-limit detection) - open-source-readiness.md (OSS community files) - remove-legacy-personas-and-filesystem-prd.md (Dead code cleanup) - prd-provider-schedule-overrides.md (Provider schedule overrides) - provider-agnostic-instructions.md (Instructions directory refactoring) - night-watch/provider-aware-queue.md (Per-bucket concurrency) Delete obsolete PRD: - refactor-interaction-listener.md (packages/slack/ removed in commit 46637a0) Add architecture docs: - docs/architecture/job-registry.md (Job registry architecture) - docs/architecture/analytics-job.md (Analytics job architecture) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add e2e-validated label for automated acceptance proof on PRs Implements the E2E Validated Label PRD across 4 phases: Phase 1 — Label Definition & Config: - Add 'e2e-validated' (green #0e8a16) to NIGHT_WATCH_LABELS in labels.ts - Add validatedLabel: string to IQaConfig in types.ts - Add DEFAULT_QA_VALIDATED_LABEL constant and update DEFAULT_QA in constants.ts - Add validatedLabel extra field to QA job definition in job-registry.ts - Config normalization picks it up automatically via normalizeJobConfig Phase 2 — QA Script Integration: - Pass NW_QA_VALIDATED_LABEL env var from qa.ts buildEnvVars() - Show Validated Label in qa --dry-run config table - Read VALIDATED_LABEL in QA bash script with e2e-validated default - Add ensure_validated_label() helper (idempotent gh label create --force) - Apply label on 'passing' outcome, remove on 'issues_found'/'no_tests_needed' - Track VALIDATED_PRS_CSV and include in emit_result and Telegram messages Phase 3 — Init Label Sync: - Add step 11 to night-watch init: sync all NIGHT_WATCH_LABELS to GitHub - Skips gracefully when no GitHub remote or gh not authenticated - Updates totalSteps from 13 to 14 and adds Label Sync to summary table Phase 4 — Dry-Run & Summary Integration: - Validated Label shown in qa --dry-run config table (merged into Phase 2) - emit_result includes validated= field for all success/warning outcomes Tests: - packages/core/src/__tests__/board/labels.test.ts (new): e2e-validated presence - packages/core/src/__tests__/jobs/job-registry.test.ts: validatedLabel extra field - packages/cli/src/__tests__/commands/qa.test.ts: NW_QA_VALIDATED_LABEL env var - packages/cli/src/__tests__/commands/init.test.ts: label sync preconditions Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Test User <test@test.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
e2e-validatedGitHub label (green #0e8a16) that is automatically applied to PRs when QA tests pass, proving acceptance requirements are metvalidatedLabelfield inIQaConfig(default:e2e-validated) — configurable viaNW_QA_VALIDATED_LABELenv varpassingoutcome, removes it onissues_found/no_tests_needednight-watch initsyncs all Night Watch labels to GitHub (new step 11)Changes
packages/core/src/board/labels.ts—e2e-validatedadded toNIGHT_WATCH_LABELSpackages/core/src/types.ts—validatedLabel: stringadded toIQaConfigpackages/core/src/constants.ts—DEFAULT_QA_VALIDATED_LABELconstant, updatedDEFAULT_QApackages/core/src/jobs/job-registry.ts—validatedLabelextra field in QA job definitionpackages/cli/src/commands/qa.ts— passesNW_QA_VALIDATED_LABEL, shows in dry-run tablescripts/night-watch-qa-cron.sh—ensure_validated_label(), label add/remove per outcome,VALIDATED_PRS_CSVtracking in emit_resultpackages/cli/src/commands/init.ts— step 11 syncs all Night Watch labels to GitHubTest plan
yarn verifypassesnight-watch qa --dry-run→ "Validated Label: e2e-validated" appears in config tablenight-watch initon a repo with GitHub remote → all Night Watch labels created includinge2e-validatede2e-validatedlabel applied on GitHube2e-validatedlabel removed🤖 Generated with Claude Code