Skip to content

Implementation Plan: Add Deterministically Gated Overengineering Auditors to review-pr - #4426

Merged
Trecek merged 46 commits into
developfrom
impl-add-deterministically-gated-overengineering-auditors-plan-2026-07-29-202704-20260729-220819
Jul 31, 2026
Merged

Implementation Plan: Add Deterministically Gated Overengineering Auditors to review-pr#4426
Trecek merged 46 commits into
developfrom
impl-add-deterministically-gated-overengineering-auditors-plan-2026-07-29-202704-20260729-220819

Conversation

@Trecek

@Trecek Trecek commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

Extend the existing snapshot-bound metrics_{pr}.json authority with a native JSON
run_overengineering_audits boolean computed in Python from exact unified-diff
churn:

closes #4421

(added_lines + removed_lines) > 2000

The change adds packless, read-only reachability and abstraction-surface auditors,
keeps their dispatch independent from standard and deletion-regression review,
and carries accepted evidence through deterministic aggregation, publication, and
the resolve-review handoff.

Verification

  • task test-check: 34,257 passed, 630 skipped, 27 expected xfails
  • pre-commit run --all-files: all hooks passed
  • independent implementation audit: verified GO with 32/32 requirements covered

Implementation Plan

Plan file: /home/talon/projects/generic_automation_mcp/.autoskillit/temp/make-plan/add_deterministically_gated_overengineering_auditors_plan_2026-07-29_202704.md

🤖 Generated with Claude Code via AutoSkillit

@Trecek Trecek left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AutoSkillit PR Review — Verdict: changes_requested

Comment thread src/autoskillit/smoke_utils/_review.py Outdated
Comment thread src/autoskillit/skills_extended/review-pr/SKILL.md Outdated
Comment thread src/autoskillit/skills_extended/review-pr/SKILL.md Outdated
Comment thread src/autoskillit/skills_extended/review-pr/SKILL.md
Comment thread src/autoskillit/skills_extended/review-pr/SKILL.md
Comment thread src/autoskillit/smoke_utils/_experimental_review.py
Comment thread src/autoskillit/skills_extended/review-pr/SKILL.md Outdated
Comment thread tests/skills/test_review_pr_adaptive_dispatch_guards.py
Comment thread tests/test_smoke_utils.py Outdated
Comment thread tests/arch/test_agent_prompt_structure.py Outdated

@Trecek Trecek left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AutoSkillit review: blocking findings detected. See the 11 inline comments; changes are required.

Non-blocking adversarial observations:

  • src/autoskillit/smoke_utils/_review.py:L78 — reachability: the new non-bytes stderr fallback is unreachable because the sole runner fixes capture_output=True and text=False; supported callers cannot replace the nested runner. Removing the fallback preserves reachable behavior and error context.
  • src/autoskillit/smoke_utils/_experimental_review.py:L706 — abstraction surface: effect_artifacts duplicates documents already held by artifacts; the only production publisher reads artifact_order and artifacts, while direct effect_artifacts consumers are tests for the new projection. Removing the duplicate leaves ordering, rollback, concurrency, validation, and supported callers unchanged.

@Trecek Trecek left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AutoSkillit PR Review — Verdict: changes_requested (self-authored PR; GitHub requires COMMENT event)

Comment thread src/autoskillit/skills_extended/resolve-review/SKILL.md Outdated
Comment thread src/autoskillit/skills_extended/review-pr/SKILL.md Outdated
Comment thread src/autoskillit/skills_extended/review-pr/SKILL.md Outdated
Comment thread src/autoskillit/skills_extended/review-pr/SKILL.md
Comment thread src/autoskillit/smoke_utils/_experimental_review.py
Comment thread src/autoskillit/smoke_utils/_experimental_review.py Outdated
Comment thread tests/recipe/test_review_loop_routing_integration.py
Comment thread tests/skills/test_review_pr_inline_comment_guards.py
Comment thread tests/test_smoke_utils.py Outdated
Comment thread tests/test_smoke_utils.py

@Trecek Trecek left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AutoSkillit review: changes requested. One critical and ten warning findings were posted inline. Three informational observations are retained in the audit summary. Because this PR is self-authored, GitHub requires a COMMENT review event rather than REQUEST_CHANGES.

@Trecek Trecek left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AutoSkillit PR Review — Verdict: changes_requested

Comment thread tests/skills/test_review_pr_inline_comment_guards.py
Comment thread tests/skills/test_review_pr_inline_comment_guards.py
Comment thread tests/skills/test_review_pr_adaptive_dispatch_guards.py
Comment thread tests/skills/test_review_pr_adaptive_dispatch_guards.py
Comment thread tests/skills/test_review_pr_adaptive_dispatch_guards.py
Comment thread src/autoskillit/smoke_utils/_review.py
Comment thread src/autoskillit/skills_extended/resolve-review/SKILL.md
Comment thread src/autoskillit/smoke_utils/_review.py
Comment thread tests/test_smoke_utils.py
Comment thread src/autoskillit/smoke_utils/_experimental_review.py Outdated

@Trecek Trecek left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AutoSkillit PR Review — Verdict: changes_requested

Comment thread tests/skills/test_review_pr_inline_comment_guards.py
Comment thread tests/skills/test_review_pr_inline_comment_guards.py
Comment thread tests/skills/test_review_pr_adaptive_dispatch_guards.py
Comment thread tests/skills/test_review_pr_adaptive_dispatch_guards.py
Comment thread tests/skills/test_review_pr_adaptive_dispatch_guards.py
Comment thread src/autoskillit/smoke_utils/_review.py
Comment thread src/autoskillit/skills_extended/resolve-review/SKILL.md
Comment thread src/autoskillit/smoke_utils/_review.py
Comment thread tests/test_smoke_utils.py
Comment thread src/autoskillit/smoke_utils/_experimental_review.py Outdated

@Trecek Trecek left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test

Comment thread tests/skills/test_review_pr_inline_comment_guards.py
Comment thread tests/skills/test_review_pr_inline_comment_guards.py

@Trecek Trecek left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AutoSkillit PR Review — Verdict: changes_requested

Comment thread tests/skills/test_review_pr_inline_comment_guards.py
Comment thread tests/skills/test_review_pr_inline_comment_guards.py
Comment thread tests/skills/test_review_pr_adaptive_dispatch_guards.py
Comment thread tests/skills/test_review_pr_adaptive_dispatch_guards.py
Comment thread tests/skills/test_review_pr_adaptive_dispatch_guards.py
Comment thread src/autoskillit/smoke_utils/_review.py
Comment thread src/autoskillit/skills_extended/resolve-review/SKILL.md
Comment thread src/autoskillit/smoke_utils/_review.py
Comment thread tests/test_smoke_utils.py
Comment thread src/autoskillit/smoke_utils/_experimental_review.py Outdated

@Trecek Trecek left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AutoSkillit PR Review — Verdict: changes_requested (batch 3/4)

Comment thread src/autoskillit/skills_extended/review-pr/SKILL.md
Comment thread src/autoskillit/server/tools/tools_agents.py
Comment thread src/autoskillit/execution/diff_annotator.py
Comment thread src/autoskillit/smoke_utils/_review.py
Comment thread src/autoskillit/execution/diff_annotator.py
Comment thread src/autoskillit/recipe/rules/graph/rules_graph_review.py
Comment thread src/autoskillit/skills_extended/review-pr/SKILL.md
Comment thread src/autoskillit/skills_extended/review-pr/SKILL.md
Comment thread src/autoskillit/skills_extended/review-pr/SKILL.md
Comment thread src/autoskillit/skills_extended/review-pr/SKILL.md

@Trecek Trecek left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AutoSkillit PR Review — Verdict: changes_requested (batch 4/4)

Comment thread tests/recipe/test_bundled_recipes_review_pr.py Outdated
Comment thread src/autoskillit/smoke_utils/_review.py
Comment thread src/autoskillit/skills_extended/resolve-review/SKILL.md
Comment thread src/autoskillit/smoke_utils/_review.py
Comment thread tests/test_smoke_utils.py
Comment thread src/autoskillit/smoke_utils/_experimental_review.py Outdated

@Trecek Trecek left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AutoSkillit PR Review — Verdict: changes_requested (retry batch 1/4)

Comment thread tests/skills/test_review_pr_adaptive_dispatch_guards.py
Comment thread tests/skills/test_review_pr_adaptive_dispatch_guards.py
Comment thread tests/skills/test_review_pr_adaptive_dispatch_guards.py
Comment thread tests/skills/test_review_pr_adaptive_dispatch_guards.py
Comment thread tests/skills/test_review_pr_adaptive_dispatch_guards.py
Comment thread tests/skills/test_review_pr_adaptive_dispatch_guards.py
Comment thread tests/skills/test_review_pr_adaptive_dispatch_guards.py
Comment thread tests/skills/test_review_pr_artifact_idempotency.py
Comment thread tests/skills/test_review_pr_artifact_idempotency.py
Comment thread tests/skills/test_review_pr_inline_comment_guards.py

@Trecek Trecek left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AutoSkillit PR Review — Verdict: changes_requested (retry batch 2/4)

Comment thread tests/skills/test_review_pr_inline_comment_guards.py
Comment thread tests/skills/test_review_pr_inline_comment_guards.py
Comment thread tests/skills/test_review_pr_local_mode.py
Comment thread tests/skills/test_review_pr_local_mode.py
Comment thread tests/skills/test_review_pr_verdict_guards.py
Comment thread tests/skills/test_review_pr_verdict_guards.py
Comment thread tests/skills/test_review_pr_prior_thread_awareness.py
Comment thread tests/skills/test_review_pr_diff_context_handoff.py
Comment thread tests/skills/test_review_pr_inline_comment_guards.py
Comment thread tests/skills/test_review_pr_inline_comment_guards.py

@Trecek Trecek left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AutoSkillit PR Review — Verdict: changes_requested (retry batch 3/4)

Comment thread tests/skills/test_review_pr_adaptive_dispatch_guards.py
Comment thread tests/skills/test_review_pr_adaptive_dispatch_guards.py
Comment thread tests/test_smoke_utils.py
Comment thread tests/test_smoke_utils.py
Comment thread tests/test_smoke_utils.py
Comment thread tests/test_smoke_utils.py
Comment thread tests/skills/test_review_pr_adaptive_dispatch_guards.py
Comment thread tests/skills/test_review_pr_artifact_idempotency.py
Comment thread tests/arch/test_agent_prompt_structure.py
Comment thread tests/skills/test_resolve_review_local_mode.py

@Trecek Trecek left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AutoSkillit PR Review — Verdict: changes_requested (retry batch 4/4)

Comment thread tests/skills/test_resolve_review_local_mode.py Outdated
Comment thread tests/test_smoke_utils.py
Comment thread tests/recipe/test_bundled_recipes_review_pr.py
Comment thread tests/recipe/test_bundled_recipes_review_pr.py
Comment thread src/autoskillit/smoke_utils/_experimental_review.py
Comment thread src/autoskillit/smoke_utils/_experimental_review.py
Comment thread src/autoskillit/smoke_utils/_experimental_review.py
Comment thread src/autoskillit/smoke_utils/_experimental_review.py
Comment thread src/autoskillit/smoke_utils/_experimental_review.py Outdated
Comment thread src/autoskillit/smoke_utils/_experimental_review.py Outdated

@Trecek Trecek left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AutoSkillit PR Review — Verdict: changes_requested

74 findings after dedup; 2 critical, 52 warning, 20 info.
68 inline comments posted across 7 reviews (info/warning/critical).
6 findings could not be posted inline (lines outside diff hunks).
2 findings suppressed as duplicates of prior resolved threads.

Outside Diff Range

These findings target lines not in the diff and could not be posted as inline comments:

src/autoskillit/recipes/implementation.yaml

  • L1082 [info/cohesion]: The same clear_review_annotation_context step block (callable, capture set, on_success/on_failure, note) is duplicated f

src/autoskillit/skills_extended/review-pr/SKILL.md

  • L165 [warning/cohesion]: The hunk-ranges artifact is named inconsistently across layers: the on-disk file is ranges_{pr}.json (per _review.py L46

src/autoskillit/smoke_utils/_experimental_review.py

  • L2630 [warning/cohesion]: EXPERIMENTAL_REVIEW_AUDITORS, _EXPERIMENTAL_DIMENSIONS, and the closed-set validator constants duplicate agent definitio
  • L3025 [info/slop]: assert isinstance(candidate, dict) is dead code: _candidate_validation_error already verified the candidate is a dict an

src/autoskillit/smoke_utils/_review.py

  • L3681 [warning/defense]: assert refs_before is not None after _read_pr_refs(required=True) is invoked. Under python -O the assertion disapp
  • L3674 [warning/defense]: _required_scalar decodes git output with errors="strict". If a git rev-parse result contains non-UTF8 bytes, the Runti

@Trecek
Trecek force-pushed the impl-add-deterministically-gated-overengineering-auditors-plan-2026-07-29-202704-20260729-220819 branch from 67e226b to 2a4ec87 Compare July 31, 2026 03:10
@Trecek
Trecek added this pull request to the merge queue Jul 31, 2026
Merged via the queue into develop with commit 3282c0c Jul 31, 2026
3 checks passed
@Trecek
Trecek deleted the impl-add-deterministically-gated-overengineering-auditors-plan-2026-07-29-202704-20260729-220819 branch July 31, 2026 03:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant