Rectify: Marker Substring Match False Positives — Centralized Code-Region-Aware Detection - #4408
Conversation
019ea24 to
c010e28
Compare
Trecek
left a comment
There was a problem hiding this comment.
AutoSkillit PR Review — Verdict: approved_with_comments
| _INLINE_CODE_RE = re.compile(r"`[^`\n]*`") | ||
|
|
||
|
|
||
| def strip_markdown_code_regions(text: str) -> str: |
There was a problem hiding this comment.
[warning] bugs: strip_markdown_code_regions requires a matched opening+closing fence pair via the (|~~~).*?\1 pattern. An unclosed/odd-count fence (e.g. an issue body opens a block and never closes it) leaves everything after the dangling opening delimiter completely unstripped, including any marker text that follows it. On GitHub, an unterminated fence causes all subsequent content to render as code (until EOF), so a marker appearing there is visually in code and should not be treated as live — but detect_body_marker(body, marker) still returns True in this case, reproducing the class of false-positive this PR is meant to fix. Repro: strip_markdown_code_regions("```\ncode\n\nmore, unclosed") leaves the marker present in the stripped text. Consider treating an unmatched fence delimiter as extending to EOF (matching GitHub's rendering) so trailing content is also stripped.
There was a problem hiding this comment.
Valid observation — confirmed the repro (strip_markdown_code_regions("```\ncode\n<!-- investigation_complete: true -->\nmore, unclosed") leaves the marker present). A literal fix (extend the unmatched fence to EOF, e.g. (```|~~~).*?(?:\1|\Z)) works for this repro, but strip_markdown_code_regions is a single shared implementation — tests/_helpers.py re-exports the same function for tests/docs/test_banned_phrases.py, tests/docs/test_glossary_spelling.py, tests/skills/test_investigate_design_intent_contracts.py, and tests/workspace/test_skills.py, which strip code fences from SKILL.md/docs prose before banned-phrase/mention-guard scanning. tests/test_helpers_strip.py::test_unclosed_fence_is_fail_open explicitly locks the opposite (fail-open) behavior as intentional for that use case — flipping the shared regex to fail-closed would silently drop trailing prose in doc scans, inverting the safety direction those consumers rely on. Flagging for a design decision: fork a stricter fail-closed helper scoped to detect_body_marker, or accept the shared behavior change and update the fail-open test/consumers.
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
…stray backtick from swallowing markers
c010e28 to
d51474e
Compare
Summary
INVESTIGATION_COMPLETE_MARKERandREVIEW_APPROACH_MARKERare detected via naivestr.__contains__against raw, unsanitized GitHub issue bodies at two duplicated call sites. This produces false positives when the marker text appears inside markdown fenced code blocks or inline code spans. A production-gradestrip_markdown_code_regionsutility does not exist insrc/— only a test-only version intests/_helpers.py. Additionally, thebuild-execution-mapSKILL.md propagates the same vulnerability in prose by instructing the LLM to perform a "simple substring check."The architectural weakness is twofold: (1) no centralized marker-detection predicate exists — each call site reimplements raw
inchecks independently, and (2) there is no shared code-region-stripping utility in production code. The fix promotes the stripping utility into IL-0core/, creates a centralizeddetect_body_marker()predicate that strips before matching, replaces both duplicated call sites, fixes the SKILL.md prose, and adds false-positive rejection tests for both markers.Implementation Plan
Plan file:
/home/talon/projects/autoskillit-runs/remediation-20260728-142843-376918/.autoskillit/temp/rectify/rectify_marker_substring_match_2026-07-28_223749.mdCloses #4386
🤖 Generated with Claude Code via AutoSkillit
Audit gate bypassed — #4406
audit_impldid not gate this PR. It crashed withAuditCycleVerificationError: authority is not strict canonical versioned JSON— that is#4406:
audit-implwritesauthority.jsonpretty-printed, while its own verifier requiresthe strict canonical profile (
sort_keys=True,separators=(",", ":")). The crash is in thecycle-artifact machinery after the audit itself completed, so it is not a quality finding
against this change.
The bypass was explicitly authorized by the repository owner. Evidence standing in for the gate:
task test-check— 31848 passed, 557 skipped, 26 xfailedpre-commit run --all-files— all hooks passeddry-walkthrough— plan verified and stamped after three auditor passes