Skip to content

feat(review): tag tracking comments for classification - #133

Open
factory-andrew-seo wants to merge 6 commits into
devfrom
andrew/factory-pr-review
Open

feat(review): tag tracking comments for classification#133
factory-andrew-seo wants to merge 6 commits into
devfrom
andrew/factory-pr-review

Conversation

@factory-andrew-seo

@factory-andrew-seo factory-andrew-seo commented Aug 25, 2026

Copy link
Copy Markdown

Summary

  • define a centralized DroidRunType enum for default, review, security review, fill, full security scan, and CI Steward runs
  • resolve run identity before creating the initial tracking comment, export it when known, and pass it through each mode and the MCP comment server
  • distinguish a bare @droid command as droid-default and leave the run type null when no command or automatic mode identifies one
  • serialize exact run identity in the hidden marker, for example <!-- factory-pr-validation: run-type=droid-review -->
  • mark default, code review, security review, and full security scan comments while leaving fill and CI Steward outside PR-validation tagging
  • preserve the trusted run-type marker when Droid sanitizes and rewrites comments
  • check automatic-security run-once state before comment creation, avoiding no-op comments and downgrading dual-review messages when security is skipped
  • cover run-type resolution, marker generation, initial comments, final summaries, MCP wiring, nullable runs, and automatic-security skip behavior with tests

Why

GitHub issue comments do not retain structured metadata linking them to the workflow or validation that created them. The hidden PR validation marker lets downstream consumers identify the exact originating run type without coupling classification to presentation-oriented comment text.

Reviewer notes

The action derives the marker from its trusted DroidRunType rather than accepting it from model output. The comment server restores it after sanitization, so arbitrary HTML comments remain stripped and the trusted marker survives every update. Existing CommentType values and user-facing status messages remain presentation-only.

The emitted marker values are:

  • run-type=droid-default
  • run-type=droid-review
  • run-type=droid-security-review
  • run-type=droid-security-scan

When no command is parsed and neither automatic mode applies, run-type resolution returns null; no run type is exported or added to the marker.

Automatic security reviews now check the existing security summary before creating a tracking comment. A security-only no-op creates no comment, while a combined run that skips security creates a code-review-only comment.

Testing

  • npx --yes bun@1.2.11 test (599 passed)
  • npm run typecheck
  • npm run format:check

Add a dedicated hidden marker to PR review tracking comments and preserve it across sanitized summary updates.

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
@factory-droid

factory-droid Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Droid finished @factory-andrew-seo's task —— View job


Security Review

The PR review marker tagging is cohesive and well-tested. One correctness gap remains: tag execution can create a tracking comment before the automatic security “run once” logic, leaving a misleading “running a security check” message even when security is skipped.

Comment thread src/tag/index.ts
factory-andrew-seo and others added 5 commits August 25, 2026 16:45
Replace the review-specific marker with a generic PR validation marker carrying an explicit source value for future validation comment types.

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Resolve a typed run identity before tracking comments are created, pass it through each mode and MCP server, and derive PR validation metadata independently of comment presentation.

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Check automatic security run-once state before creating the tracking comment. Skip comment creation for security-only no-op runs and downgrade dual-review comments when security will not run.

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Replace the single-value validation source with exact run-type metadata. Mark code review, security review, and full security scan comments while keeping fill and CI Steward outside the PR validation marker.

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Add a dedicated droid-default run type for bare @droid commands and return null when no command or automatic mode identifies a run. Propagate unresolved runs without exporting or marking a false review identity.

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant