Add type-specific outcome evaluators for create_issue, add_comment, and add_labels#35113
Draft
Copilot wants to merge 4 commits into
Draft
Add type-specific outcome evaluators for create_issue, add_comment, and add_labels#35113Copilot wants to merge 4 commits into
create_issue, add_comment, and add_labels#35113Copilot wants to merge 4 commits into
Conversation
Open
7 tasks
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Implement dedicated outcome evaluators for create_issue, add_comment, and add_labels
Add type-specific outcome evaluators for May 27, 2026
create_issue, add_comment, and add_labels
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR replaces existence-based outcome classification for
create_issue,add_comment, andadd_labelswith dedicated evaluators that use engagement/lifecycle signals and persisted label state. It also adds branch-complete coverage for accepted/rejected/pending/unknown outcomes across these types.Outcome evaluation: dedicated classifiers
actions/setup/js/evaluate_outcomes.cjsnow routescreate_issue,add_comment, andadd_labelsthrough type-specific evaluators.create_issuenow classifies based on engagement and closure behavior:add_commentnow classifies based on comment lifecycle and thread activity:add_labelsnow evaluates retention using before-state + added labels with a retention window:Manifest/data model updates for
add_labelsactions/setup/js/add_labels.cjscaptureslabelsBeforebefore label mutation.actions/setup/js/safe_output_manifest.cjspersistslabelsBeforeandlabelsAddedfor downstream evaluation.Focused test coverage for outcome branches
actions/setup/js/evaluate_outcomes.test.cjsto cover outcome branches for all three evaluators.actions/setup/js/safe_output_manifest.test.cjsto assert new label metadata extraction.