Enhance triage skills with updated scoring, confidence, and P-scale labels#6294
Enhance triage skills with updated scoring, confidence, and P-scale labels#6294lauren-ciha wants to merge 16 commits intomainfrom
Conversation
guimafelipe
left a comment
guimafelipe
left a comment
There was a problem hiding this comment.
As this is a script that we use internally, I don't think any of my comments should be blocking. But maybe they will help making this overall process more maintainable and consistent.
Feel free to take them as suggestions.
|
|
||
| Normalized Score = (Total Score / 100) × 100 | ||
| Total Score = Reactions + Age + Comments + Severity | ||
| = 30 + 30 + 30 + 10 = 100 max |
There was a problem hiding this comment.
I would like to review that later. I feel like severity should have a way bigger impact on the score. If we have a security issue, we want to act on it ASAP regardless of reactions, comments and age.
There was a problem hiding this comment.
I'm going to loop in @ssparach on this one. I'm open to updating the criteria.
There was a problem hiding this comment.
Severity handling needs to be adjusted. Once severity is appropriately tracking actual security and data loss issues, its weight can be increased to reflect their higher priority
.github/skills/triage-meeting-prep/scripts/Get-IssueDetails.ps1
Outdated
Show resolved
Hide resolved
| | None of above | 0 | | ||
| | Severity Tier | Labels | Score | | ||
| |---------------|--------|-------| | ||
| | Critical | `regression`, `crash`, `hang`, `data-loss`, `security`, `P0` | 10 | |
There was a problem hiding this comment.
Do we plan to add labels for crash, hang, and data loss and some others? These labels do not exist today.
Also, regarding the security label, the skill is currently referencing area-security, which is intended to track bugs related to OAuth2Manager APIs. Severity falls outside that scope, so a different area label should be used here. Happy to align on which label makes the most sense. Thanks!
.github/skills/triage-meeting-prep/scripts/Get-IssueDetails.ps1
Outdated
Show resolved
Hide resolved
.github/skills/triage-meeting-prep/scripts/Get-IssueDetails.ps1
Outdated
Show resolved
Hide resolved
lauren-ciha
force-pushed
the
user/laurenciha/weekly-sync-scoring
branch
2 times, most recently
from
1fcc9b3 to
c0833b4
Compare
…abels
Scoring System:
- New weights: reactions=30%, age=30%, comments=30%, severity=10%
- P-scale severity labels: P0=critical, P1=high, P2=medium, P3=low
- Confidence scoring with grep-friendly [confidence:XX] format (0-100)
Label Consolidation:
- Merged Hot + Popular into Popular (>=5 reactions threshold)
Contact Schema:
- Simplified from {primary, secondary} to single {contact} field
- Removed legacy schema backward compatibility
Area Suggestions:
- Get-IssueDetails.ps1 dynamically fetches area labels via Get-RepositoryLabels.ps1
- area-Notifications covers all notification types (toast, badge, push, wns)
- Fixed area-PowerManagement naming
Documentation:
- Added PowerShell examples alongside Bash for confidence filtering
- Updated all SKILL.md files with new configuration details
…onfig.json for severity labels
… fails fast on errors
…script handle human-readable formatting
…s the source of truth
lauren-ciha
force-pushed
the
user/laurenciha/weekly-sync-scoring
branch
from
25e581c to
763cff9
Compare
- Add RawUpdateAgeDays to Get-IssueScore (computed from updatedAt) - Require comments >= trending_comments AND updated within trending_days in both Get-HighlightLabels (ReportLib) and Get-DetailedIssueScore - Fix scoring-algorithm.md: comments >= 5 -> >= 10, add trending_days to highlight table and thresholds reference - Add missing trending_days to SKILL.md config example
3 participants
This PR builds upon the issue-triage and feature-area-report skills.
The key changes are:
Validate-FeatureAreaReport.ps1that checks the report against the live GitHub label data usingghcli