Skip to content

Sync .github/skills directory with azure-sdk-tools for PR 16380 - #48470

Open
azure-sdk-automation[bot] wants to merge 1 commit into
mainfrom
sync-.github-skills-fix/vally-skill-eval-environment-skills-16380
Open

Sync .github/skills directory with azure-sdk-tools for PR 16380#48470
azure-sdk-automation[bot] wants to merge 1 commit into
mainfrom
sync-.github-skills-fix/vally-skill-eval-environment-skills-16380

Conversation

@azure-sdk-automation

Copy link
Copy Markdown
Contributor

Sync .github/skills directory with azure-sdk-tools for PR Azure/azure-sdk-tools#16380 See eng/common workflow

@azure-sdk-automation
azure-sdk-automation Bot requested review from Helen Gao (helen229) and a balanced review from Copilot August 6, 2026 02:01
@azure-sdk-automation azure-sdk-automation Bot added EngSys This issue is impacting the engineering system. Central-EngSys This issue is owned by the Engineering System team. labels Aug 6, 2026
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
11 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot could not run the full agentic suite for this review because it was automatically requested on a bot-authored pull request. Request a review from Copilot under Reviewers to retry with the full agentic suite. Improved support for bot-authored pull requests is coming soon.

Updates Azure SDK “skills” evaluation fixtures and trigger docs to improve routing accuracy across release, release-plan, pipeline analysis/fix, local generation, and APIView feedback resolution scenarios.

Changes:

  • Standardizes eval headers (defaults) and adds explicit environment.skills loading for stimuli across multiple skills.
  • Expands/clarifies trigger prompts and anti-trigger routing (including requiring alternative skills/tools where appropriate).
  • Updates SKILL.md trigger descriptions to include additional “how-to” release phrasing and examples.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
.github/skills/azsdk-common-sdk-release/evals/trigger.eval.yaml Adds per-stimulus skill-loading environment and expands trigger/anti-trigger prompts for release routing.
.github/skills/azsdk-common-sdk-release/SKILL.md Broadens documented triggers to include checklist/process/how-to release questions.
.github/skills/azsdk-common-prepare-release-plan/evals/trigger.eval.yaml Adds environment loading; introduces a tool-call-based stimulus to verify release-plan linking route.
.github/skills/azsdk-common-prepare-release-plan/SKILL.md Refines description and trigger phrases for release plan management.
.github/skills/azsdk-common-pipeline-fixer/evals/trigger.eval.yaml Adds environment loading and strengthens anti-trigger routing to pipeline-analysis and other skills.
.github/skills/azsdk-common-pipeline-analysis/evals/trigger.eval.yaml Adds environment loading and improves prompts for analysis-only routing and anti-triggers.
.github/skills/azsdk-common-generate-sdk-locally/evals/eval.yaml Adds environment loading and strengthens negative case to route to pipeline generation tool/skill.
.github/skills/azsdk-common-apiview-feedback-resolution/evals/trigger.eval.yaml Adds environment loading and improves URL-based prompts plus anti-trigger routing.
.github/skills/azsdk-common-apiview-feedback-resolution/evals/eval.yaml Adds shared environment anchor/alias usage and requires specific APIView tool calls in graders.

Comment on lines +23 to +26
environment:
&apiviewSkill # Load the real skill so the executor can route to it and call its tools.
skills:
- ..
Comment on lines +23 to +25
environment:
skills:
- ..
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Central-EngSys This issue is owned by the Engineering System team. EngSys This issue is impacting the engineering system.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants