Skip to content

Python: Support skill scripts execution#4558

Open
SergeyMenshykh wants to merge 17 commits intomicrosoft:mainfrom
SergeyMenshykh:support-skills-script-execution
Open

Python: Support skill scripts execution#4558
SergeyMenshykh wants to merge 17 commits intomicrosoft:mainfrom
SergeyMenshykh:support-skills-script-execution

Conversation

@SergeyMenshykh
Copy link
Member

@SergeyMenshykh SergeyMenshykh commented Mar 9, 2026

Summary

Adds support for executing agent skill scripts — both file-based scripts (delegated to a SkillScriptRunner) and code-defined scripts (registered via the @skill.script decorator and executed in-process).

Changes

Core (packages/core/agent_framework/_skills.py)

  • Added SkillScript model with optional function (code-based) and path (file-based) fields
  • Added SkillScriptRunner protocol for file-based script execution strategies
  • Added run_skill_script tool with optional user-approval gate (require_script_approval)
  • Script-execution instructions injected into the system prompt when scripts are present
  • Updated Skill model with mutable scripts list
  • Added @skill.script decorator for registering code-defined scripts

Exports (__init__.py)

  • Exported new public types: SkillScript, SkillScriptRunner

Tests (tests/core/test_skills.py)

  • Comprehensive test coverage for script execution scenarios (file-based, code-defined, approval, error handling)

Samples

  • Restructured skill samples with clearer naming (file_based_skill, code_defined_skill, mixed_skills, script_approval)
  • Added shared subprocess_script_runner.py utility for file-based script execution via subprocess
  • Each sample has a dedicated README

Design Decisions

  • File-based scripts require an explicit SkillScriptRunner — the framework does not assume a specific execution strategy (subprocess, sandbox, hosted interpreter, etc.)
  • Code-defined scripts execute in-process and bypass the runner entirely
  • Script execution is opt-in: skills without scripts work exactly as before
  • The run_skill_script tool supports an optional approval gate for safety

Closes: #4349

@SergeyMenshykh SergeyMenshykh self-assigned this Mar 9, 2026
Copilot AI review requested due to automatic review settings March 9, 2026 11:48
@SergeyMenshykh SergeyMenshykh moved this to In Review in Agent Framework Mar 9, 2026
@markwallace-microsoft markwallace-microsoft added the documentation Improvements or additions to documentation label Mar 9, 2026
@markwallace-microsoft
Copy link
Member

markwallace-microsoft commented Mar 9, 2026

Python Test Coverage

Python Test Coverage Report •
FileStmtsMissCoverMissing
packages/core/agent_framework
   _skills.py4061995%182, 912–919, 973, 1012–1013, 1086, 1091, 1140, 1145, 1297–1298, 1546
TOTAL22802258788% 

Python Unit Test Overview

Tests Skipped Failures Errors Time
5040 20 💤 0 ❌ 0 🔥 1m 24s ⏱️

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds first-class support for executing skill scripts in the Python agent framework, covering both file-based scripts (delegated to a configurable executor) and code-defined scripts (registered via a decorator and executed in-process). This extends the existing SkillsProvider progressive-disclosure model to include script execution and introduces new samples demonstrating file skills, mixed skills, and human approval gating.

Changes:

  • Core: add SkillScript, @skill.script, script discovery, and an execute_skill_script tool with optional approval gating.
  • Public API: export new script-related types from agent_framework.__init__.
  • Ecosystem: add/refresh samples for file skills, code-defined skills, mixed skills, and script approval; expand test coverage for script behavior.

Reviewed changes

Copilot reviewed 27 out of 27 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
python/packages/core/agent_framework/_skills.py Adds script model/executor APIs, script discovery, prompt/tool wiring, and runtime execution path.
python/packages/core/agent_framework/init.py Exports new public script-related types.
python/packages/core/tests/core/test_skills.py Adds extensive tests for script discovery/execution, approval mode, and edge cases.
python/packages/core/AGENTS.md Updates package layout docs to include the skills module.
python/samples/02-agents/skills/subprocess_script_runner.py Adds a shared subprocess-based file-script runner callback for samples.
python/samples/02-agents/skills/README.md Adds a top-level index README for the skills learning path and concepts.
python/samples/02-agents/skills/file_based_skill/file_based_skill.py New file-based skill sample demonstrating script execution via callback executor.
python/samples/02-agents/skills/file_based_skill/README.md Documentation for running and understanding the file-based skill sample.
python/samples/02-agents/skills/file_based_skill/skills/password-generator/SKILL.md Adds a file-based skill definition that references a script.
python/samples/02-agents/skills/file_based_skill/skills/password-generator/references/PASSWORD_GUIDELINES.md Adds reference guidelines for the password-generator file skill.
python/samples/02-agents/skills/file_based_skill/skills/password-generator/scripts/generate.py Adds the executable password generator script used by the file-based sample.
python/samples/02-agents/skills/mixed_skills/mixed_skills.py New mixed sample combining code-defined scripts/resources with a file-based scripted skill.
python/samples/02-agents/skills/mixed_skills/README.md Documentation for the mixed skills sample and how it wires both skill types.
python/samples/02-agents/skills/mixed_skills/skills/password-generator/SKILL.md Adds a file-based password-generator skill used by the mixed sample.
python/samples/02-agents/skills/mixed_skills/skills/password-generator/references/PASSWORD_GUIDELINES.md Adds reference guidelines for the mixed-sample password-generator skill.
python/samples/02-agents/skills/mixed_skills/skills/password-generator/scripts/generate.py Adds the password generator script used by the mixed sample.
python/samples/02-agents/skills/script_approval/script_approval.py New sample showing how to require human approval before executing scripts.
python/samples/02-agents/skills/script_approval/README.md Documentation for the script approval workflow sample.
python/samples/02-agents/skills/code_skill/code_skill.py Removes/renames the older code skill sample in favor of the new code-defined-skill sample.
python/samples/02-agents/skills/code_skill/README.md Removes the older code skill sample README.
python/samples/02-agents/skills/code_defined_skill/code_defined_skill.py Adds the updated code-defined skill sample including @skill.script.
python/samples/02-agents/skills/code_defined_skill/README.md Documentation for the code-defined skill sample.
python/samples/02-agents/skills/basic_skill/basic_skill.py Removes the older “basic_skill” sample.
python/samples/02-agents/skills/basic_skill/README.md Removes the older “basic_skill” README.
python/samples/02-agents/skills/basic_skill/skills/expense-report/SKILL.md Removes the older expense-report file skill.
python/samples/02-agents/skills/basic_skill/skills/expense-report/references/POLICY_FAQ.md Removes the older expense-report FAQ resource.
python/samples/02-agents/skills/basic_skill/skills/expense-report/assets/expense-report-template.md Removes the older expense-report template asset.

Copilot AI review requested due to automatic review settings March 10, 2026 09:47
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated 4 comments.

Copilot AI review requested due to automatic review settings March 10, 2026 13:28
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated 2 comments.

Copilot AI review requested due to automatic review settings March 10, 2026 15:41
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated 10 comments.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 10, 2026 16:48
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated 4 comments.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 10, 2026 17:07
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated 3 comments.

…n every call when the result was None

   2. Updating the arguments tool description to be more generic (not CLI-specific)
Copilot AI review requested due to automatic review settings March 10, 2026 17:39
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated 3 comments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation python

Projects

Status: In Review

Development

Successfully merging this pull request may close these issues.

Python: Support the execution of agent skill scripts

5 participants