Skip to content

Python: Core: notify agent of external AgentModeProvider mode changes#5650

Open
eavanvalkenburg wants to merge 1 commit intomicrosoft:mainfrom
eavanvalkenburg:harness/mode-notification
Open

Python: Core: notify agent of external AgentModeProvider mode changes#5650
eavanvalkenburg wants to merge 1 commit intomicrosoft:mainfrom
eavanvalkenburg:harness/mode-notification

Conversation

@eavanvalkenburg
Copy link
Copy Markdown
Member

Description

Follow-up to #5611 (already merged) that addresses reviewer feedback about external mode changes confusing the agent.

When the user changes the mode externally (e.g. via a slash-command handler calling set_agent_mode), the chat history still shows the agent's prior set_mode tool call near the end. Updating only the system instructions wasn't enough — models tend to anchor on the recent tool call and ignore the new mode. The reviewer's example:

system: mode is plan
toolcall: mode is execute

The agent typically thought it was still in execute mode.

This PR aligns with the existing .NET AgentModeProvider behavior:

  • set_agent_mode (the public helper used by external callers) now records the previous mode in provider state when the mode actually changes.
  • On the next before_run, the provider pops that flag and injects a user-role notification message into the context (Mode changed: ... switched from "X" to "Y". You must now adjust your behavior...), so the most recent context unambiguously reflects the current mode.
  • The agent-driven set_mode tool bypasses this — it writes current_mode directly so it doesn't trigger a redundant self-notification.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • I have added tests covering my changes
  • I have updated the documentation accordingly

Tests

5 new tests in test_harness_mode.py:

  • External change records previous mode; no-op set does not.
  • After external change, next run injects exactly one user message and clears the flag.
  • Agent-driven set_mode tool does not queue a notification.
  • Subsequent runs without further external changes do not re-inject.

When the operating mode is changed externally (e.g. via a slash-command handler
calling set_agent_mode), the agent's chat history still shows the prior set_mode
tool call near the end. Updating only the system instructions is insufficient —
models tend to anchor on the recent tool call and ignore the new mode.

Mirror the .NET AgentModeProvider behavior: when set_agent_mode detects an actual
mode change, record the previous mode in provider state. On the next before_run,
the provider pops that flag and injects a user-role notification message
announcing the switch, so the most recent context unambiguously reflects the
current mode. The agent-driven set_mode tool path bypasses this so it does not
trigger a redundant notification on its own change.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 5, 2026 11:39
@moonbox3 moonbox3 added the python label May 5, 2026
@eavanvalkenburg eavanvalkenburg enabled auto-merge May 5, 2026 11:40
@moonbox3
Copy link
Copy Markdown
Contributor

moonbox3 commented May 5, 2026

Python Test Coverage

Python Test Coverage Report •
FileStmtsMissCoverMissing
packages/core/agent_framework/_harness
   _mode.py102199%68
TOTAL32269371188% 

Python Unit Test Overview

Tests Skipped Failures Errors Time
6299 30 💤 0 ❌ 0 🔥 1m 41s ⏱️

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Python AgentModeProvider (Agent Harness) so that externally-triggered mode changes (e.g., via set_agent_mode from a slash-command handler) are surfaced to the model as a fresh user-role notification message on the next run, preventing the model from anchoring on a recent set_mode tool call in the prior chat history.

Changes:

  • Track external mode transitions by recording the previous mode in session state when set_agent_mode actually changes the mode.
  • Inject a one-time user notification message during the next AgentModeProvider.before_run, then clear the marker so it won’t repeat.
  • Add unit tests validating the state marker behavior, one-time injection, and that agent-driven set_mode does not queue notifications.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
python/packages/core/agent_framework/_harness/_mode.py Records previous mode for external changes and injects a one-time user notification message on the next run; agent tool path bypasses the notification marker.
python/packages/core/tests/core/test_harness_mode.py Adds tests covering the external-change marker, no-op behavior, single injection + clearing, and no reinjection on subsequent runs.

Comment thread python/packages/core/tests/core/test_harness_mode.py
Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Code Review

Reviewers: 4 | Confidence: 90% | Result: All clear

Reviewed: Correctness, Security Reliability, Test Coverage, Design Approach


Automated review by eavanvalkenburg's agents

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants