feat: warn on MCP tool response content-type drift between calls#3364
Draft
julian-risch wants to merge 1 commit into
Draft
feat: warn on MCP tool response content-type drift between calls#3364julian-risch wants to merge 1 commit into
julian-risch wants to merge 1 commit into
Conversation
Track the set of content block types each MCP tool returns and emit a warning when a subsequent invocation introduces a previously unseen type. This surfaces a class of server-side rug-pull where a benign tool silently substitutes different content (e.g. a ResourceLink with a sensitive URI) on later calls. Detection only — does not block. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
github-actions
Bot
added
integration:mcp
type:documentation
Contributor
Coverage report (mcp)
Click to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related Issues
Fixes https://github.com/deepset-ai/haystack-private/issues/362
Proposed Changes:
_check_response_shapeinmcp_toolset.pythat records the set of content block types each MCP tool returns on its first invocation and emits a warning whenever a later call introduces a previously unseen type.MCPToolset._response_shapes) throughcreate_invoke_toolso every tool produced by anMCPToolsetparticipates in the check.invoke_tooland share it with the existingoutputs_to_stateextraction path; on parse failure we still return the raw string so the no-outputs_to_statecontract is preserved.RugPullfixture server that returnsTextContenton the first call and aResourceLinkon the second, plus unit tests for the helper directly and one integration-style test against the rug-pull server.This is intentionally a detection-only signal — it does not block the response — because the MCP protocol allows servers to legitimately vary their content types between calls, and an attacker who keeps the content type stable will not trip this check. The aim is to give pipeline owners a clear, loggable signal when a tool's response shape shifts under them.
How did you test it?
TestCheckResponseShape::test_first_call_establishes_baselineTestCheckResponseShape::test_drift_emits_warning_and_extends_baselineTestCheckResponseShape::test_same_shape_does_not_warnTestCheckResponseShape::test_non_dict_parsed_is_ignoredTestCheckResponseShape::test_missing_or_malformed_content_field_is_ignoredTestMCPToolset::test_response_shape_drift_logs_warningNotes for the reviewer
Checklist
fix:,feat:,build:,chore:,ci:,docs:,style:,refactor:,perf:,test:.