Skip to content

[bot] Missing instrumentation for the official Model Context Protocol (MCP) Java SDK #141

Description

@braintrust-bot

Summary

The Braintrust Java SDK does not instrument the official Model Context Protocol (MCP) Java SDK (io.modelcontextprotocol.sdk:mcp). MCP is the open standard for connecting LLM applications to external tools/data sources, and its Java SDK is maintained by the Model Context Protocol org in collaboration with Spring AI — a framework this repo already instruments (springai_1_0_0). The MCP client's tool-invocation execution surface (the mechanism models actually use to call tools) is completely uninstrumented, even though Spring AI chat model spans are captured.

What is missing

The io.modelcontextprotocol.sdk:mcp SDK provides these execution-oriented surfaces, none of which are instrumented:

API Class Description
callTool() / callToolAsync() McpSyncClient / McpAsyncClient Executes a tool on an MCP server and returns its result — the core "tool-calling during model execution" surface
listTools() McpSyncClient / McpAsyncClient Discovers available tools from a server
getPrompt() / listPrompts() McpSyncClient Retrieves/executes server-defined prompt templates
readResource() / listResources() McpSyncClient Reads server-exposed resources referenced during agent execution
Server-side CallToolResult handlers McpServer (sync/async) Server-side tool execution when this JVM acts as an MCP server

Because Spring AI's MCP integration (spring-ai-mcp) wraps this same SDK to expose MCP tools as ToolCallbacks to ChatModel, any tool call made through MCP during an instrumented Spring AI chat completion still produces no dedicated tool-execution span — the model call is traced, but the tool invocation it triggers is not.

Why existing instrumentation does not cover this

Braintrust docs status

Upstream sources

Local files inspected

  • settings.gradle (lines 28–33) — lists all instrumentation modules (openai_2_15_0, anthropic_2_2_0, genai_1_18_0, langchain_1_8_0, springai_1_0_0, aws_bedrock_2_30_0); no MCP module present
  • braintrust-sdk/instrumentation/springai_1_0_0/src/main/java/dev/braintrust/instrumentation/springai/v1_0_0/BraintrustSpringAI.java — only handles OpenAiChatModel$Builder / AnthropicChatModel$Builder; no MCP tool-callback handling
  • Full-repo grep for modelcontextprotocol, io.modelcontextprotocol — zero matches

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions