Anthropic: server-side tool usage metrics (`server_tool_use`) not captured

[braintrust-bot]

Summary

The Anthropic integration captures core token metrics (input_tokens, output_tokens, cache_read_input_tokens, cache_creation_input_tokens) but silently drops the server_tool_use field from the usage object. This field tracks how many server-side tool invocations (web searches, code executions, etc.) occurred during a request.

What's missing

The Anthropic API Usage object now includes:

server_tool_use: Optional[ServerToolUsage]
# ServerToolUsage has:
#   web_search_requests: int
#   web_fetch_requests: int  (added recently)

When a user enables Anthropic's built-in server-side tools (web_search, code_execution, web_fetch, etc.), the usage object reports how many of each tool type were invoked. This is important for cost tracking (server-side tool usage has separate pricing) and for understanding model behavior.

The current extract_anthropic_usage() function in _utils.py only extracts:

• input_tokens → prompt_tokens
• output_tokens → completion_tokens
• cache_read_input_tokens → prompt_cached_tokens
• cache_creation_input_tokens → prompt_cache_creation_tokens

The server_tool_use field is not extracted or logged anywhere.

Braintrust docs status

The Anthropic integration page documents that "Braintrust handles streaming, metric collection (including cached tokens), and other details" and that "tool use (function calling) is fully supported." Server-side tools are not mentioned. Status: not_found.

Upstream sources

• Anthropic API — Tool use (server-side tools) documents web_search, code_execution, and other built-in tools
• Anthropic API — Messages response documents the server_tool_use field in the Usage object
• Anthropic Python SDK — Usage model includes server_tool_use: Optional[ServerToolUsage]

Local files inspected

• py/src/braintrust/integrations/anthropic/_utils.py — extract_anthropic_usage() (lines 16–70) only handles input_tokens, output_tokens, cache_read_input_tokens, cache_creation_input_tokens; no reference to server_tool_use
• py/src/braintrust/integrations/anthropic/tracing.py — uses extract_anthropic_usage() for all metric extraction
• py/src/braintrust/integrations/anthropic/test_anthropic.py — no tests for server-side tool usage metrics