Anthropic: `messages.count_tokens()` endpoint not instrumented

[braintrust-bot]

Summary

The Anthropic Python SDK exposes client.messages.count_tokens() (and client.beta.messages.count_tokens()), a stable GA endpoint that returns the input token count for a given set of messages, tools, and system prompt. The Braintrust Anthropic integration does not wrap this method, so calls to it produce no tracing spans.

What's missing

POST /v1/messages/count_tokens accepts the same parameters as the Messages API (messages, model, system, tools, tool_choice, thinking) and returns {"input_tokens": <int>}. It is useful for:

• Pre-flight cost estimation before sending expensive requests
• Prompt optimization workflows where users iterate on prompt length
• Budget enforcement in production pipelines

The endpoint is free to use (subject to rate limits) and is stable/GA. Instrumenting it would create a span capturing the model, input parameters, and returned token count — giving users visibility into their token counting calls alongside their generation calls.

The current Messages and AsyncMessages wrapper classes in tracing.py only wrap create() and stream(). The count_tokens() method falls through to the unwrapped SDK via Wrapper.__getattr__.

Braintrust docs status

The Anthropic integration page does not mention count_tokens(). Status: not_found.

Upstream sources

• Anthropic API — Count tokens documents the stable POST /v1/messages/count_tokens endpoint
• Anthropic Python SDK exposes client.messages.count_tokens() and client.beta.messages.count_tokens()

Local files inspected

• py/src/braintrust/integrations/anthropic/tracing.py — Messages class (lines ~140–180) wraps create() and stream() only; AsyncMessages class (lines ~60–125) same; no count_tokens method in either
• py/src/braintrust/integrations/anthropic/_utils.py — no reference to count_tokens
• py/src/braintrust/integrations/anthropic/test_anthropic.py — no tests for count_tokens()