Context
Live validation of compression.cache_hints (anthropic provider) required standing up an external recording proxy between forge and api.anthropic.com, because forge itself gives operators no way to see whether prompt caching is working.
Gap
The anthropic provider parses only input_tokens / output_tokens from the API response usage block. Anthropic also returns:
cache_creation_input_tokens — tokens written to the prompt cache (billed at 1.25x)
cache_read_input_tokens — tokens served from cache (billed at 0.1x)
Both are currently dropped on the floor (forge-core/llm/providers/anthropic.go, non-streaming Usage struct and the streaming message_start event). Consequences:
- An operator enabling
compression.cache_hints cannot verify it does anything — llm_call audit events show identical fields whether the cache hits, misses, or the hints are silently malformed.
- Cost dashboards built on the audit stream over-estimate spend: cache-read tokens are counted (via
input_tokens context) as if they were full-rate.
Proposal
- Parse both fields in the anthropic provider (non-streaming response and streaming
message_start), and populate two new optional fields on llm.UsageInfo (e.g. CacheCreationTokens, CacheReadTokens) — zero-valued for providers that don't report them.
- Stamp them on the
llm_call audit event when non-zero, e.g. cache_creation_input_tokens, cache_read_input_tokens — additive fields, no breaking change to the audit contract.
- Optionally roll them up on
invocation_complete (e.g. cache_read_tokens_total) next to the existing compression_saved_tokens_total, so a single event answers "what did compression + caching save this invocation".
- OpenAI equivalent for parity:
usage.prompt_tokens_details.cached_tokens is already returned on cache hits and is likewise unparsed today.
Evidence from the validation run
An external proxy observed what forge could not: call 1 wrote 3,447 tokens to cache, calls 2–4 each read 3,447 back (cache_read_input_tokens: 3447, cache_creation: 0) — none of it visible in forge's own logs or audit stream.
Context
Live validation of
compression.cache_hints(anthropic provider) required standing up an external recording proxy between forge andapi.anthropic.com, because forge itself gives operators no way to see whether prompt caching is working.Gap
The anthropic provider parses only
input_tokens/output_tokensfrom the API responseusageblock. Anthropic also returns:cache_creation_input_tokens— tokens written to the prompt cache (billed at 1.25x)cache_read_input_tokens— tokens served from cache (billed at 0.1x)Both are currently dropped on the floor (
forge-core/llm/providers/anthropic.go, non-streamingUsagestruct and the streamingmessage_startevent). Consequences:compression.cache_hintscannot verify it does anything —llm_callaudit events show identical fields whether the cache hits, misses, or the hints are silently malformed.input_tokenscontext) as if they were full-rate.Proposal
message_start), and populate two new optional fields onllm.UsageInfo(e.g.CacheCreationTokens,CacheReadTokens) — zero-valued for providers that don't report them.llm_callaudit event when non-zero, e.g.cache_creation_input_tokens,cache_read_input_tokens— additive fields, no breaking change to the audit contract.invocation_complete(e.g.cache_read_tokens_total) next to the existingcompression_saved_tokens_total, so a single event answers "what did compression + caching save this invocation".usage.prompt_tokens_details.cached_tokensis already returned on cache hits and is likewise unparsed today.Evidence from the validation run
An external proxy observed what forge could not: call 1 wrote 3,447 tokens to cache, calls 2–4 each read 3,447 back (
cache_read_input_tokens: 3447,cache_creation: 0) — none of it visible in forge's own logs or audit stream.