fix(ai-anthropic): populate cachedInputTokens in streaming responses#6288
Merged
tim-smart merged 2 commits intoJun 24, 2026
Merged
Conversation
🦋 Changeset detectedLatest commit: 512af66 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
The streaming response builder set inputTokens/outputTokens/totalTokens but never copied cache_read_input_tokens, so cachedInputTokens was always undefined for streamText/streamObject even when prompt caching was active. The non-streaming path already maps it; mirror that at message_start, where the Anthropic API reports the cache-read count. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
bf21ede to
1ca7194
Compare
tim-smart
reviewed
Jun 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The Anthropic streaming response builder (
makeStreamResponse) populatesinputTokens,outputTokens, andtotalTokens, but never copiescache_read_input_tokensinto the normalizedusage. As a resultusage.cachedInputTokensis alwaysundefinedforstreamText/streamObject, even when prompt caching is active and the API reports a non-zero cache read atmessage_start. The non-streaming path already maps this field, so streaming and non-streaming disagree.Fix
Set
cachedInputTokensfromcache_read_input_tokensatmessage_start— whereinput_tokensis already read — mirroring the non-streaming mapping. The output payload shape is unchanged; only the previously-missing field is now populated.event.message.usageisGenerated.BetaUsage, which already declarescache_read_input_tokens, so this matches the existing non-streaming mapping (?? undefined).Notes
I didn't add a streaming test —
packages/ai/anthropichas no test harness for the language model adapter (notest/dir, and the siblingai-openaipackage has none either), so there's no pattern yet for mocking the Anthropic SSE stream. Happy to follow up with a focused stream test once the maintainers point at the preferred mocking shape.Cache writes (
cache_creation_input_tokens) are out of scope —@effect/ai's coreUsagetype has no corresponding field.A changeset is included (
@effect/ai-anthropic, patch).