fix: Update cache token tracking for GenerateText GenerateObject calls#12783
Closed
nelsonauner wants to merge 1 commit intovercel:mainfrom
Closed
fix: Update cache token tracking for GenerateText GenerateObject calls#12783nelsonauner wants to merge 1 commit intovercel:mainfrom
nelsonauner wants to merge 1 commit intovercel:mainfrom
Conversation
tigent
Bot
added
ai/core
nelsonauner
changed the title
1 task
Author
|
@aayush-kapoor I know you're really busy but I'd love some quick feedback on if this is worth fixing or its a red herring or will be obsoleted by your changes in #12784 |
Collaborator
|
@nelsonauner thanks for raising this! don't think it makes sense to add it here rn. i'll make a note of it to ensure all tokens (cache read, write) are accounted for in the new telemetry setup. and if you still think a gap exists, you can raise a pr against the new setup. marking this as closed for now |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
The streamText and streamObject paths emit the newer telemetry span attributes (ai.usage.inputTokens, ai.usage.outputTokens, ai.usage.totalTokens, ai.usage.reasoningTokens, ai.usage.cachedInputTokens) on both their doGenerate and step/root spans. However, generateText and generateObject only emit the legacy ai.usage.promptTokens and ai.usage.completionTokens attributes, missing the cache and reasoning token breakdown entirely. The TODO comments in the code (
// TODO rename telemetry attributes to inputTokens and outputTokens) suggest this was planned but not completed for the non-streaming paths.Summary
Added the 5 missing ai.usage.* telemetry span attributes to generateText and generateObject, matching what streamText and streamObject already emit:
These are added at both the doGenerate span (per-model-call) and the root span levels. The legacy ai.usage.promptTokens and ai.usage.completionTokens attributes are preserved for backward compatibility.
Manual Verification
Compared the span attributes set in all four code paths (generateText, streamText, generateObject, streamObject) to confirm they now emit a consistent set of ai.usage.* attributes. The new attributes are undefined (and thus omitted from spans) when the provider doesn't return cache/reasoning data, so this is backward-compatible.
Checklist
Future Work
The legacy ai.usage.promptTokens / ai.usage.completionTokens attributes are preserved for backward compatibility and could be removed in a future major version.
Related Issues