docs(sdk): context_over_200k is not always a 200K threshold - #5277
Open
xyzs996 wants to merge 1 commit into
Open
docs(sdk): context_over_200k is not always a 200K threshold#5277xyzs996 wants to merge 1 commit into
xyzs996 wants to merge 1 commit into
Conversation
Contributor
|
No actionable findings. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Docstring only — no data, schema or generator change.
The problem
ModelCost.context_over_200kis documented in the SDK as "pricing applied beyond 200K context". For most of the catalog that is true, but for more than half of the entries that carry the field it isn't, and the generator says so on purpose:AGENTS.mdalso gets this right ("legacy output-only … always author tiers"). The SDK type is the one place a consumer reads, and it's the one place that states a flat 200K.How often it differs, from today's
api.json357 entries carry
context_over_200k:193 of 357 (54%), across 40 providers, have a threshold above 200K.
What it costs a consumer that trusts the name
The field carries the dear rate, so applying it from 200K upward over-prices every request between 200K and the real threshold. It never under-prices, which is why nothing has caught fire — it silently overstates.
Worst case in today's data is
minimax/MiniMax-M3(1M context, tier starts at 512K):A 400K-token prompt bills at $0.60/Mtok instead of $0.30 — exactly 2x, across a 312K-token band.
gpt-5.5and the whole GPT-5.4/5.5/5.6 family have the same shape with a 72K-token band.The change
Replaces the one-line docstring with one that states the threshold varies, names three concrete examples, points at
tiers[].tier.size, and says the field is only there for consumers predatingtiers. The comment ingenerate.tsalready carries this knowledge; this just moves it to where API users see it.Found while cross-checking long-context tiers across pricing catalogs — models.dev's numbers for the six Grok/GPT-5.5 rows I was verifying matched xAI's and OpenAI's docs exactly, including
grok-4.5'scache_readof 0.30 andgrok-4.20-multi-agentat the same rate as plaingrok-4.20. The data is right; only this docstring is misleading.