Skip to content

docs(sdk): context_over_200k is not always a 200K threshold - #5277

Open
xyzs996 wants to merge 1 commit into
anomalyco:devfrom
xyzs996:doc-context-over-200k-threshold
Open

docs(sdk): context_over_200k is not always a 200K threshold#5277
xyzs996 wants to merge 1 commit into
anomalyco:devfrom
xyzs996:doc-context-over-200k-threshold

Conversation

@xyzs996

@xyzs996 xyzs996 commented Aug 22, 2026

Copy link
Copy Markdown

Docstring only — no data, schema or generator change.

The problem

ModelCost.context_over_200k is documented in the SDK as "pricing applied beyond 200K context". For most of the catalog that is true, but for more than half of the entries that carry the field it isn't, and the generator says so on purpose:

// packages/core/src/generate.ts
// context_over_200k is a legacy compatibility field. It intentionally
// includes higher thresholds; cost.tiers carries the exact threshold.
 size >= 200_000

AGENTS.md also gets this right ("legacy output-only … always author tiers"). The SDK type is the one place a consumer reads, and it's the one place that states a flat 200K.

How often it differs, from today's api.json

357 entries carry context_over_200k:

real threshold entries
200 000 164
256 000 48
262 144 2
272 000 137
512 000 6

193 of 357 (54%), across 40 providers, have a threshold above 200K.

What it costs a consumer that trusts the name

The field carries the dear rate, so applying it from 200K upward over-prices every request between 200K and the real threshold. It never under-prices, which is why nothing has caught fire — it silently overstates.

Worst case in today's data is minimax/MiniMax-M3 (1M context, tier starts at 512K):

base            $0.30 / $1.20 per Mtok
tier @ 512 000  $0.60 / $2.40
context_over_200k  $0.60 / $2.40   ← same numbers, wrong starting point

A 400K-token prompt bills at $0.60/Mtok instead of $0.30 — exactly 2x, across a 312K-token band. gpt-5.5 and the whole GPT-5.4/5.5/5.6 family have the same shape with a 72K-token band.

The change

Replaces the one-line docstring with one that states the threshold varies, names three concrete examples, points at tiers[].tier.size, and says the field is only there for consumers predating tiers. The comment in generate.ts already carries this knowledge; this just moves it to where API users see it.

Found while cross-checking long-context tiers across pricing catalogs — models.dev's numbers for the six Grok/GPT-5.5 rows I was verifying matched xAI's and OpenAI's docs exactly, including grok-4.5's cache_read of 0.30 and grok-4.20-multi-agent at the same rate as plain grok-4.20. The data is right; only this docstring is misleading.

@github-actions

Copy link
Copy Markdown
Contributor

No actionable findings.

@github-actions github-actions Bot added the reviewer: ready Automated review found no actionable items label Aug 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

reviewer: ready Automated review found no actionable items

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant