From a956346f7f25f823014264d35451d7c7ee2ee7d6 Mon Sep 17 00:00:00 2001 From: Sihan Wang Date: Thu, 9 Jul 2026 22:33:22 +0000 Subject: [PATCH 1/3] Document the flex service tier Split the Service Tier section into Priority and Flex subsections. Flex: service_tier="flex", 20% discount, standard scheduling, best-effort (may be queued or 429'd under load). Adds Python/JS/cURL examples and generalizes the parameter row and fallback note to both tiers. Keeps the #service-tier anchor used by the model pricing table. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_013pg5hqXmjMK99hggVC6UF9 --- chat/overview.mdx | 52 +++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 50 insertions(+), 2 deletions(-) diff --git a/chat/overview.mdx b/chat/overview.mdx index 7f4ebdc..4f0b9a5 100644 --- a/chat/overview.mdx +++ b/chat/overview.mdx @@ -167,7 +167,7 @@ The longer the conversation, the more tokens it uses. The maximum conversation l | `frequency_penalty` | Penalizes tokens based on how often they've appeared so far, reducing repetition. Range: -2.0 to 2.0. Default: `0` | | `response_format` | See [Structured Outputs](/chat/structured-outputs) | | `tools`, `tool_choice` | See [Tool Calling](/chat/tool-calling) | -| `service_tier` | Priority inference for tagged models. See [Service Tier](#service-tier) below. | +| `service_tier` | Select a service tier (`"priority"` or `"flex"`) for tagged models. See [Service Tier](#service-tier) below. | | `reasoning_effort` | Controls reasoning depth for reasoning models. See [Reasoning Models](/chat/reasoning). | @@ -178,6 +178,10 @@ For the complete parameter reference, see the [API reference](/api-reference/cha ## Service tier +Set the optional `service_tier` parameter to run a request on a non-standard tier. Two tiers are available on tagged models: **priority** (faster, at a premium) and **flex** (cheaper, best-effort). Leave `service_tier` unset for standard real-time scheduling and pricing. + +### Priority + Set `service_tier` to `"priority"` to request priority inference on supported models. Priority requests get faster time-to-first-token and higher throughput during peak demand. @@ -220,7 +224,51 @@ curl "https://api.deepinfra.com/v1/openai/chat/completions" \ -The response includes a `service_tier` field confirming which tier was actually used. Not all models support priority tiers — check the model page for availability. If a model doesn't support priority inference, the request is served at the standard tier and billed at the standard price; no error is returned. +### Flex + +Set `service_tier` to `"flex"` for latency-tolerant traffic at a discount. Flex requests run at standard scheduling priority and are **best-effort**: when a model is busy they may be queued or rejected with an HTTP 429, so use flex for work you can retry rather than interactive traffic. + + + Flex inference is billed at a 20% discount off the model's standard per-token price. + + + + +```python Python +response = client.chat.completions.create( + model="deepseek-ai/DeepSeek-V3", + messages=[{"role": "user", "content": "Hello!"}], + extra_body={"service_tier": "flex"}, +) +``` + +```javascript JavaScript +const response = await openai.chat.completions.create({ + model: "deepseek-ai/DeepSeek-V3", + messages: [{ role: "user", content: "Hello!" }], + service_tier: "flex", +}); +``` + +```bash cURL +curl "https://api.deepinfra.com/v1/openai/chat/completions" \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer $DEEPINFRA_TOKEN" \ + -d '{ + "model": "deepseek-ai/DeepSeek-V3", + "service_tier": "flex", + "messages": [ + { + "role": "user", + "content": "Hello!" + } + ] + }' +``` + + + +The response includes a `service_tier` field confirming which tier was actually used. Not all models support these tiers — check the model page for availability. If a model doesn't support the requested tier, the request is served at the standard tier and billed at the standard price; no error is returned. ## Max output tokens From 7a218bb4584f5eb07b3723677f7d7ad9b2ba86f6 Mon Sep 17 00:00:00 2001 From: Sihan Wang Date: Thu, 9 Jul 2026 23:05:53 +0000 Subject: [PATCH 2/3] Drop scheduling-mechanism detail from flex docs Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_013pg5hqXmjMK99hggVC6UF9 --- chat/overview.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/chat/overview.mdx b/chat/overview.mdx index 4f0b9a5..b92aea5 100644 --- a/chat/overview.mdx +++ b/chat/overview.mdx @@ -226,7 +226,7 @@ curl "https://api.deepinfra.com/v1/openai/chat/completions" \ ### Flex -Set `service_tier` to `"flex"` for latency-tolerant traffic at a discount. Flex requests run at standard scheduling priority and are **best-effort**: when a model is busy they may be queued or rejected with an HTTP 429, so use flex for work you can retry rather than interactive traffic. +Set `service_tier` to `"flex"` for latency-tolerant traffic at a discount. Flex requests are **best-effort**: when a model is busy they may be queued or rejected with an HTTP 429, so use flex for work you can retry rather than interactive traffic. Flex inference is billed at a 20% discount off the model's standard per-token price. From 4858600ab866196b84f587608bca587ee48d0d46 Mon Sep 17 00:00:00 2001 From: Sihan Wang Date: Thu, 9 Jul 2026 23:08:01 +0000 Subject: [PATCH 3/3] Reword flex docs in OpenAI flex-tier voice Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_013pg5hqXmjMK99hggVC6UF9 --- chat/overview.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/chat/overview.mdx b/chat/overview.mdx index b92aea5..7a5a121 100644 --- a/chat/overview.mdx +++ b/chat/overview.mdx @@ -226,7 +226,7 @@ curl "https://api.deepinfra.com/v1/openai/chat/completions" \ ### Flex -Set `service_tier` to `"flex"` for latency-tolerant traffic at a discount. Flex requests are **best-effort**: when a model is busy they may be queued or rejected with an HTTP 429, so use flex for work you can retry rather than interactive traffic. +Set `service_tier` to `"flex"` to run Chat Completions requests at a lower cost in exchange for slower response times and occasional resource unavailability. It's ideal for non-production or lower-priority tasks such as model evaluations, data enrichment, and asynchronous workloads. When a model is busy, a flex request may wait up to 10 minutes for available capacity before it runs or is rejected with an HTTP 429, so use it for work you can retry. Flex inference is billed at a 20% discount off the model's standard per-token price.