Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 50 additions & 2 deletions chat/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ The longer the conversation, the more tokens it uses. The maximum conversation l
| `frequency_penalty` | Penalizes tokens based on how often they've appeared so far, reducing repetition. Range: -2.0 to 2.0. Default: `0` |
| `response_format` | See [Structured Outputs](/chat/structured-outputs) |
| `tools`, `tool_choice` | See [Tool Calling](/chat/tool-calling) |
| `service_tier` | Priority inference for tagged models. See [Service Tier](#service-tier) below. |
| `service_tier` | Select a service tier (`"priority"` or `"flex"`) for tagged models. See [Service Tier](#service-tier) below. |
| `reasoning_effort` | Controls reasoning depth for reasoning models. See [Reasoning Models](/chat/reasoning). |

<Note>
Expand All @@ -178,6 +178,10 @@ For the complete parameter reference, see the [API reference](/api-reference/cha

## Service tier

Set the optional `service_tier` parameter to run a request on a non-standard tier. Two tiers are available on tagged models: **priority** (faster, at a premium) and **flex** (cheaper, best-effort). Leave `service_tier` unset for standard real-time scheduling and pricing.

### Priority

Set `service_tier` to `"priority"` to request priority inference on supported models. Priority requests get faster time-to-first-token and higher throughput during peak demand.

<Warning>
Expand Down Expand Up @@ -220,7 +224,51 @@ curl "https://api.deepinfra.com/v1/openai/chat/completions" \

</CodeGroup>

The response includes a `service_tier` field confirming which tier was actually used. Not all models support priority tiers — check the model page for availability. If a model doesn't support priority inference, the request is served at the standard tier and billed at the standard price; no error is returned.
### Flex

Set `service_tier` to `"flex"` to run Chat Completions requests at a lower cost in exchange for slower response times and occasional resource unavailability. It's ideal for non-production or lower-priority tasks such as model evaluations, data enrichment, and asynchronous workloads. When a model is busy, a flex request may wait up to 10 minutes for available capacity before it runs or is rejected with an HTTP 429, so use it for work you can retry.

<Note>
Flex inference is billed at a 20% discount off the model's standard per-token price.
</Note>

<CodeGroup>

```python Python
response = client.chat.completions.create(
model="deepseek-ai/DeepSeek-V3",
messages=[{"role": "user", "content": "Hello!"}],
extra_body={"service_tier": "flex"},
)
```

```javascript JavaScript
const response = await openai.chat.completions.create({
model: "deepseek-ai/DeepSeek-V3",
messages: [{ role: "user", content: "Hello!" }],
service_tier: "flex",
});
```

```bash cURL
curl "https://api.deepinfra.com/v1/openai/chat/completions" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $DEEPINFRA_TOKEN" \
-d '{
"model": "deepseek-ai/DeepSeek-V3",
"service_tier": "flex",
"messages": [
{
"role": "user",
"content": "Hello!"
}
]
}'
```

</CodeGroup>

The response includes a `service_tier` field confirming which tier was actually used. Not all models support these tiers — check the model page for availability. If a model doesn't support the requested tier, the request is served at the standard tier and billed at the standard price; no error is returned.

## Max output tokens

Expand Down