Skip to content

Commit dadfa85

Browse files
committed
Stop recommending run streams
1 parent 59f9dea commit dadfa85

3 files changed

Lines changed: 14 additions & 37 deletions

File tree

docs/ai-chat/error-handling.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -399,7 +399,7 @@ A specific run-failing error worth flagging on its own. Anything written through
399399

400400
The error carries `chunkType`, `chunkSize`, and `maxSize`. Catch with the `isChatChunkTooLargeError` guard and route oversized values out-of-band.
401401

402-
See [Large payloads in chat.agent](/ai-chat/patterns/large-payloads) for the two patterns that work around the cap (ID-reference + run-scoped `streams.writer()`).
402+
See [Large payloads in chat.agent](/ai-chat/patterns/large-payloads) for the ID-reference pattern that works around the cap, plus guidance on transient data parts and out-of-band logging.
403403

404404
## See also
405405

docs/ai-chat/overview.mdx

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -40,10 +40,10 @@ sequenceDiagram
4040
Task->>Task: onTurnStart({ chatId, messages })
4141
Task->>LLM: streamText({ model, messages, abortSignal })
4242
LLM-->>Task: Stream response chunks
43-
Task->>API: streams.pipe("chat", uiStream)
43+
Task->>API: Write chunks to session.out
4444
API-->>useChat: SSE: UIMessageChunks
4545
useChat-->>User: Render streaming text
46-
Task->>API: Write trigger:turn-complete
46+
Task->>API: Write turn-complete control record
4747
API-->>useChat: SSE: turn complete + refreshed token
4848
useChat->>useChat: Close stream, update session
4949
Task->>Task: onTurnComplete({ messages, stopped: false })
@@ -73,10 +73,10 @@ sequenceDiagram
7373
Task->>Task: onTurnStart({ turn: 1 })
7474
Task->>LLM: streamText({ messages: [all accumulated] })
7575
LLM-->>Task: Stream response
76-
Task->>API: streams.pipe("chat", uiStream)
76+
Task->>API: Write chunks to session.out
7777
API-->>useChat: SSE: UIMessageChunks
7878
useChat-->>User: Render streaming text
79-
Task->>API: Write trigger:turn-complete
79+
Task->>API: Write turn-complete control record
8080
Task->>Task: onTurnComplete({ turn: 1 })
8181
Task->>Task: Wait for next message (idle → suspend)
8282
```

docs/ai-chat/patterns/large-payloads.mdx

Lines changed: 9 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
title: "Large payloads in chat.agent"
33
sidebarTitle: "Large payloads"
4-
description: "Why a single chunk on the chat stream is capped at ~1 MiB, what error you'll see, and the two patterns that work around it: ID references and out-of-band run streams."
4+
description: "Why a single chunk on the chat stream is capped at ~1 MiB, what error you'll see, and how to work around it with ID references."
55
---
66

77
The realtime stream that backs `chat.agent` enforces a **per-record cap of ~1 MiB** (`1048576` bytes minus a small envelope reserve). Anything written through the chat output — auto-piped LLM chunks, `chat.response.write`, custom `writer.write` parts — counts as one record per chunk and is rejected if it crosses the cap.
@@ -61,7 +61,7 @@ const fetchPage = tool({
6161

6262
If the size is unbounded by input, fix the tool — not the stream.
6363

64-
## Pattern 1: ID-reference (recommended)
64+
## ID-reference pattern
6565

6666
Store the large value in your own database (or object store) and emit only an identifier through the chat stream. The frontend fetches the full payload separately on demand.
6767

@@ -134,42 +134,19 @@ chat.response.write({ type: "data-report", data: { id, summary: shortSummary } }
134134
Persist the large value **before** you emit the id chunk. If the chunk reaches the UI before the row is written, the frontend gets a 404 on the follow-up fetch.
135135
</Tip>
136136

137-
## Pattern 2: Out-of-band `streams.writer()`
137+
## Transient UI parts
138138

139-
If the value is **only useful for the lifetime of the run** (a long log tail, a transient progress dump, a per-turn debug trace) and you don't want to persist it, write it to a **separate run-scoped stream** instead. Run-scoped `streams.writer()` is its own channel — chunks go through the same per-record cap, but the chat stream stays untouched, and `useRealtimeRunWithStreams` consumes them independently of the chat UI.
139+
For progress indicators or status data that should stream to the UI but not persist into the response message, use `chat.response.write` with `transient: true`. The chunk still travels on the chat stream (so the 1 MiB per-record cap still applies), but it never lands in `responseMessage` or `uiMessages`:
140140

141141
```ts
142-
import { task, streams } from "@trigger.dev/sdk";
143-
import { chat } from "@trigger.dev/sdk/ai";
144-
145-
const debugLog = streams.define<{ line: string }>("debug-log");
146-
147-
export const myChat = chat.agent({
148-
id: "my-chat",
149-
run: async ({ messages, signal }) => {
150-
// Heavy diagnostic stream lives on its own channel.
151-
const log = debugLog.writer();
152-
log.write({ line: "starting turn" });
153-
154-
return streamText({ /* ... */ });
155-
},
142+
chat.response.write({
143+
type: "data-progress",
144+
data: { percent: 50 },
145+
transient: true,
156146
});
157147
```
158148

159-
Frontend:
160-
161-
```tsx
162-
import { useRealtimeRunWithStreams } from "@trigger.dev/react-hooks";
163-
164-
function DebugPanel({ runId }: { runId: string }) {
165-
const { streams } = useRealtimeRunWithStreams<typeof myChat>(runId);
166-
return (
167-
<pre>{streams?.["debug-log"]?.map((c) => c.line).join("\n")}</pre>
168-
);
169-
}
170-
```
171-
172-
Same 1 MiB cap applies per record, so split long content across multiple writes (one record per line, per page, per progress tick) rather than one large blob.
149+
For genuinely high-volume diagnostic data (per-token traces, large debug dumps), don't try to ship it through the realtime stream at all. Log to your own store (DB, object storage, OTel logger) and surface it through a separate UI route that isn't tied to the chat session.
173150

174151
## What does **not** trigger the cap
175152

0 commit comments

Comments
 (0)