|
1 | 1 | --- |
2 | 2 | title: "Large payloads in chat.agent" |
3 | 3 | sidebarTitle: "Large payloads" |
4 | | -description: "Why a single chunk on the chat stream is capped at ~1 MiB, what error you'll see, and the two patterns that work around it: ID references and out-of-band run streams." |
| 4 | +description: "Why a single chunk on the chat stream is capped at ~1 MiB, what error you'll see, and how to work around it with ID references." |
5 | 5 | --- |
6 | 6 |
|
7 | 7 | The realtime stream that backs `chat.agent` enforces a **per-record cap of ~1 MiB** (`1048576` bytes minus a small envelope reserve). Anything written through the chat output — auto-piped LLM chunks, `chat.response.write`, custom `writer.write` parts — counts as one record per chunk and is rejected if it crosses the cap. |
@@ -61,7 +61,7 @@ const fetchPage = tool({ |
61 | 61 |
|
62 | 62 | If the size is unbounded by input, fix the tool — not the stream. |
63 | 63 |
|
64 | | -## Pattern 1: ID-reference (recommended) |
| 64 | +## ID-reference pattern |
65 | 65 |
|
66 | 66 | Store the large value in your own database (or object store) and emit only an identifier through the chat stream. The frontend fetches the full payload separately on demand. |
67 | 67 |
|
@@ -134,42 +134,19 @@ chat.response.write({ type: "data-report", data: { id, summary: shortSummary } } |
134 | 134 | Persist the large value **before** you emit the id chunk. If the chunk reaches the UI before the row is written, the frontend gets a 404 on the follow-up fetch. |
135 | 135 | </Tip> |
136 | 136 |
|
137 | | -## Pattern 2: Out-of-band `streams.writer()` |
| 137 | +## Transient UI parts |
138 | 138 |
|
139 | | -If the value is **only useful for the lifetime of the run** (a long log tail, a transient progress dump, a per-turn debug trace) and you don't want to persist it, write it to a **separate run-scoped stream** instead. Run-scoped `streams.writer()` is its own channel — chunks go through the same per-record cap, but the chat stream stays untouched, and `useRealtimeRunWithStreams` consumes them independently of the chat UI. |
| 139 | +For progress indicators or status data that should stream to the UI but not persist into the response message, use `chat.response.write` with `transient: true`. The chunk still travels on the chat stream (so the 1 MiB per-record cap still applies), but it never lands in `responseMessage` or `uiMessages`: |
140 | 140 |
|
141 | 141 | ```ts |
142 | | -import { task, streams } from "@trigger.dev/sdk"; |
143 | | -import { chat } from "@trigger.dev/sdk/ai"; |
144 | | - |
145 | | -const debugLog = streams.define<{ line: string }>("debug-log"); |
146 | | - |
147 | | -export const myChat = chat.agent({ |
148 | | - id: "my-chat", |
149 | | - run: async ({ messages, signal }) => { |
150 | | - // Heavy diagnostic stream lives on its own channel. |
151 | | - const log = debugLog.writer(); |
152 | | - log.write({ line: "starting turn" }); |
153 | | - |
154 | | - return streamText({ /* ... */ }); |
155 | | - }, |
| 142 | +chat.response.write({ |
| 143 | + type: "data-progress", |
| 144 | + data: { percent: 50 }, |
| 145 | + transient: true, |
156 | 146 | }); |
157 | 147 | ``` |
158 | 148 |
|
159 | | -Frontend: |
160 | | - |
161 | | -```tsx |
162 | | -import { useRealtimeRunWithStreams } from "@trigger.dev/react-hooks"; |
163 | | - |
164 | | -function DebugPanel({ runId }: { runId: string }) { |
165 | | - const { streams } = useRealtimeRunWithStreams<typeof myChat>(runId); |
166 | | - return ( |
167 | | - <pre>{streams?.["debug-log"]?.map((c) => c.line).join("\n")}</pre> |
168 | | - ); |
169 | | -} |
170 | | -``` |
171 | | - |
172 | | -Same 1 MiB cap applies per record, so split long content across multiple writes (one record per line, per page, per progress tick) rather than one large blob. |
| 149 | +For genuinely high-volume diagnostic data (per-token traces, large debug dumps), don't try to ship it through the realtime stream at all. Log to your own store (DB, object storage, OTel logger) and surface it through a separate UI route that isn't tied to the chat session. |
173 | 150 |
|
174 | 151 | ## What does **not** trigger the cap |
175 | 152 |
|
|
0 commit comments