Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
eb4dcd1
feat(ai-client): add message-queue config types
AlemTuzlak Jul 6, 2026
7da583f
feat(ai-client): export message-queue public types
AlemTuzlak Jul 6, 2026
e294159
feat(ai-client): add message-queue config normalization + tests
AlemTuzlak Jul 6, 2026
9bd6bcb
feat(ai-client): enqueue/cancel/flush + whenBusy policy in ChatClient
AlemTuzlak Jul 6, 2026
62e7d5c
feat(ai-client): auto-drain queue (fifo/batch) + flush on stop/clear/…
AlemTuzlak Jul 6, 2026
ac8ef62
fix(ai-client): flush message queue when a stream errors
AlemTuzlak Jul 6, 2026
2c4bb89
feat(ai-svelte): expose queue + cancelQueued + per-send whenBusy
AlemTuzlak Jul 6, 2026
833e712
feat(ai-vue): expose queue + cancelQueued + per-send whenBusy
AlemTuzlak Jul 6, 2026
17bb417
feat(ai-react): expose queue + cancelQueued + per-send whenBusy
AlemTuzlak Jul 6, 2026
67d16a2
feat(ai-solid): expose queue + cancelQueued + per-send whenBusy
AlemTuzlak Jul 6, 2026
31a01c1
test: update framework hook tests for queue-by-default behavior
AlemTuzlak Jul 6, 2026
7e6a99e
docs: add changeset for client message queue
AlemTuzlak Jul 6, 2026
7b26506
docs(skills): document message queue in chat-experience skill
AlemTuzlak Jul 6, 2026
7b19868
fix(ai-vue,ai-solid): reflow re-export + correct queue jsdoc
AlemTuzlak Jul 6, 2026
b6376a7
docs: document client message queue (queue option, cancelQueued)
AlemTuzlak Jul 6, 2026
105b741
test(e2e): queue-while-streaming, cancel, and drain ordering
AlemTuzlak Jul 6, 2026
c1651c9
feat(ai-preact): expose queue + cancelQueued + per-send whenBusy
AlemTuzlak Jul 6, 2026
deefebe
feat: re-export queue config types from framework packages + add ai-p…
AlemTuzlak Jul 6, 2026
ddd52fc
test(ai-client): cover multimodal batch-drain merge; docs: clarify wh…
AlemTuzlak Jul 6, 2026
c551ae0
fix: sort queue-type imports alphabetically in framework types
AlemTuzlak Jul 6, 2026
6aa7207
ci: apply automated fixes
autofix-ci[bot] Jul 6, 2026
cb34848
fix: export queue types from framework package entrypoints
AlemTuzlak Jul 6, 2026
7401f07
docs(example): add queueing-strategies route to ts-react-chat
AlemTuzlak Jul 6, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .changeset/message-queue.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
'@tanstack/ai-client': minor
'@tanstack/ai-react': minor
'@tanstack/ai-solid': minor
'@tanstack/ai-vue': minor
'@tanstack/ai-svelte': minor
'@tanstack/ai-preact': minor
---

Messages sent while a stream is already in flight are now queued by default and automatically sent once the in-flight stream settles, instead of being silently dropped. **This is a behavior change.**

The behavior is configurable via a new `queue` option, which accepts `whenBusy: 'queue' | 'drop' | 'interrupt'`, `drain: 'fifo' | 'batch'`, `maxSize`, and `onOverflow`, or a custom strategy function for full control.

Queued messages are exposed on the hook as `queue` and can be cancelled before they send via `cancelQueued(id)`. `sendMessage` also accepts a per-call `{ whenBusy }` override.
49 changes: 49 additions & 0 deletions docs/chat/streaming.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,13 +203,62 @@ export async function POST(request: Request) {
}
```

## Queueing Messages

By default, calling `sendMessage` while a stream is already in flight **queues** the message instead of dropping it — it sends automatically once the current run settles. Configure this with the `queue` option, which accepts a `QueueConfig` object, a plain shorthand string, or a strategy function:

```tsx group=queueing-messages
import { useChat, fetchServerSentEvents } from "@tanstack/ai-react";

const { messages, queue, sendMessage, cancelQueued, isLoading } = useChat({
connection: fetchServerSentEvents("/api/chat"),
queue: { whenBusy: "queue", drain: "fifo", maxSize: 5 },
});
```

- **`whenBusy`** — what happens to a send that arrives mid-stream:
- `"queue"` (default) — hold the message; it sends once the stream settles.
- `"drop"` — ignore the send.
- `"interrupt"` — abort the current stream (like calling `stop()`) and send the new message immediately.
- **`drain`** — how queued items leave the queue: `"fifo"` (default) sends them one at a time in order; `"batch"` merges everything currently queued into a single send once the stream settles (string contents joined with `\n`, multimodal content concatenated in order).
- **`maxSize`** — caps how many messages can be queued.
- **`onOverflow`** — `"reject"` (default) ignores a send once `maxSize` is reached; `"drop-oldest"` evicts the oldest queued item to make room.

You can also pass a plain `WhenBusy` string (e.g. `queue: "interrupt"`) as shorthand for `{ whenBusy: "interrupt" }` — this applies for any `WhenBusy` value (`"queue"`, `"drop"`, or `"interrupt"`), not just `"interrupt"` — or a `QueueStrategy` function for full control over the per-send decision (the drain order stays FIFO for the function form).

`useChat` exposes the pending queue as `queue` so you can render it distinctly from `messages`, along with `cancelQueued(id)` to cancel an item before it sends:

```tsx group=queueing-messages
function PendingQueue() {
return (
<>
{queue.map((q) => (
<div key={q.id} className="pending">
{typeof q.content === "string" ? q.content : "[attachment]"}
<button onClick={() => cancelQueued(q.id)}>Cancel</button>
</div>
))}
</>
);
}
```

Override the configured policy for a single send with the second argument to `sendMessage`:

```tsx group=queueing-messages
sendMessage("Never mind, do this instead", { whenBusy: "interrupt" });
```

> **Note:** This is a default-behavior change — messages sent while streaming used to be silently dropped. They are now queued unless you opt into `whenBusy: "drop"` or `"interrupt"`.

## Best Practices

1. **Handle loading states** - Use `isLoading` to show loading indicators
2. **Handle errors** - Check `error` state for stream failures
3. **Cancel on unmount** - Clean up streams when components unmount
4. **Optimize rendering** - Batch updates if needed for performance
5. **Show progress** - Display partial content as it streams
6. **Render queued messages distinctly** - Use `queue` to show pending sends separately from `messages`

## Next Steps

Expand Down
2 changes: 1 addition & 1 deletion docs/config.json
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@
"label": "Streaming",
"to": "chat/streaming",
"addedAt": "2026-04-15",
"updatedAt": "2026-07-03"
"updatedAt": "2026-07-06"
},
{
"label": "Connection Adapters",
Expand Down
14 changes: 14 additions & 0 deletions examples/ts-react-chat/src/components/Header.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import {
Guitar,
Home,
Image,
Layers,
Menu,
LayoutGrid,
Mic,
Expand Down Expand Up @@ -231,6 +232,19 @@ export default function Header() {
<span className="font-medium">Persistent Chats</span>
</Link>

<Link
to="/queueing"
onClick={() => setIsOpen(false)}
className="flex items-center gap-3 p-3 rounded-lg hover:bg-gray-800 transition-colors mb-2"
activeProps={{
className:
'flex items-center gap-3 p-3 rounded-lg bg-cyan-600 hover:bg-cyan-700 transition-colors mb-2',
}}
>
<Layers size={20} />
<span className="font-medium">Queueing Strategies</span>
</Link>

<Link
to="/example/runtime-context"
onClick={() => setIsOpen(false)}
Expand Down
21 changes: 21 additions & 0 deletions examples/ts-react-chat/src/routeTree.gen.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import { Route as ThreadsRouteImport } from './routes/threads'
import { Route as ServerFnChatRouteImport } from './routes/server-fn-chat'
import { Route as SandboxesRouteImport } from './routes/sandboxes'
import { Route as RealtimeRouteImport } from './routes/realtime'
import { Route as QueueingRouteImport } from './routes/queueing'
import { Route as McpDemoRouteImport } from './routes/mcp-demo'
import { Route as McpAppsRouteImport } from './routes/mcp-apps'
import { Route as Issue176ToolResultRouteImport } from './routes/issue-176-tool-result'
Expand Down Expand Up @@ -74,6 +75,11 @@ const RealtimeRoute = RealtimeRouteImport.update({
path: '/realtime',
getParentRoute: () => rootRouteImport,
} as any)
const QueueingRoute = QueueingRouteImport.update({
id: '/queueing',
path: '/queueing',
getParentRoute: () => rootRouteImport,
} as any)
const McpDemoRoute = McpDemoRouteImport.update({
id: '/mcp-demo',
path: '/mcp-demo',
Expand Down Expand Up @@ -287,6 +293,7 @@ export interface FileRoutesByFullPath {
'/issue-176-tool-result': typeof Issue176ToolResultRoute
'/mcp-apps': typeof McpAppsRoute
'/mcp-demo': typeof McpDemoRoute
'/queueing': typeof QueueingRoute
'/realtime': typeof RealtimeRoute
'/sandboxes': typeof SandboxesRoute
'/server-fn-chat': typeof ServerFnChatRoute
Expand Down Expand Up @@ -333,6 +340,7 @@ export interface FileRoutesByTo {
'/issue-176-tool-result': typeof Issue176ToolResultRoute
'/mcp-apps': typeof McpAppsRoute
'/mcp-demo': typeof McpDemoRoute
'/queueing': typeof QueueingRoute
'/realtime': typeof RealtimeRoute
'/sandboxes': typeof SandboxesRoute
'/server-fn-chat': typeof ServerFnChatRoute
Expand Down Expand Up @@ -380,6 +388,7 @@ export interface FileRoutesById {
'/issue-176-tool-result': typeof Issue176ToolResultRoute
'/mcp-apps': typeof McpAppsRoute
'/mcp-demo': typeof McpDemoRoute
'/queueing': typeof QueueingRoute
'/realtime': typeof RealtimeRoute
'/sandboxes': typeof SandboxesRoute
'/server-fn-chat': typeof ServerFnChatRoute
Expand Down Expand Up @@ -428,6 +437,7 @@ export interface FileRouteTypes {
| '/issue-176-tool-result'
| '/mcp-apps'
| '/mcp-demo'
| '/queueing'
| '/realtime'
| '/sandboxes'
| '/server-fn-chat'
Expand Down Expand Up @@ -474,6 +484,7 @@ export interface FileRouteTypes {
| '/issue-176-tool-result'
| '/mcp-apps'
| '/mcp-demo'
| '/queueing'
| '/realtime'
| '/sandboxes'
| '/server-fn-chat'
Expand Down Expand Up @@ -520,6 +531,7 @@ export interface FileRouteTypes {
| '/issue-176-tool-result'
| '/mcp-apps'
| '/mcp-demo'
| '/queueing'
| '/realtime'
| '/sandboxes'
| '/server-fn-chat'
Expand Down Expand Up @@ -567,6 +579,7 @@ export interface RootRouteChildren {
Issue176ToolResultRoute: typeof Issue176ToolResultRoute
McpAppsRoute: typeof McpAppsRoute
McpDemoRoute: typeof McpDemoRoute
QueueingRoute: typeof QueueingRoute
RealtimeRoute: typeof RealtimeRoute
SandboxesRoute: typeof SandboxesRoute
ServerFnChatRoute: typeof ServerFnChatRoute
Expand Down Expand Up @@ -635,6 +648,13 @@ declare module '@tanstack/react-router' {
preLoaderRoute: typeof RealtimeRouteImport
parentRoute: typeof rootRouteImport
}
'/queueing': {
id: '/queueing'
path: '/queueing'
fullPath: '/queueing'
preLoaderRoute: typeof QueueingRouteImport
parentRoute: typeof rootRouteImport
}
'/mcp-demo': {
id: '/mcp-demo'
path: '/mcp-demo'
Expand Down Expand Up @@ -927,6 +947,7 @@ const rootRouteChildren: RootRouteChildren = {
Issue176ToolResultRoute: Issue176ToolResultRoute,
McpAppsRoute: McpAppsRoute,
McpDemoRoute: McpDemoRoute,
QueueingRoute: QueueingRoute,
RealtimeRoute: RealtimeRoute,
SandboxesRoute: SandboxesRoute,
ServerFnChatRoute: ServerFnChatRoute,
Expand Down
Loading
Loading