Skip to content

Commit c46b859

Browse files
committed
Fixup following Paddy review
1 parent e1851d8 commit c46b859

2 files changed

Lines changed: 11 additions & 18 deletions

File tree

src/pages/docs/ai-transport/index.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -153,4 +153,4 @@ The cost of streaming token responses over Ably depends on:
153153
- the number of subscribers receiving the response.
154154
- the [token streaming pattern](/docs/ai-transport/features/token-streaming#token-streaming-patterns) you choose.
155155

156-
For example, an AI support chatbot sending a response of 250 tokens at 70 tokens/s to a single client using the [message-per-response](/docs/ai-transport/features/token-streaming/message-per-response) pattern would consume 90 inbound messages, 90 outbound messages and 90 persisted messages. See the [AI support chatbot pricing example](/docs/platform/pricing/examples/ai-chatbot) for a full breakdown of the costs in this scenario.
156+
For example, suppose an AI support chatbot sends a response of 300 tokens, each as a discrete update, using the [message-per-response](/docs/ai-transport/features/token-streaming/message-per-response) pattern, and with a single client subscribed to the channel. With AI Transport's [append rollup](/docs/ai-transport/messaging/token-rate-limits#per-response), this will result in usage of 100 inbound messages, 100 outbound messages and 100 persisted messages. See the [AI support chatbot pricing example](/docs/platform/pricing/examples/ai-chatbot) for a full breakdown of the costs in this scenario.

src/pages/docs/platform/pricing/examples/ai-chatbot.mdx

Lines changed: 10 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: AI support chatbot
2+
title: AI support chatbot pricing example
33
meta_description: "Calculate AI Transport pricing for conversations with an AI chatbot. Example shows how using the message-per-response pattern and modifying the append rollup window can generate cost savings."
44
meta_keywords: "chatbot, support chat, token streaming, token cost, AI Transport pricing, Ably AI Transport pricing, stream cost, Pub/Sub pricing, realtime data delivery, Ably Pub/Sub pricing"
55
intro: "This example uses consumption-based pricing for an AI support chatbot use case, where a single agent is publishing tokens to user over AI Transport."
@@ -12,7 +12,7 @@ The scale and features used in this calculation.
1212
| Scale | Features |
1313
|-------|----------|
1414
| 4 user prompts to get to resolution | ✓ Message-per-response |
15-
| 250 tokens per LLM response | |
15+
| 300 token events per LLM response | |
1616
| 75 appends per second from agent | |
1717
| 3 minute average chat duration | |
1818
| 1 million chats | |
@@ -23,32 +23,25 @@ The high level cost breakdown for this scenario is given in the table below. Mes
2323

2424
| Item | Calculation | Cost |
2525
|------|-------------|------|
26-
| Messages | 1092M × $2.50/M | $2730 |
26+
| Messages | 1212M × $2.50/M | $3030 |
2727
| Connection minutes | 6M × $1.00/M | $6 |
2828
| Channel minutes | 3M × $1.00/M | $3 |
2929
| Package fee | | [See plans](/pricing) |
30-
| **Total** | | **~$2739/M chats** |
30+
| **Total** | | **~$3039/M chats** |
3131

3232
### Message usage breakdown
3333

3434
Several factors influence the total message usage. The message-per-response pattern includes [automatic rollup of append events](/docs/ai-transport/features/token-streaming/token-rate-limits#per-response) to reduce consumption costs and avoid rate limits.
3535

36+
- Agent stream time: 300 token events ÷ 75 appends per second = 4 seconds of streaming per response
37+
- Messages published after rollup: 4 seconds x 25 messages/s = **100 messages per response**
38+
3639
| Type | Calculation | Inbound | Outbound | Total messages | Cost |
3740
|------|-------------|---------|----------|----------------|------|
3841
| User prompts | 1M chats × 4 prompts | 4M | 4M | 8M | $20 |
39-
| Agent responses | 1M chats x 4 responses x 250 token events per response | 360M | 360M | 720M | $1800 |
40-
| Persisted messages | Every inbound message is persisted | 364M | 0 | 364M | $910 |
41-
| **Total** | | **728M** | **364M** | **1092M** | **$2730** |
42-
43-
### Effect of append rollup
44-
45-
The calculation above uses the default append rollup window of 40ms, chosen to control costs with minimum impact on responsiveness. For a text chatbot use case, you could increase the window to 200ms without noticably impacting the user experience.
46-
47-
| Rollup window | Inbound response messages | Total messages | Cost |
48-
|---------------|---------------------------|----------------|------|
49-
| 40ms | 360 per chat | 1092M | $2730/M chats |
50-
| 100ms | 144 per chat | 444M | $1110/M chats |
51-
| 200ms | 72 per chat | 228M | $570/M chats |
42+
| Agent responses | 1M chats x 4 responses x 100 messages per response | 400M | 400M | 800M | $2000 |
43+
| Persisted messages | Every inbound message is persisted | 404M | 0 | 404M | $1010 |
44+
| **Total** | | **808M** | **404M** | **1212M** | **$3030** |
5245

5346
<Aside data-type='further-reading'>
5447
Useful links for exploring this topic in more detail.

0 commit comments

Comments
 (0)