feat: Cloudflare-native AI tracing (agents/observability/ai)

[mattzcarey]

Cloudflare-native tracing for the repository's installed AI SDK v6 using the Workers runtime tracing API. Spans use the scalar subset of the current OpenTelemetry GenAI semantic conventions that Workers custom spans can represent and flow directly to Workers Observability and configured OTLP destinations. No OTel SDK, exporter, or collector dependency is added.

AI SDK v6 instrumentation

Enable Worker traces:

{
  "observability": { "traces": { "enabled": true } }
}

Wrap the namespace once and use it normally:

import * as ai from "ai";
import { wrapAISDK } from "agents/observability/ai";

const { generateText, streamText } = wrapAISDK(ai);

await streamText({
  model,
  prompt: "book a table for two",
  tools: { searchRestaurants, reserve },
  experimental_telemetry: {
    functionId: "booking-agent", // AI SDK canonical mapping: gen_ai.agent.name
    metadata: { conversationId: "conv-42" }
  }
});

The wrapper instruments generateText, streamText, generateObject, and streamObject:

invoke_agent booking-agent
├── chat gpt-5.5
├── execute_tool searchRestaurants
└── chat gpt-5.5

Model objects are instrumented through wrapLanguageModel, so provider work and its automatic subrequests run under chat {model}. Tool execution runs under execute_tool {tool}. Streams close on completion, cancellation, an in-band error, or early consumer return. Async-generator tools stay open until iteration and cleanup finish.

AI SDK v6 exposes experimental_context; selected scalar keys can be emitted with wrapAISDK(ai, { includeRuntimeContext: [...] }).

Think: traced out of the box

Think routes every turn through the wrapper with no new customer-facing option. Its default identity is:

gen_ai.agent.name      = this.constructor.name
gen_ai.agent.id        = this.name
gen_ai.conversation.id = this.ctx.id.toString()

These are defaults, not restrictions: beforeTurn may override functionId or identity metadata for applications with another identity model. Think also adds exact cloudflare.agents.turn.* metadata for request ID, trigger, admission, channel, continuation, and generation. Ordinary customer keys such as requestId remain ordinary metadata and are not silently reclassified as Think turn fields.

Drain loops consume the abandoned AI SDK tee after early exit (in-stream error, stall abort, or user abort), preventing open operation spans.

Attribute contract

• invoke_agent: agent/conversation identity, output type, request settings, aggregate standard token usage, dashboard-ready total/tool counts, and scalar finish reason.
• chat: provider, requested model/settings, actual response ID/model when reported, model-call usage and total/tool counts, and gen_ai.response.time_to_first_chunk in seconds.
• execute_tool: tool name/type/call ID and real execution duration via span duration.
• Original SDK operation fields use cloudflare.agents.* where no standard attribute exists.

Semantic corrections included in the final audit:

• functionId → gen_ai.agent.name is retained because it is the AI SDK's canonical OpenTelemetry projection.
• gen_ai.response.finish_reasons and gen_ai.request.stop_sequences are omitted: OTel defines arrays, while Workers Span.setAttribute currently accepts only scalar values. A single finish reason is exposed accurately as cloudflare.agents.response.finish_reason instead of JSON text under an array-typed key.
• Response ID/model and first-chunk timing are placed on model-call spans rather than duplicated onto the enclosing operation when a model child exists.
• Requested modelId is never presented as the served gen_ai.response.model.
• cloudflare.agents.tool.count and cloudflare.agents.usage.total_tokens remain as useful precomputed dashboard dimensions; only low-value output-presence flags were removed.
• Span names retain the Workers Observability 64 UTF-8-byte guard, falling back to the bare operation while preserving the full target in attributes.
• Provider aliases match the AI SDK/OpenTelemetry mapping (azure vs azure-openai, google-vertex, bedrock, etc.).
• Failures emit error.type; no fake otel.status_code attribute is created because OTel status is span state and Workers exposes no custom-span status setter. Cancellations remain non-errors with cloudflare.agents.canceled: true.
• Payload storage is explicit and off by default: storeMessages writes full input/output messages to chat; storeTools writes tool arguments/results to execute_tool. Schemas, request headers, provider options, the dedicated system parameter, and raw error messages remain excluded.

Payload storage

Two independent flags control payload attributes:

• storeMessages writes gen_ai.input.messages and gen_ai.output.messages on chat, including tool-call parts. Values over budget drop the oldest unprotected message (index 2) repeatedly, preserving the first two messages and newest tail.
• storeTools writes gen_ai.tool.call.arguments and gen_ai.tool.call.result on execute_tool.

Both default to false. Think exposes the same fields; agent-think opts into both. The flags configure the wrapper directly and are never copied into telemetry metadata or span attributes.

AI Gateway log references

When an actual gateway-backed response exposes a log ID, its chat span includes cloudflare.ai_gateway.log.id. The adapter reads only explicit response headers, provider metadata, gateway errors, or the Workers AI binding; it omits absent, stale, untrusted, and oversized values. The enclosing invoke_agent span does not receive the attribute, and the adapter never makes an extra request or guesses an ID.

Live verification used the Wrangler-authenticated account against the default AI Gateway:

• unified REST requests succeeded but did not expose cf-aig-log-id, so that path remains omitted;
• a real Workers AI binding request exposed env.AI.aiGatewayLogId;
• env.AI.gateway("default").getLog(id) returned the same log;
• the temporary probe Worker was deleted after verification.

Storage span grouping

SDK-managed Durable Object storage spans are grouped beneath semantic lifecycle phases instead of flooding the same level as inference. Named buckets cover agent/chat initialization, Think startup and hydration, chat request persistence, turn preparation/result persistence, recovery/fibers, alarms, and durable submission acceptance/execution. Raw SQLite spans remain available beneath those collapsible groups.

Every grouping span carries agent identity plus cloudflare.agents.storage.grouped = true, cloudflare.agents.storage.system = "durable_object", and a phase. This keeps invoke_agent, chat, and execute_tool prominent inside chat_turn. Contributed by @Ankcorn in Ankcorn/agents#2.

Approval and durable-submission lifetimes

AI SDK v6 tool approvals are represented by bounded requested, approved, and denied segments under execute_tool, correlated with gen_ai.tool.call.id. No approval span remains open while waiting for a human across invocations.

Think durable submissions are alarm-owned: acceptance only persists and schedules work, while the awaited alarm invocation owns model execution. This prevents model/chat spans from outliving the short acceptance RPC and being force-closed by the Durable Object invocation boundary.

Public surface

agents/observability/ai exports only:

• wrapAISDK (AI SDK v6) and its wrapper option type
• createAISDKTelemetry (AI SDK v7) and its telemetry type

The Workers tracing adapter, tracer/span handles, builders, and runtime seams remain internal. agents/observability continues to expose diagnostics-channel events only.

AI SDK v7

AI SDK v7 (now GA) ships a first-class telemetry lifecycle, so v7 is instrumented through that lifecycle rather than by wrapping the SDK. createAISDKTelemetry() returns an integration you register once, or pass per call:

import { registerTelemetry } from "ai";
import { createAISDKTelemetry } from "agents/observability/ai";

registerTelemetry(createAISDKTelemetry());
// or: generateText({ ..., experimental_telemetry: { integrations: [createAISDKTelemetry()] } })

The v7 integration projects into the same invoke_agent / chat / execute_tool span schema and the same scalar attribute contract as v6, so both SDK versions are dashboard-compatible. One topology difference follows from v7's event-driven telemetry model (vs. v6's operation-wrapping): the operation, model, and tool spans are correlated by a shared cloudflare.agents.call.id rather than parented under one another. The executeLanguageModelCall / executeTool execution hooks still nest real work — provider subrequests run under chat, and a tool's own nested inference runs under execute_tool.

The adapter defines the v7 telemetry shapes structurally (no ai import), so it compiles and unit-tests in this v6 repository, and it is a structural match for AI SDK v7's exported Telemetry interface (a compile-time satisfies Telemetry check against ai@7.0.22 passes with no loosening).

Production validation. Deployed a Worker on ai@7.0.22 (@ai-sdk/provider@4.0.3) with observability.traces.enabled, registered the v7 integration, and ran a two-step tool loop. Workers Observability recorded the expected spans (invoke_agent, two chat, execute_tool) with correct gen_ai.* / cloudflare.agents.* attributes and no raw prompt/message/tool content; the provider subrequest's persisted parentSpanId confirmed it nested under its chat span via executeLanguageModelCall.

Testing

• focused AI observability coverage: 3 files / 74 tests
• full agents suite: 118 files / 2357 tests
• full ai-chat suite: 49 files / 728 tests
• Think Workers suite: 43 files / 910 tests
• pnpm check: exports, formatting, lint, and all 116 TypeScript projects
• agents and @cloudflare/think package builds pass
• generated declarations expose only the focused AI tracing API and the two storage options

Production validation

Compared raw Workers Observability events for the same agent-think smoke shape:

• baseline: de7395981c140629b3219f5f41cb9d29
• audited build: 6f3737d3788029ae4b23dff165ed3383
• topology unchanged: 1 invoke_agent ThinkAgent, 4 chat gpt-5.5, 3 execute_tool; all children correctly parented
• model TTFC corrected from four 0 values to 9.462s, 6.144s, 4.356s, 1.011s
• invalid JSON-string gen_ai.response.finish_reasons removed
• scalar application metadata (repository, issueNumber, requestedBy) visible on the root
• turn reached done, added no tool errors, and terminal cleanup left exactly one warm container on container-application image version 7

Evidence: #1883 (comment)

Provenance

Ported from @msmps's feat/ai-tracing branch with Co-authored-by credit, then folded into agents, aligned with current GenAI semantics, and integrated into Think.

[changeset-bot]

🦋 Changeset detected

Latest commit: 15b7c49

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages

Name	Type
agents	Minor
@cloudflare/ai-chat	Patch
@cloudflare/think	Minor
@cloudflare/agent-think	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[pkg-pr-new]

Open in StackBlitz

agents

npm i https://pkg.pr.new/agents@1860

@cloudflare/ai-chat

npm i https://pkg.pr.new/@cloudflare/ai-chat@1860

@cloudflare/codemode

npm i https://pkg.pr.new/@cloudflare/codemode@1860

create-think

npm i https://pkg.pr.new/create-think@1860

hono-agents

npm i https://pkg.pr.new/hono-agents@1860

@cloudflare/shell

npm i https://pkg.pr.new/@cloudflare/shell@1860

@cloudflare/think

npm i https://pkg.pr.new/@cloudflare/think@1860

@cloudflare/voice

npm i https://pkg.pr.new/@cloudflare/voice@1860

@cloudflare/worker-bundler

npm i https://pkg.pr.new/@cloudflare/worker-bundler@1860

commit: 15b7c49

[abhagsain]

@mattzcarey Would it show a tracing UI like langsmith/braintrust and also show sub-agent calls separately?

[devin-ai-integration]

Devin Review found 1 potential issue.

[mattzcarey]

Production trace comparison

Final v6-only build deployed to agent-think Worker version a612d7b1-d349-4b9b-8601-31901ca8b93e (container application image version 7, digest sha256:1ceff71cad30f3207e4380de6ceaff1b7f89c6908505f49aeba49574344a9d3b).

Raw Workers Observability comparison:

• baseline: de7395981c140629b3219f5f41cb9d29
• final v6-only build: 6ca72a82c7813bfc313c765ea3e86eeb
• topology unchanged: 1 invoke_agent ThinkAgent, 4 chat gpt-5.5, 3 execute_tool; all children correctly parented
• real model TTFC: 8.575s, 2.093s, 4.758s, 1.452s rather than four 0 values
• invalid JSON-string gen_ai.response.finish_reasons removed
• response ID/model/TTFC retained on model spans rather than copied to the root
• dashboard counts retained: root cloudflare.agents.tool.count=3, cloudflare.agents.usage.total_tokens=86336; per-model total/tool counts also present
• scalar application metadata (repository, issueNumber, requestedBy) visible on the root
• turn reached done, added no tool errors, and terminal cleanup left exactly one warm container

The untestable AI SDK v7 adapter was removed from the package and preserved as an explicitly unverified design gist: https://gist.github.com/mattzcarey/cc7265966dd8eb58e7b230688a3f4140

Validation evidence: #1883 (comment)