Skip to content

fix(tanstack-ai): handle image ContentPart in workers-ai adapter#435

Open
mdhruvil wants to merge 2 commits intocloudflare:mainfrom
mdhruvil:mdhruvil/fix-tanstack-ai-handle-image-content-part
Open

fix(tanstack-ai): handle image ContentPart in workers-ai adapter#435
mdhruvil wants to merge 2 commits intocloudflare:mainfrom
mdhruvil:mdhruvil/fix-tanstack-ai-handle-image-content-part

Conversation

@mdhruvil
Copy link

@mdhruvil mdhruvil commented Mar 4, 2026

Fix: Workers AI adapter drops type: "image" content parts

Closes #431

Problem

TanStack AI's multimodal content uses ContentPart types where images are represented as:

{
  type: "image",
  source: { type: "url" | "data", value: string, mimeType?: string }
}

The workers-ai adapter's buildUserContent function only handled the legacy type: "image_url" format (workers-ai.ts#L98-L122), which TanStack AI never sends. This caused all image parts to be silently dropped from user messages.

The detection logic on line 98 correctly checked for both type: "image_url" and type: "image", but the conversion logic on lines 112-122 only handled type: "image_url" — so images were detected as present but never converted.

Solution

Rewrote the message-building helpers to use TanStack AI's canonical types directly, following the patterns from the reference OpenAI adapter (as recommended by TanStack's community adapter guide).

Type changes

  • Removed the ad-hoc MessageLike interface — replaced with ModelMessage from @tanstack/ai
  • Removed the inline Array<{ type: string; content?: string; image_url?: unknown }> content type — replaced with ModelMessage["content"] which is string | null | Array<ContentPart>
  • Imported ContentPart and ModelMessage from @tanstack/ai (already a peer dependency)

convertContentPart (new)

Extracted a dedicated function that converts a single ContentPartOpenAI.Chat.ChatCompletionContentPart:

Input Output
type: "text" { type: "text", text: part.content }
type: "image", source.type: "url" { type: "image_url", image_url: { url: source.value } }
type: "image", source.type: "data" { type: "image_url", image_url: { url: "data:{mimeType};base64,{value}" } }
audio / video / document console.warn + skip (Workers AI chat API doesn't support these)

Includes a guard against source.value already containing a data: prefix (matching the reference adapter's pattern) to prevent malformed double-prefixed URIs.

Files changed

File Change
packages/tanstack-ai/src/adapters/workers-ai.ts Core fix: use ContentPart/ModelMessage, handle type: "image"
packages/tanstack-ai/test/message-builder.test.ts Update image tests to ContentPart format
packages/tanstack-ai/test/workers-ai-adapter.test.ts Update image integration test, add data source test

buildUserContent only handled type:"image_url" - TanStack AI
sends type:"image" with source field, so image parts were silently dropped.
- replace MessageLike with ModelMessage, ad-hoc content type with ContentPart
- extract convertContentPart(): image source.type "url"/"data" → OpenAI image_url
- guard against already-prefixed data URIs (matches reference OpenAI adapter)
- warn + skip unsupported modalities (audio/video/document)
- remove dead type:"image_url" codepath and type assertion
- update tests to ContentPart format, add edge cases
@changeset-bot
Copy link

changeset-bot bot commented Mar 4, 2026

🦋 Changeset detected

Latest commit: 561b1fc

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@cloudflare/tanstack-ai Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@mdhruvil mdhruvil changed the title fix(tanstack-ai): handle image ContentPart in workers-ai adapter (#431) fix(tanstack-ai): handle image ContentPart in workers-ai adapter Mar 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

@cloudflare/tanstack-ai workers-ai adapter drops image parts (type: "image")

1 participant