Skip to content

fix(provider): do not place Bedrock cachePoint after reasoning blocks#36532

Open
giggling-ginger wants to merge 2 commits into
anomalyco:devfrom
giggling-ginger:bedrock-cache-reasoning
Open

fix(provider): do not place Bedrock cachePoint after reasoning blocks#36532
giggling-ginger wants to merge 2 commits into
anomalyco:devfrom
giggling-ginger:bedrock-cache-reasoning

Conversation

@giggling-ginger

@giggling-ginger giggling-ginger commented Jul 12, 2026

Copy link
Copy Markdown

Issue for this PR

Closes #36517

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

On Amazon Bedrock (Anthropic Claude with extended thinking), opencode puts a prompt-cache breakpoint on the last couple of messages. The AI SDK emits message-level bedrock.cachePoint after every content part. When an assistant message ends with a reasoning block, that produces:

ValidationException: Cache point cannot be inserted after reasoning block.

The turn fails, and every retry rebuilds the same invalid request, so the session is stuck until that message ages out of history.

This PR:

  1. For Bedrock assistant messages that end in reasoning, anchors cachePoint on the last non-reasoning content part (or skips it for reasoning-only messages) instead of message-level options.
  2. Updates the LLM cache policy so it never marks a pure-reasoning assistant message, and prefers text / last non-reasoning parts.
  3. Adds regression tests for both the AI SDK transform path and the Bedrock Converse / cache-policy path.

Bug path

flowchart LR
  A[applyCaching<br/>message-level cachePoint] --> B[AI SDK convert]
  B --> C[push cache after<br/>all content parts]
  C --> D{last part is<br/>reasoning?}
  D -->|yes| E[wire: text → reasoning → cachePoint]
  E --> F[Bedrock 400<br/>ValidationException]
  F --> G[message stays in history]
  G --> H[retry rebuilds same request]
  H --> F
  D -->|no| I[OK — cache after text/tool]
Loading

Fix decision

flowchart TD
  S[Bedrock assistant message<br/>needs a cache breakpoint] --> Q1{last content part<br/>is reasoning?}
  Q1 -->|no| M[keep message-level<br/>cachePoint — old behavior]
  Q1 -->|yes| Q2{any non-reasoning<br/>part exists?}
  Q2 -->|yes| A[anchor cachePoint on<br/>last non-reasoning part]
  Q2 -->|no| K[skip cachePoint<br/>reasoning-only message]
  A --> OK[wire: text → cachePoint → reasoning ✅]
  K --> OK2[no illegal cache after reasoning ✅]
  M --> OK3[wire ends with text/tool ✅]
Loading

Before / after wire layout

flowchart TB
  subgraph before [Before — illegal]
    direction TB
    B1[text] --> B2[reasoning] --> B3[cachePoint ❌]
  end
  subgraph after [After — legal]
    direction TB
    A1[text] --> A2[cachePoint ✅] --> A3[reasoning]
  end
Loading

How did you verify your code works?

New regression coverage (6 tests, all pass):

Package Test
opencode transform anchors cachePoint on last non-reasoning part
opencode transform skips cachePoint for reasoning-only assistant
opencode transform keeps message-level cache when not trailing reasoning
llm cache-policy latest-assistant anchors on text before reasoning
llm cache-policy skips markers for reasoning-only assistant
llm bedrock-converse does not place cachePoint after trailing reasoning

Screenshots / recordings

Flow diagrams: Mermaid charts above (bug path, fix decision, before/after wire).

Test results (local bun test, 6 new cases, all green):

Test results for #36517 — 6 new tests pass

packages/opencode  transform.test.ts          3 pass
packages/llm       cache-policy + converse    3 pass

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

@github-actions github-actions Bot added needs:compliance This means the issue will auto-close after 2 hours. and removed needs:compliance This means the issue will auto-close after 2 hours. labels Jul 12, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for updating your PR! It now meets our contributing guidelines. 👍

Bedrock rejects ValidationException when a cachePoint sits immediately
after a reasoning block. Message-level bedrock.cachePoint was emitted
after all content parts by the AI SDK, so assistant turns that ended in
reasoning wedged the session on every retry.

Anchor the breakpoint on the last non-reasoning part (or skip for
reasoning-only messages), and mirror the same safety in the LLM cache
policy / Converse lowering path.

Fixes anomalyco#36517
@giggling-ginger giggling-ginger force-pushed the bedrock-cache-reasoning branch from 06c0313 to 149c2a8 Compare July 12, 2026 16:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(bedrock): cachePoint is placed after a reasoning block → ValidationException "Cache point cannot be inserted after reasoning block"

1 participant