fix(provider): do not place Bedrock cachePoint after reasoning blocks#36532
Open
giggling-ginger wants to merge 2 commits into
Open
fix(provider): do not place Bedrock cachePoint after reasoning blocks#36532giggling-ginger wants to merge 2 commits into
giggling-ginger wants to merge 2 commits into
Conversation
Contributor
|
Thanks for updating your PR! It now meets our contributing guidelines. 👍 |
Bedrock rejects ValidationException when a cachePoint sits immediately after a reasoning block. Message-level bedrock.cachePoint was emitted after all content parts by the AI SDK, so assistant turns that ended in reasoning wedged the session on every retry. Anchor the breakpoint on the last non-reasoning part (or skip for reasoning-only messages), and mirror the same safety in the LLM cache policy / Converse lowering path. Fixes anomalyco#36517
06c0313 to
149c2a8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue for this PR
Closes #36517
Type of change
What does this PR do?
On Amazon Bedrock (Anthropic Claude with extended thinking), opencode puts a prompt-cache breakpoint on the last couple of messages. The AI SDK emits message-level
bedrock.cachePointafter every content part. When an assistant message ends with a reasoning block, that produces:The turn fails, and every retry rebuilds the same invalid request, so the session is stuck until that message ages out of history.
This PR:
cachePointon the last non-reasoning content part (or skips it for reasoning-only messages) instead of message-level options.Bug path
flowchart LR A[applyCaching<br/>message-level cachePoint] --> B[AI SDK convert] B --> C[push cache after<br/>all content parts] C --> D{last part is<br/>reasoning?} D -->|yes| E[wire: text → reasoning → cachePoint] E --> F[Bedrock 400<br/>ValidationException] F --> G[message stays in history] G --> H[retry rebuilds same request] H --> F D -->|no| I[OK — cache after text/tool]Fix decision
flowchart TD S[Bedrock assistant message<br/>needs a cache breakpoint] --> Q1{last content part<br/>is reasoning?} Q1 -->|no| M[keep message-level<br/>cachePoint — old behavior] Q1 -->|yes| Q2{any non-reasoning<br/>part exists?} Q2 -->|yes| A[anchor cachePoint on<br/>last non-reasoning part] Q2 -->|no| K[skip cachePoint<br/>reasoning-only message] A --> OK[wire: text → cachePoint → reasoning ✅] K --> OK2[no illegal cache after reasoning ✅] M --> OK3[wire ends with text/tool ✅]Before / after wire layout
flowchart TB subgraph before [Before — illegal] direction TB B1[text] --> B2[reasoning] --> B3[cachePoint ❌] end subgraph after [After — legal] direction TB A1[text] --> A2[cachePoint ✅] --> A3[reasoning] endHow did you verify your code works?
cd packages/opencode && bun test test/provider/transform.test.ts(includes three new cases for bug(bedrock): cachePoint is placed after a reasoning block → ValidationException "Cache point cannot be inserted after reasoning block" #36517)cd packages/llm && bun test test/provider/bedrock-converse.test.ts test/cache-policy.test.tscd packages/opencode && bun typecheckcd packages/llm && bun typecheckNew regression coverage (6 tests, all pass):
Screenshots / recordings
Flow diagrams: Mermaid charts above (bug path, fix decision, before/after wire).
Test results (local
bun test, 6 new cases, all green):Checklist