Skip to content

fix(anthropic): accept done stream terminator#136

Merged
ayushag-nv merged 1 commit into
mainfrom
bhuvana/fix-openrouter-anthropic-stream
Jul 24, 2026
Merged

fix(anthropic): accept done stream terminator#136
ayushag-nv merged 1 commit into
mainfrom
bhuvana/fix-openrouter-anthropic-stream

Conversation

@bhuvan002

@bhuvan002 bhuvan002 commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

What

Teach the native Anthropic streaming backend to recognize the OpenAI-style
[DONE] SSE sentinel and stop the stream when it appears.

Add a regression test that verifies events after the sentinel are not emitted.
Existing coverage still verifies normal Anthropic streams that end without
[DONE].

Why

OpenRouter's Anthropic-compatible endpoint can append data: [DONE] after
otherwise valid Anthropic SSE events. Switchyard previously tried to parse that
sentinel as JSON, turning a successful response into a mid-stream parse error.

Reproduction

With Switchyard running on port 4001 and openrouter-opus configured as an
Anthropic-format route to anthropic/claude-opus-4.8 on OpenRouter:

curl --no-buffer http://127.0.0.1:4001/v1/messages \
  -H 'content-type: application/json' \
  -H 'anthropic-version: 2023-06-01' \
  --data '{
    "model": "openrouter-opus",
    "max_tokens": 16,
    "stream": true,
    "messages": [
      {"role": "user", "content": "Reply with exactly: OK"}
    ]
  }'

Before this fix, the response contains the normal Anthropic events through
message_stop, then ends with:

event: error
data: {"type": "error", "error": {"message": "SwitchyardProcessorError('processor failed: SwitchyardUpstreamError: upstream failed: Anthropic stream emitted invalid JSON frame: expected value at line 1 column 2')", "type": "internal_error"}}

OpenRouter has appended data: [DONE]; Switchyard tries to decode it as JSON
and emits the error above. After this fix, [DONE] terminates the stream
cleanly and no error event is emitted.

How tested

  • uv run ruff check . clean
  • uv run mypy switchyard clean
  • uv run pytest tests/ -v -m 'not integration' — 1949 passed, 12 skipped
  • cargo fmt --all --check clean
  • cargo clippy --workspace --all-targets -- -D warnings clean
  • cargo test --workspace green
  • User-confirmed OpenRouter streaming smoke against /api/v1/messages

Checklist

  • One class per file; filename = snake_case of the primary class. (N/A: no classes added.)
  • New public symbols exported from switchyard/__init__.py.__all__ if intended for downstream use. (N/A: no public symbols added.)
  • Unit tests added for new components / bug fixes.
  • README / --help updated if customer-facing surface changed. (N/A: compatibility fix only.)
  • Commits signed off (Signed-off-by: Bhuvan Agrawal <11240550+bhuvan002@users.noreply.github.com>) per the DCO.

Notes for reviewers

The full live E2E suite reached 17 passing tests, then hit its five-error limit
because the shared fixture on current main still invokes the removed
switchyard passthrough command. That harness issue is unrelated to this
two-file parser change; 19 live tests were therefore not run.

Summary by CodeRabbit

  • Bug Fixes

    • Fixed Anthropic streaming responses to stop processing immediately at the [DONE] marker.
    • Prevented events sent after stream completion from appearing in responses.
    • Improved handling when a connection closes without a final separator.
  • Tests

    • Added coverage verifying that events after [DONE] are ignored.

Signed-off-by: Bhuvan Agrawal <11240550+bhuvan002@users.noreply.github.com>
@bhuvan002
bhuvan002 requested a review from a team as a code owner July 24, 2026 18:53
@ayushag-nv
ayushag-nv enabled auto-merge (squash) July 24, 2026 18:55
@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown

Review Change Stack

Walkthrough

Anthropic native SSE streaming now treats [DONE] as a terminal marker, including buffered final-frame handling. An integration test verifies frames after [DONE] are not emitted.

Changes

Anthropic SSE termination

Layer / File(s) Summary
SSE termination and validation
crates/switchyard-components/src/backends/anthropic.rs, crates/switchyard-components/tests/adversarial_native_backends.rs
anthropic_sse_stream stops on ParsedSseFrame::Done, applies the sentinel to buffered frames, and verifies that later events are excluded.

Estimated code review effort: 2 (Simple) | ~10 minutes

Poem

I’m a rabbit hopping through the stream,
[DONE] ends the SSE dream.
No later carrots pass the gate,
The test confirms they’re stopped in wait.
Thump, thump—clean termination’s great!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: Anthropic streaming now accepts the [DONE] terminator.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
crates/switchyard-components/tests/adversarial_native_backends.rs (1)

1043-1077: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Cover the unterminated [DONE] frame.

This test exercises only the normal SSE-frame path because [DONE] is followed by \n\n. Add a case where the stream ends with data: [DONE] without a final separator, so the buffered-frame handling at anthropic.rs Line 539 is regression-tested.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/switchyard-components/tests/adversarial_native_backends.rs` around
lines 1043 - 1077, The test `anthropic_streaming_stops_at_done_marker` currently
covers only a `[DONE]` frame terminated by a blank-line separator. Add a
separate streaming test case whose SSE payload ends exactly with `data: [DONE]`
without a trailing newline or separator, then collect the response and assert it
emits only the preceding `message_start` event while exercising the
buffered-frame handling in the Anthropic backend.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@crates/switchyard-components/tests/adversarial_native_backends.rs`:
- Around line 1043-1077: The test `anthropic_streaming_stops_at_done_marker`
currently covers only a `[DONE]` frame terminated by a blank-line separator. Add
a separate streaming test case whose SSE payload ends exactly with `data:
[DONE]` without a trailing newline or separator, then collect the response and
assert it emits only the preceding `message_start` event while exercising the
buffered-frame handling in the Anthropic backend.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: cd36e2e8-2a0b-40b0-a899-473640bd05e7

📥 Commits

Reviewing files that changed from the base of the PR and between 3269dce and 45a8acf.

📒 Files selected for processing (2)
  • crates/switchyard-components/src/backends/anthropic.rs
  • crates/switchyard-components/tests/adversarial_native_backends.rs

@ayushag-nv
ayushag-nv merged commit 324e8fe into main Jul 24, 2026
16 checks passed
@ayushag-nv
ayushag-nv deleted the bhuvana/fix-openrouter-anthropic-stream branch July 24, 2026 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants