Skip to content

refactor(server): attach the SSE keepalive through one constructor - #1136

Merged
inureyes merged 3 commits into
mainfrom
refactor/issue-1107-shared-sse-response
Aug 13, 2026
Merged

refactor(server): attach the SSE keepalive through one constructor#1136
inureyes merged 3 commits into
mainfrom
refactor/issue-1107-shared-sse-response

Conversation

@inureyes

@inureyes inureyes commented Aug 13, 2026

Copy link
Copy Markdown
Member

Summary

Five route handlers each ended with the byte-identical tail Sse::new(stream).keep_alive(keepalive.into_inner()).into_response(). A sixth route that forgot the .keep_alive(..) would have compiled and passed the whole suite. This replaces the five copies with one shared constructor, so forgetting the keepalive stops being expressible, and deletes the TODO that asked for an axum integration harness instead.

Why not the test the TODO asked for

The TODO at streaming_tests.rs named a real risk, but the property splits in two and neither half wants that test. Whether axum emits a comment frame while the stream is idle tests axum's KeepAlive, which upstream already covers. Whether every route attaches the keepalive is the half that can actually regress, and an integration test catches it only for routes someone remembers to write a test for. That is five tests guarding five copy-paste sites: the same duplication, moved into the test module, and still blind to the sixth route.

What changed

  • src/server/streaming.rs: new IntoKeepAlive trait and sse_response constructor. sse_response takes the keepalive by value, so a route cannot call it without one.
  • src/server/streaming_responses.rs, src/server/streaming_anthropic.rs: a one-line IntoKeepAlive impl each. The three newtypes stay distinct, which is the property refactor(server): consolidate the three duplicated SSE keepalive constants under one proxy-timeout assertion #1105 asked to keep: a route still cannot attach another surface's keepalive, because the value it holds came from the channel constructor it called. The trait is the only thing they share.
  • The five route tails become sse_response(stream, keepalive).
  • src/server/router_front.rs: its two sites go through the same constructor. They are outside src/server/routes/ and so outside the letter of the criterion, but they were the last hand-assembled Sse::new in the tree, and including them is what makes the one-attach-site property hold tree-wide rather than per-directory.
  • src/server/streaming_tests.rs: the TODO is replaced by a note recording that the hazard went away structurally.

Acceptance criteria

Criterion Evidence
No Sse::new(..).keep_alive(..) remains in src/server/routes/ grep -rn "keep_alive" src/server/routes/ returns nothing
All three keepalive newtypes reach the shared constructor One IntoKeepAlive impl each; all five routes call sse_response
Deleting .keep_alive(..) from the shared constructor is the only way to break the invariant, and it breaks it for every route at once `grep -rn "Sse::new(" src/
The TODO at streaming_tests.rs is removed Replaced with a note on why

Deleting the tails made sse::Sse unused in all five route files and clippy flagged it under -D warnings. That is the compiler confirming no route builds an SSE response on its own any more; the imports were removed.

Test plan

  • cargo test --profile test-fast --features cuda --lib server::streaming: 25 passed, 0 failed
  • cargo test --profile test-fast --features cuda --lib server::routes: 116 passed, 0 failed
  • cargo clippy --profile test-fast --features cuda --lib --tests -- -D warnings
  • cargo fmt --all -- --check

Behaviour is unchanged: every route emits the same frames on the same schedule, so there is no CHANGELOG.md entry. The acceptance criterion names --features metal,accelerate; this is a Linux CUDA box and cannot build that feature set, and no feature gate reaches this code. The criterion's server:: selector was run as its two populated halves, server::streaming and server::routes, because an unfiltered server selector matches hundreds of tests and the full lib suite aborts under parallel execution on this CUDA host for unrelated reasons.

Closes #1107


Review round

A review pass verified behaviour equivalence against the vendored axum 0.7.9 source: sse_response expands to the byte-identical expression the five tails carried, with no boxing, buffering, or adapter inserted, and the error type narrowed rather than widened (all seven call sites feed Infallible). It also confirmed the three newtypes remain non-interchangeable, since default_for_long_prefill is private on both the Responses and Anthropic types, so each is obtainable only from its own channel constructor.

Two things were fixed as a result.

Stale docs, in the file this PR touched most. The hand-assembly sentence was updated in streaming_responses.rs and streaming_anthropic.rs but four more copies remained in streaming.rs. The worst was on sse_channel itself, telling a contributor adding a sixth route to write Sse::new(stream).keep_alive(keepalive.into_inner()) by hand, which is precisely the duplication this PR removes. All four now point at sse_response.

The last public route to a raw KeepAlive is closed. The three into_inner() accessors are gone and the IntoKeepAlive impls return self.0 directly. Each accessor's only caller was its own trait impl, and two were pub inside pub modules, so they were the remaining way for outside code to obtain a bare KeepAlive and hand-assemble an Sse. Raised as optional in review; taken because this PR's central claim is that hand assembly is unreachable, and a public accessor would have left it resting on convention at the one point it should be structural.

Also folded in: the router_front.rs comments regained the sentence stating that KeepAlive::default() is likewise 15s under axum 0.7.9, which is the evidence for the no-behaviour-change claim at those two sites; the IntoKeepAlive for SseKeepAlive impl moved next to impl SseKeepAlive to match the other two files.

Noted, not done. axum::response::sse::Sse is a public third-party type, so nothing mechanically stops a future route from importing it directly. A clippy.toml with a disallowed-types entry pointing at sse_response would make this PR's claim compiler-checked rather than reviewer-checked. The repo has no clippy.toml, and adding a workspace-wide lint config is a larger decision than this issue asked for; it is recorded as follow-up in the technical report.

Five route handlers each ended with the byte-identical tail `Sse::new(stream).keep_alive(keepalive.into_inner()).into_response()`. No route varied the stream type, the keepalive, or the ordering, so this was five copies of one expression, and a sixth route that forgot the `.keep_alive(..)` would have compiled and passed the whole suite. A `TODO` in `streaming_tests.rs` asked for an axum integration harness to close that gap; a test per route is five tests guarding five copy-paste sites, and it still only covers the routes someone remembers to write a test for, which is the same hazard moved into the test module.

`streaming::sse_response` is now the only way to turn one of these streams into a `Response`. It takes the keepalive by value, so forgetting to attach it stops being expressible rather than being something a test has to notice. `IntoKeepAlive` is the one thing the three newtypes share: a one-line impl each, so all three reach the shared constructor while staying distinct types, which is the property #1105 asked to keep. A route still cannot attach another surface's keepalive, because the value it holds came from the channel constructor it called.

The result is one attach site in the whole tree. `grep -rn "Sse::new(" src/` and `grep -rn "\.keep_alive(" src/` each return exactly one line, both in `streaming.rs`, with seven call sites routed through them. Deleting the `.keep_alive(..)` there is the one remaining way to break the invariant and it breaks it for every route at once, which is what a per-route test could not deliver.

`router_front.rs` is included even though it is outside `src/server/routes/`. Its two streams do not come from `sse_channel`, so they build the newtype directly, but there was no reason for them to keep assembling the attach by hand once a shared constructor existed. Removing the last hand-assembled `Sse::new` there is what makes the one-attach-site claim true tree-wide rather than true of one directory.

Deleting the tails made `sse::Sse` unused in all five route files, which clippy flagged under `-D warnings`. That is a useful confirmation rather than a nuisance: it is the compiler agreeing that no route constructs an SSE response on its own any more.

The `TODO` is removed with a note recording that the hazard went away structurally. Its other half, whether axum emits a comment frame while the stream is idle, tests axum's `KeepAlive` rather than mlxcel, and upstream already covers it.

Behaviour is unchanged. Every route emits the same frames on the same schedule.

Validated on GB10: `cargo test --profile test-fast --features cuda --lib server::streaming` is 25 passed and 0 failed, `--lib server::routes` is 116 passed and 0 failed, `cargo clippy --profile test-fast --features cuda --lib --tests -- -D warnings` is clean, and `cargo fmt --all -- --check` is clean.

Closes #1107
@inureyes inureyes added type:refactor Code restructuring without changing functionality priority:low Low priority status:review Under review labels Aug 13, 2026
Review found this PR fixed the same stale sentence in two of three files and missed the third. `streaming_responses.rs` and `streaming_anthropic.rs` stopped describing the hand assembly; `streaming.rs` carried four more copies of it, and the worst one was on `sse_channel` itself: "The `keepalive` value must be attached to the `Sse` response via `Sse::new(stream).keep_alive(keepalive.into_inner())` in the route handler". That is the first doc a contributor adding a sixth route reads, and it instructed them to reintroduce exactly the duplication this PR deletes. The module doc, the `SseKeepAlive` doc, and the `into_inner` doc had the same problem. All four now point at `sse_response`. The two sentences that are still accurate, about `router_front.rs` and `KeepAlive::default()`, are kept.

The three `into_inner()` accessors are removed and the `IntoKeepAlive` impls return `self.0` directly. Once every route went through `sse_response`, each accessor's only caller was its own trait impl, and two of them were `pub` inside `pub` modules: the last way for code outside these modules to obtain a bare `KeepAlive` and hand-assemble an `Sse`. Review raised this as optional. It is worth taking, because this PR's central claim is that hand assembly is unreachable, and a public accessor would have left that claim resting on convention at the one point it is meant to be structural. `grep -rn "fn into_inner" src/server/streaming*.rs` now returns nothing, and the "applied exactly once" rationale moved to the trait method where the consuming receiver now lives.

The `router_front.rs` comments regained the sentence this PR had dropped: `KeepAlive::default()` is also 15s under axum 0.7.9, so neither #1105 nor #1107 moves behaviour at those two sites. That sentence was the evidence for the no-behaviour-change claim there, and the replacement wording no longer said the two values are currently equal.

`impl IntoKeepAlive for SseKeepAlive` moved up next to `impl SseKeepAlive`, matching the placement the other two files use. The note replacing the removed integration-test item no longer contains the literal token it described, so a future `grep -rn TODO src/` sweep does not resurface it.

Invariant re-checked after the change: one `Sse::new` and one `.keep_alive` in non-comment code across `src/`, both in `streaming.rs`, and zero keepalive `into_inner` accessors.

Validated on GB10: `cargo test --profile test-fast --features cuda --lib server::streaming` is 25 passed and 0 failed, `--lib server::routes` is 116 passed and 0 failed, clippy with `-D warnings` is clean, and `cargo fmt --all -- --check` is clean.

Refs #1107
@inureyes inureyes added status:done Completed and removed status:review Under review labels Aug 13, 2026
@inureyes
inureyes merged commit 9c154ff into main Aug 13, 2026
8 checks passed
@inureyes
inureyes deleted the refactor/issue-1107-shared-sse-response branch August 13, 2026 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

priority:low Low priority status:done Completed type:refactor Code restructuring without changing functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor(server): make the SSE keepalive attach unforgettable via a shared constructor instead of testing for it

1 participant