refactor(server): attach the SSE keepalive through one constructor - #1136
Merged
Conversation
Five route handlers each ended with the byte-identical tail `Sse::new(stream).keep_alive(keepalive.into_inner()).into_response()`. No route varied the stream type, the keepalive, or the ordering, so this was five copies of one expression, and a sixth route that forgot the `.keep_alive(..)` would have compiled and passed the whole suite. A `TODO` in `streaming_tests.rs` asked for an axum integration harness to close that gap; a test per route is five tests guarding five copy-paste sites, and it still only covers the routes someone remembers to write a test for, which is the same hazard moved into the test module. `streaming::sse_response` is now the only way to turn one of these streams into a `Response`. It takes the keepalive by value, so forgetting to attach it stops being expressible rather than being something a test has to notice. `IntoKeepAlive` is the one thing the three newtypes share: a one-line impl each, so all three reach the shared constructor while staying distinct types, which is the property #1105 asked to keep. A route still cannot attach another surface's keepalive, because the value it holds came from the channel constructor it called. The result is one attach site in the whole tree. `grep -rn "Sse::new(" src/` and `grep -rn "\.keep_alive(" src/` each return exactly one line, both in `streaming.rs`, with seven call sites routed through them. Deleting the `.keep_alive(..)` there is the one remaining way to break the invariant and it breaks it for every route at once, which is what a per-route test could not deliver. `router_front.rs` is included even though it is outside `src/server/routes/`. Its two streams do not come from `sse_channel`, so they build the newtype directly, but there was no reason for them to keep assembling the attach by hand once a shared constructor existed. Removing the last hand-assembled `Sse::new` there is what makes the one-attach-site claim true tree-wide rather than true of one directory. Deleting the tails made `sse::Sse` unused in all five route files, which clippy flagged under `-D warnings`. That is a useful confirmation rather than a nuisance: it is the compiler agreeing that no route constructs an SSE response on its own any more. The `TODO` is removed with a note recording that the hazard went away structurally. Its other half, whether axum emits a comment frame while the stream is idle, tests axum's `KeepAlive` rather than mlxcel, and upstream already covers it. Behaviour is unchanged. Every route emits the same frames on the same schedule. Validated on GB10: `cargo test --profile test-fast --features cuda --lib server::streaming` is 25 passed and 0 failed, `--lib server::routes` is 116 passed and 0 failed, `cargo clippy --profile test-fast --features cuda --lib --tests -- -D warnings` is clean, and `cargo fmt --all -- --check` is clean. Closes #1107
10 tasks
Review found this PR fixed the same stale sentence in two of three files and missed the third. `streaming_responses.rs` and `streaming_anthropic.rs` stopped describing the hand assembly; `streaming.rs` carried four more copies of it, and the worst one was on `sse_channel` itself: "The `keepalive` value must be attached to the `Sse` response via `Sse::new(stream).keep_alive(keepalive.into_inner())` in the route handler". That is the first doc a contributor adding a sixth route reads, and it instructed them to reintroduce exactly the duplication this PR deletes. The module doc, the `SseKeepAlive` doc, and the `into_inner` doc had the same problem. All four now point at `sse_response`. The two sentences that are still accurate, about `router_front.rs` and `KeepAlive::default()`, are kept. The three `into_inner()` accessors are removed and the `IntoKeepAlive` impls return `self.0` directly. Once every route went through `sse_response`, each accessor's only caller was its own trait impl, and two of them were `pub` inside `pub` modules: the last way for code outside these modules to obtain a bare `KeepAlive` and hand-assemble an `Sse`. Review raised this as optional. It is worth taking, because this PR's central claim is that hand assembly is unreachable, and a public accessor would have left that claim resting on convention at the one point it is meant to be structural. `grep -rn "fn into_inner" src/server/streaming*.rs` now returns nothing, and the "applied exactly once" rationale moved to the trait method where the consuming receiver now lives. The `router_front.rs` comments regained the sentence this PR had dropped: `KeepAlive::default()` is also 15s under axum 0.7.9, so neither #1105 nor #1107 moves behaviour at those two sites. That sentence was the evidence for the no-behaviour-change claim there, and the replacement wording no longer said the two values are currently equal. `impl IntoKeepAlive for SseKeepAlive` moved up next to `impl SseKeepAlive`, matching the placement the other two files use. The note replacing the removed integration-test item no longer contains the literal token it described, so a future `grep -rn TODO src/` sweep does not resurface it. Invariant re-checked after the change: one `Sse::new` and one `.keep_alive` in non-comment code across `src/`, both in `streaming.rs`, and zero keepalive `into_inner` accessors. Validated on GB10: `cargo test --profile test-fast --features cuda --lib server::streaming` is 25 passed and 0 failed, `--lib server::routes` is 116 passed and 0 failed, clippy with `-D warnings` is clean, and `cargo fmt --all -- --check` is clean. Refs #1107
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Five route handlers each ended with the byte-identical tail
Sse::new(stream).keep_alive(keepalive.into_inner()).into_response(). A sixth route that forgot the.keep_alive(..)would have compiled and passed the whole suite. This replaces the five copies with one shared constructor, so forgetting the keepalive stops being expressible, and deletes theTODOthat asked for an axum integration harness instead.Why not the test the TODO asked for
The
TODOatstreaming_tests.rsnamed a real risk, but the property splits in two and neither half wants that test. Whether axum emits a comment frame while the stream is idle tests axum'sKeepAlive, which upstream already covers. Whether every route attaches the keepalive is the half that can actually regress, and an integration test catches it only for routes someone remembers to write a test for. That is five tests guarding five copy-paste sites: the same duplication, moved into the test module, and still blind to the sixth route.What changed
src/server/streaming.rs: newIntoKeepAlivetrait andsse_responseconstructor.sse_responsetakes the keepalive by value, so a route cannot call it without one.src/server/streaming_responses.rs,src/server/streaming_anthropic.rs: a one-lineIntoKeepAliveimpl each. The three newtypes stay distinct, which is the property refactor(server): consolidate the three duplicated SSE keepalive constants under one proxy-timeout assertion #1105 asked to keep: a route still cannot attach another surface's keepalive, because the value it holds came from the channel constructor it called. The trait is the only thing they share.sse_response(stream, keepalive).src/server/router_front.rs: its two sites go through the same constructor. They are outsidesrc/server/routes/and so outside the letter of the criterion, but they were the last hand-assembledSse::newin the tree, and including them is what makes the one-attach-site property hold tree-wide rather than per-directory.src/server/streaming_tests.rs: theTODOis replaced by a note recording that the hazard went away structurally.Acceptance criteria
Sse::new(..).keep_alive(..)remains insrc/server/routes/grep -rn "keep_alive" src/server/routes/returns nothingIntoKeepAliveimpl each; all five routes callsse_response.keep_alive(..)from the shared constructor is the only way to break the invariant, and it breaks it for every route at onceTODOatstreaming_tests.rsis removedDeleting the tails made
sse::Sseunused in all five route files and clippy flagged it under-D warnings. That is the compiler confirming no route builds an SSE response on its own any more; the imports were removed.Test plan
cargo test --profile test-fast --features cuda --lib server::streaming: 25 passed, 0 failedcargo test --profile test-fast --features cuda --lib server::routes: 116 passed, 0 failedcargo clippy --profile test-fast --features cuda --lib --tests -- -D warningscargo fmt --all -- --checkBehaviour is unchanged: every route emits the same frames on the same schedule, so there is no
CHANGELOG.mdentry. The acceptance criterion names--features metal,accelerate; this is a Linux CUDA box and cannot build that feature set, and no feature gate reaches this code. The criterion'sserver::selector was run as its two populated halves,server::streamingandserver::routes, because an unfilteredserverselector matches hundreds of tests and the full lib suite aborts under parallel execution on this CUDA host for unrelated reasons.Closes #1107
Review round
A review pass verified behaviour equivalence against the vendored axum 0.7.9 source:
sse_responseexpands to the byte-identical expression the five tails carried, with no boxing, buffering, or adapter inserted, and the error type narrowed rather than widened (all seven call sites feedInfallible). It also confirmed the three newtypes remain non-interchangeable, sincedefault_for_long_prefillis private on both the Responses and Anthropic types, so each is obtainable only from its own channel constructor.Two things were fixed as a result.
Stale docs, in the file this PR touched most. The hand-assembly sentence was updated in
streaming_responses.rsandstreaming_anthropic.rsbut four more copies remained instreaming.rs. The worst was onsse_channelitself, telling a contributor adding a sixth route to writeSse::new(stream).keep_alive(keepalive.into_inner())by hand, which is precisely the duplication this PR removes. All four now point atsse_response.The last public route to a raw
KeepAliveis closed. The threeinto_inner()accessors are gone and theIntoKeepAliveimpls returnself.0directly. Each accessor's only caller was its own trait impl, and two werepubinsidepubmodules, so they were the remaining way for outside code to obtain a bareKeepAliveand hand-assemble anSse. Raised as optional in review; taken because this PR's central claim is that hand assembly is unreachable, and a public accessor would have left it resting on convention at the one point it should be structural.Also folded in: the
router_front.rscomments regained the sentence stating thatKeepAlive::default()is likewise 15s under axum 0.7.9, which is the evidence for the no-behaviour-change claim at those two sites; theIntoKeepAlive for SseKeepAliveimpl moved next toimpl SseKeepAliveto match the other two files.Noted, not done.
axum::response::sse::Sseis a public third-party type, so nothing mechanically stops a future route from importing it directly. Aclippy.tomlwith adisallowed-typesentry pointing atsse_responsewould make this PR's claim compiler-checked rather than reviewer-checked. The repo has noclippy.toml, and adding a workspace-wide lint config is a larger decision than this issue asked for; it is recorded as follow-up in the technical report.