feat(sync-service): add WebSocket wake multiplexing#4701
Open
icehaunter wants to merge 1 commit into
Open
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4701 +/- ##
==========================================
- Coverage 60.20% 60.04% -0.16%
==========================================
Files 412 397 -15
Lines 44396 43766 -630
Branches 12586 12585 -1
==========================================
- Hits 26727 26279 -448
+ Misses 17587 17406 -181
+ Partials 82 81 -1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #4700.
Motivation
Each ordinary live shape request keeps an HTTP request process and its closure state alive for the full long-poll interval. At roughly 50–80 KB per reader, 200,000 mostly-silent readers can retain around 10 GB while only a small fraction receive changes.
This adds an internal transport for multiplexing only the waiting portion of those requests. Shape rows and changelog bodies remain on the existing HTTP endpoint, preserving the public client contract, caching, streaming, and proxy authorization model.
What changed
GET /v1/shape/multiplexusing WebSocket subprotocolelectric.shape-multiplex.v1.watchandunwatchframes keyed by an opaque request id, existing handle, concrete live offset, and prior raw Electric cursor.0_infquiet-shape offset, while rejecting snapshot and other special offsets.no_changeenvelope built through the existing HTTP response-header machinery, including the advancing cursor, cache policy, ETag, handle, offset, and up-to-date body.availability_guardfor embedded deployments such as Electric Cloud.Protocol and operational decisions
Electric owns every logical deadline. There is no timeout HTTP probe: after a terminal no-change response, the unchanged client naturally sends its next request, whose new watch performs the same subscribe-then-head check.
The endpoint intentionally does not send shape data, expose a new public client API, or move authentication into Electric. It uses the same service-secret protection as the shape endpoint and is intended for trusted proxies.