Pool gossip connections per peer - #40
Open
plotnick wants to merge 2 commits into
Open
Conversation
rumors hands a completed stream's connection back through Dial::recycle. A qorb pool per peer holds it for the next stream, so the attested handshake is paid per connection, not per stream. qorb returns a dropped claim to the pool either way, so a dirty bit stands in for its missing detach and only clean returns are reused. Pools drop at prune. The health interval must be finite: qorb's timer arithmetic overflows on Duration::MAX and kills the slot tasks. max_slots covers the link contract's worst case. Two warm spares keep the rebalancer from culling recycled connections.
A recycled connection is not free until the peer's reader lets it go, and rumors readers can be lazy. The pool handed connections out as soon as the writer finished, so a stream could sit undelivered behind the previous stream's consumer. Mutual gossip sessions wait on each other's streams, and the extra dependency closed a wait cycle. The CI conformance suite deadlocked there. The router now writes one byte on a recovered connection when it starts reading for the next header. Recycle reads that byte in a spawned task and clears the dirty bit only once it arrives, so the pool never hands out a connection the peer still holds. max_slots grows to cover claims parked on the byte.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With oxidecomputers/rumors#13, it hands a completed stream's connection back through
Dial::recycle. A qorb pool per peer holds it for the next stream, so the attested handshake is paid per connection, not per stream. qorb returns a dropped claim to the pool either way, so a dirty bit stands in for its missing detach and only clean returns are reused. Pools drop at prune.The health interval must be finite, as qorb's timer arithmetic overflows on
Duration::MAXand kills the slot tasks (follow-up issue to be filed).max_slotscovers the link contract's worst case. Two warm spares keep the rebalancer from culling recycled connections.