drpcstream,drpcmanager: enable per-stream flow control (grant-on-consume) - #89
Draft
suj-krishnan wants to merge 2 commits into
Draft
drpcstream,drpcmanager: enable per-stream flow control (grant-on-consume)#89suj-krishnan wants to merge 2 commits into
suj-krishnan wants to merge 2 commits into
Conversation
suj-krishnan
force-pushed
the
sujatha/flow-control-recv-window-consume
branch
4 times, most recently
from
July 29, 2026 08:47
12ae0a8 to
5a99ec2
Compare
Base automatically changed from
sujatha/flow-control-recv-window-consume
to
main
July 29, 2026 08:49
suj-krishnan
force-pushed
the
sujatha/flow-control-consume-enable
branch
5 times, most recently
from
July 29, 2026 14:56
b7f5aee to
eaa5f1e
Compare
…option
Add drpcopts.FlowControl (Enabled, StreamWindow, GrantThreshold) and
install the per-stream windows at stream creation when it is set: the
send window seeded with StreamWindow credit and watching the send signal
for termination, and the receive window returning consumed-byte credit
coalesced at GrantThreshold.
Enablement is internal-only for merge safety: the option lives in
internal/drpcopts, not on the public drpcstream.Options, so a consumer
bumping the dependency cannot enable flow control before the deliberate
promotion to a public option.
The installation site normalizes the configuration rather than failing.
The frame size follows SplitData: zero uses the 64 KiB default, and a
negative SplitSize means unbounded frames, which carry no per-frame
bound. A valid config needs positive sizes and, for bounded frames,
GrantThreshold plus one frame fitting in StreamWindow -- so credit
withheld by coalescing cannot strand the sender below the next frame's
cost. Anything that cannot make progress resorts to
FlowControl.SetDefaults and logs the override, so a misconfiguration
degrades to a working stream instead of crashing a process that embeds
drpc.
End-to-end tests in drpcmanager prove the mechanism over a real
connection:
- Managers on both ends of a net.Pipe with flow control enabled move
four window-sized messages -- twice the stream window -- so the
transfer only completes if consume-driven grants flow back across the
connection.
- Context cancellation wakes a sender parked on credit via the
production path: manageStream sees the cancellation and terminates,
setting sigs.send, the send window's done signal.
- Tripwire: a default-configuration connection never puts a
KindWindowUpdate on the wire, so flow control cannot be enabled by
accident.
Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>
suj-krishnan
marked this pull request as draft
July 30, 2026 02:21
Per-frame credit gating deadlocks under grant-on-consume: credit is returned only when a complete message is consumed, so a multi-frame message that runs out of credit mid-way strands the receiver with an incomplete message it cannot consume, and no grant is ever issued. Even a message that fits the window can deadlock once coalesced credit is withheld below the grant threshold. Gate at the message boundary instead. The first frame of a message acquires credit (parking until it arrives, or the window closes on termination); once committed, later frames overdraft -- debit without parking, letting the balance go negative -- so a started message always completes on the wire. The receiver then consumes the whole message and returns the credit, and applyGrant repays the deficit before any credit accrues. Backpressure is preserved at message granularity: a new message parks on its first frame until the overdraft is repaid. The overdraft is bounded by MaxMessageSize, a new FlowControl field. A send larger than the bound fails fast, and the receiver's PacketAssembler rejects an assembling message that exceeds it before buffering it whole. Only the first frame need fit the window, so MaxMessageSize may be smaller than StreamWindow (a stricter, safer bound); an unset bound defaults to 64 MiB independently, so omitting it does not force the window and threshold to their defaults. Because a message may overdraft past the window up to MaxMessageSize, peak per-stream memory is roughly StreamWindow + MaxMessageSize rather than StreamWindow. An oversized message surfaces as a dedicated MessageSizeError class, not a protocol fault: ToRPCErr maps it to codes.ResourceExhausted, as gRPC does. An oversized receive fails only its own stream -- HandleFrame sends the peer an abortive terminal error so a credit-gated send there wakes instead of hanging, then returns without tearing down the multiplexed connection (the framing faults that do tear it down rely on the transport close to notify the peer). Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>
suj-krishnan
force-pushed
the
sujatha/flow-control-consume-enable
branch
from
July 30, 2026 04:10
eaa5f1e to
88c548d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Installs per-stream flow-control windows from a
FlowControloption and proves the mechanism end to end over a real connection.Enablement is internal-only (merge safety). The option lives in
internal/drpcopts(SetStreamFlowControl), not on the publicdrpcstream.Options— so a consumer bumping the drpc dependency mid-implementation cannot enable flow control: no public API reaches it.Promotion to a public option is a later, deliberate change.
Validation at the installation site :
StreamWindow,GrantThresholdpositive;SplitSizebounded (negative = unsplit frames, no bound holds; zero usesSplitData's 64 KiB default as the frame size);GrantThreshold+ frame size ≤StreamWindow