|
| 1 | +# Wire Protocol: Frame Handling |
| 2 | + |
| 3 | +Frames on the wire look like `[stream S, msg M, kind K, done D]`. A packet is |
| 4 | +one or more frames that share the same stream and message ID. Each frame carries |
| 5 | +a chunk of data; as frames arrive, their data is appended together to assemble |
| 6 | +the packet. The packet is complete when the final frame has `done=true`. |
| 7 | + |
| 8 | +Validation happens at two layers: **stream constraints** are per-stream and |
| 9 | +shared between mux and non-mux, while **connection constraints** are at the |
| 10 | +manager level and differ between mux and non-mux. |
| 11 | + |
| 12 | +--- |
| 13 | + |
| 14 | +## Stream Constraints |
| 15 | + |
| 16 | +These are enforced by the stream itself and apply the same way for both mux and |
| 17 | +non-mux managers. The stream keeps track of: |
| 18 | + |
| 19 | +- `nextMessageID`: the minimum accepted message ID, bumped to `messageID + 1` |
| 20 | + when a packet completes. |
| 21 | +- `assembling`: whether frames are currently being accumulated into a packet. |
| 22 | + Set when the first frame of a packet arrives, cleared when `done=true` |
| 23 | + completes it. |
| 24 | +- `pktKind`: the kind of the packet being assembled, set by the first frame. |
| 25 | + |
| 26 | +1. **Stream ID must match.** A misrouted frame is a protocol error. |
| 27 | +2. **Message ID must not go backwards.** If the incoming frame's message ID |
| 28 | + (`messageID`) is less than `nextMessageID`, that's a monotonicity violation. |
| 29 | +3. **New packet vs continuation.** A frame starts a new packet if: |
| 30 | + - `messageID > nextMessageID` (a higher message arrived; any in-progress |
| 31 | + packet data is discarded), or |
| 32 | + - `messageID == nextMessageID` and `assembling` is false (previous packet |
| 33 | + completed, or this is the first frame on the stream). |
| 34 | + |
| 35 | + A frame is a continuation if `messageID == nextMessageID` and `assembling` |
| 36 | + is true. |
| 37 | + |
| 38 | + Discarding in-progress data when a higher message ID arrives supports |
| 39 | + asynchronous interrupts, where the sender abandons a packet mid-write (e.g., |
| 40 | + due to context cancellation) and moves to a new message. Message IDs do not |
| 41 | + have to be contiguous; a jump from message 1 to message 5 is valid. |
| 42 | + |
| 43 | +4. **Kind must stay consistent within a packet.** Continuation frames (same |
| 44 | + message ID while assembling) must carry the same kind as the first frame of |
| 45 | + that packet. |
| 46 | +5. **Append frame data** to the in-progress packet buffer. |
| 47 | +6. **Done completes the packet.** `nextMessageID` is bumped to `messageID + 1` |
| 48 | + and the assembling flag is cleared. Any future frame with the same or lower |
| 49 | + message ID gets rejected by rule 2. |
| 50 | + |
| 51 | +## Non-mux Connection Constraints |
| 52 | + |
| 53 | +These live in the non-mux manager's reader loop. A future mux manager will have |
| 54 | +its own set of rules since interleaved streams are expected there. |
| 55 | + |
| 56 | +1. **Global frame monotonicity.** Frame IDs on the wire must be non-decreasing. |
| 57 | + The manager tracks the last frame ID it saw, and any frame with a lesser ID |
| 58 | + is a protocol error. |
| 59 | +2. **Old-stream frames are silently ignored.** When an incoming frame's stream |
| 60 | + ID is less than the locally created current stream's ID, it gets dropped |
| 61 | + without error. This comes up on the client side where the local stream ID can |
| 62 | + advance before the remote finishes responding. See the examples for how this |
| 63 | + differs from rule 1. |
| 64 | +3. **First frame of a stream must be Invoke.** The first frame for a new stream |
| 65 | + has to be `KindInvokeMetadata` or `KindInvoke`. Anything else is a protocol |
| 66 | + error. |
| 67 | + |
| 68 | +## Layering |
| 69 | + |
| 70 | +Both the manager and the stream enforce ordering, but at different scopes. |
| 71 | + |
| 72 | +The stream works at the per-stream level and is the authority on message-level |
| 73 | +correctness. |
| 74 | + |
| 75 | +The manager (non-mux) works at the connection level. It catches cross-stream |
| 76 | +ordering violations and malformed stream starts. For invoke frames, which the |
| 77 | +manager consumes directly and never forwards to a stream, the manager's check is |
| 78 | +the only line of defense. Similar to the stream, manager also bumps the ID when |
| 79 | +a packet completes. This protection could be left for the streams as it also |
| 80 | +does the same, but it's needed at manager level too to protect against the |
| 81 | +replay of an Invoke message if a stream is not created by the time next frame |
| 82 | +arrives. |
| 83 | + |
| 84 | +--- |
| 85 | + |
| 86 | +## Examples |
| 87 | + |
| 88 | +Frames are shown as `[stream S, msg M, kind K, done]` where done is `d=t` (true) |
| 89 | +or `d=f` (false). Only relevant fields are included. |
| 90 | + |
| 91 | +### Stream rule 2: message ID must not go backwards |
| 92 | + |
| 93 | +``` |
| 94 | +[s1, m3, Message, d=t] <- OK, packet complete, nextMessageID becomes 4 |
| 95 | +[s1, m2, Message, d=t] <- error: m2 < nextMessageID(4) |
| 96 | +``` |
| 97 | + |
| 98 | +### Stream rule 3: new packet vs continuation |
| 99 | + |
| 100 | +``` |
| 101 | +[s1, m1, Message, d=f] <- start accumulating |
| 102 | +[s1, m1, Message, d=f] <- continuation, append |
| 103 | +[s1, m2, Message, d=f] <- m1 data silently discarded, m2 starts fresh |
| 104 | +``` |
| 105 | + |
| 106 | +### Stream rule 4: kind consistency within a packet |
| 107 | + |
| 108 | +``` |
| 109 | +[s1, m1, Message, d=f] <- start packet, pktKind=Message |
| 110 | +[s1, m1, Error, d=t] <- error: kind changed mid-packet |
| 111 | +``` |
| 112 | + |
| 113 | +Kind is only checked for continuation frames. Different messages can have |
| 114 | +different kinds without issue: |
| 115 | + |
| 116 | +``` |
| 117 | +[s1, m1, Message, d=t] <- OK, packet complete |
| 118 | +[s1, m2, Close, d=t] <- OK, new message, no kind check against m1 |
| 119 | +``` |
| 120 | + |
| 121 | +### Stream rule 6: done prevents replay |
| 122 | + |
| 123 | +``` |
| 124 | +[s1, m1, Message, d=t] <- packet complete, nextMessageID becomes 2 |
| 125 | +[s1, m1, Message, d=t] <- error: m1 < nextMessageID(2) |
| 126 | +``` |
| 127 | + |
| 128 | +### Stream rule 6: multi-frame then next message |
| 129 | + |
| 130 | +``` |
| 131 | +[s1, m1, Message, d=f] <- assembling=true, accumulate |
| 132 | +[s1, m1, Message, d=f] <- continuation, append (kind matches) |
| 133 | +[s1, m1, Message, d=t] <- packet complete, assembling=false, nextMessageID=2 |
| 134 | +[s1, m2, Close, d=t] <- not assembling, new packet, no kind check, OK |
| 135 | +``` |
| 136 | + |
| 137 | +### Connection rule 1: global monotonicity |
| 138 | + |
| 139 | +``` |
| 140 | +[s1, m5, d=t] |
| 141 | +[s1, m4, d=t] <- error: {1,4} < lastFrameID {1,5} |
| 142 | +``` |
| 143 | + |
| 144 | +Cross-stream: |
| 145 | + |
| 146 | +``` |
| 147 | +[s1, m3, d=t] |
| 148 | +[s2, m1, d=t] <- OK, new stream |
| 149 | +[s1, m4, d=t] <- error: {1,4} < lastFrameID {2,1} |
| 150 | +``` |
| 151 | + |
| 152 | +Stream 2's frame implicitly starts a new stream. A frame for stream 1 showing up |
| 153 | +after that means the sender is writing to a stream it should consider closed. |
| 154 | + |
| 155 | +### Connection rule 2: old-stream frames silently ignored (client side) |
| 156 | + |
| 157 | +This handles a case that rule 1 does not catch. On the client side, the local |
| 158 | +stream ID advances independently of the wire: |
| 159 | + |
| 160 | +``` |
| 161 | +Client creates stream 1, sends invoke. |
| 162 | +Server responds: |
| 163 | + [s1, m1, Message, d=t] <- delivered to stream 1 |
| 164 | +
|
| 165 | +Client finishes stream 1, creates stream 2 (curr.ID=2). |
| 166 | +Server still responding: |
| 167 | + [s1, m2, Close, d=t] <- s1 < curr(2), silently ignored |
| 168 | +``` |
| 169 | + |
| 170 | +Global monotonicity passes here (m2 > m1, both on stream 1). But the client has |
| 171 | +already moved on. No error is raised because the server doesn't know yet. |
| 172 | + |
| 173 | +### Connection rule 3: first frame must be Invoke |
| 174 | + |
| 175 | +``` |
| 176 | +[s1, m1, InvokeMetadata, d=t] <- OK, forwarded |
| 177 | +[s1, m2, Invoke, d=t] <- OK, stream created |
| 178 | +[s1, m3, Message, d=t] <- OK, delivered to stream 1 |
| 179 | +``` |
| 180 | + |
| 181 | +``` |
| 182 | +[s1, m4, Message, d=t] <- OK, delivered to stream 1 |
| 183 | +[s2, m1, Cancel, d=t] <- error: first frame is not Invoke |
| 184 | +``` |
0 commit comments