feat(common): add serialized error payloads for data-bearing error#3751
Draft
jiengup wants to merge 1 commit into
Draft
feat(common): add serialized error payloads for data-bearing error#3751jiengup wants to merge 1 commit into
jiengup wants to merge 1 commit into
Conversation
variants Server now serializes the fields of data-bearing error variants (e.g. PartitionNotFound, StreamIdNotFound) into a binary payload and sends it alongside the error code. Client transports (QUIC, TCP, WebSocket, VSR) read this payload and reconstruct the full error, enabling contextual error messages on the client side. The Python SDK e2e test is updated to match the new dynamic IDs.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #3751 +/- ##
============================================
- Coverage 74.53% 70.99% -3.55%
Complexity 969 969
============================================
Files 1306 1306
Lines 150028 144974 -5054
Branches 125462 120482 -4980
============================================
- Hits 111829 102927 -8902
- Misses 34700 38401 +3701
- Partials 3499 3646 +147
🚀 New features to boost your workflow:
|
Contributor
Author
|
The discussion should be considered. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
variants
Server now serializes the fields of data-bearing error variants (e.g. PartitionNotFound, StreamIdNotFound) into a binary payload and sends it alongside the error code. Client transports (QUIC, TCP, WebSocket, VSR) read this payload and reconstruct the full error, enabling contextual error messages on the client side. The Python SDK e2e test is updated to match the new dynamic IDs.
Which issue does this PR address?
Closes #3735
Rationale
Due to
&[]payload insend_error_response<T>incore/server/src/sender/mod.rshere:iggy/core/server/src/sender/mod.rs
Lines 199 to 207 in d9635b6
Error payloads (
partition_id,topic_id,stream_id, etc.) were systematically dropped on the wire forTCP, QUIC, WebSocket, and VSR transports. Clients received fielded variants filled with
Defaultvalues,producing byte-identical errors for completely different requests.
The HTTP transport was unaffected (
error.to_string()serialized in JSON), proving the data existedand was merely not transported.
What changed?
Server (
send_error_response) now serializes the error’s fields into a structured binary payloadinstead of an empty body. Client (TCP, QUIC, WebSocket, VSR) reads the body and reconstructs the
original variant with real field values via
from_code_with_payload. The old empty-body path fallsback to
from_codefor backward compatibility with pre-fix servers.A declarative macro (
implement_error_payload!) generates both serialization and deserializationfrom a single list of variant field declarations. An
ErrorPayloadFieldtrait encapsulatesthe wire encoding for
usize,u32,u64,u8,u16,String, andIdentifier.90 data-bearing variants are covered; unit variants and unsupported custom types produce empty payloads.
Local Execution
AI Usage
Discussion
The current approach uses a declarative macro to cover 90 data-bearing variants.
A more elegant implementation should be considered:
Option 1: Proc macro / unified serialization module
A
#[derive(ErrorPayload)]proc macro would eliminate hand-listing variants—newfield-bearing variants get payload support automatically. The read/write logic could
be further generalized into a
WireFieldtrait shared withWireEncode/WireDecodein
iggy_binary_protocol, avoiding duplicate serialization infrastructure.Option 2: External crate
Replace the custom
ErrorPayloadFieldtrait and macro with a third-party binaryserialization crate.
Identifieralready derivesSerialize/Deserialize, sointegration cost is low.