feat(cs): ASAPv1 wire payload for Count Sketch - #79
Draft
GordonYuanyc wants to merge 1 commit into
Draft
Conversation
Self-describing ASAPv1 serialization for Count Sketch, kind_id `0x04 0x00`, following the Count-Min pattern from #65. `serialize_to_bytes` / `deserialize_from_bytes` now emit/read the ASAPv1 envelope (replacing the old plain-rmp form) and live in a per-sketch `wire` submodule. Metadata mirrors Count-Min's — the hash-spec group derived from the hasher's `HashProfile`, then `rows` / `cols` / `mode` — with **one key removed: no `counter_type`** (Q-CS). Count Sketch counters must be signed and negatable (`CountSketchCounter` requires `Neg` + `From<i32>`), so `i64` is the only wire-eligible type and the kind_id already implies it; a key with a single legal value describes nothing. Payload is the same 1-element positional array `[counts]`, packed row-major, signed cells. The wire methods are implemented only for the wire-eligible config (`Vector2D<i64>` storage, fast/regular mode, a profiled hasher), so the old generic-over-all-storage rmp helpers are gone — exotic counters (i32/i128) and non-`Vector2D` storage must be converted first, matching what CMS did. `MessagePackCodec` is narrowed to match. Fail-closed decode: kind_id check, metadata equality check (hash spec + mode), non-zero dimensions (crafted `cols == 0` would panic in `Vector2D::from_fn` via `ilog2`), and a counts-length check that runs before the allocation so crafted dimensions cannot drive a huge reserve. Golden byte-vectors `cs_i64_regular_2x4` / `cs_i64_fast_2x4` hold the same matrix and differ only by `mode`, so the pair also pins that mode reaches the bytes. Their values sweep the msgpack integer widths in both directions (negative fixint / int8 / int16 / int32 alongside the positive widths), since Count Sketch cells are signed. Docs: new §3.4, registry row flipped to implemented, Q-CS recorded. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Self-describing ASAPv1 wire serialization for Count Sketch → kind_id
0x04 0x00, following the Count-Min pattern from #65 and the KLL pattern from #75.serialize_to_bytes/deserialize_from_bytesnow emit/read the ASAPv1 envelope (replacing the old plain-rmp form), authored in a per-sketchwiresubmodule (src/sketches/countsketch/wire.rs).Design decision (docs/asapv1_wire_format.md §3.4 + Q-CS)
Count Sketch mirrors Count-Min's metadata and
[counts]payload with exactly one key removed: nocounter_type.CountSketchCounterrequiresNeg + From<i32>, so counters must be signed and negatable — which leavesi64as the only wire-eligible type (i32widens,i128is not a wire type,f64doesn't qualify at all). The kind_id already implies it, and a metadata key with a single legal value describes nothing. The variance from Q-CMS is deliberate: the two sketches are not forced symmetric where their type domains differ.Everything else is identical to §3.2 — hash-spec group derived from the hasher's
HashProfile, thenrows/cols/mode; payload is the same 1-element positional array[counts], packed row-major. Cells are signed (the sketch adds±weight), so a decoder must not assume monotonicity.API narrowing
The wire methods are implemented only for the wire-eligible config (
Vector2D<i64>storage, fast/regular mode, a profiled hasher), so the old generic-over-all-storage rmp helpers are gone andMessagePackCodecis narrowed to match. Exotic counters (i32/i128) and non-Vector2Dstorage must be converted first — same cut CMS made in #65. Decode is a hard cut with no legacy-rmp fallback, consistent with every other sketch.Fail-closed decode
kind_id check; metadata equality check (hash spec + mode); non-zero dimensions (a crafted
cols == 0would otherwise panic inVector2D::from_fnviailog2); and a counts-length check that runs before the allocation, so crafted dimensions cannot drive a huge reserve.Tests
Nine new tests in the
wiremodule: round-trip; negative-counter round-trip; custom-hasher profile self-describing + cross-rejection; mode in metadata + cross-mode rejection; foreign kind_id (CMS bytes) rejection; zero dimension; dimension/length mismatch; unknown metadata key; and one asserting that addingcounter_typeback is rejected.Golden byte-vectors
cs_i64_regular_2x4/cs_i64_fast_2x4hold the same matrix and differ only bymode, so the pair also pins that mode reaches the bytes. Their values sweep the msgpack integer widths in both directions — negative fixint / int8 / int16 / int32 alongside positive fixint / uint8 / uint32 — since Count Sketch cells are signed:Full suite green (491 + 9 golden);
clippy --all-targetsclean;fmt --checkclean;cargo docclean under CI's-D warnings.Not in this PR
0x04 0x00and copy the twocs_*.hexfixtures. Deliberately out of scope here (same standing item as KLL's0x06 0x00/0x06 0x01from feat(kll): ASAPv1 wire payload for KLL and KLLDynamic #75).0x0a 0x00Count-Sketch-with-heap (CSHeap) is a separate kind_id and untouched.🤖 Generated with Claude Code