Add schema-derived binary codec - #7366
Open
tim-smart wants to merge 20 commits into
Open
Conversation
🦋 Changeset detectedLatest commit: f104ae4 The changes in this PR will be included in the next version bump. This PR includes changesets to release 30 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
Bundle Size AnalysisGenerated from PR build output; treat the content below as untrusted.
|
This was referenced Aug 20, 2026
IMax153
reviewed
Aug 20, 2026
IMax153
reviewed
Aug 20, 2026
Removes the per-field allocations that dominated both directions and gives the hot loops cheaper primitives. On the benchmark cases this is 2.0x to 3.4x faster than before, and ahead of Effect's Msgpack codec everywhere except the smallest payload. Encode - Length-prefixed values are written straight into the output buffer and the prefix is backfilled, replacing a scratch buffer and a copy per nested value. - The field, element and extra-key loops track their path on an ambient stack instead of allocating a closure and a try/catch per entry. - Cycle detection walks an ancestor stack rather than adding to and deleting from a WeakSet at every level. - Field id varints are encoded once at layout compile time. - One writer is pooled across top-level encodes so the buffer keeps its high-water mark. Decode - Nested values narrow the reader's extent and restore it, rather than allocating a child reader per field. - Varints are read with an unrolled loop over shifts; field ids are 32-bit hashes, so five-byte varints were the common case through a loop that used exponentiation per byte. This alone was 24% of decode time. - Struct fields are found by walking a cursor over the id-sorted field list, falling back to the map for reordered or unknown ids. - Duplicate-id and presence tracking use a bit mask instead of a Set per value. - Short ASCII runs are decoded from char codes, avoiding the fixed cost of TextDecoder and its subarray view. The acyclic-value guard now applies only to recursive schemas. It walked the whole value on every encode to stop a cyclic value driving the parser into unbounded recursion, which only a schema containing a Suspend can do. As a side effect, non-recursive schemas now report the real issue where they previously reported "Expected acyclic value" for any failure. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
`EMPTY_READER_VIEW` was built from `EMPTY_READER_BUFFER.buffer`. A member access in argument position is not effect-free to a bundler, so the `@__PURE__` annotation the build adds could not be honoured and the pair survived tree shaking in every consumer that imports `effect/unstable/encoding` without ever touching this module. Both placeholders now read one named `ArrayBuffer`, which keeps `EMPTY_READER_VIEW.buffer === EMPTY_READER_BUFFER.buffer` exactly as before. Add a tracked bundle fixture so the module's size is compared on every PR by the existing Bundle job. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tim-smart
force-pushed
the
agent/codex-engineer/58cacc24
branch
from
August 21, 2026 00:41
87d2fa4 to
68df691
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
SchemaBinaryfromeffect/unstable/encodingand cover framing, evolution, parser behavior, native values, declarations, recursion, parse options, and error surfacesVerification
nix develop -c pnpm lintnix develop -c pnpm checknix develop -c pnpm vitest run packages/effect/test/unstable/encodingnix develop -c pnpm --dir packages/effect exec node benchmark/schema/SchemaBinary.tsCloses EFF-772
Closes EFF-773
Closes EFF-779
Closes EFF-784