From 226499c467f3d0cb2879c9274a94188932b838ac Mon Sep 17 00:00:00 2001 From: Robert Kruszewski Date: Tue, 11 Aug 2026 18:37:34 +0100 Subject: [PATCH 1/2] Add docs about object evolution in editions Signed-off-by: Robert Kruszewski --- docs/specs/editions.md | 137 +++++++++++++++++++++++++++++------------ 1 file changed, 98 insertions(+), 39 deletions(-) diff --git a/docs/specs/editions.md b/docs/specs/editions.md index bded2c7e488..ea6ac5c56a6 100644 --- a/docs/specs/editions.md +++ b/docs/specs/editions.md @@ -1,17 +1,15 @@ # Editions Vortex defines an evergrowing set of serializable array encodings, once written this can be read back by any future -version of vortex. -**Editions** are used to keep track of these encodings and talk about groups of encodings. +version of vortex. **Editions** are used to keep track of these encodings and talk about groups of encodings. The first edition, `core2025.05.0`, contains the stable encodings that could be written by Vortex `0.36.0`. This is the release from which the Vortex file format is considered stable. Later `core` -editions add stable encodings released after that compatibility boundary. -Editions are additive so an edition that comes after a previous one contains all the encodings from the previous one -and more. -The writer can be configured with a set of different editions (for example, `core2026.07.0` and -`unstable2026.06.0` select stable encodings released through July 2026 and unstable encodings -released through June 2026). +editions add stable encodings released after that compatibility boundary. Editions are additive so an edition that comes +after a previous one contains all the encodings from the previous one and more. The writer can be configured with a set +of different editions (for example, `core2026.07.0` and +`unstable2026.06.0` select stable encodings released through July 2026 and unstable encodings released through June +2026). Editions can be used to constrain your minimum required vortex reader, since latest version over vortex across all editions is the earliest version of vortex required to read that file. @@ -46,53 +44,114 @@ this filter does not govern. A session that registers components outside `core`, such as the spatial extension types, enables its own edition family alongside `core`, and the writer may emit the union. -## Resolving an unknown-encoding error +## Resolving an unknown-object error -If a read failed with an unknown encoding ID and pointed you here, the reader met an array encoding -it does not support. Find the encoding ID in the [registry](#edition-registry) below: +If a read failed with an unknown ID (an encoding, layout, aggregate function, or extension dtype) and pointed you here, +the reader met an object it does not support: -1. **The ID is listed under an edition.** The file is newer than your Vortex build. Upgrade to - at least that edition's required Vortex release and the file will read. -2. **The ID is not listed anywhere.** The file was written outside the editions system, with a - custom, third-party, or experimental encoding. Ask the producer of the file how to read it, - or register the encoding with your session before reading. Tools that only inspect or - relocate data (rather than query it) can opt in to `allow_unknown`, which decodes - unrecognised encodings into inert placeholders. +1. **The ID is listed under an edition.** The file is newer than your Vortex build. Upgrade to at least that edition's + required Vortex release and the file will read. +2. **The ID is not listed anywhere.** The file was written outside the editions system, with a custom, third-party, or + experimental object. Ask the producer of the file how to read it, or register the object with your session before + reading. Tools that only inspect or relocate data (rather than query it) can opt in to `allow_unknown`, which decodes + unrecognised encodings into inert placeholders and disables pruning for zone maps whose aggregates it cannot resolve. ## Writing with an edition The default the writer targets a `core` edition lagging the latest vortex release by a few version giving delay before -writing the latest vortex encodings to disk. -Every file you write carries the read-forever guarantee. If a file would contain an encoding -outside the targeted edition, the write fails immediately; edition violations never surface as +writing the latest vortex encodings to disk. Every file you write carries the read-forever guarantee. If a file would +contain an encoding outside the targeted edition, the write fails immediately; edition violations never surface as someone else's read error later. -The enabled editions are stored on the writer's Vortex session. Registering an edition makes -its declaration available to the session; enabling it separately allows the writer to emit its -array encodings. Enabling another edition from the same family replaces the earlier selection. +The enabled editions are stored on the writer's Vortex session. Registering an edition makes its declaration available +to the session; enabling it separately allows the writer to emit its encodings. Enabling another edition from the same +family replaces the earlier selection. Two knobs exist when the default is not what you want: - **Pin an older edition** when files must stay readable by deployments running older Vortex. -- **Opt in to additional edition families.** Editions come in independently versioned, - additive families — `core` today, with families for more specialised encoding groups (for - example spatial encodings) possible later. A writer targets at most one edition per family - and may emit any encoding in their union; each member belongs to exactly one family. +- **Opt in to additional edition families.** Editions come in independently versioned, additive families — `core` today, + with families for more specialised encoding groups (for example spatial encodings) possible later. A writer targets at + most one edition per family and may emit any encoding in their union; each encoding belongs to exactly one family. -Lower-level sessions without an enabled-editions store opt out of editions entirely and can write -custom or experimental encodings. A raw `with_allow_encodings` writer policy is another explicit -opt-out. Either choice gives up the standardization guarantee — only readers that know those -encodings can read the files. +Lower-level sessions without an enabled-editions store opt out of editions entirely and can write custom or experimental +encodings. A raw `with_allow_encodings` writer policy is another explicit opt-out. Either choice gives up the +standardization guarantee — only readers that know those encodings can read the files. ## How editions change -A published edition is frozen — its member list never grows or shrinks. New members are -staged in a **draft** edition and become guaranteed only when that draft is frozen as the next -edition; each member's registry entry records its component kind and the edition it joined in. -Declaring the first member of a kind arms that kind's write-time filter, so a kind gains -enforcement at the edition that first declares one. -In the future an encoding may be *deprecated*, meaning writers stop emitting it — but readers -keep decoding it indefinitely, so deprecation never invalidates existing files. +A published edition is frozen — its member list never grows or shrinks, for any object kind. New objects are staged in a +**draft** edition and become guaranteed only when that draft is frozen as the next edition; each object's registry entry +records the edition it joined in. In the future an object may be *deprecated*, meaning writers stop emitting it — but +readers keep decoding it indefinitely, so deprecation never invalidates existing files. + +## How serialized objects evolve + +Editions name *which* objects a file may contain. This section defines how the serialized form of those objects — array +metadata, layout metadata, aggregate options, extension dtype metadata — is allowed to change over time. + +An object may evolve in place, keeping its id and its registry entry, as long as every change is both **backward and +forward compatible**: a new reader must still understand data written by an old writer, and an old reader must still +understand data written by a new writer. Adding an optional field that older readers ignore and newer readers default +when absent is the canonical example. + +A change that breaks either direction is not an evolution of the object — it is a **new object**. Removing or +repurposing a field, redefining what existing bytes mean, or requiring information older writers never emitted all fall +in this bucket, and each demands a new id, a new registry entry, and its own edition membership. The old object stays in +the registry and stays readable forever; writers stop emitting it only if it is deprecated. This is the rule that makes +the reading and writing halves below tractable: readers only ever accumulate compatible forms of an object, and writers +only ever have to choose between distinct objects the target edition does or does not guarantee. + +### Reading: deserialize to the latest version + +Vortex maintains exactly one in-memory representation per object: the latest one. Deserialization always targets it, +from **every serialized form that has ever existed**: + +- A serialized form, once shipped in a release, stays readable **forever**. Deserializers accumulate historical forms; + they never drop one. +- Old forms deserialize *into the latest in-memory version*, not into parallel legacy code paths. For example, zone maps + written before aggregate descriptors existed (and whole + `vortex.stats` layouts) deserialize into the same zone-map machinery that modern + `vortex.zoned` layouts use; the reader upgrades on the way in. +- Consequently there is no version negotiation at read time: a reader either knows the object (and then reads all of its + historical forms) or reports an unknown-object error covered [above](#resolving-an-unknown-object-error). + +A compatible change therefore lands as *additional* accepted input: the new form joins the deserializer alongside every +earlier one, and its new fields are optional, so data written before they existed still deserializes. A change that +cannot be expressed that way is a new object, not a new form of this one. + +### Writing: translate down, or convert to canonical and recompress + +Writers emit the **newest serialized form permitted by the target editions**. An object (or object version) newer than +the target edition never leaks into the file; the writer resolves the conflict in one of two ways: + +1. **Translate.** If the newer in-memory version has a defined translation to a serialized form the target edition + guarantees, the writer emits the older form. This is preferred when the translation is lossless, e.g. re-emitting a + newer layout's zone statistics using an older stats schema. +2. **Convert to canonical and recompress.** Otherwise the writer decompresses the data to a canonical representation and + recompresses it using only the configured compressors, filtered to the target editions. This is how arrays are + handled today: the write pipeline normalizes each chunk (recursively executing any encoding outside the permitted set + down to canonical) and the configured compressor — default, BtrBlocks-style, or custom — is restricted to choose + encodings from the enabled editions. + +Both paths run inside the ordinary write pipeline, so the configured compressors are always what produces the final +bytes. If neither path can express the data inside the target editions, the write fails immediately rather than emitting +a file the target reader could not load. + +### What this means per kind + +- **Arrays.** Enforced at write time today. The writer's array context only permits encodings from the enabled editions; + anything else is normalized to canonical and recompressed by the edition-filtered compressor. +- **Layouts.** The layout strategy decides the layout tree at write time. Layout membership declares which layout + encodings (in their current serialized form) a target reader understands; strategies must degrade to older structures + (e.g. plain chunked data instead of newer auxiliary layouts) when targeting editions that predate them. +- **Aggregations.** Zone maps and file statistics serialize aggregate function ids plus their options. Writers targeting + an edition without a given aggregate must omit it or translate to an older stats schema; readers meeting an unknown + aggregate disable pruning for that zone map (under `allow_unknown`) rather than failing the scan, since dropping + statistics is always sound. +- **Extension dtypes.** Every serialized `DType` — including every file's schema — embeds the ids and metadata of the + extension dtypes it uses, so an extension dtype in durable data needs the same guarantee as an encoding. Readers + resolve ids against the session's dtype registry. ## Edition registry From 4d7d286182d1058666836b2297f66921318d4466 Mon Sep 17 00:00:00 2001 From: Robert Kruszewski Date: Wed, 12 Aug 2026 15:29:52 +0100 Subject: [PATCH 2/2] changes Signed-off-by: Robert Kruszewski --- docs/specs/editions.md | 255 ++++++++++++++++++++++++----------------- 1 file changed, 147 insertions(+), 108 deletions(-) diff --git a/docs/specs/editions.md b/docs/specs/editions.md index ea6ac5c56a6..e700c8c7397 100644 --- a/docs/specs/editions.md +++ b/docs/specs/editions.md @@ -1,18 +1,22 @@ # Editions -Vortex defines an evergrowing set of serializable array encodings, once written this can be read back by any future -version of vortex. **Editions** are used to keep track of these encodings and talk about groups of encodings. - -The first edition, `core2025.05.0`, contains the stable encodings that could be written by Vortex -`0.36.0`. This is the release from which the Vortex file format is considered stable. Later `core` -editions add stable encodings released after that compatibility boundary. Editions are additive so an edition that comes -after a previous one contains all the encodings from the previous one and more. The writer can be configured with a set -of different editions (for example, `core2026.07.0` and -`unstable2026.06.0` select stable encodings released through July 2026 and unstable encodings released through June -2026). - -Editions can be used to constrain your minimum required vortex reader, since latest version over vortex across all -editions is the earliest version of vortex required to read that file. +Vortex defines an ever-growing set of serialized formats for arrays and other durable objects. +Once a format is published, every future version of Vortex can read it. **Editions** group these +formats and record when each joined that compatibility guarantee. + +An edition member is a **serialized format**: the ID written into the file, its metadata schema, +and the meaning of its buffers, children, options, or other payload. Array and layout encodings, +aggregate functions, and extension dtypes are different kinds of serialized format covered by the +same rule. The read-forever guarantee attaches to those bytes and their meaning, not to the +in-memory implementation that produced them. + +The first edition, `core2025.05.0`, contains the stable serialized array formats that Vortex +`0.36.0` could write. This is the release from which the Vortex file format is considered stable. +Later `core` editions add stable formats released after that boundary. Editions are additive: each +edition contains every member of the preceding edition in its family, plus new members. For +example, selecting `core2026.07.0` and `unstable2026.06.0` enables stable formats released through +July 2026 and unstable formats released through June 2026. The most recent required Vortex release +among the selected editions is the earliest Vortex version that is guaranteed to read the file. ## What an edition contains @@ -46,114 +50,149 @@ family alongside `core`, and the writer may emit the union. ## Resolving an unknown-object error -If a read failed with an unknown ID (an encoding, layout, aggregate function, or extension dtype) and pointed you here, -the reader met an object it does not support: +If a read failed with an unknown ID for an array encoding, layout encoding, aggregate function, or +extension dtype and pointed you here, the reader encountered a serialized format it does not +support. Find the ID in the [registry](#edition-registry) below: -1. **The ID is listed under an edition.** The file is newer than your Vortex build. Upgrade to at least that edition's - required Vortex release and the file will read. -2. **The ID is not listed anywhere.** The file was written outside the editions system, with a custom, third-party, or - experimental object. Ask the producer of the file how to read it, or register the object with your session before - reading. Tools that only inspect or relocate data (rather than query it) can opt in to `allow_unknown`, which decodes - unrecognised encodings into inert placeholders and disables pruning for zone maps whose aggregates it cannot resolve. +1. **The ID is listed under an edition.** The file is newer than your Vortex build. Upgrade to at + least that edition's required Vortex release. +2. **The ID is not listed anywhere.** The file was written outside the editions system with a + custom, third-party, or experimental format. Ask the producer how to read it, or register its + implementation with your session. Tools that only inspect or relocate data can opt in to + `allow_unknown`, which decodes unrecognised encodings into inert placeholders and disables + pruning for zone maps whose aggregate functions it cannot resolve. ## Writing with an edition -The default the writer targets a `core` edition lagging the latest vortex release by a few version giving delay before -writing the latest vortex encodings to disk. Every file you write carries the read-forever guarantee. If a file would -contain an encoding outside the targeted edition, the write fails immediately; edition violations never surface as -someone else's read error later. +By default, the writer targets a `core` edition that lags the latest Vortex release by a few +versions, leaving an adoption period before it writes new formats. Every file written under this +policy carries the read-forever guarantee. If serialization would emit a format ID outside the +target editions, the write fails immediately; edition violations never surface as someone else's +read error later. -The enabled editions are stored on the writer's Vortex session. Registering an edition makes its declaration available -to the session; enabling it separately allows the writer to emit its encodings. Enabling another edition from the same -family replaces the earlier selection. +The enabled editions are stored on the writer's Vortex session. Registering an edition makes its +declaration available to the session; enabling it separately allows the writer to emit its +members. Enabling another edition from the same family replaces the earlier selection. Two knobs exist when the default is not what you want: - **Pin an older edition** when files must stay readable by deployments running older Vortex. -- **Opt in to additional edition families.** Editions come in independently versioned, additive families — `core` today, - with families for more specialised encoding groups (for example spatial encodings) possible later. A writer targets at - most one edition per family and may emit any encoding in their union; each encoding belongs to exactly one family. +- **Opt in to additional edition families.** Families are independently versioned and additive: + `core` exists today, and specialised families, for example for spatial formats, may be added + later. A writer targets at most one edition per family and may emit any format in their union; + each format belongs to exactly one family. -Lower-level sessions without an enabled-editions store opt out of editions entirely and can write custom or experimental -encodings. A raw `with_allow_encodings` writer policy is another explicit opt-out. Either choice gives up the -standardization guarantee — only readers that know those encodings can read the files. +Lower-level sessions without an enabled-editions store opt out of editions entirely and can write +custom or experimental formats. A raw `with_allow_encodings` array-writer policy is another +explicit opt-out. Either choice gives up the standardization guarantee: only readers that know +those formats can read the files. ## How editions change -A published edition is frozen — its member list never grows or shrinks, for any object kind. New objects are staged in a -**draft** edition and become guaranteed only when that draft is frozen as the next edition; each object's registry entry -records the edition it joined in. In the future an object may be *deprecated*, meaning writers stop emitting it — but -readers keep decoding it indefinitely, so deprecation never invalidates existing files. - -## How serialized objects evolve - -Editions name *which* objects a file may contain. This section defines how the serialized form of those objects — array -metadata, layout metadata, aggregate options, extension dtype metadata — is allowed to change over time. - -An object may evolve in place, keeping its id and its registry entry, as long as every change is both **backward and -forward compatible**: a new reader must still understand data written by an old writer, and an old reader must still -understand data written by a new writer. Adding an optional field that older readers ignore and newer readers default -when absent is the canonical example. - -A change that breaks either direction is not an evolution of the object — it is a **new object**. Removing or -repurposing a field, redefining what existing bytes mean, or requiring information older writers never emitted all fall -in this bucket, and each demands a new id, a new registry entry, and its own edition membership. The old object stays in -the registry and stays readable forever; writers stop emitting it only if it is deprecated. This is the rule that makes -the reading and writing halves below tractable: readers only ever accumulate compatible forms of an object, and writers -only ever have to choose between distinct objects the target edition does or does not guarantee. - -### Reading: deserialize to the latest version - -Vortex maintains exactly one in-memory representation per object: the latest one. Deserialization always targets it, -from **every serialized form that has ever existed**: - -- A serialized form, once shipped in a release, stays readable **forever**. Deserializers accumulate historical forms; - they never drop one. -- Old forms deserialize *into the latest in-memory version*, not into parallel legacy code paths. For example, zone maps - written before aggregate descriptors existed (and whole - `vortex.stats` layouts) deserialize into the same zone-map machinery that modern - `vortex.zoned` layouts use; the reader upgrades on the way in. -- Consequently there is no version negotiation at read time: a reader either knows the object (and then reads all of its - historical forms) or reports an unknown-object error covered [above](#resolving-an-unknown-object-error). - -A compatible change therefore lands as *additional* accepted input: the new form joins the deserializer alongside every -earlier one, and its new fields are optional, so data written before they existed still deserializes. A change that -cannot be expressed that way is a new object, not a new form of this one. - -### Writing: translate down, or convert to canonical and recompress - -Writers emit the **newest serialized form permitted by the target editions**. An object (or object version) newer than -the target edition never leaks into the file; the writer resolves the conflict in one of two ways: - -1. **Translate.** If the newer in-memory version has a defined translation to a serialized form the target edition - guarantees, the writer emits the older form. This is preferred when the translation is lossless, e.g. re-emitting a - newer layout's zone statistics using an older stats schema. -2. **Convert to canonical and recompress.** Otherwise the writer decompresses the data to a canonical representation and - recompresses it using only the configured compressors, filtered to the target editions. This is how arrays are - handled today: the write pipeline normalizes each chunk (recursively executing any encoding outside the permitted set - down to canonical) and the configured compressor — default, BtrBlocks-style, or custom — is restricted to choose - encodings from the enabled editions. - -Both paths run inside the ordinary write pipeline, so the configured compressors are always what produces the final -bytes. If neither path can express the data inside the target editions, the write fails immediately rather than emitting -a file the target reader could not load. - -### What this means per kind - -- **Arrays.** Enforced at write time today. The writer's array context only permits encodings from the enabled editions; - anything else is normalized to canonical and recompressed by the edition-filtered compressor. -- **Layouts.** The layout strategy decides the layout tree at write time. Layout membership declares which layout - encodings (in their current serialized form) a target reader understands; strategies must degrade to older structures - (e.g. plain chunked data instead of newer auxiliary layouts) when targeting editions that predate them. -- **Aggregations.** Zone maps and file statistics serialize aggregate function ids plus their options. Writers targeting - an edition without a given aggregate must omit it or translate to an older stats schema; readers meeting an unknown - aggregate disable pruning for that zone map (under `allow_unknown`) rather than failing the scan, since dropping - statistics is always sound. -- **Extension dtypes.** Every serialized `DType` — including every file's schema — embeds the ids and metadata of the - extension dtypes it uses, so an extension dtype in durable data needs the same guarantee as an encoding. Readers - resolve ids against the session's dtype registry. +A published edition is frozen: neither its member list nor the meaning of any member ID may change. +New formats are staged in a **draft** edition and become guaranteed only when that draft is frozen +as the next edition; each registry entry records the edition it joined in. A format may later be +*deprecated*, meaning writers stop emitting it, but readers keep decoding it indefinitely. +Deprecation therefore never invalidates existing files. + +## How serialized formats evolve + +In-memory representations are unversioned implementation details. They may gain capabilities or be +replaced without changing an edition. Serialization plugin registries map between the two worlds: +on read, the plugin selected by a serialized ID constructs whichever in-memory representation the +reader prefers, and on write, the implementation chooses a serialized format that can represent +the value and is permitted by the target editions. Most in-memory representations have one format +and use the same ID in memory and on disk, but that is not required: several serialized IDs may +map to one in-memory representation. The edition check applies to the ID written into the file, so +it constrains exactly what the target reader will encounter. + +### Compatible evolution keeps the ID + +A serialized format may evolve under its existing ID only when the change is both **backward and +forward compatible**: an old reader must still interpret data from a new writer correctly, and a +new reader must still interpret data from an old writer correctly. Adding an optional field is +compatible only when old readers can safely ignore it and new readers have the correct default when +it is absent. + +Compatible evolution may add accepted input, but it cannot change the meaning of bytes that +existing readers already accept. Removing or repurposing a field, redefining existing bytes, or +requiring information that old writers never emitted is incompatible. + +### Incompatible evolution creates a new format + +An incompatible revision is a **new serialized format** with a new ID, registry entry, and edition +membership. The old format remains in the registry and readable forever. The in-memory +representation does not need to fork: it can read and write both formats, choosing between them +according to the value and the target editions. + +Name successive incompatible revisions as a version chain on the same base name: +`vortex.foo`, `vortex.foo_v2`, `vortex.foo_v3`. Do not use descriptively named successor variants. +This gives each format at most one successor, so its serialized history is a list rather than a +tree of competing revisions. + +#### Example: multi-part decimals + +`vortex.decimal_byte_parts` froze into `core2025.05.0` representing each decimal value as a single +signed integer child. Its metadata includes a `lower_part_count` field, but readers of the frozen +format require it to be zero. Suppose its in-memory representation gains support for wide decimals +as a signed most-significant part plus unsigned 64-bit lower parts: + +- A single-part array still serializes as `vortex.decimal_byte_parts` with + `lower_part_count = 0`, indistinguishable from files written before the change. +- An array with lower parts serializes as `vortex.decimal_byte_parts_v2`, staged in a draft + edition. +- A new reader deserializes both IDs into the same in-memory representation. A reader that predates + the second format reports an unknown-ID error for `vortex.decimal_byte_parts_v2`, rather than + entering a decoder that was never taught about lower parts. + +### Reading: deserialize into the current representation + +Each serialized format that ships in a release remains readable forever. Its deserializer may +upgrade the data into the current in-memory representation instead of preserving a parallel legacy +representation. For example, a serialized `vortex.alp` array with interior patches is deserialized +as a `Patched` array wrapping a patch-free ALP array, and zone maps written before aggregate +descriptors existed, including whole `vortex.stats` layouts, deserialize into the same zone-map +machinery used by modern `vortex.zoned` layouts. There is no version negotiation at read time: the +reader resolves the serialized ID and deserializes it, or reports the relevant +[unknown-ID error](#resolving-an-unknown-object-error). + +### Writing: select a permitted format + +Writers choose a serialized format that can represent the current in-memory value and is permitted +by the target editions. This need not be the newest format: a value that the older frozen format +represents exactly may continue to use its older ID. If the preferred format is newer than the +target edition, the writer resolves the conflict in one of two ways: + +1. **Translate.** If the value has a lossless translation to a permitted serialized format, emit + that format. For example, a newer layout may re-emit its zone statistics using an older stats + schema. +2. **Convert to canonical and recompress.** Otherwise, decompress the data to a canonical + representation and recompress it with the configured compressors, filtered to the target + editions. This is how arrays are handled today: the write pipeline normalizes each chunk, + recursively executing an encoding outside the permitted set down to canonical, and then lets + the edition-filtered compressor choose the final encoding. + +Both paths run inside the ordinary write pipeline, so the configured compressors produce the final +bytes. If neither path can express the data within the target editions, the write fails rather than +emitting a file that the target reader cannot load. + +### What this means for each kind + +- **Arrays.** Enforced at write time today: the writer's array context only permits serialized + array encodings from the enabled editions. +- **Layouts.** The layout strategy decides the layout tree at write time. Layout membership + declares which serialized layout formats a target reader understands. Strategies must degrade + to older structures, for example plain chunked data instead of newer auxiliary layouts, when + targeting editions that predate them. +- **Aggregate functions.** Zone maps and file statistics serialize aggregate function IDs and + their options. Writers targeting an edition without a function must omit it or translate to an + older stats schema; readers handle an unknown aggregate under `allow_unknown` as described + [above](#resolving-an-unknown-id-error), which is sound because dropping statistics only + weakens pruning. +- **Extension dtypes.** Every serialized `DType`, including every file's schema, embeds the IDs and + metadata of its extension dtypes. An extension dtype in durable data therefore needs the same + guarantee as an array encoding. Readers resolve its ID against the session's dtype registry. ## Edition registry -Coming soon.. It will list each edition's members with their component kind, the edition they -joined in, and the Vortex release required to read them. +Coming soon.