Skip to content

state: automatic serialization of a processor's state struct#180

Open
jcelerier wants to merge 7 commits into
mainfrom
feature/auto-state-solo
Open

state: automatic serialization of a processor's state struct#180
jcelerier wants to merge 7 commits into
mainfrom
feature/auto-state-solo

Conversation

@jcelerier

Copy link
Copy Markdown
Member

Serialize a processor's persistent state by reflecting over a state struct, the way the value bindings already reflect over inputs and outputs -- no hand-written byte code.

struct MyProcessor
{
  struct { … } inputs;
  struct { … } outputs;
  struct { std::vector<uint8_t> program; int mode{}; std::string label; } state;
};

The presence of the state member is enough; avnd::save_object_state / load_object_state do the rest. This is the core only -- concepts, the serializer and its tests. Wiring it into the CLAP and VST3 state containers is a follow-up.

What is covered

Member types are classified with the same concepts the value bindings use (avnd/concepts/generic.hpp), so anything a processor can carry round-trips: scalars and enums, strings, vector and other list containers, set and map, optional, variant, pair and tuple, std::array, aggregates, and any nesting of those. Verified on GCC 16 and MSVC across 21 type combinations.

Refused at compile time: anything that only borrows its data -- a pointer, reference or string_view -- since it would be written out as an address that means nothing to the process reading it back. A processor needing one provides save/load on its state instead, which takes precedence over reflection.

Format and schema changes

Members are encoded positionally, each as a self-describing record [tag][scalar kind][size][payload], behind a header carrying a format marker, a layout hash and the field count. The shape of the data absorbs the common edits without any version bookkeeping:

  • appending a member: older data just has fewer records, the new member keeps its default;
  • dropping a trailing member: its record is skipped;
  • changing a member's type: the record no longer matches, so it is skipped rather than reinterpreted -- the scalar kind distinguishes an int from a float of the same width.

An edit the format cannot follow on its own -- reordering or inserting in the middle -- changes the layout hash and is refused unless the processor declares a state_version and a migrate_state hook, which is also how a change of meaning (same type, new unit) is handled. Data written by a newer version than the binary understands is refused rather than misread.

The one edit nothing here can see is a swap of two members of the identical type; that is exactly what the version bump exists for, and it is documented as such. In practice this suggests the same discipline as inlets: add state members at the end.

Override and versioning

// full control of the bytes -- takes precedence over reflection
struct { std::vector<uint8_t> save() const; bool load(const uint8_t*, std::size_t); } state;

// or reflect, with a migration for the changes reflection cannot follow
halp_meta(state_version, 3)
static void migrate_state(auto& state, int from) { … }

Testing

test_state_serial and test_state_object: the round-trip, the full type surface, the schema-change matrix (append / drop / retype / reorder), truncated and corrupt input, and the override and migration paths. All pass on MSVC; the type-coverage matrix was also run on GCC 16. FindBoost was not touched here (it lives in the transport/state PR).

🤖 Generated with Claude Code

jcelerier and others added 7 commits July 22, 2026 01:23
A processor declares its persistent data as a state member, like its port
groups, and the bindings serialize it: members are written as records keyed
by field name, so adding, removing or reordering them leaves existing data
readable. A record whose member changed type is skipped rather than
reinterpreted, and unknown records are stepped over by size.

An object that provides save/load on its state keeps full control of its
bytes. Semantic changes that names cannot express are handled by declaring
state_version and a migrate_state hook, called once with the version the
data was written at; data from a newer version is refused rather than
misread.

The state struct must be a named type: field names cannot be reflected
from an unnamed one.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Positional records let the state struct be declared inline like the other
port groups, which reflecting field names cannot do. Appending a member,
dropping a trailing one and changing a member's type stay readable on their
own; every other structural edit changes a compile-time hash of the layout,
which the reader compares against its own before trusting positions.

Each record carries the member's scalar kind as well as its tag, so an int
that became a float of the same width is skipped rather than reinterpreted.
A load across a changed layout is reported to the caller, which refuses it
unless the processor declared a migration.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Appending a member changes the layout hash, so keying refusal off the hash
rejected exactly the edit the format is meant to absorb -- and the one
processors are asked to make. Refusal now keys off a record describing
something other than the member at its position, which is what actually
means the positions after it cannot be trusted. Appending and dropping
trailing members are reported as a layout change and load normally.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ake apart

A pointer, reference or view is trivially copyable, so it would have been
written into a session file as an address and read back meaning nothing.
Types holding one are now rejected at compile time, transitively; owning
containers that look like views -- std::array in particular -- are not.

std::pair and std::tuple are trivially copyable on some standard libraries
and not others, so accepting them on that basis compiled a state struct on
one toolchain and not the next; they are refused everywhere. std::array of
non-trivial elements is refused as well: the aggregate reflection cannot
decompose a tuple-like type portably. Both are cases for a processor's own
save/load.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Classify state members with the same concepts the value bindings use, so
everything a processor can carry round-trips: optionals, variants, maps,
sets, pairs, tuples, fixed arrays and any nesting, alongside the scalars,
strings, vectors and aggregates already handled. Only borrowed memory --
pointers, references, string_view -- is refused, transitively.

Fixes found writing the coverage tests: a trivially-copyable aggregate was
hashed by size alone, so {int,float} and {float,int} shared a layout hash;
the hash now recurses into aggregate fields regardless. string_view is
string_ish but borrows its characters, so the owned-string path now
requires resize().

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The per-field position fold in the layout hash was never load-bearing --
the field shapes already distinguish {int, float} from {float, int} once
trivially-copyable aggregates recurse into the hash.
Calling boost::pfr::get<I> on a struct goes through std::forward_like,
which clang cannot use in this constexpr context with libstdc++. The
framework's own field iteration takes the structured-binding path on that
toolchain and avoids it; route through it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant