Dynamic SynthDef instance size

[kaoskorobase]

Motivation

Today a SynthDef's instance memory is a static size_t instance_size field on Methcla_SynthDef (include/methcla/plugin.h:200). Plugins whose per-instance state size depends on OSC options either hard-code a worst-case upper bound or do a second methcla_world_alloc from construct.

Concrete driving case: a planned matrix plugin generalising patch_cable and a gain stage, exposing an N×M connection matrix with per-cell controllable gains. It needs N * M * sizeof(GainState) bytes of per-instance state, with N and M arriving as OSC options. Other shapes that fit the same pattern: delay lines sized by maxDelay, FFT processors sized by fftSize, ring buffers, any plugin that takes a sample-count or channel-count option.

Goal: let plugins declare an instance size that's a pure function of the populated options struct, so the engine folds that storage into the single rtMem allocation it already does in Synth::construct (src/Methcla/Audio/Synth.cpp:119-136).

Scope

• In scope: Size is a pure function of the options struct populated by configure(). Computed exactly once, before construct. Used to size the contiguous synth allocation.
• Out of scope: Construct-time-dynamic sizing (size only knowable inside construct). Lifetime-dynamic sizing (grow/shrink after construct).

Design decisions

#	Decision
1	Scope: configure-time dynamic only; size is a pure function of populated options
2	New optional size_t (*get_instance_size)(const Methcla_SynthOptions*) field on Methcla_SynthDef; wins over static instance_size when present
3	Engine calls get_instance_size exactly once between configure and allocOf; no cap, rt allocator is the safety net
4	Engine rounds the instance region up to alignof(AudioInputConnection); same alignment cleanup applied to AudioInputConnection[] → AudioOutputConnection[] → sample_t[controls] transitions
5	construct signature gains size_t instance_size as the fifth parameter (all in-tree plugins updated)
6	destroy signature unchanged
7	C++ wrapper: pass size to the C++ Synth constructor; do not auto-construct trailing members; plugin owns placement-new/destruct of trailing
8	TrailingArray<T> helper deferred until matrix plugin proves it's worth extracting
9	Ship API + alignment cleanup + plugin updates as one PR; matrix plugin in a follow-up issue

C ABI

Methcla_SynthDef gains:

//* Compute dynamic instance size from populated options.
//* If non-NULL, called once after `configure` and before `construct`.
//* Result supersedes the static `instance_size` field.
//* Must be a pure function of `options`.
size_t (*get_instance_size)(const Methcla_SynthOptions* options);

construct becomes:

void (*construct)(Methcla_World* world, const Methcla_SynthDef* def,
                  const Methcla_SynthOptions* options,
                  Methcla_Synth* synth,
                  size_t instance_size);

Source-only break. Plugins rebuilt in lockstep with the engine; no ABI versioning shim required.

C++ wrapper

include/methcla/plugin.hpp SynthDef<...>::construct shim passes instance_size to the C++ Synth constructor. Detection mechanism for an optional static size_t Synth::instanceSize(const Options::Type&) is to be decided at implementation time — either a new kSynthDefHasDynamicSize flag (consistent with kSynthDefHasActivate/kSynthDefHasCleanup) or SFINAE / if constexpr detection (cleaner, fits C++17 baseline). No auto-construction of trailing members; the plugin's Synth constructor does its own placement-new and the destructor does its own destruct.

Example shape for the matrix plugin:

class MatrixSynth {
    GainState* m_gains;
    size_t m_N, m_M;
public:
    static size_t instanceSize(const Options& o) {
        return sizeof(MatrixSynth) + o.N * o.M * sizeof(GainState);
    }
    MatrixSynth(World<MatrixSynth>, const Methcla_SynthDef*,
                const Options& o, size_t /*size*/)
      : m_gains(reinterpret_cast<GainState*>(this + 1))
      , m_N(o.N), m_M(o.M)
    {
        for (size_t i = 0; i < m_N * m_M; ++i) new (&m_gains[i]) GainState();
    }
    ~MatrixSynth() {
        for (size_t i = m_N * m_M; i-- > 0;) m_gains[i].~GainState();
    }
};

Engine changes

src/Methcla/Audio/Synth.cpp:119 becomes (sketch):

const size_t pluginInstanceSize =
    synthDef.descriptor()->get_instance_size
        ? synthDef.descriptor()->get_instance_size(synthOptions)
        : synthDef.instanceSize();
const size_t synthAllocSize = sizeof(Synth) + pluginInstanceSize;
const size_t audioInputOffset =
    alignUp(synthAllocSize, alignof(AudioInputConnection));
// ...same alignUp() applied at each subsequent region transition

Also pass pluginInstanceSize to synthDef.construct(...), which forwards to the plugin's construct callback.

In-tree plugin updates

Every plugin's construct callback signature changes (even those keeping a static size): sine.c, sampler.cpp, disksampler.cpp, patch_cable.cpp, node_control.cpp, soundfile_api_*.cpp (if any register synthdefs — most are pure soundfile APIs). C++ plugins using the wrapper get the size via their Synth constructor.

Rejected alternatives

• Overload instance_size == 0 to mean "call a function elsewhere." Magic, easy to miss.
• Change configure to return the size. Breaks every existing plugin's configure signature and conflates parsing with sizing.
• Stash the size on the Methcla_SynthDef* passed to construct. Mutates what should be a read-only descriptor.
• Cap the returned size at some engine-defined maximum. Introduces a magic number that will be wrong for someone's use case; rt allocator already fails cleanly on oversize requests.
• Have the C++ wrapper auto-construct one trailing array type via using TrailingType = …; static size_t trailingCount(…);. Default-construction is rarely what plugins want (matrix gains start at 0/1 depending on diagonal; delay lines need explicit zero-init for non-trivial types). The placement-new boilerplate it saves is ~3 lines per plugin. Can be added as a non-breaking extension later if the matrix plugin proves it worthwhile.
• std::vector<GainState> as a Synth member. The vector's elements live in heap memory allocated by its allocator (default: malloc), which defeats both the "single allocation" and "no malloc on rt thread" goals. Custom allocators that pull from the trailing region recover the layout but inherit broken vector semantics (grow/shrink/swap make no sense on borrowed memory).

Out of scope (follow-ups)

• The matrix plugin itself. Lands in a separate PR/issue on top of the API change so the API change is reviewable on its own merits and the ABI break is one atomic step.
• An optional TrailingArray<T> helper in the C++ wrapper. Extract from the matrix plugin if its constructor/destructor boilerplate proves painful.

CHANGELOG entry (draft)

### Added
- `get_instance_size` callback on `Methcla_SynthDef` for dynamic per-instance memory sized from options (#<this issue>).

### Changed
- `Methcla_SynthDef.construct` signature gains a `size_t instance_size` parameter; all plugins must be rebuilt (#<this issue>).
- Engine pads transitions between synth instance memory and trailing connection/buffer regions to satisfy alignment in all cases (#<this issue>).

ADR draft (to be moved to docs/adr/0002-dynamic-synthdef-instance-size.md at implementation time)

Dynamic SynthDef instance size

Option-dependent instance memory via a new callback

Methcla_SynthDef gains an optional function pointer size_t (*get_instance_size)(const Methcla_SynthOptions*). When present, the engine calls it exactly once after configure() populates the options struct and uses the result instead of the static instance_size field. The plugin's per-instance memory is then sized to fit, and remains part of the single contiguous rtMem allocation that already backs the Synth, its audio/control connection arrays, and its audio buffers. Plugins that don't need dynamic sizing leave the field NULL and keep using the static instance_size.

Alternatives considered and rejected: changing configure to return the size (signature break across all plugins for no benefit over an additive field); overloading instance_size == 0 (magic); mutating the descriptor from construct (breaks the read-only contract of Methcla_SynthDef*); capping the returned size (magic number; rt allocator already enforces a real limit via std::bad_alloc).

construct signature carries instance size

The construct callback gains size_t instance_size as a fifth parameter. Plugins that need to know their allocation size at construct time get it directly rather than having to recompute from options. Static-size plugins ignore it. This is a source-only break; plugins are rebuilt in lockstep with the engine, and Methcla does not maintain a versioned plugin ABI.

Alternative considered: have plugins recompute size from options via a shared helper. Rejected because every plugin would need that helper anyway, and one parameter is cheaper than per-plugin convention.

Engine pads inter-region transitions for alignment

Previously the offsets between the Synth instance memory, AudioInputConnection[], AudioOutputConnection[], and sample_t[] control buffers were computed with no padding, relying on plugin authors to choose instance_size values whose layout happened to satisfy AudioInputConnection's alignment. The asserts at Synth.cpp:55-58 caught the failure mode but did not prevent it. With dynamic sizing, the plugin's returned size can land at any byte boundary, making the implicit contract untenable. The engine now applies alignUp() at each region transition. Cost is at most a few padding bytes per Synth; benefit is that the plugin contract for get_instance_size is "return the bytes you need" with no engine-layout leakage.

C++ wrapper plumbs the size; does not auto-construct trailing storage

SynthDef<...>::construct in plugin.hpp passes instance_size to the C++ Synth constructor. The wrapper does not attempt to default-construct trailing arrays of a declared element type, even though that pattern (using TrailingType = T; static size_t trailingCount(...)) would cover simple cases. Reasons: default-construction is rarely what the plugin wants (non-trivial element initialisation is common); the placement-new boilerplate the sugar would save is ~3 lines per plugin; and the wrapper can grow such a helper later as a non-breaking extension if a concrete plugin shows it's worth the abstraction. Plugins instead own their own placement-new/destruct loops in the Synth constructor and destructor.

Vector-like in-place storage. std::vector<T> is not a viable member type for the trailing region because its allocator allocates separately (defeating the "single allocation" goal) and its growth/shrink semantics make no sense on borrowed memory. The right shape for vector-like access is a non-owning view ((T*, size_t)), either bare or wrapped in a future TrailingArray<T> helper.