diff --git a/changelog.d/sui-t36.7.md b/changelog.d/sui-t36.7.md new file mode 100644 index 0000000..5f60979 --- /dev/null +++ b/changelog.d/sui-t36.7.md @@ -0,0 +1,9 @@ +### Added + +- `StatifierUI.DatamodelExplorer.build_authoring/3` and `build_live/2` build + a read-only datamodel tree - document `` declarations, spec + 5.10.1 system variables, predicator provider functions in scope, and + either a fixture scenario or a live session's datamodel with entries + marked `changed?` per macrostep - and + `StatifierUI.DatamodelExplorer.Markdown.render/2` renders it as Markdown + for `Kino.Markdown`. diff --git a/docs/plans/260822-sui-t36.7-datamodel-explorer-pane.md b/docs/plans/260822-sui-t36.7-datamodel-explorer-pane.md new file mode 100644 index 0000000..5c0687d --- /dev/null +++ b/docs/plans/260822-sui-t36.7-datamodel-explorer-pane.md @@ -0,0 +1,1010 @@ +--- +date: 2026-08-22 +issue: sui-t36.7 +title: Datamodel explorer pane +status: draft +tags: [plan, kino, datamodel, fixtures, wire-format] +--- + +# Datamodel explorer pane implementation plan + +## Overview + +The model behind the inspector's datamodel tree: one component with two data +sources. **Authoring mode** builds the merged fixture scope from a compiled +`%Statifier.Machine{}` and an ADR-0003 fixture bundle - document ``s, +the SCXML 5.10.1 system variables, the predicator provider functions in scope, +and one named fixture scenario - each entry carrying an inferred +`StatifierUI.Shape` type label. **Live mode** folds a subscribed session's +trace messages - the `session.datamodel` name snapshot plus every +`effect.datamodel_change` - into the same entry shape, marking the entries +that changed in the latest macrostep. + +Read-only in this milestone: no editing affordance in either mode. Four +modules land, all pure, with **no `Kino` reference in code**; the widget, the +scenario switcher, and the composition with the other panes belong to +`sui-t36.8`. This plan follows the shape `sui-t36.4` (`StatifierUI.Diagram`), +`sui-t36.5` (`StatifierUI.EventLog`) and `sui-t36.6` +(`StatifierUI.EventInjection`) already set: a pure `build/*` fold, a pure +`render/2`, `{:ok, struct} | {:error, reason}` out, one diagnostic per bad +entry rather than a failed build, and `Kino` named only in moduledoc prose. + +Beads issue: `sui-t36.7` (parent `sui-t36`, the Livebook inspector; depends on +`sui-t36.2` fixtures core, `sui-t36.3` subscriber/normalizer, `sui-h92` +datamodel_change serialization, and `sui-bpb` the 9.0 refresh - all closed; +blocks `sui-t36.8`). + +Source research: `docs/research/260822-sui-t36.7-datamodel-explorer-pane.md`. +That document ends with ten open questions, each carrying a recorded default. +**This plan adopts every one of those defaults rather than re-opening them**, +and each phase below names the ones it rests on. + +## Current State Analysis + +Everything this bead consumes exists and is closed; nothing is blocked on an +upstream gap. + +### What is already built here + +- `lib/statifier_ui/shape.ex` - `infer/1` (total over `term()`, always + recursing fully) and `label/2` (`:max_keys` 4, `:max_depth` 3, truncation in + rendering only). `:undefined` is already a first-class member of the + vocabulary (`shape.ex:28-41`), which is what makes a name-only snapshot + representable. This is the "inferred type label" machinery the bead asks + for, already shipped by `sui-t36.2`. +- `lib/statifier_ui/value.ex` - the ADR-0005 codec. `decode/1` + (`{:ok, term} | {:error, term}`) is the wire-to-Elixir step; JSON `null` + decodes to `nil`, never `:undefined` (`value.ex:12-14`). +- `lib/statifier_ui/fixtures.ex` - `%Fixtures{scenarios:, events:, + diagnostics:}`, `scenario/2` (`{:ok, datamodel} | :error`), + `scenario_names/1` (sorted), and the `diagnostic()` type + (`fixtures.ex:44-62`). Scenario keys are validated as strings at every + depth. +- `lib/statifier_ui/event_log.ex` - the fold precedent: `build/1` over a + message list, order-independent by construction, refusing + `{:error, {:mixed_sessions, ids}}` rather than merging two timelines + (`event_log.ex:87-93`), and `truncated?` from `min(seq) > 0` + (`event_log.ex:95-101`). Per-wire-type dispatch is one `put_message/2` + clause per type in the top module (`event_log.ex:146-195`). +- `lib/statifier_ui/event_log/markdown.ex` - the pure-renderer precedent: + `render(t(), keyword()) :: String.t()` with `Keyword.get/3` defaults, and a + moduledoc that states outright that it "names `Kino` only in this moduledoc + and calls nothing under `Kino.*`". +- `lib/statifier_ui/event_injection/palette.ex:97-126` - the + diagnostic-per-bad-entry precedent: one bad sample takes down one entry, not + the pane. +- `lib/statifier_ui/trace/manifest.ex:193-201` - the in-repo idiom for + enumerating a Machine's `` elements: + `machine.data_elements |> Tuple.to_list() |> Enum.map(...)`. +- `test/support/trace/session_case.ex` - `compile!/1`, `start_early!/3`, + `start_late!/3`, `wait_for_seq/3`: the live-session harness. + +There is no datamodel-explorer code of any kind, and **no state-diffing +precedent anywhere in `lib/`**. `trace.exit_set` / `trace.entry_set` are +deltas the *engine* computed and the pane passes through verbatim. Computing a +delta between two steps is this bead's own new machinery. + +### What the engine provides, verified read-only + +- `Statifier.Machine.data/2` (`machine.ex:206-207`) returns the `%Machine.Data{}` + at a `d_index`; `data_elements` is a tuple field on the public struct and + `Tuple.to_list/1` over it is already this repo's enumeration idiom. A + `%Machine.Data{}` carries `d_index`, `id`, `value`, `location`, and + `value_location` - identity and spans, never an evaluated value. +- `Statifier.Evaluator.SystemVariables.initial/2` + (`system_variables.ex:75-84`) seeds exactly four variables: `_sessionid`, + `_name`, `_event`, `_ioprocessors`. `_x` is deliberately seeded nowhere. + `event/1` (`:100-110`) is the schema of `_event`'s 5.10.1 sub-fields. +- `Statifier.Evaluator.Functions.base_context()` (`functions.ex:63-78`) is a + compile-time `%Predicator.Context{}` whose `functions` field is the fully + resolved `%{name => {arity, {module, atom}}}` map: the four predicator + builtin providers plus `In/1`, already merged in predicator's + shadowing order (`context.ex:216-237`). +- `Predicator.FunctionProvider.functions/0` carries **name and arity only** - + no return type, no parameter names, no doc string. + +### What the wire format provides, verified + +- `session.datamodel` (`docs/wire-format.md:737-762`): one payload field, + `datamodel`, keyed by variable name; emitted exactly once per session, + unconditionally, always at `seq: 1`; envelope `macrostep`/`microstep`/`round` + all `nil` by deliberate choice. **Every `` element reads + `{"$undefined": true}`** because the snapshot precedes the binding fold. It + is the name list, not a source of values. +- `effect.datamodel_change` (`docs/wire-format.md:629-681`, shipped by + `sui-h92`): `location_path` (always; heterogeneous array, strings are object + keys, integers 0-based array indexes, already resolved), `location_source` + (always, display only), `new_value` (**absent** when the write stored + `:undefined`, `null` when it stored null), `prior_value` (absent when + nothing stood at the path), and the optional `d_index` / `c_index` / `owner` + attribution. Emitted once per **successful** write; a failed write emits + nothing. Envelope carries `macrostep` and `microstep` but `round` is `nil` + (`normalizer.ex:271`). +- `session.start` carries a `data` identity table in `d_index` order with + `id`, `location`, `value_location` (`docs/wire-format.md:311-348`) - + deliberately identity only, no declared value. +- `Message.payload` is typed `%{String.t() => json()}` + (`trace/message.ex:21-42`): **already `$`-tagged wire-encoded**. This is the + sharpest hazard in the bead. `Shape.infer/1` applied straight to a payload + value classifies `%{"$undefined" => true}` as a one-key map, not as + `:undefined`. `Value.decode/1` must run first. + +### What is already decided and must be honored + +- **ADR-0003** - fixtures as the example-data contract. The three-tier model is + contract, not a design choice; the two-mode explorer exists *because* only + tier 3 switches source between modes + (`docs/adr/0003-fixtures-as-the-example-data-contract.md:82-88`). Types are + inferred shapes from example values, never a declared schema (`:56-62`). + Richer provider metadata is named there as upstream `px-`/`st-` work, not to + be synthesized here (`:110-116`). +- **ADR-0005** - the value codec and the envelope. Every wire value crosses + into this pane through `StatifierUI.Value`, and there is no second spelling. +- **ADR-0006** - datasets and expressions are accepted but explicitly **not + `sui-t36` scope** and not implemented. Authoring mode reads `scenarios` + only (and `events` not at all - that is `sui-t36.6`'s). +- **ADR-0011** - an engine-ordered sequence is never re-sorted by a consumer. + The `datamodel_change` fold applies writes in the producer's stamped order, + never re-sorted by path, by `d_index`, or by name. +- **ADR-0004** - `kino` and `phoenix_live_view` stay optional. Nothing here + references either. +- **ADR-0002 / CLAUDE.md** - the engine and predicator are read-only from here. + A gap found upstream is an `st-` or `px-` bead, never a patch applied from + this repo. +- **The bead's read-only clause wins over `docs/architecture.md:214-217`.** + That paragraph says authoring-mode example values are "editable in place"; + it describes the eventual LiveComponent, and the bead scopes this milestone + to read-only. Research Open Question 10, default adopted. + +### Key Discoveries + +- `Message.payload` values are wire-encoded, `Fixtures` values are not + (`sidecar.ex:171-179` decodes at load). The two modes therefore differ in + **one more place than "tier 3 source"**: live mode needs a decode pass that + authoring mode must not have. Research Open Question 5; the call is to + decode at live-mode ingestion so both modes hold decoded Elixir values + internally and share exactly one `Shape.infer/1` pass. +- A tier-1 entry has **no runtime value in authoring mode** and the pane must + not evaluate a declared expression (predicator is non-evaluative, ADR-0004 + upstream, adopted by ADR-0002). Research Open Question 3; the call is + `:undefined` unless the selected scenario supplies a value of the same name, + plus the declared source text as a separate display-only field. +- `Statifier.Evaluator.Functions.base_context().functions` is the only source + that reflects predicator's shadowing/merge order, and therefore the only + honest answer to "what is in scope for this chart". Walking + `FunctionProvider.builtin_providers/0` and calling `functions/0` on each + reproduces ADR-0003's literal wording but can report a shadowed function as + live. Research Open Question 2; the call is the resolved context. +- `round` is `nil` on `effect.datamodel_change`, so change marking is + available at **macrostep** granularity and no finer. The bead says "per + macrostep", so this is a match rather than a gap - but it is a bound worth + writing into the moduledoc. Research Open Question 6. +- An **absent** `new_value` is a real state change to `:undefined`, not a + no-op: `put_defined/3` (`normalizer.ex:558-565`) is deliberately distinct + from `put_value/3`. A fold that treats absence as "unchanged" will silently + miss every unbinding write. Research Open Question 7. + +## Desired End State + +Four modules under `StatifierUI.DatamodelExplorer`: + +- `Entry` - the tree node: name, tier, decoded value, inferred shape, label, + `changed?`, and the display-only attribution fields. +- `Scope` - the mode-independent tiers: `build(machine, opts)` returning the + `` entries (tier 1), the four system variables plus `_event`'s + 5.10.1 sub-field names (tier 2a), and the resolved provider functions as + `name/arity` nodes (tier 2b). +- `StatifierUI.DatamodelExplorer` - the pane model, with the two constructors: + `build_authoring(machine, fixtures, opts)` and `build_live(messages, opts)`. +- `Markdown` - a pure `render(t(), keyword()) :: String.t()`. + +Verified by: `mix quality` green; a shuffle-invariance test proving +`build_live/2` over five shuffles of the same message list returns identical +structs; a live-session test that runs a real chart with an `` and +observes the assigned value and its `changed?` mark on the folded pane; and +`grep -rn "Kino" lib/` still matching only moduledoc prose. + +## What We're NOT Doing + +- **No `Kino`.** No tree widget, no frame, no control, no subscription. + `sui-t36.8` owns every line of that, and these modules are the data it + drives. +- **No editing.** No `` from the pane, no write-back, no editable + value affordance in either mode. Live datamodel editing waits for a + recordable-channel design (statifier ADR-0029) and is explicitly out of + scope; `docs/architecture.md:214-217`'s "editable in place" describes the + eventual LiveComponent, and the bead's read-only clause governs this + milestone. No edit to `docs/architecture.md` either - the sentence is not + wrong, it is about a later component. +- **No expression evaluation.** A `` shows as + `:undefined` with its declared *source text* alongside; the pane never + computes `42`. Predicator is non-evaluative by ADR-0004 upstream. +- **No ADR-0006 consumption.** No `datasets`, no `expressions`, no new sidecar + key. `sui-bob` owns that; ADR-0006:115-119 puts it outside `sui-t36`. +- **No engine or predicator change.** No new callback, no metadata table, no + patch to `deps/`. The provider-metadata ceiling (name and arity only) is + recorded in Open Questions, not built around. +- **No correlation between an injected event and a datamodel write.** That is + `sui-t36.8`'s question, and `sui-t36.6`'s plan already records it. +- **No Subscriber change and no late-attach catch-up.** `build_live/2` is a + pure fold over whatever message list it is handed, exactly as + `EventLog.build/1` is; closing the `messages/1`-then-`add_listener/2` gap is + `sui-t36.8`'s problem. Research Open Question 9, default adopted. +- **No per-round change marking.** `effect.datamodel_change` carries no + `round`. Macrostep granularity is what the bead asks for and what the wire + supports. +- **No `.quality.exs`, `coveralls.json`, or `.doctor.exs` edit, and no version + bump.** +- **No golden-file snapshots** for the renderer. The house convention is to + split the rendered string on `"\n"`, trim, and assert membership + (`markdown_test.exs:135-198`). + +## Implementation Approach + +Four phases, split on module boundaries. Each ships its own tests, so no phase +leaves a structure nothing exercises, and each leaves the gate green on its +own. + +1. **Entry and Scope** - the mode-independent tiers, built from a compiled + Machine and the engine's own constants. Depends on nothing but `Shape` and + two read-only engine modules. Fully unit-testable with a compiled heredoc + chart and no session, no fixtures, no messages. +2. **Authoring mode** - tier 3 from a `Fixtures` scenario, merged onto the + scope, plus the top-level pane struct and `build_authoring/3`. +3. **Live mode** - the fold: `session.start`'s data table for tier + attribution, `session.datamodel` for the name set, `effect.datamodel_change` + for values and change marking, with `Value.decode/1` at ingestion. +4. **Renderer, live-session test, changelog** - the Markdown view, the + end-to-end proof against a real session, and the fragment. + +The dependency chain is strictly linear: Phase 2 depends on Phase 1 (it merges +onto `Scope`'s output), **Phase 3 depends on Phases 1 and 2** - it adds +`build_live/2` to the module Phase 2 creates and fills the struct Phase 2 +defines - and Phase 4 depends on 2 and 3. Nothing here can be reordered or +run in parallel; the phase numbers are the execution order. + +--- + +## Phase 1: Entry and Scope - the mode-independent tiers + +### Overview + +The three tiers that do not switch source between modes: document `` +declarations, the SCXML system variables, and the provider functions in scope. +All three come from a compiled `%Statifier.Machine{}` and two engine +constants, so this phase needs no session, no fixtures, and no wire messages. + +Rests on research defaults **1** (authoring mode takes a `%Statifier.Machine{}`, +not a manifest), **2** (enumerate functions from the resolved context), and +**3** (a valueless tier-1 entry is `:undefined` with its declared source text +carried separately). + +### Changes Required: + +#### 1. The entry struct + +**File**: `lib/statifier_ui/datamodel_explorer/entry.ex` +**Changes**: New module `StatifierUI.DatamodelExplorer.Entry` - struct and +types only, zero functions, the way `event_log/round.ex` and +`event_injection/entry.ex` are. + +``` +@type tier :: :data | :system | :function | :scenario | :runtime + +@type t :: %__MODULE__{ + name: String.t(), + tier: tier(), + value: term(), + shape: StatifierUI.Shape.t(), + label: String.t(), + changed?: boolean(), + children: [t()], + d_index: non_neg_integer() | nil, + arity: non_neg_integer() | [non_neg_integer()] | nil, + declared_source: String.t() | nil, + location_source: String.t() | nil + } +``` + +Field meanings, stated in the `@typedoc`s because they are the contract +`sui-t36.8` renders against: + +- `tier` maps onto ADR-0003's tiers: `:data` and `:runtime` are tier 1 and its + live-mode sibling (a location written at runtime that the snapshot did not + name), `:system` is tier 2a, `:function` is tier 2b, `:scenario` is tier 3. +- `value` is **always a decoded Elixir value**, never a `$`-tagged wire term. + This is the invariant that lets one `Shape.infer/1` pass serve both modes. +- `changed?` is live-mode only and defaults to `false`; authoring mode never + sets it. +- `arity` is set on `:function` entries only, and is predicator's + `non_neg_integer() | [non_neg_integer()]` multi-arity form verbatim. +- `declared_source` is the tier-1 declared expression text, display only, + present only when the pane was given the chart source and + `value_location != location`. +- `location_source` is `effect.datamodel_change`'s raw author string, live + mode only, display only. + +#### 2. The scope builder + +**File**: `lib/statifier_ui/datamodel_explorer/scope.ex` +**Changes**: New module `StatifierUI.DatamodelExplorer.Scope`. + +``` +@type opt :: {:source, String.t() | nil} | {:session_id, String.t()} +@spec build(Statifier.Machine.t(), [opt()]) :: {:ok, t()} | {:error, term()} +``` + +`t()` is `%Scope{data: [Entry.t()], system: [Entry.t()], +functions: [Entry.t()], diagnostics: [StatifierUI.Fixtures.diagnostic()]}`. + +**Tier 1 - `` declarations.** Enumerate with +`machine.data_elements |> Tuple.to_list()`, the idiom +`trace/manifest.ex:193-201` already uses, in `d_index` order (never re-sorted +by id - ADR-0011's rule applied to an engine-ordered sequence). Each element +yields an entry with `name: element.id`, `tier: :data`, `d_index`, +`value: :undefined`, `shape: :undefined`, and `declared_source` sliced from +the `:source` option when one was given **and** `element.value_location` +differs from `element.location` - the guard +`docs/wire-format.md:335-348` states, because when the two are equal there is +no value span and the slice would be the whole element. Slicing goes through +`Statifier.Parser.Location.slice/2`, the engine's own primitive for exactly +this (`deps/statifier/lib/statifier/parser/location.ex:64-76`), rather than +open-coding `binary_part/3` over the offsets. With no `:source`, +`declared_source` is `nil` and nothing else changes. + +**Tier 2a - system variables.** Call +`Statifier.Evaluator.SystemVariables.initial(machine, session_id)` and turn +each pair into an entry with `tier: :system`, inferring the shape from the +seeded value. The session id comes from the `:session_id` option, defaulting +to the documented placeholder `"(authoring)"`; the point of going through +`initial/2` rather than re-listing the four names here is that the engine +stays the single source of the list, so a fifth seeded variable upstream +appears in the tree without a change in this repo. + +`_event` additionally gets `children`: one `:system` entry per key of +`Statifier.Evaluator.SystemVariables.event(Statifier.Event.external("(none)"))`, +each with `value: :undefined`. That is spec 5.10.1's six fields plus `data`, +taken from the engine's own schema function so the key list cannot drift. +`_ioprocessors`'s nested value is left to `Shape.infer/1` rather than +expanded into children - it is a value, not a schema. + +The moduledoc records that **`_x` is deliberately absent** +(`deps/statifier/lib/statifier/evaluator.ex:359-361`) and that `In(stateId)` +is a tier-2b function, not a datamodel key, because both are things a reader +will look for and not find. + +**Tier 2b - provider functions.** Read +`Statifier.Evaluator.Functions.base_context().functions`, a +`%{name => {arity, {module, atom}}}` map, and emit one entry per key sorted by +name, with `tier: :function`, `arity` from the tuple, `value: :undefined`, +`shape: :unknown`, and `label` rendered as `"name/arity"` (or +`"name/2|3"` for the multi-arity form). The moduledoc states why the resolved +context rather than a walk over `functions/0`: it is the only source that +reflects predicator's builtins-then-providers-then-`:functions` shadowing +order (`context.ex:216-237`), and ADR-0003's `functions/0` phrasing is +satisfied transitively, since that callback is what the resolution consumed. + +**Diagnostics, not failures.** A tier-1 element whose `location`/ +`value_location` offsets fall outside the given `:source` string yields an +entry with `declared_source: nil` plus one `Fixtures.diagnostic()` with +`kind: :unsliceable_declared_source`, following +`event_injection/palette.ex:97-126`. One bad span takes down one label, not +the pane. Anything that is not a `%Statifier.Machine{}` returns +`{:error, {:invalid_machine, other}}`. + +#### 3. Tests + +**File**: `test/statifier_ui/datamodel_explorer/scope_test.exs` +**Changes**: New test module. Compile heredoc charts with +`StatifierUI.Test.Support.Trace.SessionCase.compile!/1` (triple-quoted, 4-space +base indent per `CLAUDE.md`). Cases: + +- a chart with three `` elements: entries in `d_index` order, all + `:undefined`, `d_index` set. +- `` with `:source` given: `declared_source` + is `"41 + 1"` and `value` is still `:undefined` - the assertion that would + catch an accidental evaluation. +- `` (no value source, `value_location == location`): + `declared_source` is `nil`. +- `:source` omitted: every `declared_source` is `nil`, no diagnostics. +- system variables: exactly the four names `initial/2` seeds, `_sessionid` + inferring `:string`, `_ioprocessors` inferring a nested map, `_event` and a + nameless chart's `_name` inferring `:undefined`; `_event`'s children are + exactly the keys of `SystemVariables.event/1`'s result; **no `_x`**. +- functions: `"In"` present with arity 1; at least one multi-arity builtin + (`"substring"`, `[2, 3]`) rendering `"substring/2|3"`; entries sorted by + name. +- `{:error, {:invalid_machine, _}}` for a non-Machine. + +### Success Criteria: + +#### Automated Verification: +- [x] `mix format` run, then full `mix quality` passes with no `○` line beyond + the two permanent ones named in `CLAUDE.md` +- [x] `lib/statifier_ui/datamodel_explorer/entry.ex`, + `lib/statifier_ui/datamodel_explorer/scope.ex`, and + `test/statifier_ui/datamodel_explorer/scope_test.exs` exist +- [x] A test asserts a `` entry's `value` is `:undefined` + while its `declared_source` holds the expression text +- [x] A test asserts the system-variable name set is exactly + `["_event", "_ioprocessors", "_name", "_sessionid"]` sorted, with no + `"_x"` +- [x] `grep -rn "Kino" lib/statifier_ui/datamodel_explorer/` matches nothing +- [x] Every public function in the new modules carries a `@spec`, checked by + counting: `grep -c "^ def " ` equals `grep -c "^ @spec " ` + for each new `lib/` file. **The gate does not decide this**: + `.doctor.exs` sets `min_module_spec_coverage: 75`, not 100, deliberately, + so a single unspecced public function still passes a green run. The + repo's `@spec`-on-public-functions convention (`CLAUDE.md`) is enforced + here by this count, not by the gate. + +#### Manual Verification: +- [ ] In `iex -S mix`, `Scope.build/2` over a real corpus chart produces a + tree a person would recognize as "what is in scope here" +- [ ] The function labels read usefully at `name/arity` and the ceiling (no + return type, no parameter names) is not misleading +- [ ] No regressions in related features + +**Implementation Note**: Run `mix format` within the phase - the gate's format +stage is check-mode only (ex_quality 0.14) and rewrites nothing. Use +`mix quality --profile loop` between edits; run the full `mix quality` as the +phase gate. In interactive execution, pause here for the human to confirm the +manual testing before moving to the next phase. In looped (`--loop`) +execution, this phase's Automated Verification gates advancement automatically +(via `/wurk:commit --auto`), and Manual Verification items are deferred and +surfaced once at the end instead of blocking here. + +--- + +## Phase 2: Authoring mode - the scenario tier and the pane struct + +### Overview + +Tier 3 in authoring mode: one named fixture scenario, merged onto the scope, +with the tier-1 entries taking a scenario value of the same name when one +exists. This phase also introduces the top-level pane struct that Phase 3's +live constructor will fill differently. + +Rests on research defaults **3** (a scenario value promotes a tier-1 entry out +of `:undefined`), **4** (one selected scenario, defaulting to the first sorted +name, with `scenario_names/1` exposed for a host-side switcher), **5** +(authoring-mode values are already decoded, so no `Value.decode/1` here), and +**10** (read-only throughout). + +### Changes Required: + +#### 1. The pane module + +**File**: `lib/statifier_ui/datamodel_explorer.ex` +**Changes**: New module `StatifierUI.DatamodelExplorer`. + +``` +@type mode :: :authoring | :live + +@type t :: %__MODULE__{ + mode: mode(), + session: String.t() | nil, + entries: [Entry.t()], + scenario: String.t() | nil, + scenario_names: [String.t()], + macrostep: non_neg_integer() | nil, + truncated?: boolean(), + diagnostics: [StatifierUI.Fixtures.diagnostic()] + } + +@spec build_authoring(Statifier.Machine.t(), StatifierUI.Fixtures.t() | nil, [opt()]) :: + {:ok, t()} | {:error, term()} +@spec entries(t()) :: [Entry.t()] +@spec entries(t(), Entry.tier()) :: [Entry.t()] +@spec diagnostics(t()) :: [StatifierUI.Fixtures.diagnostic()] +``` + +`build_authoring/3` behavior: + +1. `Scope.build(machine, opts)`; propagate its error. +2. Select the scenario: the `:scenario` option when given, else the first of + `Fixtures.scenario_names/1` (sorted), else `nil` when the bundle is `nil` + or holds no scenarios. A named `:scenario` that the bundle does not hold is + `{:error, {:unknown_scenario, name}}` - a typo in a host's switcher should + say so rather than silently show the first one. +3. For each `{name, value}` in the selected scenario (already decoded Elixir + values - the sidecar decoded at load, `sidecar.ex:171-179`, so **no** + `Value.decode/1` call in this path): if a tier-1 entry of that name exists, + set its `value`, `shape` and `label` from the scenario value and leave its + tier `:data`; otherwise emit a new `tier: :scenario` entry. +4. `entries` is the concatenation in tier order - `:data`, `:system`, + `:function`, `:scenario` - with each group in the order its builder + produced (tier 1 in `d_index` order, the rest sorted by name). +5. `scenario_names` is `Fixtures.scenario_names/1` verbatim, so a host can + render a switcher without holding the bundle. `macrostep` is `nil` and + `truncated?` is `false` in this mode; `session` is `nil`. +6. Bundle diagnostics are carried onto the pane ahead of the scope's, so a + sidecar that already had complaints does not lose them here. + +The moduledoc states the read-only rule where a reader of `sui-t36.8` will hit +it: this pane is a projection, there is no write path, and live datamodel +editing waits for a recordable-channel design (statifier ADR-0029). It also +states the three-tier mapping and which tier switches source between modes +(ADR-0003:82-88). + +#### 2. Tests + +**File**: `test/statifier_ui/datamodel_explorer_test.exs` +**Changes**: New test module. Cases: `build_authoring/3` with `nil` fixtures +(scenario `nil`, `scenario_names` `[]`, every tier-1 entry `:undefined`); a +bundle whose scenario names a declared `` (that entry's shape becomes +the scenario value's, tier stays `:data`); a scenario naming something the +chart does not declare (a `:scenario` entry appears); explicit `:scenario` +selection; `{:error, {:unknown_scenario, _}}`; default selection is the first +sorted name with a two-scenario bundle whose insertion order differs from +sorted order; `entries/2` filtering by tier; bundle diagnostics carried +through. Build one bundle via `Fixtures.from_source/1` over the existing +`test/support/fixtures/payment_source.ex` so the pane is exercised against the +same fixtures the rest of the suite uses. Assert on whole structs where +practical, per the repo's convention. + +### Success Criteria: + +#### Automated Verification: +- [x] `mix format` run, then full `mix quality` passes +- [x] `lib/statifier_ui/datamodel_explorer.ex` and + `test/statifier_ui/datamodel_explorer_test.exs` exist +- [x] A test asserts a scenario value promotes a tier-1 entry's shape while + leaving its `tier` `:data` and its `d_index` intact +- [x] A test asserts default scenario selection is the first **sorted** name, + not the first inserted +- [x] A test asserts a scenario value that is already a decoded Elixir term (a + `Date`) infers `:date` rather than a `$`-tagged map shape - the guard + that authoring mode neither decodes nor re-encodes +- [x] `grep -rn "Kino" lib/statifier_ui/datamodel_explorer*` matches nothing + +#### Manual Verification: +- [ ] In `iex -S mix`, a chart plus its `.fixtures.json` sidecar produces a + tree where the scenario's contribution is obvious at a glance +- [ ] `{:error, {:unknown_scenario, name}}` reads well enough for a form to + show verbatim +- [ ] No regressions in related features + +**Implementation Note**: Run `mix format` within the phase - the gate's format +stage is check-mode only and rewrites nothing. Use `mix quality --profile loop` +between edits; run the full `mix quality` as the phase gate. In interactive +execution, pause here for the human to confirm the manual testing before +moving to the next phase. In looped (`--loop`) execution, this phase's +Automated Verification gates advancement automatically, and Manual +Verification items are deferred and surfaced once at the end. + +--- + +## Phase 3: Live mode - the datamodel fold and change marking + +### Overview + +The genuinely new machinery: fold a message list into the same entry shape, +decoding at ingestion, applying writes in the producer's stamped order, and +marking the entries that changed in the latest macrostep. + +Rests on research defaults **5** (decode at ingestion; a decode failure is a +diagnostic on that entry, not a failed build), **6** (macrostep granularity, +stated in the moduledoc), **7** (an absent `new_value`/`prior_value` reads as +`:undefined`, so a first binding counts as a change), **8** (materialize +intermediate containers along a deep `location_path`; an inapplicable path +becomes a diagnostic), and **9** (a pure fold, with a `truncated?` flag; +late-attach catch-up is `sui-t36.8`'s). + +### Changes Required: + +#### 1. The fold + +**File**: `lib/statifier_ui/datamodel_explorer.ex` +**Changes**: Add to the existing module. + +``` +@spec build_live([StatifierUI.Trace.Message.t()], [opt()]) :: + {:ok, t()} | {:error, {:mixed_sessions, [String.t()]}} +``` + +Behavior, in order: + +1. **Refuse mixed sessions.** Same rule and same error shape as + `EventLog.build/1` (`event_log.ex:87-93`), because + `docs/wire-format.md:96-102` forbids merging two sessions' stamps onto one + timeline. An empty list returns an empty `:live` pane with `session: nil`. +2. **`truncated?`** from `min(seq) > 0`, the `EventLog` precedent - the head of + the stream was dropped and the fold's starting point is not the session's. +3. **Tier attribution** from `session.start`'s `data` table when the list + carries one: `id -> d_index`, giving those names `tier: :data`. Absent a + `session.start` (a late attach), tier-1 names are indistinguishable from + others and fall through to rule 5's classification; the moduledoc says so. +4. **Seed the name set** from `session.datamodel`'s `datamodel` payload map. + Each value goes through `StatifierUI.Value.decode/1` **before** + `Shape.infer/1` - which is the whole point, since every `` element + reads `{"$undefined": true}` in that snapshot and an undecoded pass would + classify it as a one-key map. A decode failure yields an entry with + `value: :undefined` plus one diagnostic + (`kind: :undecodable_datamodel_value`), never a failed build. +5. **Classify** each seeded name: in the `data` table means `:data`; matching a + name `SystemVariables.initial/2` seeds means `:system`; otherwise + `:runtime`. Function entries come from `Scope`'s tier-2b builder unchanged - + they are a compile-time constant and need no session - so both modes show + the same tier 2b. +6. **Apply the writes.** Take every `effect.datamodel_change`, order by the + producer's stamps `{macrostep, microstep, seq}`, and apply in that order. + ADR-0011: never re-sorted by `location_path`, by `d_index`, or by name. For + each: + - `new_value` **absent** reads as `:undefined`; present reads as + `Value.decode/1` of the payload term (so a `null` reads as `nil`). + `prior_value` follows the same rule. This is + `normalizer.ex:558-565`'s three-way absence rule honored on the consumer + side. + - The head of `location_path` is the root variable name; a non-binary head + is a malformed path and becomes a diagnostic + (`kind: :unresolvable_location_path`) with the write skipped. + - The remaining segments are applied into the root value, materializing a + map for a string segment and a list for an integer segment when the + container is missing or `:undefined` - the snapshot's `user` reads + `$undefined`, so a write to `["user", "items", 0, "name"]` has to build + the intermediate structure. An integer segment beyond the end of an + existing list, or a segment kind that contradicts an existing container, + is a diagnostic and a skipped write rather than a crash. + - A name not in the seeded set is added as `tier: :runtime` (an + `` or an empty `` can write a location the + snapshot did not name). + - `location_source` is stored on the entry from the most recent write, for + display. +7. **Mark changes.** `macrostep` on the pane is the highest macrostep any + applied `datamodel_change` carried (`nil` when none did). An entry is + `changed?: true` when it received at least one write stamped at that + macrostep whose decoded `prior_value` differs from its decoded `new_value` + under rule 6's absence reading - so a first binding, which has no + `prior_value` at all, counts as a change from `:undefined`. Every other + entry is `changed?: false`. +8. `shape` and `label` are recomputed from the final decoded value once, after + the fold, so an entry written three times in one macrostep is labelled from + its end state. + +The moduledoc states the macrostep bound explicitly: `effect.datamodel_change` +carries `macrostep` and `microstep` but a `nil` `round` +(`normalizer.ex:271`, consistent with every `effect.*` type but +`effect.budget_exhausted`), so "changed since the previous step" is answerable +at macrostep granularity and no finer. Per-round marking would be an upstream +wire change, not a change here. + +#### 2. Tests + +**File**: `test/statifier_ui/datamodel_explorer/live_test.exs` +**Changes**: New test module built on a `@worked_example` module attribute +holding literal `%Message{}` structs transcribed from +`docs/wire-format.md:737-762` and `:629-681`, the way `event_log_test.exs` +does. Cases: + +- the `session.datamodel` snapshot alone: every entry `:undefined`, none + `changed?`, names in the payload's key order sorted. +- a `{"$undefined": true}` snapshot value inferring `:undefined` and **not** + `{:map, %{"$undefined" => :boolean}}` - the regression guard for a missing + decode pass. +- one `` binding write: value present, `changed?` true, `macrostep` set. +- a write with `new_value` absent: the entry becomes `:undefined` and is + marked `changed?` - the unbinding case. +- a write with `new_value` `null`: the entry becomes `nil`, distinctly from + the case above. +- two macrosteps of writes: only the later macrostep's entries are + `changed?`. +- a deep `location_path` into a `$undefined` root: containers materialized, + final value at the leaf. +- an integer segment into a map and a string segment into a list: one + diagnostic each, the sibling entries intact. +- a name written that the snapshot never carried: `tier: :runtime`. +- **shuffle invariance**: five shuffles of the same list fold to identical + structs, the assertion `event_log_test.exs:188-198` established. +- `{:error, {:mixed_sessions, ids}}` for two session ids. +- `truncated?` true when the lowest `seq` is greater than 0. + +### Success Criteria: + +#### Automated Verification: +- [x] `mix format` run, then full `mix quality` passes +- [x] `test/statifier_ui/datamodel_explorer/live_test.exs` exists and holds a + `@worked_example` transcribed from `docs/wire-format.md` +- [x] A shuffle-invariance test folds at least five permutations and asserts + struct equality +- [x] A test asserts a `{"$undefined": true}` snapshot value infers + `:undefined`, not a map shape +- [x] Separate tests distinguish absent `new_value` (`:undefined`) from a + `null` `new_value` (`nil`) +- [x] A test asserts only the highest macrostep's writes set `changed?` +- [x] `grep -rn "Kino" lib/statifier_ui/datamodel_explorer*` matches nothing + +#### Manual Verification: +- [ ] In `iex -S mix`, folding a captured trace (`test/support/trace/two_state.jsonl` + or a live run) produces values a person would recognize as the chart's + datamodel +- [ ] The diagnostics for a malformed path name the variable and the path + clearly enough to act on +- [ ] No regressions in related features + +**Implementation Note**: Run `mix format` within the phase - the gate's format +stage is check-mode only and rewrites nothing. Use `mix quality --profile loop` +between edits; run the full `mix quality` as the phase gate. In interactive +execution, pause here for the human to confirm the manual testing before +moving to the next phase. In looped (`--loop`) execution, this phase's +Automated Verification gates advancement automatically, and Manual +Verification items are deferred and surfaced once at the end. + +--- + +## Phase 4: Renderer, live-session proof, changelog + +### Overview + +The textual view `sui-t36.8` wraps in a widget, the end-to-end test against a +real session, and the changelog fragment for the bead. + +### Changes Required: + +#### 1. The renderer + +**File**: `lib/statifier_ui/datamodel_explorer/markdown.ex` +**Changes**: New module `StatifierUI.DatamodelExplorer.Markdown`. + +``` +@type opt :: + {:tiers, [Entry.tier()]} + | {:collapsible, boolean()} + | {:changed_marker, String.t()} + | {:max_keys, pos_integer()} + | {:max_depth, pos_integer()} + +@spec render(StatifierUI.DatamodelExplorer.t(), [opt()]) :: String.t() +``` + +A header naming the mode and either the scenario (authoring) or the session +and macrostep (live), a drop warning when `truncated?`, then one section per +tier in the pane's tier order, each a Markdown table of name, type label, and +value, with the `:changed_marker` (default `"*"`) on a `changed?` entry and a +nested child row per `Entry.children` element. A diagnostics section last when +the pane carries any. `:max_keys` and `:max_depth` pass straight through to +`Shape.label/2`. Nothing is re-sorted here: rows come out in `entries/1` order +(ADR-0011). + +Values are rendered with `inspect/2` rather than re-encoded to JSON: this is a +debugging view of decoded Elixir terms, and `StatifierUI.Trace.Json` is for +the wire. + +The moduledoc carries the same `Kino`-in-prose-only statement +`event_log/markdown.ex:1-9` does, naming `sui-t36.8` as the owner of the +widget. + +**File**: `test/statifier_ui/datamodel_explorer/markdown_test.exs` +**Changes**: New test module. Split the rendered string on `"\n"`, trim, and +assert membership - **no golden files**, per `markdown_test.exs:135-198`. +Cases: an authoring render names the scenario; a live render names the session +and macrostep; a `changed?` entry carries the marker and an unchanged one does +not; `:tiers` filtering omits a section entirely; `truncated?` produces the +warning line; a diagnostics section appears only when there are diagnostics; +`max_keys: 1` truncates a label. + +#### 2. Live-session round trip + +**File**: `test/statifier_ui/datamodel_explorer/session_test.exs` +**Changes**: New test using `StatifierUI.Test.Support.Trace.SessionCase`. + +Compile a heredoc chart with a `` and a +transition whose `` fires on an +external event. Start a real session with a subscriber via `start_early!/3`, +`wait_for_seq/3` past the initial macrostep, fold `Subscriber.messages/1` with +`build_live/2`, and assert `count` holds `42` and is `changed?`. Then send the +event with `Statifier.Session.send_event/2`, wait again, re-fold, and assert +`count` holds `43`, is `changed?` at the new macrostep, and that an entry +untouched by that macrostep is **not** marked. This is the test that proves +the fold matches what the engine actually emits, rather than what the wire doc +says it emits. + +Also assert `_sessionid` holds the pinned session id, which is the cheapest +proof that the tier-2a entries survive the live path. + +#### 3. Changelog fragment + +**File**: `changelog.d/sui-t36.7.md` +**Changes**: One fragment for the whole bead (public API addition), following +`changelog.d/README.md` and the `sui-t36.6` fragment's shape. + +### Success Criteria: + +#### Automated Verification: +- [x] `mix format` run, then full `mix quality` passes - this is the last + phase, so this run is also the bead's gate +- [x] `lib/statifier_ui/datamodel_explorer/markdown.ex`, + `test/statifier_ui/datamodel_explorer/markdown_test.exs`, + `test/statifier_ui/datamodel_explorer/session_test.exs`, and + `changelog.d/sui-t36.7.md` exist +- [x] The renderer test asserts by line membership over the split string and + adds no fixture file of expected output (no golden snapshots) +- [x] The live-session test asserts on the folded pane, not on raw messages +- [x] `grep -rn "Kino" lib/` matches only moduledoc prose - the sibling + renderers name `Kino.Markdown` in prose in order to say they do not call + it, so this is read, not grepped for a call site + +#### Manual Verification: +- [ ] The rendered Markdown reads as something a person would want in a + Livebook cell, in both modes +- [ ] The changed marker is legible without a legend +- [ ] The changelog fragment describes the addition in terms a library user + cares about +- [ ] No regressions in related features + +**Implementation Note**: Run `mix format` within the phase - the gate's format +stage is check-mode only and rewrites nothing. Use `mix quality --profile loop` +between edits; run the full `mix quality` as the phase gate. In interactive +execution, pause here for the human to confirm the manual testing. This is the +last phase, so the full gate here is also the bead's gate. + +--- + +## Testing Strategy + +### Unit Tests: + +- `Scope.build/2`: tier-1 order and `d_index`; declared source sliced only + when `value_location != location`; never evaluated; the exact four system + variables with no `_x`; `_event`'s children from the engine's own schema; + function entries sorted with multi-arity rendering; `{:error, + {:invalid_machine, _}}`; one diagnostic per unsliceable span. +- `DatamodelExplorer.build_authoring/3`: `nil` fixtures; scenario promoting a + tier-1 entry; a scenario key with no declaration becoming a `:scenario` + entry; explicit and defaulted scenario selection; `{:error, + {:unknown_scenario, _}}`; bundle diagnostics carried through. +- `DatamodelExplorer.build_live/2`: the decode-first guard; the three-way + absence rule (absent vs `null` vs a value) on both `new_value` and + `prior_value`; deep-path materialization; contradictory-segment diagnostics; + `:runtime` entries; change marking confined to the highest macrostep; + shuffle invariance over five permutations; `{:error, {:mixed_sessions, _}}`; + `truncated?`. +- `Markdown.render/2`: line-membership assertions only, both modes, the + changed marker, tier filtering, truncation warning, diagnostics section, + label option pass-through. + +### Manual Testing Steps: + +1. `iex -S mix`; `Statifier.compile/1` a chart with two or three `` + elements, `Fixtures.from_source/1` a bundle, then + `DatamodelExplorer.build_authoring/3` and + `DatamodelExplorer.Markdown.render/1`; confirm the tree reads as "what is + in scope". +2. Start a session with `trace: true` and a subscriber, fire an event that + assigns, fold `Subscriber.messages/1` with `build_live/2`, and render; + confirm the changed marker lands on exactly the variable that moved. +3. Fire a second event that assigns a different variable; confirm the first + variable is no longer marked and the second is. +4. Fold a deliberately truncated message list (drop the first few); confirm + the drop warning renders and nothing crashes. + +## Open Questions + +Recorded rather than resolved. None blocks implementation, and each is either +someone else's decision or an upstream one. + +1. **Provider metadata is name and arity only.** `functions/0` carries no + return type, no parameter names, and no doc string, so a tier-2b node can + be labelled `name/arity` and nothing richer. **Call made**: label + `name/arity` and stop there. ADR-0003:110-116 already names richer provider + metadata as upstream `px-`/`st-` work; no bead is filed from this plan, and + the research deliberately did not file one either. If the inspector demo + makes the ceiling obviously painful, that is a `px-` bead and an ADR-0003 + amendment, not a synthesis here. +2. **No `round` on `effect.datamodel_change`.** Change marking is available at + macrostep granularity and no finer. **Call made**: mark per macrostep, + which is what the bead asks for, and state the bound in the moduledoc. + Propagating `round` onto the remaining `effect.*` types is already noted as + tracked separately (`event_log.ex:32-34`) and is an upstream wire change. +3. **No accessor for a Machine's `` count.** This plan enumerates with + `machine.data_elements |> Tuple.to_list()`, reading a public struct field, + because `Statifier.Machine.data/2` takes a `d_index` and there is no + `data_count/1` or `data_elements/1`. **Call made**: use the same idiom + `lib/statifier_ui/trace/manifest.ex:193-201` already uses in this repo, so + there is one spelling here rather than two. A `Machine.data_elements/1` + accessor upstream would be a nicety, not a need; no `st-` bead filed. +4. **Authoring mode has no session id**, but + `SystemVariables.initial/2` requires one to seed `_sessionid` and + `_ioprocessors`. **Call made**: a `:session_id` option defaulting to the + documented placeholder `"(authoring)"`, so the engine stays the single + source of the variable list rather than this repo re-listing four names + that could drift. The placeholder is visible in the tree, which is honest - + there is no session. +5. **Late-attach gaps are not detectable atomically.** `Subscriber` offers no + push-based catch-up and no atomicity between `messages/1` and + `add_listener/2`. **Call made**: not this bead's problem. `build_live/2` is + a pure fold over whatever list it is handed and surfaces `truncated?` the + way `EventLog` does; closing the gap belongs to `sui-t36.8`. +6. **No correlation between an injected event and a resulting write.** + `sui-t36.6`'s plan records the same question from the sending side. **Call + made**: out of scope here and owned by `sui-t36.8`; nothing in this plan + depends on it. +7. **`docs/architecture.md:214-217` says authoring values are "editable in + place".** **Call made**: the bead wins for this milestone - read-only in + both modes, no editing affordance, and no edit to `docs/architecture.md`, + since that sentence describes the eventual LiveComponent rather than the + Kino milestone. + +## References + +- Bead: `sui-t36.7` (parent `sui-t36`; blocks `sui-t36.8`; depends on + `sui-t36.2`, `sui-t36.3`, `sui-h92`, `sui-bpb`, all closed) +- Source research: + `docs/research/260822-sui-t36.7-datamodel-explorer-pane.md` +- ADRs: `docs/adr/0003-fixtures-as-the-example-data-contract.md` (tiers, + inferred types, the two-mode justification at `:82-88`), + `docs/adr/0005-language-neutral-trace-wire-format.md`, + `docs/adr/0006-datasets-and-expression-fixtures.md` (out of `sui-t36` + scope), `docs/adr/0011-exit-and-entry-sets-are-sequences.md` (never re-sort + a producer-ordered sequence), + `docs/adr/0004-one-package-with-optional-integrations.md`, + `docs/adr/0002-adopt-upstream-decisions-by-reference.md` +- Wire format: `docs/wire-format.md:311-348` (`session.start`'s `data` + table), `:629-681` (`effect.datamodel_change`), `:737-762` + (`session.datamodel` and the `$undefined` rule), `:135-165` (the value + codec), `:96-102` (one session per timeline) +- Reused modules: `lib/statifier_ui/shape.ex`, `lib/statifier_ui/value.ex`, + `lib/statifier_ui/fixtures.ex`, `lib/statifier_ui/trace/message.ex`, + `lib/statifier_ui/trace/manifest.ex:193-201` +- Sibling precedents: `lib/statifier_ui/event_log.ex:64-101,146-195`, + `lib/statifier_ui/event_log/markdown.ex:1-63`, + `lib/statifier_ui/event_injection/palette.ex:97-126`, and their plans + `docs/plans/260822-sui-t36.5-event-log-pane.md`, + `docs/plans/260822-sui-t36.6-event-injection-pane.md` +- Upstream (read-only): `deps/statifier/lib/statifier/machine.ex:206-207`, + `deps/statifier/lib/statifier/parser/location.ex:64-76` (`slice/2`), + `deps/statifier/lib/statifier/machine/data.ex`, + `deps/statifier/lib/statifier/evaluator/system_variables.ex:75-110`, + `deps/statifier/lib/statifier/evaluator/functions.ex:20-78`, + `deps/predicator/lib/predicator/functions/provider.ex:19-51`, + `deps/predicator/lib/predicator/context.ex:216-237` +- Test harness: `test/support/trace/session_case.ex`, + `test/support/fixtures/payment_source.ex`, + `test/statifier_ui/event_log_test.exs:188-198` (shuffle invariance) + +## Deferred Manual Verification + +Manual verification items are deferred during looped (--loop) execution and +surfaced here once, rather than blocking after each phase. Confirm these +before considering the plan fully landed. + +### Phase 1 + +- [ ] In `iex -S mix`, `Scope.build/2` over a real corpus chart produces a + tree a person would recognize as "what is in scope here" +- [ ] The function labels read usefully at `name/arity` and the ceiling (no + return type, no parameter names) is not misleading +- [ ] No regressions in related features + +--- + +### Phase 2 + +- [ ] In `iex -S mix`, a chart plus its `.fixtures.json` sidecar produces a + tree where the scenario's contribution is obvious at a glance +- [ ] `{:error, {:unknown_scenario, name}}` reads well enough for a form to + show verbatim +- [ ] No regressions in related features + +--- + +### Phase 3 + +- [ ] In `iex -S mix`, folding a captured trace produces values a person would + recognize as the chart's datamodel +- [ ] The diagnostics for a malformed path name the variable and the path + clearly enough to act on +- [ ] No regressions in related features + +--- + +### Phase 4 + +- [ ] The rendered Markdown reads as something a person would want in a + Livebook cell, in both modes +- [ ] The changed marker is legible without a legend +- [ ] The changelog fragment describes the addition in terms a library user + cares about +- [ ] No regressions in related features + +--- diff --git a/docs/research/260822-sui-t36.7-datamodel-explorer-pane.md b/docs/research/260822-sui-t36.7-datamodel-explorer-pane.md new file mode 100644 index 0000000..e422834 --- /dev/null +++ b/docs/research/260822-sui-t36.7-datamodel-explorer-pane.md @@ -0,0 +1,594 @@ +--- +date: 2026-08-22T12:27:55-0600 +researcher: Claude +git_commit: edf983b813bc6175666299a5e29db7972c82d5c8 +branch: sui-t36.7-datamodel-explorer +repository: statifier-ui +beads_issue: sui-t36.7 +topic: "How the datamodel explorer pane (sui-t36.7) is built: authoring-mode tier scope, live-mode datamodel fold, and the pane conventions it follows" +tags: [research, codebase, datamodel, fixtures, wire-format, kino] +status: complete +last_updated: 2026-08-22 +last_updated_by: Claude +--- + +# Research: the datamodel explorer pane (sui-t36.7) + +**Date**: 2026-08-22T12:27:55-0600 +**Git Commit**: edf983b813bc6175666299a5e29db7972c82d5c8 +**Branch**: sui-t36.7-datamodel-explorer +**Bead**: sui-t36.7 (parent `sui-t36`, blocks `sui-t36.8`) + +## Research Question + +sui-t36.7 asks for a datamodel explorer tree view: **one component, two data +sources**. Authoring mode shows the merged fixture scope - document ``s, SCXML 5.10.1 system variables from `Statifier.Evaluator.SystemVariables`, +provider functions enumerated through the predicator `Provider` behaviour's +`functions/0`, and fixture scenarios - each with an inferred type label. Live +mode shows the subscribed session datamodel, updating per macrostep, marking +entries changed since the previous step. Read-only in this milestone. + +What exists today for each of those inputs, what shape the two existing panes +established, and where the pieces do not yet meet? + +## Summary + +Everything the bead needs is present and closed; nothing is blocked on an +upstream gap. Concretely: + +- **The tier model is settled contract, not a design choice to make.** + ADR-0003 names the three tiers explicitly and states that the two-mode + explorer exists *because* only tier 3 switches source between modes + ([`docs/adr/0003-fixtures-as-the-example-data-contract.md:82-88`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/docs/adr/0003-fixtures-as-the-example-data-contract.md#L82-L88)). + [`docs/architecture.md:207-239`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/docs/architecture.md#L207-L239) restates it. +- **Tier 2 is smaller than "spec 5.10.1" suggests.** + `Statifier.Evaluator.SystemVariables.initial/2` seeds exactly four variables - + `_sessionid`, `_name`, `_event`, `_ioprocessors`. `_x` is deliberately seeded + nowhere. `event/1` is the schema for `_event`'s six 5.10.1 sub-fields. +- **Provider enumeration returns name and arity only.** `functions/0` is + `%{binary() => {arity, atom()}}`. There is no return type, no parameter names, + no doc string. Richer provider metadata is named by ADR-0003 as upstream + `px-`/`st-` work, not something to synthesize here. +- **Live mode is a fold, and both halves of it are on the wire.** + `session.datamodel` (seq 1, always) names every variable with every value + reading `{"$undefined": true}`; `effect.datamodel_change` (sui-h92, merged as + `2991987`) carries `location_path`, `location_source`, `new_value`, + `prior_value` per successful write. [`docs/wire-format.md:751-754`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/docs/wire-format.md#L751-L754) states the + fold as the contract. +- **The house pane shape is a pure `build/1` fold plus a pure renderer**, no + Kino anywhere in `lib/`. `lib/statifier_ui/event_log.ex` and + `lib/statifier_ui/event_injection.ex` are the two precedents; `sui-t36.8` owns + wrapping a rendered string in an actual widget. +- **The one genuinely new thing in this bead is diffing.** No existing pane + computes a delta between two steps; every "what changed" fact rendered today + (`trace.exit_set`, `trace.entry_set`) was already computed by the engine and + passed through verbatim. Marking changed entries per macrostep is this pane's + own work. + +The sharpest practical hazard found: **`Message.payload` values are already +`$`-tagged wire-encoded**, so `Shape.infer/1` applied directly to a payload +value would classify `%{"$undefined" => true}` as a one-key map, not as +`:undefined`. `StatifierUI.Value.decode/1` has to run first. + +## Detailed Findings + +### Tier 1 - document `` declarations + +Two candidate sources exist and they are not interchangeable. + +**On the wire**, `session.start` carries a `data` identity table, one object per +compiled `` element in `d_index` order, with `id`, `location`, and +`value_location` ([`docs/wire-format.md:311-348`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/docs/wire-format.md#L311-L348)). It is **deliberately identity +only**: no declared value, because three of the four `Machine.Data.value` +variants have no honest language-neutral encoding +(`docs/plans/260819-sui-o5c-data-identity-table-in-session-start.md`). A +consumer wanting the declared value slices `source` at `value_location`, and +must first compare `value_location` against `location` - when they are equal +there is no value span and the slice is the whole element +([`docs/wire-format.md:335-348`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/docs/wire-format.md#L335-L348)). + +**In process**, the compiled `%Statifier.Machine{}` carries the same elements; +sui-o5c's plan names `Statifier.Machine.data/2` as the accessor. This is the +only route available in authoring mode, which by definition has no session and +therefore no `session.start` message. + +Consequence for type labels: a tier-1 entry has **no runtime value to infer a +shape from** unless a fixture scenario supplies one, or the pane slices declared +source text and evaluates it (which it must not - predicator is non-evaluative +by ADR-0004 upstream, adopted by ADR-0002). See Open Question 3. + +`d_index` is the join key: `effect.datamodel_change` carries `d_index` for a +`` binding, resolving through `session.start`'s `data` table to a human +name ([`docs/wire-format.md:699-702`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/docs/wire-format.md#L699-L702)). + +### Tier 2a - system variables + +`deps/statifier/lib/statifier/evaluator/system_variables.ex` is the whole +schema, and it is prose plus code rather than a queryable metadata table: + +- `initial/2` (`:75-84`) returns exactly + `%{"_sessionid" => session_id, "_name" => absent(machine.name), "_event" => + :undefined, "_ioprocessors" => %{@scxml_event_processor => %{"location" => + scxml_location(session_id)}}}`. +- `event/1` (`:100-110`) is the shape of `_event`: `"name"`, `"type"`, + `"sendid"`, `"origin"`, `"origintype"`, `"invokeid"`, `"data"` - the six + 5.10.1 fields plus data, each `nil` translated to `:undefined`. +- `scxml_event_processor/0` (`:28`) and `scxml_location/1` (`:36`) are the two + constants `_ioprocessors` is built from. +- **`_x` is not seeded.** The evaluator says so directly + (`deps/statifier/lib/statifier/evaluator.ex:359-361`: "`_x`... which this repo + seeds nowhere"). +- `In(stateId)` is **not** a datamodel key. It is a predicator function + provider, `Statifier.Evaluator.Functions` - tier 2b, not 2a. + +The engine carries no per-variable description or type metadata, so a tree +label for a system variable is either hardcoded here or inferred from the value +`initial/2` produces. Two of the four are concrete at init +(`_sessionid` a string, `_ioprocessors` a nested map); two are `:undefined` +(`_event`, and `_name` when the chart has no name). The worked example confirms +this on the wire ([`docs/wire-format.md:837`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/docs/wire-format.md#L837)). + +### Tier 2b - provider functions + +`Predicator.FunctionProvider` (`deps/predicator/lib/predicator/functions/provider.ex:1-52`) +defines exactly one callback: + +```elixir +@callback functions() :: %{name() => entry()} +# name :: binary() +# entry :: {Predicator.Evaluator.function_arity(), atom()} +# function_arity :: non_neg_integer() | [non_neg_integer()] +``` + +So an entry is `{arity_or_arities, impl_atom}` and nothing more - `"substring" +=> {[2, 3], :call_substring}` is the multi-arity form +(`deps/predicator/lib/predicator/functions/system_functions.ex:77-95`). There is +no `variables/0`-style callback anywhere in predicator; a repo-wide `@callback` +grep finds only this one plus `Predicator.Visitor.visit/2`. + +`FunctionProvider.builtin_providers/0` (`provider.ex:43-51`) is public and +returns the four defaults: `SystemFunctions`, `DateFunctions`, `JSONFunctions`, +`MathFunctions`. + +**Statifier does not register through `providers:`.** It hand-writes the +resolved tuple into `:functions` at compile time, because a module cannot +validate itself while compiling +(`deps/statifier/lib/statifier/evaluator/functions.ex:29-62`): + +```elixir +@base_context Predicator.Context.new(%{}, + builtins: true, + functions: %{"In" => {1, {__MODULE__, :in_state}}}, + on_unbound: :error) +def base_context, do: @base_context +``` + +`Predicator.Context.resolve_functions/1` +(`deps/predicator/lib/predicator/context.ex:225-237`) produces the fully +resolved map `%{name => {arity, {module, atom}}}` on the context's `functions` +field. That means **the honest enumeration of "every function in scope for this +chart" is `Statifier.Evaluator.Functions.base_context().functions`**, a public +function returning a public struct field - not a walk over provider modules, +which would miss the shadowing/merge order (`context.ex:216-223`: builtins, +then `:providers` left-to-right, then `:functions`, each shadowing the last). +sui-bpb's refresh review confirmed `functions/0` itself is unchanged under +predicator 9.0. + +Also note `Predicator.Types` (`deps/predicator/lib/predicator/types.ex:66-76`) +has **no type-tag atom vocabulary**: a value's type is its Elixir term shape. +`types_match?/2` (`:304-315`) compares two values structurally; there is no +`type_of/1`. The type vocabulary this repo uses is its own, +`StatifierUI.Shape`. + +### Tier 3 - fixtures + +`StatifierUI.Fixtures` ([`lib/statifier_ui/fixtures.ex:56-62`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/lib/statifier_ui/fixtures.ex#L56-L62)) is three fields: + +```elixir +defstruct scenarios: %{}, events: %{}, diagnostics: [] +``` + +A "scenario" is not a struct - it is a validated `%{String.t() => term()}` map +under a name. Access is `scenario/2` and `scenario_names/1` (sorted, +`fixtures.ex:112-120`). Validation is deliberately stricter than the engine's: +string keys at every depth (`check_keys/2`, `:191-216`), and a duration-shaped +map inside a scenario is rejected outright as +`{:error, {:duration_in_scenario, path}}` (`:205-214`). + +Two delivery paths converge on the one struct (ADR-0003): +`StatifierUI.Fixtures.Source` (behaviour: `scenarios/0`, `example_events/0`) and +`StatifierUI.Fixtures.Sidecar` (`.fixtures.json`, version 1, unknown +top-level keys become diagnostics rather than errors, +`fixtures/sidecar.ex:28,135-192`). `test/statifier_ui/fixtures/convergence_test.exs` +asserts the two paths produce equal structs. + +**ADR-0006's `datasets` and `expressions` are not implemented and are explicitly +out of sui-t36 scope** (ADR-0006:115-119: "The Livebook inspector needs +scenarios and events only"). `@known_top_level_keys` is `~w(version scenarios +events)` (`sidecar.ex:28`); the `Source` behaviour has two callbacks. `sui-bob` +owns that work. + +### Type labels - `StatifierUI.Shape` + +`lib/statifier_ui/shape.ex` is exactly the "inferred type label" machinery the +bead asks for, already built by sui-t36.2, and it depends on nothing in +`Fixtures` (`shape.ex:13-14`). + +Vocabulary (`shape.ex:28-41`): + +``` +:boolean | :integer | :float | :string | :date | :datetime | :duration +| :null | :undefined | :unknown +| {:list, t() | :empty} | {:map, %{String.t() => t()}} | {:union, [t()]} +``` + +- `infer/1` (`:61-85`) is total over `term()`, always recursing fully. +- `label/2` (`:201-206`) renders a shape as a short string, truncating with + `:max_keys` (default 4) and `:max_depth` (default 3) - truncation lives in + rendering, never in inference (`:187`). +- List elements are unified (`:138-175`): one shape if uniform, a key-by-key + merge if all are maps (missing keys fill as `:undefined`), otherwise + `{:union, sorted}`. +- `duration?/1` (`:114-122`) accepts a *subset* of the eight unit atoms because + predicator's parser emits seven, omitting `:milliseconds`. + +`:undefined` is already a first-class member of the vocabulary, which matters: +every tier-1 entry in the `session.datamodel` snapshot infers as `:undefined` +until a `datamodel_change` supplies a value. + +### The value codec - `StatifierUI.Value` + +`lib/statifier_ui/value.ex` implements ADR-0005's four `$`-tagged forms: +`{"$undefined": true}`, `{"$date": iso}`, `{"$datetime": iso}`, +`{"$duration": {8 units}}`. `decode/1` (`:71-115`) is wire-to-Elixir; +`encode/1` (`:137-171`) is the inverse; anything outside predicator's closed +domain returns `{:error, {:unsupported_value, term}}`. JSON `null` always +decodes to `nil`, never `:undefined` (`:12-14`). + +**This is the load-bearing sequencing constraint for the pane.** +`StatifierUI.Trace.Message.payload` is typed `%{String.t() => json()}` +([`lib/statifier_ui/trace/message.ex:21-39`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/lib/statifier_ui/trace/message.ex#L21-L39)) - already encoded. So the live-mode +pipeline is: + +``` +payload value -> Value.decode/1 -> Shape.infer/1 -> Shape.label/2 +``` + +Skipping `decode/1` would classify `%{"$undefined" => true}` as +`{:map, %{"$undefined" => :boolean}}`. Authoring mode has the opposite +situation: `Fixtures` holds already-decoded Elixir values (the sidecar decoded +them at load, `sidecar.ex:171-179`), so `Shape.infer/1` applies directly with no +decode step. The two modes therefore differ in one more place than "tier 3 +source" - a point worth settling explicitly in the plan. + +### Live mode input 1 - `session.datamodel` + +Produced by `StatifierUI.Trace.Normalizer.datamodel_message/1` +([`lib/statifier_ui/trace/normalizer.ex:378-391`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/lib/statifier_ui/trace/normalizer.ex#L378-L391)) from +`Statifier.Effect.DatamodelInit`: + +```elixir +with {:ok, encoded} <- Value.encode(p.datamodel) do + {:ok, {"session.datamodel", nil, nil, nil, %{"datamodel" => encoded}}} +end +``` + +- One payload field, `datamodel`, keyed by variable name + ([`docs/wire-format.md:760-762`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/docs/wire-format.md#L760-L762)). +- Envelope `macrostep`/`microstep`/`round` are all `nil` by deliberate choice, + even though the effect carries them internally + ([`docs/wire-format.md:727-730`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/docs/wire-format.md#L727-L730)). +- Emitted exactly once per session, unconditionally, even under `trace: false`, + always at `seq: 1`, right after `session.start` at `seq: 0` + ([`lib/statifier_ui/trace/subscriber.ex:278-284`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/lib/statifier_ui/trace/subscriber.ex#L278-L284), worked example + [`docs/wire-format.md:837`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/docs/wire-format.md#L837)). +- **Every `` element reads `{"$undefined": true}`**, including one written + ``, because the snapshot precedes the binding + fold ([`docs/wire-format.md:746-750`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/docs/wire-format.md#L746-L750)). The two system variables bound at init + (`_sessionid`, `_ioprocessors`) do carry real values; `_event` and `_name` do + not. + +So this message is the **name list**, and the source of the tree's key set. It +is not a source of values. + +### Live mode input 2 - `effect.datamodel_change` + +Produced by `Normalizer.core_message(%DatamodelChange{})` +([`lib/statifier_ui/trace/normalizer.ex:258-273`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/lib/statifier_ui/trace/normalizer.ex#L258-L273)), merged as `2991987` under +sui-h92. Payload ([`docs/wire-format.md:639-647`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/docs/wire-format.md#L639-L647)): + +| Field | Notes | +|---|---| +| `location_path` | always; heterogeneous array, strings are object keys, integers are 0-based array indexes. Already *resolved* - `items[i]` arrives as `..., "items", 0, ...`. Emitted structurally, not through `Value.encode/1` (`normalizer.ex:259-262`) | +| `location_source` | always; the raw author string, for display only | +| `new_value` | absent when the write stored `:undefined`; `null` when it stored null | +| `prior_value` | absent when nothing stood at the path before - the normal case for a first write | +| `d_index` | only for a `` binding; then `owner` is absent too | +| `c_index` | only when a content node wrote (``, ``) | +| `owner` | only when known; the two runner-side writes (``, empty ``) are identified by `owner` alone | + +Emitted once per **successful** write: ``, a `` binding during the +binding fold, ``, ``, or an empty +`` auto-assign. A failed write emits nothing +([`docs/wire-format.md:631-637`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/docs/wire-format.md#L631-L637)). + +**Envelope stamping**: `macrostep` and `microstep` yes, `round` **no** - +`normalizer.ex:271` passes `nil` for round, consistent with every `effect.*` +type except `effect.budget_exhausted`. So "changed since the previous step" is +directly answerable at **macrostep** granularity and not at round granularity +from this message alone. The bead says "per macrostep", so this is a match, but +it is worth writing down as a bound. + +The three-way absence rule is implemented as `put_defined/3` +(`normalizer.ex:558-565`), distinct from `put_value/3` (which collapses `nil` +and `:undefined`) and `put_present/3` (non-value optional fields). A consumer +folding these must honor the same distinction: **`new_value` absent means the +write stored the unbound sentinel**, which is a real state change to +`:undefined`, not a no-op. + +### The subscriber seam + +`StatifierUI.Trace.Subscriber` (`lib/statifier_ui/trace/subscriber.ex`) is a +GenServer offering both a pull and a push surface: + +- `messages/1` (`:194-195`) - synchronous, returns `Buffer.to_list/1` + oldest-first. +- `add_listener/2` (`:201-204`) - registered pids receive + `{:statifier_ui, session_id, %Message{}}` for every *subsequent* message + (`buffer_and_fanout/2`, `:407-412`). +- There is **no push-based catch-up** and no atomicity between the two: a late + joiner calls `messages/1` for history, then `add_listener/2` for the live + stream, and a message arriving between the two calls is a gap the pane cannot + see. `Buffer` is fixed-capacity drop-oldest (default 1000, `:90`); loss is + detected by comparing the first buffered `seq` against the expected start + (`buffer.ex:12-16`), and counted by `stats/1`'s `dropped`. +- The `truncated?` flag on `StatifierUI.EventLog` (`event_log.ex:95-101`) is the + precedent for surfacing that: `min(seq) > 0` means the head of the stream was + dropped. + +Macrostep boundaries are marked by `trace.macrostep_stable`, with the **last** +one for a given macrostep being the true quiescence point (ADR-0044, +[`docs/wire-format.md:104-123`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/docs/wire-format.md#L104-L123)). For this pane the boundary marker is largely +unnecessary though: `effect.datamodel_change` carries `macrostep` itself, so +grouping by macrostep needs no marker. + +### The pane house pattern + +Two precedents, both landed in the last week (`5cf771c`..`c3594c5`): + +**`StatifierUI.EventLog` (`lib/statifier_ui/event_log.ex`)** - a batch fold. +`build([Message.t()]) :: {:ok, t()} | {:error, {:mixed_sessions, ids}}`. Order +independent by construction, and tested for it +([`test/statifier_ui/event_log_test.exs:188-198`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/test/statifier_ui/event_log_test.exs#L188-L198) folds five shuffles and asserts +struct equality). Per-message-type dispatch lives in a private `put_message/2` +with one clause per wire type (`:146-195`) in the top module, not distributed +across sub-files. It refuses a message list naming more than one session rather +than corrupting the timeline (`:87-93`, per ADR-0050 / [`docs/wire-format.md:96-102`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/docs/wire-format.md#L96-L102)). + +**`StatifierUI.EventInjection` (`lib/statifier_ui/event_injection.ex`)** - a +`build(Fixtures.t() | nil)` constructor plus thin accessors, with side effects +concentrated in one function it calls "the one door in". + +Conventions both follow, and this pane should: + +- Pane modules live at the **top** of `lib/statifier_ui/`, with a subdirectory + for sub-structs. No `lib/statifier_ui/kino/` exists - confirmed by glob. +- **No Kino call sites anywhere in `lib/`.** `grep -rn Kino lib/` returns only + moduledoc prose (`statifier_ui.ex:7`, `diagram.ex:4-5`, + `event_log/markdown.ex:4-9`). `event_log/markdown.ex` states the split + outright: it "names `Kino` only in this moduledoc and calls nothing under + `Kino.*`. `sui-t36.8` owns wrapping the returned string in an actual + `Kino.Markdown` widget." +- Rendering is a pure `render(t(), keyword()) :: String.t()` with `Keyword.get/3` + defaults (`event_log/markdown.ex:58-63` takes `:labels`, `:collapsible`, + `:open`). +- A sub-module earns a file when it is a node in the data tree (`Round`, + `Macrostep`, `Entry` - `round.ex` and `entry.ex` are struct-and-type only, + zero functions), a self-contained build/parse step with its own diagnostics + (`Draft`, `Palette`), a rendering concern (`Markdown`), or an index resolver + (`Labels`). Never `Helpers`/`Utils`. +- `@moduledoc` prose citing ADR numbers and `docs/wire-format.md` line ranges; + `@type t`; `@spec` on every public function and most private ones; doctests + where an example is genuinely illustrative (`Draft.build/2`, + `Diagram.render/2`) rather than uniformly. +- Tests mirror `lib/`. `describe "build/1 - "`. A `@worked_example` + module attribute holding literal `%Message{}` structs transcribed from + `docs/wire-format.md`. Pattern-match assertions over multiple asserts. No + golden-file snapshots - renderer tests split on `"\n"`, trim, and assert + membership (`markdown_test.exs:135-198`). A live-session integration test via + `StatifierUI.Test.Support.Trace.SessionCase` where the pane touches a real + session. +- One diagnostic per degraded input rather than a failed build: `Palette` + omits an unencodable event and appends a `Fixtures.diagnostic()` - + "one bad sample takes down one button, not the pane" + (`event_injection/palette.ex:18-22, 118-126`). + +**No state-diffing precedent exists.** Nothing in `lib/` compares two steps' +snapshots; `trace.exit_set` / `trace.entry_set` are deltas the *engine* computed +and the pane passes through verbatim. Marking changed entries is this pane's +own new machinery, and ADR-0011's rule applies to how it is fed: apply a +sequence of writes in the producer's stamped order, never re-sorted by path or +index. + +## Code References + +- [`lib/statifier_ui/event_log.ex:64-81`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/lib/statifier_ui/event_log.ex#L64-L81) - the `build/1` fold precedent +- [`lib/statifier_ui/event_log.ex:146-195`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/lib/statifier_ui/event_log.ex#L146-L195) - per-wire-type `put_message/2` dispatch +- [`lib/statifier_ui/event_log.ex:95-101`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/lib/statifier_ui/event_log.ex#L95-L101) - `truncated?` from `min(seq) > 0` +- [`lib/statifier_ui/event_injection.ex:64-69`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/lib/statifier_ui/event_injection.ex#L64-L69) - `build(Fixtures.t() | nil)` precedent +- [`lib/statifier_ui/event_injection/palette.ex:97-126`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/lib/statifier_ui/event_injection/palette.ex#L97-L126) - diagnostic-per-bad-entry pattern +- [`lib/statifier_ui/event_log/markdown.ex:1-63`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/lib/statifier_ui/event_log/markdown.ex#L1-L63) - pure renderer, Kino only in prose +- [`lib/statifier_ui/shape.ex:28-41`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/lib/statifier_ui/shape.ex#L28-L41) - the inferred-type vocabulary +- [`lib/statifier_ui/shape.ex:61-85`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/lib/statifier_ui/shape.ex#L61-L85) - `infer/1`, total over `term()` +- [`lib/statifier_ui/shape.ex:201-261`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/lib/statifier_ui/shape.ex#L201-L261) - `label/2` and its truncation options +- [`lib/statifier_ui/value.ex:71-115`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/lib/statifier_ui/value.ex#L71-L115) - `decode/1`, the wire-to-Elixir step live mode needs +- [`lib/statifier_ui/fixtures.ex:56-62`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/lib/statifier_ui/fixtures.ex#L56-L62) - the fixture bundle struct +- [`lib/statifier_ui/fixtures.ex:112-120`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/lib/statifier_ui/fixtures.ex#L112-L120) - `scenario/2`, `scenario_names/1` +- [`lib/statifier_ui/trace/message.ex:21-42`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/lib/statifier_ui/trace/message.ex#L21-L42) - the envelope struct and its `json()` payload type +- [`lib/statifier_ui/trace/normalizer.ex:258-273`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/lib/statifier_ui/trace/normalizer.ex#L258-L273) - `effect.datamodel_change` producer +- [`lib/statifier_ui/trace/normalizer.ex:378-391`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/lib/statifier_ui/trace/normalizer.ex#L378-L391) - `session.datamodel` producer +- [`lib/statifier_ui/trace/normalizer.ex:558-565`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/lib/statifier_ui/trace/normalizer.ex#L558-L565) - `put_defined/3`, the three-way absence rule +- [`lib/statifier_ui/trace/subscriber.ex:194-208`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/lib/statifier_ui/trace/subscriber.ex#L194-L208) - `messages/1`, `add_listener/2` +- [`lib/statifier_ui/trace/subscriber.ex:407-412`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/lib/statifier_ui/trace/subscriber.ex#L407-L412) - the listener fanout message shape +- [`docs/wire-format.md:311-348`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/docs/wire-format.md#L311-L348) - `session.start`'s `data` identity table +- [`docs/wire-format.md:629-681`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/docs/wire-format.md#L629-L681) - the `effect.datamodel_change` schema +- [`docs/wire-format.md:737-762`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/docs/wire-format.md#L737-L762) - the `session.datamodel` schema and the `$undefined` rule +- [`docs/wire-format.md:135-165`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/docs/wire-format.md#L135-L165) - the `$`-tagged value codec +- `deps/statifier/lib/statifier/evaluator/system_variables.ex:75-110` - `initial/2`, `event/1` +- `deps/statifier/lib/statifier/evaluator/functions.ex:20-78` - `functions/0` and `base_context/0` +- `deps/predicator/lib/predicator/functions/provider.ex:19-51` - the callback and `builtin_providers/0` +- `deps/predicator/lib/predicator/context.ex:216-237` - provider resolution and shadowing order + +## Architecture Documentation + +- **ADR-0003** (fixtures as the example-data contract) is the governing record. + The tier model is part of the contract, and the two-mode explorer is its + stated justification: "only the tier-3 source switches between fixture data + (authoring) and the live session's datamodel (debug)" + ([`docs/adr/0003-fixtures-as-the-example-data-contract.md:82-88`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/docs/adr/0003-fixtures-as-the-example-data-contract.md#L82-L88)). Types are + inferred shapes from example values, never a declared schema + (`:56-62`). Richer provider metadata is named as upstream work, not to be + synthesized here (`:110-116`). +- **ADR-0005** (language-neutral trace wire format) governs the envelope, the + four-form value codec, canonical key ordering, and the additive-change rule + that let `effect.datamodel_change` join without a version bump + ([`docs/wire-format.md:676-681`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/docs/wire-format.md#L676-L681)). +- **ADR-0006** (datasets and expression fixtures) is accepted but explicitly + **not sui-t36 scope** and not implemented. Authoring mode reads `scenarios` + and `events` only. +- **ADR-0011** (exit and entry sets are sequences) establishes the general rule + that an engine-ordered sequence is never re-sorted by a consumer - which is + how the `datamodel_change` fold must treat its input. +- **ADR-0002 / CLAUDE.md**: the engine and predicator are read-only from here. + Anything missing upstream is an `st-` or `px-` bead, never a patch applied + from this repo. +- [`docs/architecture.md:207-239`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/docs/architecture.md#L207-L239) is the prose statement of the two-mode design, + including the ADR-0029 constraint that live edits (out of scope here) must + flow through statifier's recordable input channel. + +## Historical Context + +- [`docs/research/260816-sui-kua-gui-research-and-direction.md:141-180`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/docs/research/260816-sui-kua-gui-research-and-direction.md#L141-L180) is the + founding design for this pane and the origin of the three tiers. It notes that + `Statifier.Evaluator.SystemVariables` "is literally the schema" and that + provider functions are "enumerable today via the `Provider` behaviour's + `functions/0` callback (`name -> {arity, impl}`)". +- `docs/research/260816-sui-t36.1-trace-coverage-spike.md` filed GAP 2 + (upstream `st-oef3`): `` was observable only as a `ContentExecuted` + naming a `c_index`, with nothing saying what was written. Its stated rationale + names this pane: reconstructing the datamodel by polling `Session.snapshot/1` + after each effect "races the session, defeats replay from a recorded trace + entirely... and is impossible for a non-Elixir consumer". That gap is now + fully closed on both sides. +- `docs/research/260819-sui-bpb-statifier-and-predicator-9-refresh-surface.md` + put `session.datamodel` on the wire (upstream `st-1xwh`) and recorded that + sui-t36.7 had been missing its dependencies; it also verified that predicator + 9.0 left `functions/0` unchanged. +- `docs/plans/260819-sui-o5c-data-identity-table-in-session-start.md` added the + `data` table and recorded, at length, why it carries no declared value. +- `docs/plans/260822-sui-t36.5-event-log-pane.md` and + `260822-sui-t36.6-event-injection-pane.md` are the structural template a plan + for this bead should follow: frontmatter, Overview, Current State Analysis + (with "what is already decided and must be honored" and "what the wire format + actually provides, verified"), Desired End State, What We're NOT Doing, N + design decisions this plan settles, per-phase Automated/Manual success + criteria, Testing Strategy, Open Questions each with a "Call made:", Deferred + Manual Verification, References. Both name `sui-t36.8` as the owner of Kino + assembly. + +## Related Research + +- `docs/research/260816-sui-kua-gui-research-and-direction.md` +- `docs/research/260816-sui-t36.1-trace-coverage-spike.md` +- `docs/research/260819-sui-bpb-statifier-and-predicator-9-refresh-surface.md` + +## Open Questions + +None of these blocks implementation; each has a workable default noted. + +1. **What does authoring mode take as input for tier 1?** There is no session, + so `session.start`'s `data` table is unavailable; the only route is a + compiled `%Statifier.Machine{}` (via `Statifier.Machine.data/2`). That makes + the authoring-mode entry point `build(machine, fixtures, opts)` rather than + `build(messages)`. Alternatively the pane could accept the + `StatifierUI.Trace.Manifest` payload, which would let one code path serve + both modes when a session does exist. *Default: take a `%Statifier.Machine{}` + for authoring mode; treat manifest-driven tier 1 as a live-mode convenience.* + +2. **How are provider functions enumerated - by provider module or by resolved + context?** `Statifier.Evaluator.Functions.base_context().functions` is the + only source that reflects predicator's shadowing/merge order and therefore + the only honest answer to "what is in scope". Walking + `FunctionProvider.builtin_providers/0` plus host-supplied modules and calling + `functions/0` on each reproduces ADR-0003's literal wording but can report a + shadowed function as live. *Default: read the resolved context; note in the + moduledoc that ADR-0003's `functions/0` phrasing is satisfied transitively.* + +3. **Tier 1 entries have no value in authoring mode, so what type label do they + get?** The wire `data` table is identity only, and the pane must not evaluate + a declared expression. Options: label them `:undefined`; label them from a + fixture scenario entry of the same name when one exists; or slice `source` at + `value_location` and show the declared *text* with no inferred type. + *Default: `:undefined` unless a scenario supplies a value, with the declared + source text shown as a separate display-only field where `value_location != + location`.* + +4. **Which scenario does authoring mode show?** `Fixtures` holds a map of named + scenarios and `scenario_names/1` is sorted. The tree could show one selected + scenario, or all of them as sibling subtrees. *Default: a selected scenario, + defaulting to the first sorted name, with the pane exposing + `scenario_names/1` so the host (sui-t36.8) can offer a switcher.* + +5. **Does the pane decode wire values itself, or take pre-decoded input?** + Authoring mode gets already-decoded Elixir values from `Fixtures`; live mode + gets `$`-tagged encoded values off `Message.payload`. The two modes therefore + differ in more than tier 3. *Default: the live-mode fold calls + `Value.decode/1` at ingestion so both modes hold decoded values internally + and share one `Shape.infer/1` pass; a decode failure becomes a diagnostic on + that entry rather than failing the build (`Palette`'s precedent).* + +6. **Round granularity is not available for change marking.** + `effect.datamodel_change` carries `macrostep` and `microstep` but a `nil` + `round` in the envelope (`normalizer.ex:271`), matching every `effect.*` type + but `effect.budget_exhausted`. The bead says "per macrostep", so this is + sufficient - but if per-round marking is ever wanted it is an upstream wire + change (propagating `round` onto the remaining `effect.*` types is already + noted as tracked separately in `event_log.ex:32-34`). *Default: mark changes + per macrostep, and say so in the moduledoc.* + +7. **What counts as "changed" when `new_value` is absent?** Key absence means + the write stored the unbound sentinel - a genuine change to `:undefined`, not + a no-op. Similarly an absent `prior_value` means the slot was unbound before. + *Default: treat both absences as `:undefined` internally after decoding, and + mark an entry changed whenever `prior_value != new_value` under that reading, + so a first binding counts as a change.* + +8. **Does the fold create intermediate containers for a deep + `location_path`?** A path like `["user", "items", 0, "name"]` may reach into + a structure the snapshot did not carry (the snapshot's `user` reads + `$undefined`). *Default: apply segments in order and materialize maps for + string segments and lists for integer segments as needed; a path that cannot + be applied becomes a diagnostic rather than a crash.* + +9. **Late-attach gaps are not detectable atomically.** `Subscriber` has no + push-based catch-up and no atomicity between `messages/1` and + `add_listener/2`. If the pane is a pure fold over a message list (the + `EventLog.build/1` shape), this is entirely sui-t36.8's problem, not this + bead's. *Default: build a pure fold, and surface a `truncated?`-style flag + the way `EventLog` does when the head of the stream is missing.* + +10. **[`docs/architecture.md:214-217`](https://github.com/riddler/statifier-ui/blob/edf983b813bc6175666299a5e29db7972c82d5c8/docs/architecture.md#L214-L217) says authoring mode has "example values + editable in place", while the bead says read-only in this milestone.** The + bead's read-only clause is written about *live* datamodel editing + specifically, and the architecture sentence describes the eventual + LiveComponent rather than the Kino milestone. *Default: read-only + throughout this bead; no editing affordance in either mode.* + +### Noted but not filed + +Provider metadata is name and arity only - `functions/0` carries no return type, +parameter names, or doc string, so a function node in the tree can be labelled +`name/arity` and nothing richer. ADR-0003 already names this as upstream +(`px-`/`st-`) work. It is recorded here as a known ceiling on tier 2b's display, +not as a discovered blocker, and no bead is filed for it from this research. diff --git a/lib/statifier_ui/datamodel_explorer.ex b/lib/statifier_ui/datamodel_explorer.ex new file mode 100644 index 0000000..a1468f8 --- /dev/null +++ b/lib/statifier_ui/datamodel_explorer.ex @@ -0,0 +1,669 @@ +defmodule StatifierUI.DatamodelExplorer do + @moduledoc """ + The datamodel explorer pane (`sui-t36.7`): one component, two data + sources. **Authoring mode** (`build_authoring/3`) merges the mode- + independent tiers `StatifierUI.DatamodelExplorer.Scope` builds - document + `` declarations (tier 1), the SCXML 5.10.1 system variables + (tier 2a), the predicator provider functions in scope (tier 2b) - with + one named fixture scenario (tier 3), the one tier that switches source + between modes (ADR-0003:82-88). **Live mode**'s `build_live/2` is + `sui-t36.7` Phase 3's addition to this same module. + + This pane is a **projection, not an editor**. There is no write path in + either mode: authoring-mode tier-1 entries hold their declared source + text for display but are never evaluated (predicator is non-evaluative, + ADR-0004 upstream, adopted by ADR-0002), and live datamodel editing waits + for a recordable-channel design (statifier ADR-0029). `sui-t36.8` owns + the widget and the write affordance never arrives here. + """ + + alias Statifier.Evaluator.Functions + alias StatifierUI.DatamodelExplorer.Entry + alias StatifierUI.DatamodelExplorer.Scope + alias StatifierUI.Fixtures + alias StatifierUI.Trace.Message + alias StatifierUI.Value + + @typedoc "Which data source fed this pane: the merged fixture scope, or a live session." + @type mode :: :authoring | :live + + @type t :: %__MODULE__{ + mode: mode(), + session: String.t() | nil, + entries: [Entry.t()], + scenario: String.t() | nil, + scenario_names: [String.t()], + macrostep: non_neg_integer() | nil, + truncated?: boolean(), + diagnostics: [Fixtures.diagnostic()] + } + + defstruct mode: :authoring, + session: nil, + entries: [], + scenario: nil, + scenario_names: [], + macrostep: nil, + truncated?: false, + diagnostics: [] + + @typedoc "Options shared by both constructors; `Scope.opt()` is forwarded as-is." + @type opt :: Scope.opt() | {:scenario, String.t()} + + # Live mode has no session to ask `SystemVariables.initial/2` for these - + # the four names are spec 5.10's fixed set, unchanged by any machine, so + # the name list is hardcoded rather than fabricating a placeholder + # `%Statifier.Machine{}` just to call the function. + @system_variable_names MapSet.new(~w(_sessionid _name _event _ioprocessors)) + + # Tier 1 (`:data`, `:runtime`), tier 2a (`:system`), tier 2b (`:function`) - + # live mode carries no tier 3, since a fixture scenario is an authoring-only + # concept. + @live_tier_order [:data, :runtime, :system, :function] + + @doc """ + Builds an authoring-mode pane from a compiled `machine` and an optional + fixture bundle. + + 1. Builds the mode-independent tiers with `Scope.build/2`, propagating + its error. + 2. Selects a scenario: the `:scenario` option when given, else the first + of `Fixtures.scenario_names/1` (sorted), else `nil` when `fixtures` is + `nil` or holds no scenarios. A named `:scenario` the bundle does not + hold is `{:error, {:unknown_scenario, name}}` - a typo in a host's + switcher should say so rather than silently show the first scenario. + 3. Merges the selected scenario's datamodel onto the scope: a scenario + value naming an existing tier-1 entry replaces that entry's `value`, + `shape` and `label` and leaves its `tier` `:data` and its `d_index` + intact; a scenario value naming nothing declared becomes a new + `tier: :scenario` entry. Scenario values are already decoded Elixir + terms (the sidecar decodes at load, `sidecar.ex:171-179`), so this + path never calls `StatifierUI.Value.decode/1` - doing so would + re-interpret an already-decoded `Date` as a `$`-tagged map. + + `entries/1` on the result concatenates `:data`, `:system`, `:function`, + then `:scenario`, each group in the order its builder produced. Bundle + diagnostics are carried onto the pane ahead of the scope's own, so a + sidecar that already had complaints does not lose them here. + `macrostep` is `nil`, `truncated?` is `false`, and `session` is `nil` - + none of those are meaningful outside live mode. + """ + @spec build_authoring(Statifier.Machine.t(), Fixtures.t() | nil, [opt()]) :: + {:ok, t()} | {:error, term()} + def build_authoring(machine, fixtures \\ nil, opts \\ []) do + scope_opts = Keyword.take(opts, [:source, :session_id]) + + with {:ok, scope} <- Scope.build(machine, scope_opts), + {:ok, scenario_name, scenario_values} <- select_scenario(fixtures, opts) do + entries = merge_scenario(scope, scenario_values) + + {:ok, + %__MODULE__{ + mode: :authoring, + entries: entries, + scenario: scenario_name, + scenario_names: scenario_names(fixtures), + diagnostics: bundle_diagnostics(fixtures) ++ scope.diagnostics + }} + end + end + + @doc """ + Builds a live-mode pane from `messages`, a session's captured effect + stream (in any order - this is a pure fold, not a subscription). + + 1. Refuses a `messages` list naming more than one session, the same + `EventLog.build/1` rule and error shape + (`docs/wire-format.md:96-102`). An empty list returns an empty `:live` + pane with `session: nil`. + 2. `truncated?` is `true` when the lowest `seq` present is greater than + `0` - the head of the stream was dropped, the `EventLog` precedent. + 3. Seeds the name set from `session.datamodel`'s `datamodel` payload map, + decoding each value with `StatifierUI.Value.decode/1` **before** + inferring its shape - the whole point, since every `` element + reads `{"$undefined": true}` there. A decode failure yields an entry + holding `:undefined` plus one `:undecodable_datamodel_value` + diagnostic, never a failed build. + 4. Classifies each seeded name against `session.start`'s `data` table + (`:data`, carrying that element's `d_index`) and the fixed system + variable names spec 5.10 declares (`:system`); anything else is + `:runtime`. + 5. Applies every `effect.datamodel_change` in the producer's stamped + `{macrostep, microstep, seq}` order (ADR-0011: never re-sorted by + `location_path`, by `d_index`, or by name). `new_value`/`prior_value` + absence reads as `:undefined`, matching `normalizer.ex:558-565`'s rule + on the consumer side. `location_path`'s head names the root entry (a + name the snapshot never carried is added as `tier: :runtime`); the + remaining segments are applied into the root's decoded value, + materializing a map for a string segment and a list for an integer + segment when the container is missing or `:undefined`. A non-binary + path head, an out-of-range index, or a segment that contradicts an + existing container becomes an `:unresolvable_location_path` diagnostic + and the write is skipped rather than crashing. `location_source` is + kept from the most recent write applied to that entry. + 6. `macrostep` on the pane is the highest macrostep any *applied* write + carried (`nil` when none did). An entry is `changed?: true` when at + least one write stamped at that macrostep carried a decoded + `prior_value` different from its decoded `new_value` - so a first + binding, which has no `prior_value` at all, counts as a change from + `:undefined`. This is macrostep granularity, not round: every + `effect.*` type but `effect.budget_exhausted` carries a `nil` `round` + (`normalizer.ex:271`), so finer marking would need an upstream wire + change. + 7. `shape` and `label` are computed once, from each entry's final decoded + value, so an entry written more than once in one macrostep is + labelled from its end state. + + Tier 2b (provider functions) is appended unchanged from the same source + `Scope` reads - `Statifier.Evaluator.Functions.base_context().functions` - + since it is a compile-time constant that needs no session; both modes + therefore show the same function list. `entries/1` on the result returns + `:data`/`:runtime`, then `:system`, then `:function`, each group sorted + by name; there is no tier 3 in live mode. + """ + @spec build_live([Message.t()], [opt()]) :: + {:ok, t()} | {:error, {:mixed_sessions, [String.t()]}} + def build_live(messages, opts \\ []) + + def build_live([], _opts), do: {:ok, %__MODULE__{mode: :live}} + + def build_live(messages, _opts) do + with {:ok, session} <- resolve_live_session(messages) do + data_table = data_table(messages) + {seeded, seed_diagnostics} = seed_live_entries(messages, data_table) + + fold = + messages + |> live_writes() + |> Enum.reduce(%{entries: seeded, diagnostics: [], writes: []}, &apply_datamodel_write/2) + + macrostep = highest_macrostep(fold.writes) + changed = changed_names(fold.writes, macrostep) + + entries = + fold.entries + |> finalize_live_entries(changed) + |> Kernel.++(function_entries()) + |> order_live_entries() + + {:ok, + %__MODULE__{ + mode: :live, + session: session, + entries: entries, + macrostep: macrostep, + truncated?: live_truncated?(messages), + diagnostics: seed_diagnostics ++ Enum.reverse(fold.diagnostics) + }} + end + end + + @doc """ + This pane's entries, in tier order (`:data`, `:system`, `:function`, + `:scenario` in authoring mode). + """ + @spec entries(t()) :: [Entry.t()] + def entries(%__MODULE__{entries: entries}), do: entries + + @doc """ + This pane's entries belonging to one `tier`, in the same relative order + `entries/1` returns them. + """ + @spec entries(t(), Entry.tier()) :: [Entry.t()] + def entries(%__MODULE__{entries: entries}, tier), do: Enum.filter(entries, &(&1.tier == tier)) + + @doc """ + This pane's diagnostics, bundle diagnostics ahead of any the pane's own + build produced. + """ + @spec diagnostics(t()) :: [Fixtures.diagnostic()] + def diagnostics(%__MODULE__{diagnostics: diagnostics}), do: diagnostics + + # -- Scenario selection -------------------------------------------------- + + @spec select_scenario(Fixtures.t() | nil, [opt()]) :: + {:ok, String.t() | nil, Fixtures.datamodel()} | {:error, {:unknown_scenario, term()}} + defp select_scenario(fixtures, opts) do + case Keyword.fetch(opts, :scenario) do + {:ok, name} -> + fetch_named_scenario(fixtures, name) + + :error -> + {name, datamodel} = fixtures |> default_scenario() |> fetch_default_scenario(fixtures) + {:ok, name, datamodel} + end + end + + @spec fetch_named_scenario(Fixtures.t() | nil, String.t()) :: + {:ok, String.t(), Fixtures.datamodel()} | {:error, {:unknown_scenario, String.t()}} + defp fetch_named_scenario(fixtures, name) do + case fixtures && Fixtures.scenario(fixtures, name) do + {:ok, datamodel} -> {:ok, name, datamodel} + _not_found -> {:error, {:unknown_scenario, name}} + end + end + + @spec default_scenario(Fixtures.t() | nil) :: String.t() | nil + defp default_scenario(nil), do: nil + + defp default_scenario(%Fixtures{} = fixtures) do + case Fixtures.scenario_names(fixtures) do + [] -> nil + [first | _rest] -> first + end + end + + @spec fetch_default_scenario(String.t() | nil, Fixtures.t() | nil) :: + {String.t() | nil, Fixtures.datamodel()} + defp fetch_default_scenario(nil, _fixtures), do: {nil, %{}} + + defp fetch_default_scenario(name, fixtures) do + {:ok, datamodel} = Fixtures.scenario(fixtures, name) + {name, datamodel} + end + + @spec scenario_names(Fixtures.t() | nil) :: [String.t()] + defp scenario_names(nil), do: [] + defp scenario_names(%Fixtures{} = fixtures), do: Fixtures.scenario_names(fixtures) + + @spec bundle_diagnostics(Fixtures.t() | nil) :: [Fixtures.diagnostic()] + defp bundle_diagnostics(nil), do: [] + defp bundle_diagnostics(%Fixtures{diagnostics: diagnostics}), do: diagnostics + + # -- Merging the scenario onto the scope --------------------------------- + + @spec merge_scenario(Scope.t(), Fixtures.datamodel()) :: [Entry.t()] + defp merge_scenario(%Scope{} = scope, scenario_values) do + names = MapSet.new(scope.data, & &1.name) + + data_entries = Enum.map(scope.data, &apply_scenario_value(&1, scenario_values)) + + scenario_entries = + scenario_values + |> Enum.reject(fn {name, _value} -> MapSet.member?(names, name) end) + |> Enum.sort_by(fn {name, _value} -> name end) + |> Enum.map(&scenario_entry/1) + + data_entries ++ scope.system ++ scope.functions ++ scenario_entries + end + + @spec apply_scenario_value(Entry.t(), Fixtures.datamodel()) :: Entry.t() + defp apply_scenario_value(%Entry{name: name} = entry, scenario_values) do + case Map.fetch(scenario_values, name) do + {:ok, value} -> + shape = StatifierUI.Shape.infer(value) + %{entry | value: value, shape: shape, label: StatifierUI.Shape.label(shape)} + + :error -> + entry + end + end + + @spec scenario_entry({String.t(), term()}) :: Entry.t() + defp scenario_entry({name, value}) do + shape = StatifierUI.Shape.infer(value) + + %Entry{ + name: name, + tier: :scenario, + value: value, + shape: shape, + label: StatifierUI.Shape.label(shape) + } + end + + # -- Live mode: session resolution and truncation ------------------------ + + @spec resolve_live_session([Message.t()]) :: + {:ok, String.t() | nil} | {:error, {:mixed_sessions, [String.t()]}} + defp resolve_live_session(messages) do + case messages |> Enum.map(& &1.session) |> Enum.uniq() |> Enum.sort() do + [] -> {:ok, nil} + [session] -> {:ok, session} + ids -> {:error, {:mixed_sessions, ids}} + end + end + + @spec live_truncated?([Message.t()]) :: boolean() + defp live_truncated?(messages) do + messages + |> Enum.map(& &1.seq) + |> Enum.min() + |> Kernel.>(0) + end + + # -- Live mode: tier attribution from session.start's data table -------- + + @spec data_table([Message.t()]) :: %{optional(String.t()) => non_neg_integer()} + defp data_table(messages) do + case Enum.find(messages, &(&1.type == "session.start")) do + nil -> + %{} + + %Message{payload: payload} -> + payload + |> Map.get("data", []) + |> Map.new(fn %{"id" => id, "d_index" => d_index} -> {id, d_index} end) + end + end + + @spec classify_live_tier(String.t(), %{optional(String.t()) => non_neg_integer()}) :: + Entry.tier() + defp classify_live_tier(name, data_table) do + cond do + Map.has_key?(data_table, name) -> :data + MapSet.member?(@system_variable_names, name) -> :system + true -> :runtime + end + end + + # -- Live mode: seeding the name set from session.datamodel -------------- + + @spec seed_live_entries([Message.t()], %{optional(String.t()) => non_neg_integer()}) :: + {%{optional(String.t()) => map()}, [Fixtures.diagnostic()]} + defp seed_live_entries(messages, data_table) do + case Enum.find(messages, &(&1.type == "session.datamodel")) do + nil -> + {%{}, []} + + %Message{payload: payload} -> + {entries, diagnostics} = + payload + |> Map.get("datamodel", %{}) + |> Enum.reduce({%{}, []}, &seed_live_entry(&1, &2, data_table)) + + {entries, Enum.reverse(diagnostics)} + end + end + + @spec seed_live_entry( + {String.t(), term()}, + {%{optional(String.t()) => map()}, [Fixtures.diagnostic()]}, + %{optional(String.t()) => non_neg_integer()} + ) :: {%{optional(String.t()) => map()}, [Fixtures.diagnostic()]} + defp seed_live_entry({name, raw}, {entries, diagnostics}, data_table) do + case Value.decode(raw) do + {:ok, value} -> + {Map.put(entries, name, live_entry(name, value, data_table)), diagnostics} + + {:error, reason} -> + entry = live_entry(name, :undefined, data_table) + + diagnostic = + live_diagnostic( + :undecodable_datamodel_value, + "datamodel[#{inspect(name)}] could not be decoded: #{inspect(reason)}", + ["datamodel", name] + ) + + {Map.put(entries, name, entry), [diagnostic | diagnostics]} + end + end + + @spec live_entry(String.t(), term(), %{optional(String.t()) => non_neg_integer()}) :: map() + defp live_entry(name, value, data_table) do + %{ + name: name, + tier: classify_live_tier(name, data_table), + value: value, + d_index: Map.get(data_table, name), + location_source: nil + } + end + + @spec runtime_entry(String.t()) :: map() + defp runtime_entry(name) do + %{name: name, tier: :runtime, value: :undefined, d_index: nil, location_source: nil} + end + + # -- Live mode: applying effect.datamodel_change in stamped order -------- + + @spec live_writes([Message.t()]) :: [Message.t()] + defp live_writes(messages) do + messages + |> Enum.filter(&(&1.type == "effect.datamodel_change")) + |> Enum.sort_by(&{&1.macrostep, &1.microstep, &1.seq}) + end + + @spec apply_datamodel_write(Message.t(), map()) :: map() + defp apply_datamodel_write(%Message{payload: payload} = message, acc) do + {new_value, acc} = decode_write_value(payload, "new_value", acc) + {prior_value, acc} = decode_write_value(payload, "prior_value", acc) + + case payload["location_path"] do + [root_name | segments] when is_binary(root_name) -> + apply_resolved_write(acc, message, root_name, segments, new_value, prior_value) + + path -> + diagnostic = + live_diagnostic( + :unresolvable_location_path, + "location_path #{inspect(path)} has no binary root segment", + stringify_path(List.wrap(path)) + ) + + %{acc | diagnostics: [diagnostic | acc.diagnostics]} + end + end + + @spec decode_write_value(map(), String.t(), map()) :: {term(), map()} + defp decode_write_value(payload, key, acc) do + case Map.fetch(payload, key) do + :error -> + {:undefined, acc} + + {:ok, raw} -> + case Value.decode(raw) do + {:ok, value} -> + {value, acc} + + {:error, reason} -> + diagnostic = + live_diagnostic( + :undecodable_datamodel_value, + "#{key} at #{inspect(payload["location_path"])} could not be decoded: " <> + inspect(reason), + stringify_path(List.wrap(payload["location_path"])) ++ [key] + ) + + {:undefined, %{acc | diagnostics: [diagnostic | acc.diagnostics]}} + end + end + end + + @spec apply_resolved_write( + map(), + Message.t(), + String.t(), + [String.t() | integer()], + term(), + term() + ) :: map() + defp apply_resolved_write(acc, message, root_name, segments, new_value, prior_value) do + root_entry = Map.get(acc.entries, root_name, runtime_entry(root_name)) + + case apply_path(root_entry.value, segments, new_value) do + {:ok, updated_value} -> + updated_entry = %{ + root_entry + | value: updated_value, + location_source: message.payload["location_source"] + } + + write = %{ + name: root_name, + macrostep: message.macrostep, + prior: prior_value, + new: new_value + } + + %{ + acc + | entries: Map.put(acc.entries, root_name, updated_entry), + writes: [write | acc.writes] + } + + {:error, reason} -> + diagnostic = + live_diagnostic( + :unresolvable_location_path, + "write to #{inspect([root_name | segments])} could not be applied: " <> + inspect(reason), + stringify_path([root_name | segments]) + ) + + %{acc | diagnostics: [diagnostic | acc.diagnostics]} + end + end + + # Materializes intermediate containers along a resolved `location_path`: + # a missing or `:undefined` container becomes a map for a string segment, + # a list for an integer segment. An integer segment equal to the + # container's current length appends; beyond it, or a segment kind that + # contradicts an existing container, is `{:error, reason}` rather than a + # crash. + @spec apply_path(term(), [String.t() | integer()], term()) :: {:ok, term()} | {:error, term()} + defp apply_path(_current, [], new_value), do: {:ok, new_value} + + defp apply_path(current, [key | rest], new_value) when is_binary(key) do + with {:ok, map} <- as_map(current) do + case apply_path(Map.get(map, key, :undefined), rest, new_value) do + {:ok, updated} -> {:ok, Map.put(map, key, updated)} + error -> error + end + end + end + + defp apply_path(current, [index | rest], new_value) when is_integer(index) and index >= 0 do + with {:ok, list} <- as_list(current) do + apply_list_index(list, index, rest, new_value) + end + end + + defp apply_path(_current, [segment | _rest], _new_value) do + {:error, {:invalid_segment, segment}} + end + + @spec apply_list_index(list(), non_neg_integer(), [String.t() | integer()], term()) :: + {:ok, list()} | {:error, term()} + defp apply_list_index(list, index, rest, new_value) when index < length(list) do + case apply_path(Enum.at(list, index), rest, new_value) do + {:ok, updated} -> {:ok, List.replace_at(list, index, updated)} + error -> error + end + end + + defp apply_list_index(list, index, rest, new_value) when index == length(list) do + case apply_path(:undefined, rest, new_value) do + {:ok, updated} -> {:ok, list ++ [updated]} + error -> error + end + end + + defp apply_list_index(_list, index, _rest, _new_value) do + {:error, {:index_out_of_bounds, index}} + end + + @spec as_map(term()) :: {:ok, map()} | {:error, term()} + defp as_map(:undefined), do: {:ok, %{}} + defp as_map(map) when is_map(map), do: {:ok, map} + defp as_map(other), do: {:error, {:not_a_map, other}} + + @spec as_list(term()) :: {:ok, list()} | {:error, term()} + defp as_list(:undefined), do: {:ok, []} + defp as_list(list) when is_list(list), do: {:ok, list} + defp as_list(other), do: {:error, {:not_a_list, other}} + + # -- Live mode: marking changes at the pane's macrostep ------------------- + + @spec highest_macrostep([map()]) :: non_neg_integer() | nil + defp highest_macrostep(writes) do + case writes |> Enum.map(& &1.macrostep) |> Enum.reject(&is_nil/1) do + [] -> nil + macrosteps -> Enum.max(macrosteps) + end + end + + @spec changed_names([map()], non_neg_integer() | nil) :: MapSet.t(String.t()) + defp changed_names(_writes, nil), do: MapSet.new() + + defp changed_names(writes, macrostep) do + writes + |> Enum.filter(&(&1.macrostep == macrostep and &1.prior != &1.new)) + |> Enum.map(& &1.name) + |> MapSet.new() + end + + # -- Live mode: finishing entries and tier 2b -------------------------- + + @spec finalize_live_entries(%{optional(String.t()) => map()}, MapSet.t(String.t())) :: [ + Entry.t() + ] + defp finalize_live_entries(entries_map, changed) do + Enum.map(entries_map, fn {name, entry} -> + shape = StatifierUI.Shape.infer(entry.value) + + %Entry{ + name: name, + tier: entry.tier, + value: entry.value, + shape: shape, + label: StatifierUI.Shape.label(shape), + changed?: MapSet.member?(changed, name), + d_index: entry.d_index, + location_source: entry.location_source + } + end) + end + + # Tier 2b, read from the same resolved context `Scope` reads - a + # compile-time constant that needs no session, so both modes show the + # same function list (`Scope`'s `build_functions/0`, duplicated here + # rather than exposed publicly since it takes no machine-dependent + # input). + @spec function_entries() :: [Entry.t()] + defp function_entries do + Functions.base_context().functions + |> Enum.sort_by(fn {name, _entry} -> name end) + |> Enum.map(&live_function_entry/1) + end + + @spec live_function_entry({String.t(), {non_neg_integer() | [non_neg_integer()], atom()}}) :: + Entry.t() + defp live_function_entry({name, {arity, _impl}}) do + %Entry{ + name: name, + tier: :function, + value: :undefined, + shape: :unknown, + label: live_function_label(name, arity), + arity: arity + } + end + + @spec live_function_label(String.t(), non_neg_integer() | [non_neg_integer()]) :: String.t() + defp live_function_label(name, arity) when is_integer(arity), do: "#{name}/#{arity}" + + defp live_function_label(name, arities) when is_list(arities) do + "#{name}/#{Enum.map_join(arities, "|", &to_string/1)}" + end + + @spec order_live_entries([Entry.t()]) :: [Entry.t()] + defp order_live_entries(entries) do + Enum.flat_map(@live_tier_order, fn tier -> + entries + |> Enum.filter(&(&1.tier == tier)) + |> Enum.sort_by(& &1.name) + end) + end + + # -- Live mode: diagnostics ----------------------------------------------- + + @spec live_diagnostic(atom(), String.t(), [String.t()]) :: Fixtures.diagnostic() + defp live_diagnostic(kind, message, path) do + %{kind: kind, message: message, path: path, source: nil} + end + + @spec stringify_path([String.t() | integer()]) :: [String.t()] + defp stringify_path(path), do: Enum.map(path, &to_string/1) +end diff --git a/lib/statifier_ui/datamodel_explorer/entry.ex b/lib/statifier_ui/datamodel_explorer/entry.ex new file mode 100644 index 0000000..0b223b0 --- /dev/null +++ b/lib/statifier_ui/datamodel_explorer/entry.ex @@ -0,0 +1,67 @@ +defmodule StatifierUI.DatamodelExplorer.Entry do + @moduledoc """ + One node in the datamodel explorer tree (`sui-t36.7`): a `` + declaration, a system variable, a provider function, a fixture scenario + value, or a runtime-only location - the ADR-0003 tiers, plus live mode's + `:runtime` sibling for a location `session.datamodel` never named. + + Struct and types only - the way `StatifierUI.EventLog.Round` and + `StatifierUI.EventInjection.Entry` are - because both + `StatifierUI.DatamodelExplorer.Scope` (authoring and live alike) and + `sui-t36.8`'s renderer share this one shape. + """ + + @typedoc """ + Which of ADR-0003's tiers this entry belongs to. `:data` and `:runtime` + are both tier 1 - a document `` declaration and, in live mode + only, a location written at runtime that `session.datamodel`'s snapshot + did not name (an `` or an empty `` + auto-assign). `:system` is tier 2a (spec 5.10.1's system variables), + `:function` is tier 2b (predicator provider functions in scope), and + `:scenario` is tier 3 - the one tier that switches source between modes. + """ + @type tier :: :data | :system | :function | :scenario | :runtime + + @typedoc """ + * `value` - always a **decoded** Elixir value, never a `$`-tagged wire + term. This is the invariant that lets one `StatifierUI.Shape.infer/1` + pass serve both authoring and live mode. + * `changed?` - live-mode only; defaults to `false`. Authoring mode never + sets it. + * `arity` - set on `:function` entries only, predicator's + `non_neg_integer() | [non_neg_integer()]` multi-arity form verbatim. + * `declared_source` - the tier-1 declared expression text, display only. + Present only when the pane was given the chart source and + `value_location != location` (`docs/wire-format.md:335-348`). + * `location_source` - `effect.datamodel_change`'s raw author string, + live mode only, display only. + """ + @type t :: %__MODULE__{ + name: String.t(), + tier: tier(), + value: term(), + shape: StatifierUI.Shape.t(), + label: String.t(), + changed?: boolean(), + children: [t()], + d_index: non_neg_integer() | nil, + arity: non_neg_integer() | [non_neg_integer()] | nil, + declared_source: String.t() | nil, + location_source: String.t() | nil + } + + @enforce_keys [:name, :tier, :value, :shape, :label] + defstruct [ + :name, + :tier, + :value, + :shape, + :label, + :d_index, + :arity, + :declared_source, + :location_source, + changed?: false, + children: [] + ] +end diff --git a/lib/statifier_ui/datamodel_explorer/markdown.ex b/lib/statifier_ui/datamodel_explorer/markdown.ex new file mode 100644 index 0000000..3040e04 --- /dev/null +++ b/lib/statifier_ui/datamodel_explorer/markdown.ex @@ -0,0 +1,165 @@ +defmodule StatifierUI.DatamodelExplorer.Markdown do + @moduledoc """ + Renders a `StatifierUI.DatamodelExplorer.t()` as Markdown a host hands to + `Kino.Markdown.new/1`, per the `sui-t36.7` plan's Phase 4. + + `render/2` is a pure function over the pane - it names `Kino` only in this + moduledoc and calls nothing under `Kino.*`. `sui-t36.8` owns wrapping the + returned string in an actual `Kino.Markdown` widget. + + ## Structure + + A header line naming the mode and either the scenario (authoring) or the + session and macrostep (live), a drop warning when `truncated?`, then one + section per tier **in the pane's tier order** - `entries/1` already + produces its entries grouped contiguously by tier, so this module chunks + on that grouping rather than re-deriving or re-sorting it (ADR-0011). Each + section is a Markdown table of name, type label, and value; a `changed?` + entry's name carries the `:changed_marker` suffix (default `"*"`), and + each `Entry.children` element renders as its own row, qualified as + `"parent.child"`. A diagnostics section renders last, only when the pane + carries any. + + Values render with `inspect/1` rather than re-encoded to JSON: this is a + debugging view of decoded Elixir terms, and `StatifierUI.Trace.Json` is + for the wire. + """ + + alias StatifierUI.DatamodelExplorer + alias StatifierUI.DatamodelExplorer.Entry + alias StatifierUI.Fixtures + alias StatifierUI.Shape + + @type opt :: + {:tiers, [Entry.tier()]} + | {:collapsible, boolean()} + | {:changed_marker, String.t()} + | {:max_keys, pos_integer()} + | {:max_depth, pos_integer()} + + @doc """ + Renders `pane` as a Markdown string. + + Options: + + * `:tiers` - when given, only entries whose tier is in this list get a + section; the other tiers' sections are omitted entirely rather than + rendered empty. + * `:collapsible` - wraps each tier section in `
`/`` + when `true`. Defaults to `false` - a plain `### ` heading. + * `:changed_marker` - the suffix appended to a `changed?` entry's name. + Defaults to `"*"`. + * `:max_keys`, `:max_depth` - passed straight through to + `StatifierUI.Shape.label/2` for every entry's type label, so a value's + inferred shape can be rendered at a coarser grain than the entry was + built with. + """ + @spec render(DatamodelExplorer.t(), [opt()]) :: String.t() + def render(%DatamodelExplorer{} = pane, opts \\ []) do + changed_marker = Keyword.get(opts, :changed_marker, "*") + collapsible? = Keyword.get(opts, :collapsible, false) + shape_opts = Keyword.take(opts, [:max_keys, :max_depth]) + + entries = filtered_entries(pane, Keyword.get(opts, :tiers)) + + ([header_block(pane)] ++ + tier_blocks(entries, changed_marker, shape_opts, collapsible?) ++ + diagnostics_block(pane)) + |> Enum.reject(&(&1 == "")) + |> Enum.join("\n\n") + end + + # -- Entry selection ------------------------------------------------------ + + @spec filtered_entries(DatamodelExplorer.t(), [Entry.tier()] | nil) :: [Entry.t()] + defp filtered_entries(pane, nil), do: DatamodelExplorer.entries(pane) + + defp filtered_entries(pane, tiers) do + Enum.filter(DatamodelExplorer.entries(pane), &(&1.tier in tiers)) + end + + # -- Header and truncation warning ---------------------------------------- + + @spec header_block(DatamodelExplorer.t()) :: String.t() + defp header_block(%DatamodelExplorer{mode: :authoring} = pane) do + lines = ["# Datamodel: authoring (scenario: #{pane.scenario || "(none)"})"] + Enum.join(lines ++ truncation_lines(pane), "\n") + end + + defp header_block(%DatamodelExplorer{mode: :live} = pane) do + lines = [ + "# Datamodel: live (session: #{pane.session || "(no session)"}, " <> + "macrostep: #{macrostep_text(pane.macrostep)})" + ] + + Enum.join(lines ++ truncation_lines(pane), "\n") + end + + @spec macrostep_text(non_neg_integer() | nil) :: String.t() + defp macrostep_text(nil), do: "(none)" + defp macrostep_text(macrostep), do: to_string(macrostep) + + @spec truncation_lines(DatamodelExplorer.t()) :: [String.t()] + defp truncation_lines(%DatamodelExplorer{truncated?: true}) do + ["Earliest entries dropped; this fold does not start at the session's beginning."] + end + + defp truncation_lines(%DatamodelExplorer{truncated?: false}), do: [] + + # -- Tier sections --------------------------------------------------------- + + @spec tier_blocks([Entry.t()], String.t(), keyword(), boolean()) :: [String.t()] + defp tier_blocks(entries, changed_marker, shape_opts, collapsible?) do + entries + |> Enum.chunk_by(& &1.tier) + |> Enum.map(&tier_block(&1, changed_marker, shape_opts, collapsible?)) + end + + @spec tier_block([Entry.t()], String.t(), keyword(), boolean()) :: String.t() + defp tier_block([%Entry{tier: tier} | _] = entries, changed_marker, shape_opts, true) do + body = tier_table_lines(entries, changed_marker, shape_opts) + + Enum.join( + ["
", "#{tier}", ""] ++ body ++ ["
"], + "\n" + ) + end + + defp tier_block([%Entry{tier: tier} | _] = entries, changed_marker, shape_opts, false) do + body = tier_table_lines(entries, changed_marker, shape_opts) + Enum.join(["### #{tier}", ""] ++ body, "\n") + end + + @spec tier_table_lines([Entry.t()], String.t(), keyword()) :: [String.t()] + defp tier_table_lines(entries, changed_marker, shape_opts) do + header = ["| name | type | value |", "| --- | --- | --- |"] + header ++ Enum.flat_map(entries, &entry_rows(&1, nil, changed_marker, shape_opts)) + end + + @spec entry_rows(Entry.t(), String.t() | nil, String.t(), keyword()) :: [String.t()] + defp entry_rows(%Entry{} = entry, prefix, changed_marker, shape_opts) do + name = qualified_name(prefix, entry.name) + marked_name = if entry.changed?, do: name <> changed_marker, else: name + type_label = Shape.label(entry.shape, shape_opts) + row = "| #{marked_name} | #{type_label} | #{inspect(entry.value)} |" + + [row | Enum.flat_map(entry.children, &entry_rows(&1, name, changed_marker, shape_opts))] + end + + @spec qualified_name(String.t() | nil, String.t()) :: String.t() + defp qualified_name(nil, name), do: name + defp qualified_name(prefix, name), do: "#{prefix}.#{name}" + + # -- Diagnostics ----------------------------------------------------------- + + @spec diagnostics_block(DatamodelExplorer.t()) :: [String.t()] + defp diagnostics_block(%DatamodelExplorer{diagnostics: []}), do: [] + + defp diagnostics_block(%DatamodelExplorer{diagnostics: diagnostics}) do + lines = ["### Diagnostics" | Enum.map(diagnostics, &diagnostic_line/1)] + [Enum.join(lines, "\n")] + end + + @spec diagnostic_line(Fixtures.diagnostic()) :: String.t() + defp diagnostic_line(%{kind: kind, message: message}), do: "- #{kind}: #{message}" +end diff --git a/lib/statifier_ui/datamodel_explorer/scope.ex b/lib/statifier_ui/datamodel_explorer/scope.ex new file mode 100644 index 0000000..da5edce --- /dev/null +++ b/lib/statifier_ui/datamodel_explorer/scope.ex @@ -0,0 +1,244 @@ +defmodule StatifierUI.DatamodelExplorer.Scope do + @moduledoc """ + The three ADR-0003 tiers that do not switch source between authoring and + live mode: document `` declarations (tier 1), the SCXML 5.10.1 + system variables (tier 2a), and the predicator provider functions in + scope (tier 2b). All three come from a compiled `%Statifier.Machine{}` + and two engine constants - no session, no fixtures, no wire messages. + + ## Tier 1 - `` declarations + + Enumerated with `machine.data_elements |> Tuple.to_list()`, the idiom + `StatifierUI.Trace.Manifest` already uses for the same field + (`trace/manifest.ex:193-201`), in `d_index` order - never re-sorted by + id (ADR-0011's rule applied to an engine-ordered sequence). A tier-1 + entry has no runtime value here: predicator is non-evaluative + (ADR-0004 upstream, adopted by ADR-0002), so `value` is always + `:undefined` and `declared_source` carries the declared expression + *text*, sliced from the `:source` option, only when one was given and + `value_location` differs from `location` + (`docs/wire-format.md:335-348`'s guard: when the two are equal there is + no value span and the slice would be the whole element). + + ## Tier 2a - system variables + + `Statifier.Evaluator.SystemVariables.initial/2` is the single source of + the name list: exactly `_sessionid`, `_name`, `_event`, `_ioprocessors`. + **`_x` is deliberately absent** - the evaluator seeds it nowhere + (`deps/statifier/lib/statifier/evaluator.ex:359-361`) - and `In(stateId)` + is a tier-2b *function*, not a tier-2a datamodel key, even though both + are things a reader of spec 5.10/5.9.1 will look for here. `_event` + additionally gets `children`: one entry per key of + `SystemVariables.event/1`'s schema (spec 5.10.1's six fields plus + `data`), so the key list cannot drift from the engine's own. The + session id comes from the `:session_id` option, defaulting to the + documented placeholder `"(authoring)"` - there is no session in + authoring mode, and the placeholder says so honestly rather than + hiding it. + + ## Tier 2b - provider functions + + Read from `Statifier.Evaluator.Functions.base_context().functions`, the + fully *resolved* `%{name => {arity, {module, atom}}}` map, rather than a + walk over `Predicator.FunctionProvider.builtin_providers/0`. The + resolved context is the only source that reflects predicator's + builtins-then-providers-then-`:functions` shadowing order + (`deps/predicator/lib/predicator/context.ex:216-237`); a provider-module + walk can report a shadowed function as live. ADR-0003's `functions/0` + phrasing is satisfied transitively, since that callback is what the + resolution already consumed. Provider metadata is name and arity only - + no return type, no parameter names - so a function entry's `label` is + `"name/arity"` (or `"name/2|3"` for the multi-arity form) and nothing + richer; ADR-0003:110-116 names richer metadata as upstream `px-`/`st-` + work, not something to synthesize here. + + ## Diagnostics, not failures + + A tier-1 element whose `value_location` offsets fall outside the given + `:source` string yields an entry with `declared_source: nil` plus one + `StatifierUI.Fixtures.diagnostic()` (`kind: :unsliceable_declared_source`) + - one bad span takes down one label, not the pane, following + `StatifierUI.EventInjection.Palette`'s precedent + (`event_injection/palette.ex:97-126`). + """ + + alias Statifier.Evaluator.Functions + alias Statifier.Evaluator.SystemVariables + alias Statifier.Machine + alias Statifier.Machine.Data + alias Statifier.Parser.Location + alias StatifierUI.DatamodelExplorer.Entry + alias StatifierUI.Fixtures + alias StatifierUI.Shape + + @type opt :: {:source, String.t() | nil} | {:session_id, String.t()} + + @type t :: %__MODULE__{ + data: [Entry.t()], + system: [Entry.t()], + functions: [Entry.t()], + diagnostics: [Fixtures.diagnostic()] + } + + defstruct data: [], system: [], functions: [], diagnostics: [] + + @doc """ + Builds the mode-independent tiers from a compiled `machine`. + + `:source` (default `nil`) is the chart's raw SCXML text, used only to + slice tier-1 declared-value spans for display; `:session_id` (default + `"(authoring)"`) seeds tier 2a's `_sessionid` and `_ioprocessors`. + Returns `{:error, {:invalid_machine, other}}` for anything that is not + a `%Statifier.Machine{}`. + """ + @spec build(Statifier.Machine.t(), [opt()]) :: {:ok, t()} | {:error, term()} + def build(machine, opts \\ []) do + case machine do + %Machine{} -> + source = Keyword.get(opts, :source) + session_id = Keyword.get(opts, :session_id, "(authoring)") + {data_entries, diagnostics} = build_data(machine, source) + + {:ok, + %__MODULE__{ + data: data_entries, + system: build_system(machine, session_id), + functions: build_functions(), + diagnostics: diagnostics + }} + + other -> + {:error, {:invalid_machine, other}} + end + end + + # -- Tier 1: declarations --------------------------------------- + + defp build_data(%Machine{data_elements: data_elements}, source) do + {entries, diagnostics} = + data_elements + |> Tuple.to_list() + |> Enum.map_reduce([], fn element, diagnostics -> + case build_data_entry(element, source) do + {:ok, entry} -> {entry, diagnostics} + {:ok, entry, diagnostic} -> {entry, [diagnostic | diagnostics]} + end + end) + + {entries, Enum.reverse(diagnostics)} + end + + defp build_data_entry(%Data{} = element, source) do + case declared_source(element, source) do + {:ok, declared} -> {:ok, data_entry(element, declared)} + :error -> {:ok, data_entry(element, nil), unsliceable_diagnostic(element)} + end + end + + defp data_entry(%Data{} = element, declared_source) do + %Entry{ + name: element.id, + tier: :data, + value: :undefined, + shape: :undefined, + label: Shape.label(:undefined), + d_index: element.d_index, + declared_source: declared_source + } + end + + defp declared_source(_element, nil), do: {:ok, nil} + + defp declared_source(%Data{value_location: same, location: same}, _source), do: {:ok, nil} + + defp declared_source(%Data{value_location: value_location}, source) do + if within_source?(value_location, source) do + {:ok, Location.slice(value_location, source)} + else + :error + end + end + + defp within_source?(%Location{start_offset: start_offset, end_offset: end_offset}, source) do + start_offset >= 0 and end_offset >= start_offset and end_offset <= byte_size(source) + end + + defp unsliceable_diagnostic(%Data{id: id}) do + %{ + kind: :unsliceable_declared_source, + message: + "data #{inspect(id)}'s declared-value span falls outside the given source; " <> + "declared_source is nil for this entry", + path: ["data", id], + source: nil + } + end + + # -- Tier 2a: system variables ------------------------------------------ + + defp build_system(%Machine{} = machine, session_id) do + machine + |> SystemVariables.initial(session_id) + |> Enum.sort_by(fn {name, _value} -> name end) + |> Enum.map(&build_system_entry/1) + end + + defp build_system_entry({"_event" = name, value}) do + system_entry(name, value, children: build_event_children()) + end + + defp build_system_entry({name, value}) do + system_entry(name, value, []) + end + + defp system_entry(name, value, extra) do + shape = Shape.infer(value) + + struct!( + Entry, + [name: name, tier: :system, value: value, shape: shape, label: Shape.label(shape)] ++ extra + ) + end + + defp build_event_children do + "(none)" + |> Statifier.Event.external() + |> SystemVariables.event() + |> Map.keys() + |> Enum.sort() + |> Enum.map(fn key -> + %Entry{ + name: key, + tier: :system, + value: :undefined, + shape: :undefined, + label: Shape.label(:undefined) + } + end) + end + + # -- Tier 2b: provider functions ----------------------------------------- + + defp build_functions do + Functions.base_context().functions + |> Enum.sort_by(fn {name, _entry} -> name end) + |> Enum.map(&build_function_entry/1) + end + + defp build_function_entry({name, {arity, _impl}}) do + %Entry{ + name: name, + tier: :function, + value: :undefined, + shape: :unknown, + label: function_label(name, arity), + arity: arity + } + end + + defp function_label(name, arity) when is_integer(arity), do: "#{name}/#{arity}" + + defp function_label(name, arities) when is_list(arities) do + "#{name}/#{Enum.map_join(arities, "|", &to_string/1)}" + end +end diff --git a/test/statifier_ui/datamodel_explorer/live_test.exs b/test/statifier_ui/datamodel_explorer/live_test.exs new file mode 100644 index 0000000..adf7c60 --- /dev/null +++ b/test/statifier_ui/datamodel_explorer/live_test.exs @@ -0,0 +1,406 @@ +defmodule StatifierUI.DatamodelExplorer.LiveTest do + use ExUnit.Case, async: true + + alias StatifierUI.DatamodelExplorer + alias StatifierUI.DatamodelExplorer.Entry + alias StatifierUI.Trace.Message + + # Transcribed from `docs/wire-format.md:737-762` (`session.datamodel`) and + # `:629-681` (`effect.datamodel_change`), the `event_log_test.exs` + # `@worked_example` idiom. A chart with a document `` + # (tier 1, bound during the binding fold), the four spec 5.10 system + # variables (tier 2a, seeded at session start), and one runtime-only write + # (`user`, never named by the snapshot). + @session_start %Message{ + type: "session.start", + session: "sess_dm", + seq: 0, + payload: %{ + "version" => 1, + "data" => [%{"d_index" => 0, "id" => "count", "location" => %{}}] + } + } + + @session_datamodel %Message{ + type: "session.datamodel", + session: "sess_dm", + seq: 1, + payload: %{ + "datamodel" => %{ + "count" => %{"$undefined" => true}, + "_sessionid" => "sess_dm", + "_name" => %{"$undefined" => true}, + "_event" => %{"$undefined" => true}, + "_ioprocessors" => %{ + "http://www.w3.org/TR/scxml/#SCXMLEventProcessor" => %{ + "location" => "#_scxml_sess_dm" + } + } + } + } + } + + @count_bound %Message{ + type: "effect.datamodel_change", + session: "sess_dm", + seq: 2, + macrostep: 0, + microstep: 0, + payload: %{ + "location_path" => ["count"], + "location_source" => "count", + "new_value" => 42, + "d_index" => 0 + } + } + + @worked_example [@session_start, @session_datamodel, @count_bound] + + describe "build_live/1 - the session.datamodel snapshot alone" do + test "the tier-1 entry is :undefined, none changed?, data names sorted" do + assert {:ok, pane} = DatamodelExplorer.build_live([@session_start, @session_datamodel]) + + assert %DatamodelExplorer{mode: :live, session: "sess_dm", macrostep: nil} = pane + # `_sessionid` and `_ioprocessors` are the two system variables bound + # at init (`docs/wire-format.md:746-750`); every tier-1 + # element still reads :undefined, since the snapshot precedes the + # binding fold. + assert Enum.all?(DatamodelExplorer.entries(pane, :data), &(&1.value == :undefined)) + assert Enum.all?(pane.entries, &(&1.changed? == false)) + + names = pane |> DatamodelExplorer.entries(:system) |> Enum.map(& &1.name) + assert names == Enum.sort(names) + end + + test "a {\"$undefined\": true} snapshot value infers :undefined, not a map shape" do + assert {:ok, pane} = DatamodelExplorer.build_live([@session_start, @session_datamodel]) + + [count] = Enum.filter(pane.entries, &(&1.name == "count")) + assert count.shape == :undefined + refute match?({:map, %{"$undefined" => :boolean}}, count.shape) + end + + test "tier attribution: count is :data, system names are :system" do + assert {:ok, pane} = DatamodelExplorer.build_live([@session_start, @session_datamodel]) + + assert [%Entry{tier: :data, d_index: 0}] = + Enum.filter(pane.entries, &(&1.name == "count")) + + assert [%Entry{tier: :system}] = + Enum.filter(pane.entries, &(&1.name == "_sessionid")) + end + end + + describe "build_live/1 - one binding write" do + test "value present, changed? true, macrostep set" do + assert {:ok, pane} = DatamodelExplorer.build_live(@worked_example) + + assert pane.macrostep == 0 + [count] = Enum.filter(pane.entries, &(&1.name == "count")) + assert count.value == 42 + assert count.changed? == true + assert count.tier == :data + end + end + + describe "build_live/1 - new_value absent (the unbinding case)" do + test "the entry becomes :undefined and is marked changed?" do + unbind = %Message{ + @count_bound + | payload: %{ + "location_path" => ["count"], + "location_source" => "count", + "prior_value" => 42, + "d_index" => 0 + } + } + + assert {:ok, pane} = + DatamodelExplorer.build_live([@session_start, @session_datamodel, unbind]) + + [count] = Enum.filter(pane.entries, &(&1.name == "count")) + assert count.value == :undefined + assert count.changed? == true + end + end + + describe "build_live/1 - new_value null" do + test "the entry becomes nil, distinctly from the absent case" do + nulled = %Message{ + @count_bound + | payload: %{ + "location_path" => ["count"], + "location_source" => "count", + "new_value" => nil, + "d_index" => 0 + } + } + + assert {:ok, pane} = + DatamodelExplorer.build_live([@session_start, @session_datamodel, nulled]) + + [count] = Enum.filter(pane.entries, &(&1.name == "count")) + assert count.value == nil + assert count.changed? == true + end + end + + describe "build_live/1 - two macrosteps of writes" do + test "only the later macrostep's entries are changed?" do + second_write = %Message{ + type: "effect.datamodel_change", + session: "sess_dm", + seq: 3, + macrostep: 1, + microstep: 0, + payload: %{ + "location_path" => ["count"], + "location_source" => "count", + "new_value" => 43, + "prior_value" => 42, + "d_index" => 0 + } + } + + assert {:ok, pane} = + DatamodelExplorer.build_live(@worked_example ++ [second_write]) + + assert pane.macrostep == 1 + [count] = Enum.filter(pane.entries, &(&1.name == "count")) + assert count.value == 43 + assert count.changed? == true + end + + test "an entry only written at the earlier macrostep is not changed?" do + # A second variable, "other", is written only at macrostep 0; "count" + # is written again at macrostep 1. Only "count" should read changed?. + other_write = %Message{ + type: "effect.datamodel_change", + session: "sess_dm", + seq: 3, + macrostep: 0, + microstep: 1, + payload: %{ + "location_path" => ["other"], + "location_source" => "other", + "new_value" => "x" + } + } + + later_count_write = %Message{ + type: "effect.datamodel_change", + session: "sess_dm", + seq: 4, + macrostep: 1, + microstep: 0, + payload: %{ + "location_path" => ["count"], + "location_source" => "count", + "new_value" => 99, + "prior_value" => 42, + "d_index" => 0 + } + } + + assert {:ok, pane} = + DatamodelExplorer.build_live(@worked_example ++ [other_write, later_count_write]) + + assert pane.macrostep == 1 + [other] = Enum.filter(pane.entries, &(&1.name == "other")) + [count] = Enum.filter(pane.entries, &(&1.name == "count")) + assert other.changed? == false + assert other.value == "x" + assert count.changed? == true + end + end + + describe "build_live/1 - a deep location_path into a $undefined root" do + test "containers are materialized, final value sits at the leaf" do + deep_write = %Message{ + type: "effect.datamodel_change", + session: "sess_dm", + seq: 2, + macrostep: 0, + microstep: 0, + payload: %{ + "location_path" => ["user", "items", 0, "name"], + "location_source" => "user.items[0].name", + "new_value" => "Alice" + } + } + + assert {:ok, pane} = + DatamodelExplorer.build_live([@session_start, @session_datamodel, deep_write]) + + [user] = Enum.filter(pane.entries, &(&1.name == "user")) + assert user.value == %{"items" => [%{"name" => "Alice"}]} + assert user.tier == :runtime + end + end + + describe "build_live/1 - malformed writes" do + test "an integer segment into a map yields one diagnostic, siblings intact" do + # "user" is first written as a map (root-level write), then a second + # write tries to index into it as though it were a list. + root_write = %Message{ + type: "effect.datamodel_change", + session: "sess_dm", + seq: 2, + macrostep: 0, + microstep: 0, + payload: %{ + "location_path" => ["user"], + "location_source" => "user", + "new_value" => %{"name" => "Alice"} + } + } + + bad_write = %Message{ + type: "effect.datamodel_change", + session: "sess_dm", + seq: 3, + macrostep: 0, + microstep: 1, + payload: %{ + "location_path" => ["user", 0], + "location_source" => "user[0]", + "new_value" => "oops" + } + } + + assert {:ok, pane} = + DatamodelExplorer.build_live([ + @session_start, + @session_datamodel, + root_write, + bad_write, + @count_bound + ]) + + assert [%{kind: :unresolvable_location_path}] = pane.diagnostics + [user] = Enum.filter(pane.entries, &(&1.name == "user")) + assert user.value == %{"name" => "Alice"} + [count] = Enum.filter(pane.entries, &(&1.name == "count")) + assert count.value == 42 + end + + test "a string segment into a list yields one diagnostic, siblings intact" do + root_write = %Message{ + type: "effect.datamodel_change", + session: "sess_dm", + seq: 2, + macrostep: 0, + microstep: 0, + payload: %{ + "location_path" => ["items"], + "location_source" => "items", + "new_value" => [1, 2] + } + } + + bad_write = %Message{ + type: "effect.datamodel_change", + session: "sess_dm", + seq: 3, + macrostep: 0, + microstep: 1, + payload: %{ + "location_path" => ["items", "name"], + "location_source" => "items.name", + "new_value" => "oops" + } + } + + assert {:ok, pane} = + DatamodelExplorer.build_live([ + @session_start, + @session_datamodel, + root_write, + bad_write, + @count_bound + ]) + + assert [%{kind: :unresolvable_location_path}] = pane.diagnostics + [items] = Enum.filter(pane.entries, &(&1.name == "items")) + assert items.value == [1, 2] + [count] = Enum.filter(pane.entries, &(&1.name == "count")) + assert count.value == 42 + end + end + + describe "build_live/1 - a name the snapshot never carried" do + test "gets tier: :runtime" do + invoke_write = %Message{ + type: "effect.datamodel_change", + session: "sess_dm", + seq: 2, + macrostep: 0, + microstep: 0, + payload: %{ + "location_path" => ["result"], + "location_source" => "result", + "new_value" => "ok", + "owner" => %{"kind" => "invoke", "state_index" => 0, "invoke_index" => 0} + } + } + + assert {:ok, pane} = + DatamodelExplorer.build_live([@session_start, @session_datamodel, invoke_write]) + + assert [%Entry{tier: :runtime, value: "ok"}] = + Enum.filter(pane.entries, &(&1.name == "result")) + end + end + + describe "build_live/1 - shuffle invariance" do + test "five shuffles of the same messages fold to identical structs" do + {:ok, expected} = DatamodelExplorer.build_live(@worked_example) + + for seed <- 1..5 do + :rand.seed(:exsss, {seed, seed, seed}) + shuffled = Enum.shuffle(@worked_example) + assert {:ok, ^expected} = DatamodelExplorer.build_live(shuffled) + end + end + end + + describe "build_live/1 - mixed sessions" do + test "returns {:error, {:mixed_sessions, ids}} rather than merging timelines" do + other = %Message{@session_start | session: "sess_other"} + + assert {:error, {:mixed_sessions, ["sess_dm", "sess_other"]}} = + DatamodelExplorer.build_live([@session_start, other]) + end + end + + describe "build_live/1 - truncated?" do + test "true when the lowest seq is greater than 0" do + dropped_head = %Message{@session_datamodel | seq: 5} + + assert {:ok, pane} = DatamodelExplorer.build_live([dropped_head]) + assert pane.truncated? == true + end + + test "false when seq starts at 0" do + assert {:ok, pane} = DatamodelExplorer.build_live(@worked_example) + assert pane.truncated? == false + end + end + + describe "build_live/1 - empty input" do + test "returns an empty :live pane" do + assert {:ok, %DatamodelExplorer{mode: :live, session: nil, entries: []}} = + DatamodelExplorer.build_live([]) + end + end + + describe "build_live/1 - tier 2b" do + test "provider functions appear unchanged from Scope's tier-2b source" do + assert {:ok, pane} = DatamodelExplorer.build_live(@worked_example) + + functions = DatamodelExplorer.entries(pane, :function) + refute functions == [] + assert Enum.all?(functions, &(&1.tier == :function)) + end + end +end diff --git a/test/statifier_ui/datamodel_explorer/markdown_test.exs b/test/statifier_ui/datamodel_explorer/markdown_test.exs new file mode 100644 index 0000000..fece703 --- /dev/null +++ b/test/statifier_ui/datamodel_explorer/markdown_test.exs @@ -0,0 +1,218 @@ +defmodule StatifierUI.DatamodelExplorer.MarkdownTest do + use ExUnit.Case, async: true + + alias StatifierUI.DatamodelExplorer + alias StatifierUI.DatamodelExplorer.Markdown + alias StatifierUI.Fixtures + alias StatifierUI.Test.Support.Trace.SessionCase + alias StatifierUI.Trace.Message + + @chart """ + + + + + + + + """ + + # The `sui-t36.7` live_test.exs `@worked_example` idiom, transcribed the + # same way from `docs/wire-format.md:737-762` and `:629-681`: a + # `session.start` naming one ``, its `session.datamodel` + # snapshot, and one binding write. + @session_start %Message{ + type: "session.start", + session: "sess_md", + seq: 0, + payload: %{ + "version" => 1, + "data" => [%{"d_index" => 0, "id" => "count", "location" => %{}}] + } + } + + @session_datamodel %Message{ + type: "session.datamodel", + session: "sess_md", + seq: 1, + payload: %{ + "datamodel" => %{ + "count" => %{"$undefined" => true}, + "_sessionid" => "sess_md", + "_name" => %{"$undefined" => true}, + "_event" => %{"$undefined" => true}, + "_ioprocessors" => %{ + "http://www.w3.org/TR/scxml/#SCXMLEventProcessor" => %{"location" => "#_scxml_sess_md"} + } + } + } + } + + @count_bound %Message{ + type: "effect.datamodel_change", + session: "sess_md", + seq: 2, + macrostep: 0, + microstep: 0, + payload: %{ + "location_path" => ["count"], + "location_source" => "count", + "new_value" => 42, + "d_index" => 0 + } + } + + @worked_example [@session_start, @session_datamodel, @count_bound] + + @spec rendered_lines(DatamodelExplorer.t(), keyword()) :: [String.t()] + defp rendered_lines(pane, opts \\ []) do + pane + |> Markdown.render(opts) + |> String.split("\n") + |> Enum.map(&String.trim/1) + end + + describe "render/2 - authoring mode" do + test "names the selected scenario" do + machine = SessionCase.compile!(@chart) + {:ok, fixtures} = Fixtures.new(scenarios: %{"gold" => %{"tier" => "gold"}}) + {:ok, pane} = DatamodelExplorer.build_authoring(machine, fixtures) + + lines = rendered_lines(pane) + + assert "# Datamodel: authoring (scenario: gold)" in lines + assert Enum.any?(lines, &(&1 == "| tier | string | \"gold\" |")) + end + + test "no fixtures names no scenario" do + machine = SessionCase.compile!(@chart) + {:ok, pane} = DatamodelExplorer.build_authoring(machine, nil) + + lines = rendered_lines(pane) + + assert "# Datamodel: authoring (scenario: (none))" in lines + end + end + + describe "render/2 - live mode" do + test "names the session and macrostep" do + {:ok, pane} = DatamodelExplorer.build_live(@worked_example) + + lines = rendered_lines(pane) + + assert "# Datamodel: live (session: sess_md, macrostep: 0)" in lines + end + + test "no writes yet names no macrostep" do + {:ok, pane} = DatamodelExplorer.build_live([@session_start, @session_datamodel]) + + lines = rendered_lines(pane) + + assert "# Datamodel: live (session: sess_md, macrostep: (none))" in lines + end + end + + describe "render/2 - the changed marker" do + test "a changed? entry's name carries the marker, an unchanged one does not" do + {:ok, pane} = DatamodelExplorer.build_live(@worked_example) + + lines = rendered_lines(pane) + + assert "| count* | integer | 42 |" in lines + refute Enum.any?(lines, &String.starts_with?(&1, "| count |")) + assert "| _sessionid | string | \"sess_md\" |" in lines + end + + test "a custom :changed_marker replaces the default" do + {:ok, pane} = DatamodelExplorer.build_live(@worked_example) + + lines = rendered_lines(pane, changed_marker: " (changed)") + + assert "| count (changed) | integer | 42 |" in lines + end + end + + describe "render/2 - :tiers filtering" do + test "omits a section entirely rather than rendering it empty" do + {:ok, pane} = DatamodelExplorer.build_live(@worked_example) + + lines = rendered_lines(pane, tiers: [:system]) + + refute Enum.any?(lines, &(&1 == "### data")) + assert "### system" in lines + refute Enum.any?(lines, &(&1 == "### function")) + end + end + + describe "render/2 - truncated?" do + test "produces the drop-warning line" do + dropped_head = %Message{@session_datamodel | seq: 5} + {:ok, pane} = DatamodelExplorer.build_live([dropped_head]) + + lines = rendered_lines(pane) + + assert pane.truncated? == true + + assert "Earliest entries dropped; this fold does not start at the session's beginning." in lines + end + + test "a complete fold renders no drop warning" do + {:ok, pane} = DatamodelExplorer.build_live(@worked_example) + + lines = rendered_lines(pane) + + refute Enum.any?(lines, &String.contains?(&1, "dropped")) + end + end + + describe "render/2 - diagnostics" do + test "a diagnostics section appears only when the pane carries diagnostics" do + bad_write = %Message{ + type: "effect.datamodel_change", + session: "sess_md", + seq: 3, + macrostep: 0, + microstep: 1, + payload: %{ + "location_path" => ["count", "nested"], + "location_source" => "count.nested", + "new_value" => "oops" + } + } + + {:ok, with_diagnostics} = DatamodelExplorer.build_live(@worked_example ++ [bad_write]) + {:ok, without_diagnostics} = DatamodelExplorer.build_live(@worked_example) + + with_lines = rendered_lines(with_diagnostics) + without_lines = rendered_lines(without_diagnostics) + + assert "### Diagnostics" in with_lines + assert Enum.any?(with_lines, &String.starts_with?(&1, "- unresolvable_location_path:")) + refute "### Diagnostics" in without_lines + end + end + + describe "render/2 - :max_keys pass-through" do + test "max_keys: 1 truncates a map label" do + machine = SessionCase.compile!(@chart) + + {:ok, fixtures} = + Fixtures.new(scenarios: %{"gold" => %{"tier" => %{"a" => 1, "b" => 2}}}) + + {:ok, pane} = DatamodelExplorer.build_authoring(machine, fixtures) + + default_lines = rendered_lines(pane) + truncated_lines = rendered_lines(pane, max_keys: 1) + + assert Enum.any?( + default_lines, + &(&1 == "| tier | map{a: integer, b: integer} | %{\"a\" => 1, \"b\" => 2} |") + ) + + assert Enum.any?( + truncated_lines, + &(&1 == "| tier | map{a: integer, ...} | %{\"a\" => 1, \"b\" => 2} |") + ) + end + end +end diff --git a/test/statifier_ui/datamodel_explorer/scope_test.exs b/test/statifier_ui/datamodel_explorer/scope_test.exs new file mode 100644 index 0000000..2d45b23 --- /dev/null +++ b/test/statifier_ui/datamodel_explorer/scope_test.exs @@ -0,0 +1,139 @@ +defmodule StatifierUI.DatamodelExplorer.ScopeTest do + use ExUnit.Case, async: true + + alias Statifier.Evaluator.SystemVariables + alias StatifierUI.DatamodelExplorer.Scope + alias StatifierUI.Test.Support.Trace.SessionCase + + # Three elements - one bare, one with an `expr`, one bare again - + # so d_index order, declared-source slicing, and the bare-element guard + # (`value_location == location`) are all exercised on one chart. No + # `name` attribute on ``, so tier 2a's `_name` infers `:undefined`. + @chart """ + + + + + + + + + + """ + + setup do + {:ok, machine: SessionCase.compile!(@chart)} + end + + describe "build/2 - tier 1 ( declarations)" do + test "entries come back in d_index order, all :undefined, with d_index set", %{ + machine: machine + } do + assert {:ok, %Scope{data: entries}} = Scope.build(machine) + + assert [ + %{name: "first", tier: :data, d_index: 0, value: :undefined}, + %{name: "count", tier: :data, d_index: 1, value: :undefined}, + %{name: "third", tier: :data, d_index: 2, value: :undefined} + ] = entries + end + + test "an expr-written element's declared_source holds the expression text, unevaluated", %{ + machine: machine + } do + assert {:ok, %Scope{data: entries}} = Scope.build(machine, source: @chart) + + assert %{value: :undefined, declared_source: "41 + 1"} = + Enum.find(entries, &(&1.name == "count")) + end + + test "a bare element's declared_source is nil even when :source is given", %{ + machine: machine + } do + assert {:ok, %Scope{data: entries}} = Scope.build(machine, source: @chart) + + assert %{declared_source: nil} = Enum.find(entries, &(&1.name == "first")) + assert %{declared_source: nil} = Enum.find(entries, &(&1.name == "third")) + end + + test "with :source omitted, every declared_source is nil and there are no diagnostics", %{ + machine: machine + } do + assert {:ok, %Scope{data: entries, diagnostics: diagnostics}} = Scope.build(machine) + + assert Enum.all?(entries, &is_nil(&1.declared_source)) + assert diagnostics == [] + end + + test "an out-of-range :source yields one diagnostic and a nil declared_source, not a failed build", + %{machine: machine} do + assert {:ok, %Scope{data: entries, diagnostics: diagnostics}} = + Scope.build(machine, source: "x") + + assert %{declared_source: nil} = Enum.find(entries, &(&1.name == "count")) + assert [%{kind: :unsliceable_declared_source, path: ["data", "count"]}] = diagnostics + end + end + + describe "build/2 - tier 2a (system variables)" do + test "exactly the four names initial/2 seeds, with no _x", %{machine: machine} do + assert {:ok, %Scope{system: entries}} = Scope.build(machine) + + assert Enum.map(entries, & &1.name) |> Enum.sort() == + ["_event", "_ioprocessors", "_name", "_sessionid"] + end + + test "_sessionid infers :string, _ioprocessors a nested map, _event and _name :undefined", %{ + machine: machine + } do + assert {:ok, %Scope{system: entries}} = Scope.build(machine) + by_name = Map.new(entries, &{&1.name, &1}) + + assert %{shape: :string} = by_name["_sessionid"] + assert %{shape: {:map, _pairs}} = by_name["_ioprocessors"] + assert %{shape: :undefined, value: :undefined} = by_name["_event"] + assert %{shape: :undefined, value: :undefined} = by_name["_name"] + end + + test "_event's children are exactly the keys of SystemVariables.event/1's result", %{ + machine: machine + } do + assert {:ok, %Scope{system: entries}} = Scope.build(machine) + event_entry = Enum.find(entries, &(&1.name == "_event")) + + expected = + "(none)" + |> Statifier.Event.external() + |> SystemVariables.event() + |> Map.keys() + |> Enum.sort() + + assert Enum.map(event_entry.children, & &1.name) |> Enum.sort() == expected + assert Enum.all?(event_entry.children, &(&1.value == :undefined)) + end + end + + describe "build/2 - tier 2b (provider functions)" do + test "In is present at arity 1, a multi-arity builtin renders name/a|b, entries sorted", %{ + machine: machine + } do + assert {:ok, %Scope{functions: entries}} = Scope.build(machine) + + assert %{arity: 1, label: "In/1", tier: :function} = + Enum.find(entries, &(&1.name == "In")) + + assert %{arity: [2, 3], label: "substring/2|3"} = + Enum.find(entries, &(&1.name == "substring")) + + names = Enum.map(entries, & &1.name) + assert names == Enum.sort(names) + end + end + + describe "build/2 - invalid input" do + test "returns {:error, {:invalid_machine, _}} for anything that is not a Machine" do + assert {:error, {:invalid_machine, %{}}} = Scope.build(%{}) + assert {:error, {:invalid_machine, "nope"}} = Scope.build("nope") + end + end +end diff --git a/test/statifier_ui/datamodel_explorer/session_test.exs b/test/statifier_ui/datamodel_explorer/session_test.exs new file mode 100644 index 0000000..7e58323 --- /dev/null +++ b/test/statifier_ui/datamodel_explorer/session_test.exs @@ -0,0 +1,86 @@ +defmodule StatifierUI.DatamodelExplorer.SessionTest do + use ExUnit.Case, async: true + + alias Statifier.Session + alias StatifierUI.DatamodelExplorer + alias StatifierUI.Test.Support.Trace.SessionCase + alias StatifierUI.Trace.Subscriber + + # A self-transition on "bump" that assigns `count`, so a second macrostep + # is driven by an ordinary external event rather than the initialize + # burst. `` proves the fold reads the + # engine's own binding-fold result, not a value this test computed. + @chart """ + + + + + + + + + + + + """ + + @spec macrostep_stable_count(pid()) :: non_neg_integer() + defp macrostep_stable_count(sub) do + sub + |> Subscriber.messages() + |> Enum.filter(&(&1.type == "trace.macrostep_stable")) + |> Enum.map(& &1.macrostep) + |> Enum.uniq() + |> length() + end + + # `Subscriber.stats/1` carries no macrostep information, so this polls + # `Subscriber.messages/1` directly through `wait_until/3`'s predicate + # rather than `wait_for_seq/3`, which only knows about `seq`. + @spec wait_for_macrostep_count(pid(), pos_integer()) :: :ok + defp wait_for_macrostep_count(sub, count) do + SessionCase.wait_until(sub, 1000, fn _stats -> macrostep_stable_count(sub) >= count end) + :ok + end + + @spec fold(pid()) :: DatamodelExplorer.t() + defp fold(sub) do + assert {:ok, pane} = DatamodelExplorer.build_live(Subscriber.messages(sub)) + pane + end + + @spec entry(DatamodelExplorer.t(), String.t()) :: DatamodelExplorer.Entry.t() + defp entry(pane, name), do: Enum.find(pane.entries, &(&1.name == name)) + + test "the fold tracks a real session's binding, then its " do + machine = SessionCase.compile!(@chart) + {sub, session} = SessionCase.start_early!(machine, "sess_dm_live") + + wait_for_macrostep_count(sub, 1) + pane = fold(sub) + + count = entry(pane, "count") + assert count.value == 42 + assert count.changed? == true + + session_id_entry = entry(pane, "_sessionid") + assert session_id_entry.value == "sess_dm_live" + + first_macrostep = pane.macrostep + + assert :ok = Session.send_event(session, "bump") + wait_for_macrostep_count(sub, 2) + pane = fold(sub) + + count = entry(pane, "count") + assert count.value == 43 + assert count.changed? == true + assert pane.macrostep > first_macrostep + + # _sessionid was untouched by this macrostep's writes, so it must not + # be marked changed? even though it is still present and correct. + session_id_entry = entry(pane, "_sessionid") + assert session_id_entry.value == "sess_dm_live" + assert session_id_entry.changed? == false + end +end diff --git a/test/statifier_ui/datamodel_explorer_test.exs b/test/statifier_ui/datamodel_explorer_test.exs new file mode 100644 index 0000000..1f3f4ab --- /dev/null +++ b/test/statifier_ui/datamodel_explorer_test.exs @@ -0,0 +1,176 @@ +defmodule StatifierUI.DatamodelExplorerTest do + use ExUnit.Case, async: true + + alias StatifierUI.DatamodelExplorer + alias StatifierUI.Fixtures + alias StatifierUI.Test.Support.Fixtures.PaymentSource + alias StatifierUI.Test.Support.Trace.SessionCase + + # One declaration the "gold-tier-user" scenario names ("tier") and + # one it does not name ("count") - so promotion and pass-through both + # exercise on the same chart. No `name` attribute, matching scope_test.exs. + @chart """ + + + + + + + + + """ + + setup do + machine = SessionCase.compile!(@chart) + {:ok, fixtures} = Fixtures.from_source(PaymentSource) + {:ok, machine: machine, fixtures: fixtures} + end + + describe "build_authoring/3 - nil fixtures" do + test "scenario nil, scenario_names empty, every tier-1 entry :undefined", %{machine: machine} do + assert {:ok, pane} = DatamodelExplorer.build_authoring(machine, nil) + + assert %DatamodelExplorer{ + mode: :authoring, + session: nil, + scenario: nil, + scenario_names: [], + macrostep: nil, + truncated?: false, + diagnostics: [] + } = pane + + assert Enum.all?(DatamodelExplorer.entries(pane, :data), &(&1.value == :undefined)) + end + end + + describe "build_authoring/3 - a scenario naming a declared " do + test "that entry's shape becomes the scenario value's, tier stays :data, d_index intact", %{ + machine: machine, + fixtures: fixtures + } do + assert {:ok, pane} = DatamodelExplorer.build_authoring(machine, fixtures) + + [tier_entry] = Enum.filter(DatamodelExplorer.entries(pane, :data), &(&1.name == "tier")) + + assert %{ + name: "tier", + tier: :data, + d_index: 0, + value: "gold", + shape: :string + } = tier_entry + end + + test "a tier-1 entry the scenario does not name stays :undefined", %{ + machine: machine, + fixtures: fixtures + } do + assert {:ok, pane} = DatamodelExplorer.build_authoring(machine, fixtures) + + assert %{tier: :data, value: :undefined} = + Enum.find(DatamodelExplorer.entries(pane, :data), &(&1.name == "count")) + end + end + + describe "build_authoring/3 - a scenario naming something the chart does not declare" do + test "a :scenario entry appears", %{machine: machine} do + {:ok, fixtures} = + Fixtures.new(scenarios: %{"gold-tier-user" => %{"user_id" => "u-1999"}}) + + assert {:ok, pane} = DatamodelExplorer.build_authoring(machine, fixtures) + + assert [%{name: "user_id", tier: :scenario, value: "u-1999", shape: :string}] = + DatamodelExplorer.entries(pane, :scenario) + end + end + + describe "build_authoring/3 - scenario selection" do + test "explicit :scenario selects that scenario", %{machine: machine, fixtures: fixtures} do + assert {:ok, pane} = + DatamodelExplorer.build_authoring(machine, fixtures, scenario: "gold-tier-user") + + assert pane.scenario == "gold-tier-user" + end + + test "an unknown :scenario name returns {:error, {:unknown_scenario, name}}", %{ + machine: machine, + fixtures: fixtures + } do + assert {:error, {:unknown_scenario, "no-such-scenario"}} = + DatamodelExplorer.build_authoring(machine, fixtures, scenario: "no-such-scenario") + end + + test "an unknown :scenario with nil fixtures also returns {:error, {:unknown_scenario, _}}", + %{ + machine: machine + } do + assert {:error, {:unknown_scenario, "anything"}} = + DatamodelExplorer.build_authoring(machine, nil, scenario: "anything") + end + + test "default selection is the first sorted name, not the first inserted", %{ + machine: machine + } do + {:ok, fixtures} = + Fixtures.new( + scenarios: %{ + "zeta-scenario" => %{"tier" => "zeta"}, + "alpha-scenario" => %{"tier" => "alpha"} + } + ) + + assert {:ok, pane} = DatamodelExplorer.build_authoring(machine, fixtures) + + assert pane.scenario == "alpha-scenario" + + assert %{value: "alpha"} = + Enum.find(DatamodelExplorer.entries(pane, :data), &(&1.name == "tier")) + end + end + + describe "build_authoring/3 - decoded values are not re-encoded" do + test "a scenario value that is already a decoded Date infers :date, not a $-tagged map", %{ + machine: machine + } do + {:ok, fixtures} = + Fixtures.new(scenarios: %{"gold-tier-user" => %{"tier" => ~D[2026-08-22]}}) + + assert {:ok, pane} = DatamodelExplorer.build_authoring(machine, fixtures) + + assert %{value: ~D[2026-08-22], shape: :date} = + Enum.find(DatamodelExplorer.entries(pane, :data), &(&1.name == "tier")) + end + end + + describe "entries/2" do + test "filters by tier", %{machine: machine, fixtures: fixtures} do + assert {:ok, pane} = DatamodelExplorer.build_authoring(machine, fixtures) + + assert Enum.all?(DatamodelExplorer.entries(pane, :data), &(&1.tier == :data)) + assert Enum.all?(DatamodelExplorer.entries(pane, :system), &(&1.tier == :system)) + assert Enum.all?(DatamodelExplorer.entries(pane, :function), &(&1.tier == :function)) + + assert DatamodelExplorer.entries(pane) == + DatamodelExplorer.entries(pane, :data) ++ + DatamodelExplorer.entries(pane, :system) ++ + DatamodelExplorer.entries(pane, :function) ++ + DatamodelExplorer.entries(pane, :scenario) + end + end + + describe "diagnostics/1" do + test "bundle diagnostics are carried through ahead of the scope's own", %{machine: machine} do + fixtures = %Fixtures{ + scenarios: %{"gold-tier-user" => %{"tier" => "gold"}}, + diagnostics: [ + %{kind: :some_bundle_problem, message: "pre-existing", path: ["x"], source: nil} + ] + } + + assert {:ok, pane} = DatamodelExplorer.build_authoring(machine, fixtures) + + assert [%{kind: :some_bundle_problem}] = DatamodelExplorer.diagnostics(pane) + end + end +end