Adds the Ecto storage adapter

README.md

@@ -12,7 +12,40 @@ crash semantics. This package is that loop, packaged.
 
 ## Status
 
-Nothing is implemented yet. This repository holds the scaffold only.
+Pre-release, under active development. The storage-adapter behaviour with
+its identity guard, the in-memory reference adapter, the run lifecycle,
+and the Ecto layer (configurable keys/tables, versioned migrations, and
+the Postgres adapter below) exist; nothing is published to Hex yet.
+
+## The Ecto adapter
+
+Configure a persistence module on your own repo once, and migrate:
+
+    defmodule MyApp.Persistence do
+      use StatifierPersistence.Ecto, repo: MyApp.Repo
+    end
+
+    defmodule MyApp.Repo.Migrations.AddStatifierPersistence do
+      use Ecto.Migration
+      def up, do: StatifierPersistence.Ecto.Migrations.up(for: MyApp.Persistence)
+      def down, do: StatifierPersistence.Ecto.Migrations.down(for: MyApp.Persistence)
+    end
+
+then build the guarded store the rest of the package works through:
+
+    {:ok, store} =
+      StatifierPersistence.Storage.new(
+        StatifierPersistence.Storage.Ecto,
+        persistence: MyApp.Persistence
+      )
+
+The adapter passes the same conformance suite the in-memory reference
+does (`StatifierPersistence.Testing.StorageConformance` - point it at
+your own adapter to hold it to the identical bar), stores engine
+identities verbatim, and implements the optional per-run `lock_run/3`
+as a transaction-scoped advisory-plus-row lock (ADR-0004 as amended).
+In your test suite, pass `sandbox: true` so each test runs in its own
+`Ecto.Adapters.SQL.Sandbox` checkout via the adapter's `isolate/1`.
 
 ## Running the tests
 

changelog.d/sp-4an.3.1.md

@@ -0,0 +1,17 @@
+# sp-4an.3.1
+
+## Added
+
+- `StatifierPersistence.Storage.Ecto`: the Postgres storage adapter over
+  the schemas a host generates with `use StatifierPersistence.Ecto`
+  (`Storage.new(Storage.Ecto, persistence: MyApp.Persistence)`). Passes
+  the same conformance suite as the in-memory reference adapter; engine
+  identities stored verbatim; `:run_exists` enforced atomically by the
+  unique index.
+- `Storage.Ecto.isolate/1`: with `sandbox: true`, wraps each test in its
+  own `Ecto.Adapters.SQL.Sandbox` checkout - the hook host test suites
+  (and this package's conformance suite) isolate through.
+- `Storage.Ecto.lock_run/3`: per-run mutual exclusion as a
+  transaction-scoped `pg_advisory_xact_lock` plus a `SELECT ... FOR
+  UPDATE` row lock (ADR-0004 as amended), consumed by
+  `Serialization.AdapterLock`.

docs/adr/0004-run-lifecycle-executor-seam-and-serialization.md

@@ -177,3 +177,22 @@ a host decision about the run, not a chart transition.
 - The Ecto adapter (sp-4an.3) inherits three run callbacks, two error
   arms, and an optional `lock_run/3` to implement as a transaction-scoped
   row lock, all conformance-tested through the sp-4an.1 suite.
+
+## Amendment (2026-08-22, sp-4an.3.1): the Ecto lock is advisory plus row
+
+Decision 5 (and the Consequences bullet above) named the Ecto adapter's
+`lock_run/3` a transaction-scoped row lock. Implementing it showed the
+row lock alone cannot honor the callback's contract: `SELECT ... FOR
+UPDATE` excludes nothing when no run row matches, and the contract (with
+the conformance suite's lock tests) requires mutual exclusion for a
+`run_id` that has not been inserted yet.
+
+So the Ecto adapter's transaction takes
+`pg_advisory_xact_lock(hashtextextended(run_id, 0))` first -
+unconditional per-run exclusion, row or no row - and then locks the run
+row with `SELECT ... FOR UPDATE` when it exists, keeping this decision's
+ordering against the row itself. Both are transaction-scoped, so any
+exit from `fun` (a raise included) releases them with the transaction.
+The rowless hole and the fix are pinned by a live two-connection test
+outside the SQL sandbox, whose single shared connection would otherwise
+serialize the callers by ownership and mask a broken lock.

docs/plans/260822-sp-4an.3.1-ecto-storage-adapter.md

@@ -0,0 +1,226 @@
+# Ecto storage adapter (sp-4an.3.1) Implementation Plan
+
+## Overview
+
+Implement `StatifierPersistence.Storage.Ecto`: the
+`StatifierPersistence.Storage.Adapter` behaviour over the schemas a host
+generates with `use StatifierPersistence.Ecto` (sp-02x, merged as PR #13),
+passing the identical conformance suite `InMemory` passes, against real
+Postgres per ADR-0005. Bead: sp-4an.3.1 (second task of the sp-4an.3 epic).
+
+## Current State Analysis
+
+- The adapter contract is fixed and conformance-tested:
+  `lib/statifier_persistence/storage/adapter.ex` (seven required callbacks,
+  optional `isolate/1` and `lock_run/3`),
+  `lib/statifier_persistence/testing/storage_conformance.ex` (the suite,
+  lock tests generated only when the adapter exports `lock_run/3`),
+  `lib/statifier_persistence/storage/in_memory.ex` (the reference).
+- The Ecto layer under it is on main: `use StatifierPersistence.Ecto`
+  generates `Chart`/`Position`/`Run` schema modules from a resolved
+  `Config`; `StatifierPersistence.Ecto.Migrations` V01 creates the tables
+  with unique indexes on `content_hash` / `session_id` / `run_id`
+  (`lib/statifier_persistence/ecto/migrations/v01.ex`). Primary keys
+  autogenerate through the schemas, so `Repo.insert/2` works under every
+  key scheme, `bigserial` included (`read_after_writes`).
+- The Postgres harness is live (ADR-0005): `TestRepo` on the SQL sandbox in
+  `:manual` mode, `docker compose up -d db`, storage_up in
+  `test/test_helper.exs`. `mix quality` requires the reachable server.
+- No table exists for the `EctoHosts.Default` host yet at test start - the
+  live migration tests create and drop only their own `kx_*` tables.
+
+### Key Discoveries
+
+- `runs.status` is a `text` column; the contract's `run_status` is an atom
+  (`:active | :completed | :failed`). The adapter owns an explicit two-way
+  mapping - no `String.to_atom` on database bytes.
+- `insert_run/2`'s `:run_exists` refusal must be atomic with the write
+  (adapter.ex contract). The V01 unique index on `run_id` is the mechanism;
+  a changeset `unique_constraint` (index name `<table>_run_id_index`, the
+  `unique_index/2` default) maps the violation to `{:error, :run_exists}`
+  without a raise and without check-then-insert.
+- `update_run/2` must refuse a missing `run_id` atomically too:
+  `Repo.update_all/3` on `run_id` returns the match count - `{0, _}` is
+  `:run_not_found`, no separate existence read.
+- The conformance suite locks run_ids that were **never inserted**
+  ("run-conformance-lock"). A pure `SELECT ... FOR UPDATE` row lock excludes
+  nothing when no row matches, so ADR-0004's row-lock spelling alone cannot
+  satisfy the unconditional mutual-exclusion contract in
+  `adapter.ex` (`lock_run/3`: "no interleaving ... for the same run_id").
+  Resolution (this repo owns locking per the cross-repo ownership table):
+  inside the transaction, first `pg_advisory_xact_lock(hashtextextended(
+  run_id, 0))`, then `SELECT ... FOR UPDATE` on the run row when it exists.
+  The advisory lock makes exclusion unconditional; the row lock keeps
+  ADR-0004's ordering against the run row itself. Both are
+  transaction-scoped, so any exit from `fun` - raise included - releases
+  them with the transaction. Recorded as a dated amendment to ADR-0004.
+- Under the SQL sandbox both conformance lock tests pass through connection
+  ownership alone (both tasks share the owner's one connection, and
+  `$callers` grants the allowance), which proves nothing about the SQL.
+  The real semantics need a live test in `:auto` mode with two pooled
+  connections - the same pattern `migrations_test.exs` already uses.
+
+## Desired End State
+
+```elixir
+{:ok, store} =
+  StatifierPersistence.Storage.new(
+    StatifierPersistence.Storage.Ecto,
+    persistence: MyApp.Persistence
+  )
+```
+
+behaves exactly like the same call with `InMemory`: every facade path
+(save/load chart and position, insert/fetch/update run,
+`update_run_status/4`, `load_run_position/3`) works over Postgres, engine
+identities verbatim, blobs byte-identical, identity guard on every load,
+and `Serialization.AdapterLock` gets real per-run mutual exclusion from
+`lock_run/3`. The conformance suite runs green against it in this repo's
+own gate, lock tests included.
+
+## What We're NOT Doing
+
+- **No stepper/lifecycle changes.** `Runs`, `Executor`, `Serialization`
+  land untouched; the adapter slots under them through the existing seams.
+- **No new facade surface.** `update_run_status/4` already rides
+  `fetch_run` + `update_run`; the adapter only implements the behaviour.
+- **No listing/querying API** (runs by status, etc.) - no consumer asks for
+  it yet; unexercised contract.
+- **No tenancy columns, no session_id population on runs** - unchanged
+  from sp-02x's scope cuts.
+- **No `.quality.exs` edits** (campaign consent excludes them).
+- **No backend-failure taxonomy.** Like `InMemory`, `{:adapter, term()}`
+  is produced where init can observe a failure; a dead database mid-call
+  raises (DBConnection's own exception), which the loop above treats as a
+  crash, not a value. Wrapping every call is scope for a later bead if a
+  host asks.
+
+## Implementation Approach
+
+Three phases, each independently gate-green and committable. Phase 1 is
+the adapter's CRUD surface plus `isolate/1`, passing the full conformance
+suite minus the lock tests (they generate only once `lock_run/3` exists).
+Phase 2 adds `lock_run/3`, which makes the conformance lock tests
+generate, plus the live two-connection proof and the ADR-0004 amendment.
+Phase 3 is user-facing docs and the changelog fragment.
+
+### Phase 1: adapter CRUD, isolate, conformance green
+
+**Files:**
+- `lib/statifier_persistence/storage/ecto.ex` (new, inside
+  `if Code.ensure_loaded?(Ecto)`): `@behaviour Storage.Adapter`.
+  - `init/1`: requires `persistence:` (a module exporting
+    `__statifier_persistence__/1`); refusal is
+    `{:error, {:adapter, {:not_a_persistence_host, module}}}`. Resolves
+    and stores `repo`, the three schema modules, and the runs table name
+    (for the constraint name) into the returned opts. Optional
+    `sandbox: true` opts flag consumed by `isolate/1`.
+  - `save_chart/2`: `Repo.insert(struct, on_conflict: :nothing,
+    conflict_target: [:content_hash])` - idempotent by construction.
+  - `save_position/2`: upsert,
+    `on_conflict: {:replace, [:content_hash, :identity_blob,
+    :position_blob, :updated_at]}, conflict_target: [:session_id]`.
+  - `fetch_chart/2`, `fetch_position/2`, `fetch_run/2`: `Repo.get_by/3`,
+    `nil` mapped to the contract's not-found arm, struct mapped to the
+    plain record map (status decoded through the explicit mapping).
+  - `insert_run/2`: `Ecto.Changeset.change/1` +
+    `unique_constraint(:run_id, name: "<runs_table>_run_id_index")`;
+    `{:error, changeset}` with that constraint error becomes
+    `{:error, :run_exists}`.
+  - `update_run/2`: one `Repo.update_all/3` setting every contract field
+    plus `updated_at`; `{0, _}` is `{:error, :run_not_found}`.
+  - `isolate/1`: `Ecto.Adapters.SQL.Sandbox.checkout(repo)` when
+    `sandbox: true` (tolerating `{:already, _}`), no-op `:ok` otherwise.
+- `test/test_helper.exs`: after `start_link`, before `:manual`, run the
+  V01 migration for `EctoHosts.Default` **and** `EctoHosts.Overridden`
+  through `Ecto.Migrator` (fixed versions, idempotent on `:already_up`)
+  so the default `statifier_*` tables and the prefixed
+  `workflows.wf_*`/`workflows.workflow_runs` tables exist for the whole
+  suite.
+- `test/statifier_persistence/storage/ecto_conformance_test.exs` (new):
+  `use StatifierPersistence.Testing.StorageConformance,
+  adapter: Storage.Ecto, opts: [persistence: EctoHosts.Default,
+  sandbox: true], async: true`.
+- `test/statifier_persistence/storage/ecto_test.exs` (new): adapter-level
+  unit tests the conformance suite does not cover - `init/1` refusing a
+  non-host module, all three statuses round-tripping, `save_chart`
+  repeated write not bumping the row count, and the same CRUD mechanisms
+  (`save_chart`/`insert_run` upsert, uniqueness, `update_run`) exercised
+  once against `EctoHosts.Overridden` so the `@schema_prefix`/renamed-
+  table path (`workflows.workflow_runs`) is covered, not only the
+  zero-config host. Each with a sabotage note (mutation verified red,
+  reverted).
+
+**Success criteria (automated):** full `mix quality` green (the 22
+non-lock conformance tests run against the Ecto adapter); `mix
+gate.verify`.
+**Manual:** none.
+
+### Phase 2: lock_run/3, live proof, ADR-0004 amendment
+
+**Files:**
+- `lib/statifier_persistence/storage/ecto.ex`: `lock_run/3` -
+  `repo.transaction(fn -> advisory xact lock on
+  hashtextextended(run_id, 0); SELECT ... FOR UPDATE on the run row (zero
+  rows is fine); {:ok tail} fun.() end)`; result `{:ok, fun_result}`; a
+  raising `fun` rolls back and propagates (locks release with the
+  transaction).
+- `test/statifier_persistence/storage/ecto_conformance_test.exs`: the two
+  lock tests now generate automatically - no edit, but the run count
+  changes. The generation gate is a compile-time
+  `function_exported?(adapter, :lock_run, 3)` inside the `use` expansion,
+  and nothing in this repo has yet proven Mix recompiles the test file
+  when the adapter gains the export. **Verify the tests actually appear**
+  (`mix test test/statifier_persistence/storage/ecto_conformance_test.exs
+  --trace` and see both "lock_run/3" tests listed); if they do not,
+  force the regeneration (`touch` the test file or `mix compile
+  --force`) before treating the phase as green - a silently smaller
+  suite still passes.
+- `test/statifier_persistence/storage/ecto_live_lock_test.exs` (new,
+  `async: false`, sandbox `:auto` for the module like
+  `migrations_test.exs`): with two real pooled connections, (a) two
+  concurrent `lock_run/3` bodies on an **inserted** run never overlap
+  (the FOR UPDATE row path), (b) same for a run_id **never inserted**
+  (the advisory path), (c) a raising `fun` releases the lock and a fresh
+  `lock_run/3` reacquires. Rows cleaned up after the module. Sabotage
+  notes on each.
+- `docs/adr/0004-run-lifecycle-executor-seam-and-serialization.md`: dated
+  amendment under decision 5 recording the advisory-lock-plus-row-lock
+  shape and why (unconditional exclusion for a run_id with no row yet).
+- `lib/statifier_persistence/storage/adapter.ex`: the `lock_run/3` doc's
+  "implements it as a transaction-scoped row lock" sentence gains the
+  advisory clause so the doc and the amendment agree.
+
+**Success criteria (automated):** full `mix quality` green (conformance
+now 24 tests against Ecto, the two lock tests confirmed present per the
+verification note above; live lock tests green); `mix gate.verify`.
+**Manual:** none.
+
+### Phase 3: docs and changelog
+
+**Files:**
+- `README.md`: an "Ecto adapter" subsection - the
+  `Storage.new(Storage.Ecto, persistence: MyApp.Persistence)` call, the
+  migration prerequisite, the sandbox note for host test suites.
+- `changelog.d/sp-4an.3.1.md`: Added - `StatifierPersistence.Storage.Ecto`
+  (conformance-passing Postgres adapter; `isolate/1` sandbox hook;
+  `lock_run/3` transaction-scoped locking behind
+  `Serialization.AdapterLock`).
+
+**Success criteria:** docs-only diff, reviewed; gate untouched by content
+(run the full gate anyway before the final commit per campaign rules).
+
+## Testing Strategy
+
+The conformance suite is the bar and is not edited (except that lock
+tests self-generate). New tests: adapter unit tests (Phase 1) and the
+live two-connection lock tests (Phase 2), each sabotage-verified per the
+repo convention. The live tests exist precisely because the sandbox
+serializes on one connection and would mask a broken lock.
+
+## References
+
+- ADR-0003 (behaviour + identity guard), ADR-0004 (runs vocabulary, lock),
+  ADR-0005 (Postgres harness), ADR-0002 (keys/tables).
+- `docs/plans/260822-sp-02x-keys-tables-migrations.md` - phase/gate
+  discipline mirrored here.

lib/statifier_persistence/storage/adapter.ex

@@ -238,9 +238,12 @@ defmodule StatifierPersistence.Storage.Adapter do
   This is the callback the default serialization strategy
   (`StatifierPersistence.Serialization.AdapterLock`) delegates to; an
   adapter that does not export it makes that strategy refuse with
-  `{:error, {:serialization, :not_supported}}`. An Ecto adapter implements
-  it as a transaction-scoped row lock - `SELECT ... FOR UPDATE` on the run
-  row inside a transaction that spans `fun` (sp-4an.3).
+  `{:error, {:serialization, :not_supported}}`. The Ecto adapter
+  implements it as a transaction-scoped advisory lock plus a
+  `SELECT ... FOR UPDATE` row lock inside a transaction that spans `fun`
+  (ADR-0004 decision 5 as amended 2026-08-22) - the advisory half exists
+  because a row lock alone excludes nothing for a `run_id` whose run has
+  not been inserted yet.
   """
   @callback lock_run(opts(), run_id(), (-> result)) ::
               {:ok, result} | {:error, error()}