Memory API on the record-log backends: availability, latency, and the rest of the mem0 surface

[bjmeetsfo]

The memory API on the record-log backends — the ones that actually ship. Two availability
defects that made it unusable under load, three O(store) costs on the ingest path, the missing
part of the mem0 surface, and a guard so the defect class that started this cannot come back
silently.

Availability

The background summary refresher could hold the only proxy lane

_summary_refresh_loop calls refresh_summaries({"scope": {}}) on a fixed 1000 ms timer. That
pass is O(store): it reads the whole record log and writes the refreshed summaries back through
the same proxy lane the request path uses. In shared_process_mode, build_lane_pools points
write, read and control at ONE lane behind a single permit — one permit for the whole
gateway. Once a pass costs longer than the interval, the loop holds that permit essentially
continuously: requests queue on the semaphore, block to the request timeout, then are rejected on
lane backpressure. Nothing recovers, because the store only grows.

This is why the native record-log read had to be reverted the same day it landed. Before it,
read_all() returned [] on a native backend, so a pass did no work and this always-unbounded
loop was harmless. Making the read work turned it into a permanent occupant of the lane. The
read path was never the problem.

16 ingests then 8 retrieves, refresher at the shipping default on every arm:

	ingest, first → last	retrieves
fixed interval	391 → 22392 ms	3 ok, then a 120 s hang, then every retrieve rejected at 40 s
refresher off entirely	568 → 2412 ms	8/8 ok, 179–263 ms
cost-proportional delay	699 → 2808 ms	8/8 ok, 229–760 ms

…and bounding the interval between passes was not enough

Testing at a realistic size exposed that the first fix was incomplete. The cost is inside one
pass, not between them: the pass opens with read_all(), which on a 105 MB store takes minutes and
holds the lane throughout. With no client load and nothing to refresh, eight retrieves in a row
were rejected at 40 s.

Dirty state only changes when records are appended, and the record count is a single point read. So
when the count is unchanged since the last pass AND that pass refreshed nothing, this pass would
read the entire store to reach the same conclusion — skip it. The state is persisted, because an
in-memory-only token makes every restart pay one full-store pass.

105 MB store, no load, after a restart	result
before	8/8 rejected at 40 s
in-memory token only	4 fast, 4 timed out during the first pass, then fast
persisted	8/8 answered in 67–583 ms (~104 ms warm)

Ingest cost: 19 → 5.7 full-store reads

Three scans replaced by keyed indexes, each backfilled once per store behind a persisted marker.

• Idempotency was a point question — "have I seen this key" — answered by walking
  reversed(read_all()), and asked 4× per ingest (replay + before-store, across two tool
  calls).
• Node embeddings: ensure_context_node_path read the whole log just to collect which nodes
  already had an embedding, 3× per ingest. The persisted marker is load-bearing: the native
  adapters deliberately start their context-node caches empty, so an in-process-only index would
  re-embed every node after a restart. Proven across a restart — fresh store creates 3, restart
  creates 0.
• Subjects (for users(), below) are indexed on append rather than scanned.

	full-store reads per ingest
before	19 (114 over 6 ingests)
after idempotency index	11 (68)
after node-embedding index	5.7 (34)

Ingest, 24 into one scope, median of the second half: 2431 → 1617 ms (−33%). Retrieve unchanged
at 270–400 ms.

mem0 surface

• batch_update / batch_delete — mem0's shape, capped at the documented 1000. Deliberately
  not atomic and documented as such: an update here is a supersede with no cross-memory
  transaction, so failures are reported per memory rather than aborting the batch, because a
  partial batch the caller cannot see is worse than one they can.
• users() — the users/agents/runs that hold memories, which is not the question the
  account-level user list answers. The re-scoping was subtly wrong twice: a request scope carries
  identity fields derived from the caller (user_hash, scope_key), and get_all filters on
  those, never on user_id. Swapping only user_id made every user report the caller's count
  (users with 2 and 1 memories both reported 3); dropping the hashes made user_hash == 0, read as
  "no filter", returning the whole tenant. Neither errors — both answer confidently and wrongly.
  Hashes are now recomputed with identity_hashes, the same function ingest uses.
• add() now finalizes by default — a behaviour change, stated plainly. add was a streaming
  ingest, so a memory only appeared once a debounce elapsed, and every further write to the scope
  pushed that debounce out. Three add calls followed by get_all returned 0 memories against
  a live gateway. Written against mem0, that reads as data loss. finalize=False keeps the old
  behaviour.

A guard for the defect class

The original five defects were one shape: MatrixArkLocalAdapter precedes the direct mixins in
every native adapter's MRO and defines JSONL-only readers/writers that return [] once
_local_jsonl_enabled is False — which is what a native backend sets. Four methods were broken at
once, silently. Nothing stopped a fifth, so that check is now a test.

It flags a method whose own code reaches for the local log, not merely one that resolves to a
local module (most of those are backend-agnostic orchestration going through the overridden seams).
Docstrings are stripped first, because the native overrides explain this very trap and would report
themselves. It checks all three native adapters — auditing one checks a backend nobody may be
running, which is exactly what hid the one real defect: backend_metrics on
temporalstore-direct inherited the JSONL implementation and reported mode: "local-jsonl" with
the path of an unused sentinel file, so a native deployment was told it was running the wrong
engine.

Verification

Full mem0 surface over the native path, asserted on values, not status codes — the existing
matrix harness scores get_all on count > 0 and history on http == 200, and both passed
while get_all returned twice the rows it should and history returned an empty log:

22/22 — add, get, get_all (exact count AND content), search, update, history, delete, forget,
keyed upsert, keyed recall, TTL, reset. Plus users(), batch_update, batch_delete end to end.

• mem0 suites 56/56; delete/forget 21/21; 34 new tests across the five areas above
• The module-boundary runner matches main exactly. Every other failing suite was confirmed
  pre-existing by running it in place with and without these changes, and again against main;
  test_retrieval_audit_is_off_by_default is a flaky teardown race that fires 3/5 on main and
  1/5 here
• Three comments that recorded the wrong cause for the wedge are corrected — that
  misattribution had already sent two investigations to the wrong place

Scope of evidence, so it is not overstated: temporalstore-rust is verified live throughout.
temporalstore-direct could not be exercised here — it fails at construction because the Python
SDK in this tree does not export Client — so the backend_metrics fix rests on the guard test,
which does fail without it.

Known remaining cost

About 3 full-store reads per ingest remain: drain_due_idle_session_commits, batch_extract, and
_read_raw_records via session_commit. The first looked straightforward — a native scoped scan
already exists — but _native_candidate_scan ends in compact_latest_context_state_records, so it
does not return the append-ordered log that the drain's last-write-wins logic depends on. That has
to be established before switching, not assumed.