Two Writer implementations for the search-as-pipeline design (epic: #534). Depends on the @lde/pipeline core change (transaction-aware Writer).
Scope
Mode is the Writer implementation (no marker/flag; the pipeline never branches):
BlueGreenRebuild: create the blue collection at openRun; commit swaps the alias + drops the old collection; abort drops the blue collection.InPlaceRebuild: per-source upsert stamped last_seen = runId; commit runs the registry-membership sweep + releases the lock; abort releases the lock (next run reconciles, upserts idempotent). Docs carry source + last_seen.- Deletion: per-source sweep (
delete where source = dataset AND last_seen < runId) + membership sweep (delete where source not in selectedSources()).
- Single-flight cross-pod lock in
openRun / commit / abort.
Acceptance
- Testcontainer: after a run, correct docs in the live collection; correct swap (Blue/green) / sweep (In-place);
abort drops blue / leaves live for next-run reconcile.
- Unit (sweep logic): removed sources deleted, unchanged survive, membership sweep drops departed datasets.
Design and rationale: #534.
Two
Writerimplementations for the search-as-pipeline design (epic: #534). Depends on the@lde/pipelinecore change (transaction-awareWriter).Scope
Mode is the
Writerimplementation (no marker/flag; the pipeline never branches):BlueGreenRebuild: create the blue collection atopenRun;commitswaps the alias + drops the old collection;abortdrops the blue collection.InPlaceRebuild: per-source upsert stampedlast_seen = runId;commitruns the registry-membership sweep + releases the lock;abortreleases the lock (next run reconciles, upserts idempotent). Docs carrysource+last_seen.delete where source = dataset AND last_seen < runId) + membership sweep (delete where source not in selectedSources()).openRun/commit/abort.Acceptance
abortdrops blue / leaves live for next-run reconcile.Design and rationale: #534.