From 9566a7339ca35ed228e0cbb9e37b597bfc505c7c Mon Sep 17 00:00:00 2001 From: Kris Zyp Date: Tue, 11 Aug 2026 14:52:55 -0600 Subject: [PATCH] docs(storage): document storage.blobRetention New in harper#2145: how long a superseded blob file is kept on disk so an in-flight read (or a replication peer that has not fetched it yet) can still get the bytes, and the disk cost of widening it. Co-Authored-By: Claude Opus 5 --- reference/database/storage-tuning.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/reference/database/storage-tuning.md b/reference/database/storage-tuning.md index d99153b8..498334b3 100644 --- a/reference/database/storage-tuning.md +++ b/reference/database/storage-tuning.md @@ -102,6 +102,27 @@ storage: Blobs are not relocated when `blobPaths` changes — only new blobs honor the updated configuration. Existing blob references continue to resolve at their original path. +### `storage.blobRetention` + +Type: `number` (milliseconds) + +Default: `2000` + +How long a superseded blob file is kept on disk after the record that referenced it is overwritten or removed. + +A blob's bytes are read lazily: a request resolves the record first, then opens the backing file when it starts streaming the response. If the record is overwritten in between, the file it pointed at is on its way out — and because the failure surfaces after the response headers are already committed, the client sees a truncated body rather than an error status. `blobRetention` is the window that lets those in-flight reads finish. + +Raise it when reads are slow enough to outlive the default window — large blobs, slow or heavily backpressured clients, or a busy cache table whose entries are rewritten while being served. Replication peers that have not yet fetched a superseded blob are also covered by this window, so a cluster with significant replication lag wants a value comfortably above that lag. + +The cost is disk: superseded blobs written during the window stay on disk for its duration, so the overhang is roughly your blob write rate multiplied by the retention window. Set it to `0` to reclaim as soon as the queue drains. + +```yaml +storage: + blobRetention: 30000 +``` + +If the process exits before a deferred reclamation runs, the file is left behind until the `cleanup_orphan_blobs` operation reclaims it — a longer window widens that gap. + ## Read & Write Behavior ### `storage.prefetchWrites`