-
Notifications
You must be signed in to change notification settings - Fork 9
Document copy-db's blob companion directory and restore steps #620
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -27,7 +27,7 @@ Run using the [CLI](../cli/commands.md): | |||||
| harper copy-db <source-database> <target-database-path> | ||||||
| ``` | ||||||
|
|
||||||
| The `source-database` is the database name (not a file path). The target is the full file path where the compacted copy will be written. | ||||||
| The `source-database` is the database name (not a file path). The target is the full file path where the compacted copy will be written, and it must not already exist — `copy-db` refuses to write into an existing file rather than merging the copy into whatever it holds. | ||||||
|
|
||||||
| To replace the original database with the compacted copy, move or rename the output file to the original database path after Harper is stopped. | ||||||
|
|
||||||
|
|
@@ -37,6 +37,29 @@ To replace the original database with the compacted copy, move or rename the out | |||||
| harper copy-db data /home/user/hdb/database/copy.mdb | ||||||
| ``` | ||||||
|
|
||||||
| Copy compaction applies to LMDB databases. RocksDB databases compact themselves and are skipped. | ||||||
|
|
||||||
| ### File-backed blobs travel separately | ||||||
|
|
||||||
| A database's file-backed blob values (`Blob` and large `Bytes` attributes) are not stored inside the database file. They live in the configured blob roots — `storage.blobPaths[n]`, or `<rootPath>/blobs/<database>` when `blobPaths` is not configured — and are addressed by **database name**, not by the path of the database file. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. According to the repository's guidelines, when documenting behavior changes to an existing surface (such as the new file-backed blobs behavior), we should use the
Suggested change
References
|
||||||
|
|
||||||
| `copy-db` therefore writes them alongside the copy: | ||||||
|
|
||||||
| ``` | ||||||
| <target-database-path>-blobs/<rootIndex>/… | ||||||
| ``` | ||||||
|
|
||||||
| `<rootIndex>` is the position of the source root in the database's blob-root list, preserved so a multi-root database restores each root to its original slot. A `README.md` in that directory records the mapping. | ||||||
|
|
||||||
| **The copy is not restorable without this directory.** To restore the copy under a database name, put each `<rootIndex>` tree into that name's matching blob root — for example, restoring the copy above as a database named `archive` with no `storage.blobPaths` configured: | ||||||
|
|
||||||
| ```bash | ||||||
| cp -r /home/user/hdb/database/copy.mdb /home/user/hdb/database/archive.mdb | ||||||
| cp -r /home/user/hdb/database/copy.mdb-blobs/0/. /home/user/hdb/blobs/archive/ | ||||||
| ``` | ||||||
|
|
||||||
| Restoring the copy under its original database name in the same installation needs only the database file, since the blob roots it already references are untouched. | ||||||
|
|
||||||
| ## Compact on Start | ||||||
|
|
||||||
| Automatically compacts all non-system databases when Harper starts. Harper will not start until compaction is complete. Under the hood, it loops through all user databases, creates a backup of each, compacts it, replaces the original with the compacted copy, and removes the backup. | ||||||
|
|
||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
According to the repository's guidelines, when documenting behavior changes to an existing surface (such as the new blob copying behavior and RocksDB exclusion in
copy-db), we should use the<VersionBadge type="changed" version="vX.Y.0" />format. Please add the appropriate version badge (e.g.,v5.3.0or the target release version) to indicate this change.References