test(database): cover copy-db's blob-store omission (#2048 ch.3), as todo-marked contract assertions - #2126
test(database): cover copy-db's blob-store omission (#2048 ch.3), as todo-marked contract assertions#2126kriszyp wants to merge 3 commits into
Conversation
Reproduces the measurement behind #2048 channel 3 as an integration test: seed six 256 KiB file-backed blobs plus four inline controls, run the documented `copy-db` CLI verb at an external target, and look at what lands there. Source has 10 blob files, the copy has 0, and the CLI exits 0 — records in the copy carry fileId references that resolve to nothing while inline attributes survive byte-exact, so the loss is partial and easy to miss. The two contract assertions are `todo`, deliberately. The tempting version of this test asserts what happens TODAY — "the copy contains zero blob files" — which passes now and goes red the moment #2048 is fixed, reading as a regression when it is the opposite. That is not hypothetical: it is exactly how txnlog-purge-stale-read-blast arm 5 sat red on main for a day (159b4bb). So they assert what SHOULD hold and carry `todo`, which runs them, keeps the failure off the CI result, and surfaces the moment they start passing — at which point the marker comes off and this becomes an ordinary regression anchor. Result today: 3 pass, 2 todo, 0 fail. Scope is asserted rather than assumed: compactOnStart shares the function but not the consequence, because it swaps the copy into the same rootPath and leaves blobs/{db}/ adjacent. Only an external target loses them. Two harness details worth keeping: the CLI needs ROOTPATH (not HARPER_ROOT_PATH), and an empty CLI output is treated as "never found the database" — a harness problem — rather than being scored as a finding about copy-db. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Reviewed; no blockers found. |
There was a problem hiding this comment.
Code Review
This pull request adds integration tests to verify the behavior of the copy-db command with the blob store, including test configurations, a GraphQL schema, and custom resources for seeding and verification. The review feedback suggests cleaning up the temporary directory created during tests in the after hook using rmSync within a try/finally block, and guarding the copyExit assignment against non-numeric error codes to prevent type mismatches.
| import { suite, test, before, after } from 'node:test'; | ||
| import { ok, strictEqual } from 'node:assert'; | ||
| import { resolve, join } from 'node:path'; | ||
| import { readdirSync, statSync, existsSync, mkdtempSync } from 'node:fs'; |
There was a problem hiding this comment.
| after(async () => { | ||
| await teardownHarper(ctx); | ||
| }); |
There was a problem hiding this comment.
The temporary directory copyRoot created via mkdtempSync is not cleaned up after the test suite finishes. In test teardown hooks, ensure cleanup operations for temporary resources are guarded and wrap independent teardown steps in try/finally blocks to guarantee all cleanup executes and to prevent secondary errors from masking the original setup failure.
after(async () => {
try {
await teardownHarper(ctx);
} finally {
if (copyRoot && existsSync(copyRoot)) {
rmSync(copyRoot, { recursive: true, force: true });
}
}
});References
- In test teardown hooks, ensure cleanup operations for temporary resources are guarded and wrap independent teardown steps in try/finally blocks to guarantee all cleanup executes.
| } catch (e: any) { | ||
| copyExit = e.code ?? 1; | ||
| copyOut = `${e.stdout ?? ''}\n${e.stderr ?? ''}`; | ||
| } |
There was a problem hiding this comment.
The e.code property on errors thrown by execFile can be a string (e.g., 'ENOENT') or a number. Assigning it directly to copyExit (which is typed as number | null) can cause a type mismatch. Guard the assignment to ensure it is a number.
| } catch (e: any) { | |
| copyExit = e.code ?? 1; | |
| copyOut = `${e.stdout ?? ''}\n${e.stderr ?? ''}`; | |
| } | |
| } catch (e: any) { | |
| copyExit = typeof e.code === 'number' ? e.code : 1; | |
| copyOut = (e.stdout ?? '') + '\n' + (e.stderr ?? ''); | |
| } |
harper-integration-test-run (@harperfast/integration-testing) sets
process.exitCode = 1 on node:test's raw test:fail event without checking
data.todo, and node:test still emits that event for a failing todo test.
So the two `{ todo }`-marked contract assertions in copy-db-blob-store.test.ts
(documenting known harper#2048 ch.3/4 bugs, intentionally non-blocking) were
failing CI exactly as hard as a real assertion, across every job config.
Add assertContractOrDiagnose(), which keeps { todo } for readable output but
catches the assertion internally and reports it via t.diagnostic() instead of
letting it throw out of the test body, so no raw test:fail event fires. The
upstream runner bug is filed as a Finding for a separate fix in
@harperfast/integration-testing.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MZwTDr3RLhKo9F5hzh4inf
Address pre-push review findings on the prior commit: - assertContractOrDiagnose() was wrapping the ch.3 vacuity guard (inCopy.length > 0) along with the actual contract assertion, so a harness failure (e.g. copy-db never running) would misreport as "known bug #2048" instead of throwing for real. Move the guard outside the wrapper. - The mkdtemp'd copy target was never removed; clean it up in after(). - Normalize e.code to a number on child-process failure so a signal name string can't silently satisfy `copyExit === 0` downstream. - Trim the duplicated runner-bug explanation to one place. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MZwTDr3RLhKo9F5hzh4inf
Turns the reproduction behind #2048 channel 3 into an integration test. Test-only.
Seeds six 256 KiB file-backed blobs plus four inline controls, runs the documented
copy-db <source> <target>CLI verb at an external target, and looks at what lands there:Records in the copy carry
fileIdreferences that resolve to nothing, while the inline controls survive byte-exact — which is what makes the loss partial and easy to miss.Today: 3 pass, 2 todo, 0 fail. CI stays green.
The design decision worth reviewing
The two contract assertions are
todo, and that is the point of the PR as much as the coverage is.The tempting version of this test asserts what happens today — "the copy contains zero blob files". That passes now and goes red the moment #2048 is fixed, reading as a regression when it is the opposite. That failure mode is not hypothetical here: it is exactly how
txnlog-purge-stale-read-blastarm 5 sat red onmainfor a day (159b4bb) — a QA spec characterising a bug was promoted a day after the bug was fixed.So the assertions state what should be true and carry
todo, which:If you'd rather this land alongside the fix as plain assertions, that works too — say so and I'll hold it.
Where to look
copy-db-blob-store.test.ts— the header explains the todo choice; the twotodotests are the contract. Everything else is preconditions.The precondition test is load-bearing: if the seeded blobs are ever inlined rather than file-backed, every assertion below it is vacuous, so it asserts
>= 6real files before anything else runs.Scope is asserted, not assumed.
compactOnStartsharescopyDb()but not the consequence — it swaps the copy into the samerootPathand moves onlydatabase/{db}.mdb, leavingblobs/{db}/adjacent and resolvable. A test asserts that, so nobody "fixes" compactOnStart chasing this.Two harness details that cost me time and are recorded in the test: the CLI reads
ROOTPATH(notHARPER_ROOT_PATH), and an empty CLI output is treated as "never located the database" — a harness problem — rather than being scored as a finding aboutcopy-db.What it does and does not cover
Covers #2048 channel 3 (blob store never copied) and touches channel 4 (per-record failures with exit 0) via the second todo. It does not cover channels 1 and 2 — the tombstone heuristic dropping the shared-structures dictionary, and the audit store being written into the source environment. Channel 1 is the most severe of the four and still has no test.
Generated by Claude Opus 5.