Skip to content

test(database): cover copy-db's blob-store omission (#2048 ch.3), as todo-marked contract assertions - #2126

Draft
kriszyp wants to merge 3 commits into
mainfrom
kris/test-copydb-blob-store
Draft

test(database): cover copy-db's blob-store omission (#2048 ch.3), as todo-marked contract assertions#2126
kriszyp wants to merge 3 commits into
mainfrom
kris/test-copydb-blob-store

Conversation

@kriszyp

@kriszyp kriszyp commented Aug 9, 2026

Copy link
Copy Markdown
Member

Turns the reproduction behind #2048 channel 3 into an integration test. Test-only.

Seeds six 256 KiB file-backed blobs plus four inline controls, runs the documented copy-db <source> <target> CLI verb at an external target, and looks at what lands there:

source blobs/data:  10 files
copy tree:          ["data-copy.mdb", "data-copy.mdb-lock"]   ← 0 blob files
copy-db exit code:  0

Records in the copy carry fileId references that resolve to nothing, while the inline controls survive byte-exact — which is what makes the loss partial and easy to miss.

Today: 3 pass, 2 todo, 0 fail. CI stays green.

The design decision worth reviewing

The two contract assertions are todo, and that is the point of the PR as much as the coverage is.

The tempting version of this test asserts what happens today — "the copy contains zero blob files". That passes now and goes red the moment #2048 is fixed, reading as a regression when it is the opposite. That failure mode is not hypothetical here: it is exactly how txnlog-purge-stale-read-blast arm 5 sat red on main for a day (159b4bb) — a QA spec characterising a bug was promoted a day after the bug was fixed.

So the assertions state what should be true and carry todo, which:

If you'd rather this land alongside the fix as plain assertions, that works too — say so and I'll hold it.

Where to look

copy-db-blob-store.test.ts — the header explains the todo choice; the two todo tests are the contract. Everything else is preconditions.

The precondition test is load-bearing: if the seeded blobs are ever inlined rather than file-backed, every assertion below it is vacuous, so it asserts >= 6 real files before anything else runs.

Scope is asserted, not assumed. compactOnStart shares copyDb() but not the consequence — it swaps the copy into the same rootPath and moves only database/{db}.mdb, leaving blobs/{db}/ adjacent and resolvable. A test asserts that, so nobody "fixes" compactOnStart chasing this.

Two harness details that cost me time and are recorded in the test: the CLI reads ROOTPATH (not HARPER_ROOT_PATH), and an empty CLI output is treated as "never located the database" — a harness problem — rather than being scored as a finding about copy-db.

What it does and does not cover

Covers #2048 channel 3 (blob store never copied) and touches channel 4 (per-record failures with exit 0) via the second todo. It does not cover channels 1 and 2 — the tombstone heuristic dropping the shared-structures dictionary, and the audit store being written into the source environment. Channel 1 is the most severe of the four and still has no test.

Generated by Claude Opus 5.

Reproduces the measurement behind #2048 channel 3 as an integration test: seed six
256 KiB file-backed blobs plus four inline controls, run the documented `copy-db`
CLI verb at an external target, and look at what lands there. Source has 10 blob
files, the copy has 0, and the CLI exits 0 — records in the copy carry fileId
references that resolve to nothing while inline attributes survive byte-exact, so
the loss is partial and easy to miss.

The two contract assertions are `todo`, deliberately. The tempting version of this
test asserts what happens TODAY — "the copy contains zero blob files" — which passes
now and goes red the moment #2048 is fixed, reading as a regression when it is the
opposite. That is not hypothetical: it is exactly how txnlog-purge-stale-read-blast
arm 5 sat red on main for a day (159b4bb). So they assert what SHOULD hold and
carry `todo`, which runs them, keeps the failure off the CI result, and surfaces the
moment they start passing — at which point the marker comes off and this becomes an
ordinary regression anchor. Result today: 3 pass, 2 todo, 0 fail.

Scope is asserted rather than assumed: compactOnStart shares the function but not
the consequence, because it swaps the copy into the same rootPath and leaves
blobs/{db}/ adjacent. Only an external target loses them.

Two harness details worth keeping: the CLI needs ROOTPATH (not HARPER_ROOT_PATH),
and an empty CLI output is treated as "never found the database" — a harness problem
— rather than being scored as a finding about copy-db.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@claude

claude Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Reviewed; no blockers found.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds integration tests to verify the behavior of the copy-db command with the blob store, including test configurations, a GraphQL schema, and custom resources for seeding and verification. The review feedback suggests cleaning up the temporary directory created during tests in the after hook using rmSync within a try/finally block, and guarding the copyExit assignment against non-numeric error codes to prevent type mismatches.

import { suite, test, before, after } from 'node:test';
import { ok, strictEqual } from 'node:assert';
import { resolve, join } from 'node:path';
import { readdirSync, statSync, existsSync, mkdtempSync } from 'node:fs';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Import rmSync from node:fs to clean up the temporary directory created during the test execution.

Suggested change
import { readdirSync, statSync, existsSync, mkdtempSync } from 'node:fs';
import { readdirSync, statSync, existsSync, mkdtempSync, rmSync } from 'node:fs';

Comment on lines +121 to +123
after(async () => {
await teardownHarper(ctx);
});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The temporary directory copyRoot created via mkdtempSync is not cleaned up after the test suite finishes. In test teardown hooks, ensure cleanup operations for temporary resources are guarded and wrap independent teardown steps in try/finally blocks to guarantee all cleanup executes and to prevent secondary errors from masking the original setup failure.

		after(async () => {
			try {
				await teardownHarper(ctx);
			} finally {
				if (copyRoot && existsSync(copyRoot)) {
					rmSync(copyRoot, { recursive: true, force: true });
				}
			}
		});
References
  1. In test teardown hooks, ensure cleanup operations for temporary resources are guarded and wrap independent teardown steps in try/finally blocks to guarantee all cleanup executes.

Comment on lines +149 to +152
} catch (e: any) {
copyExit = e.code ?? 1;
copyOut = `${e.stdout ?? ''}\n${e.stderr ?? ''}`;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The e.code property on errors thrown by execFile can be a string (e.g., 'ENOENT') or a number. Assigning it directly to copyExit (which is typed as number | null) can cause a type mismatch. Guard the assignment to ensure it is a number.

Suggested change
} catch (e: any) {
copyExit = e.code ?? 1;
copyOut = `${e.stdout ?? ''}\n${e.stderr ?? ''}`;
}
} catch (e: any) {
copyExit = typeof e.code === 'number' ? e.code : 1;
copyOut = (e.stdout ?? '') + '\n' + (e.stderr ?? '');
}

kriszyp and others added 2 commits August 8, 2026 19:07
harper-integration-test-run (@harperfast/integration-testing) sets
process.exitCode = 1 on node:test's raw test:fail event without checking
data.todo, and node:test still emits that event for a failing todo test.
So the two `{ todo }`-marked contract assertions in copy-db-blob-store.test.ts
(documenting known harper#2048 ch.3/4 bugs, intentionally non-blocking) were
failing CI exactly as hard as a real assertion, across every job config.

Add assertContractOrDiagnose(), which keeps { todo } for readable output but
catches the assertion internally and reports it via t.diagnostic() instead of
letting it throw out of the test body, so no raw test:fail event fires. The
upstream runner bug is filed as a Finding for a separate fix in
@harperfast/integration-testing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MZwTDr3RLhKo9F5hzh4inf
Address pre-push review findings on the prior commit:
- assertContractOrDiagnose() was wrapping the ch.3 vacuity guard
  (inCopy.length > 0) along with the actual contract assertion, so a
  harness failure (e.g. copy-db never running) would misreport as
  "known bug #2048" instead of throwing for real. Move the guard
  outside the wrapper.
- The mkdtemp'd copy target was never removed; clean it up in after().
- Normalize e.code to a number on child-process failure so a signal
  name string can't silently satisfy `copyExit === 0` downstream.
- Trim the duplicated runner-bug explanation to one place.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MZwTDr3RLhKo9F5hzh4inf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant