feat(sdk:go): Phase 11.11c — cross-pool sibling shape via path registry (SQLR-22)#134
Merged
Conversation
…ry (SQLR-22) Closes the last open item from Phase 11's SDK arc. Pre-11.11c the Go SDK took a real engine `Connection::open` for every `sql.Open(…)` call — which deadlocked against itself on `flock(LOCK_EX)` whenever two `*sql.DB` instances pointed at the same file, or a single pool grew past one connection. The C / Python / Node SDKs already had sibling-handle shapes since Phase 11.8; Go was the holdout because `database/sql`'s pool model expects `driver.Open` to be cheap + idempotent and SQLRite's exclusive flock collided with that contract. The fix: a process-level path registry in `sdk/go/sqlrite.go` keyed by canonical absolute path (`filepath.Abs` + `filepath.Clean`). For file-backed read-write opens: 1. First opener pays for a real `sqlrite_open` → handle stored as a hidden "primary" in the registry, refcount = 0. 2. Subsequent openers mint a sibling via the FFI's `sqlrite_connect_sibling(primary)` (shipped in 11.8). Each `*conn` owns its own sibling; refcount++. 3. Close: refcount--. When 0, the registry closes the primary and removes the entry. Lock order: `c.mu` → `registryMu`, never the reverse. `newConn` holds only `registryMu` (the `*conn` doesn't exist yet); `conn.Close` takes `c.mu` first, then `registryMu`. Scope (v0): - `:memory:` opens bypass the registry — each is its own DB by design (matches SQLite). - Read-only opens (`sqlrite.OpenReadOnly`) bypass too — they take a shared `flock(LOCK_SH)` that already coexists with other readers. - Symlinks are NOT resolved; the key is lexical. Callers needing symlink-equality canonicalize via `os.EvalSymlinks`. Tests (`sdk/go/sqlrite_test.go`): - `TestTwoSqlOpenOnSameFileShareState` — two `*sql.DB`s on the same path see each other's writes immediately, bidirectional. - `TestBeginConcurrentAcrossSqlOpenInstances` — pinned `*sql.Conn`s from two different pools each hold their own `BEGIN CONCURRENT`; A's commit wins, B's hits `ErrBusy` and retries; final value matches `0 + 1 + 100 = 101`. - `TestRegistryRefcountDropsToZeroOnLastClose` — after the last sibling closes, a fresh `sql.Open` on the same path succeeds (proves the flock was released and the entry removed). Docs: - `sdk/go/README.md` — new "Multi-handle reads + writes" section with the cross-pool runnable example + BEGIN CONCURRENT retry loop demo. Caveats called out (`:memory:` isolation, read-only bypass, symlinks). - `docs/concurrent-writes.md` — SDK propagation table refreshed to reflect Go cross-pool support; the "still constructs its own backing DB" note replaced with the registry's actual behaviour. - `docs/_index.md` — Phase 11 summary blurb updated to reflect the end-to-end story (only 11.10 indexes-under-MVCC and the checkpoint-drain follow-up remain). - `docs/roadmap.md` — Phase 11.11c promoted to ✅ shipped; active- frontier blurb refreshed. - `docs/design-decisions.md` — new §12i covering the registry rationale, lock order, scope decisions, and symlink caveat. Workspace: 615/615 Rust tests pass; `go test ./...` in `sdk/go` covers all existing tests plus the 3 new ones in ~1s. fmt + clippy + doc all clean. Phase 11's SDK arc is now done end-to-end: C / Python / Node / Go all mint sibling handles that share `Arc<Mutex<Database>>`; the 11.7 retryable-error machinery is exerciseable cross-pool from every shipped SDK. Only WASM remains untouched (single-threaded runtime, sibling story doesn't apply). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
… method `t.Context()` was added in Go 1.24, but `sdk/go/go.mod` declares Go 1.21 and CI runs an older toolchain. The three new tests added in 11.11c (`TestBeginConcurrentAcrossSqlOpenInstances` + `TestRegistryRefcountDropsToZeroOnLastClose`) used `t.Context()` which compiled locally on Go 1.24+ but broke CI on Go 1.21. Replace all three sites with `context.Background()` (available since Go 1.7), add the `context` import. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Connection::openfor everysql.Open(…)call — which deadlocked against itself onflock(LOCK_EX)whenever two*sql.DBinstances pointed at the same file, or a single pool grew past one connection.sqlrite_open; subsequent openers mint sibling handles off it via the FFI'ssqlrite_connect_sibling(shipped in 11.8), sharingArc<Mutex<Database>>underneath. Refcounted: the last sibling out firessqlrite_closeon the primary.*sql.DBstate sharing,BEGIN CONCURRENTacross separate pools with a real Busy + retry (final value0 + 1 + 100 = 101), and the refcount-to-zero roundtrip.The headline cross-pool demo
Architecture
Scope decisions (v0)
:memory:sqlrite.OpenReadOnly)flock(LOCK_SH)already coexists with other readersfilepath.Abs+filepath.Clean). Callers needing symlink-equality passos.EvalSymlinks-canonicalized pathsWhere this fits in the Phase 11 SDK arc
sqlrite_connect_siblingconn.connect()db.connect()sqlrite_connect_siblingDocs
sdk/go/README.md— new "Multi-handle reads + writes" section with the cross-pool runnable example + BEGIN CONCURRENT retry-loop demo + caveats.docs/concurrent-writes.md— SDK propagation table updated; the "still constructs its own backing DB" note replaced with the registry's actual behaviour.docs/_index.md— Phase 11 summary blurb refreshed to reflect end-to-end completion.docs/roadmap.md— Phase 11.11c promoted to ✅ shipped; active-frontier blurb refreshed.docs/design-decisions.md— new §12i covering the registry rationale, lock order, scope decisions, symlink caveat.Test plan
cargo build --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks --all-targetscargo test --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks— 615/615cargo fmt --all -- --checkcargo clippy --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks --all-targetscargo doc --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks --no-depscd sdk/go && go test -count=1 -timeout 120s ./...— all existing + 3 new tests pass in ~1sWhat's left of Phase 11
This wraps the SDK arc. The only remaining items are deferred-by-design or foundation work:
set_journal_mode(Mvcc → Wal)onceMvStoreis drainable.🤖 Generated with Claude Code