Enable logical replication slots (wal_level=logical + pgoutput fixes)#1047
Enable logical replication slots (wal_level=logical + pgoutput fixes)#1047riderx wants to merge 2 commits into
Conversation
- Start Postgres with wal_level=logical in defaultStartParams so pg_create_logical_replication_slot works out of the box. - Surface unexpected WASM exceptions from PostgresMainLoopOnce instead of swallowing them: previously a crash (e.g. a call into a missing symbol) was silently ignored, leaving the session returning empty results until it hit 'ERRORDATA_STACK_SIZE exceeded'. - Add tests covering slot creation/listing/drop, decoding changes with pgoutput, error recovery, and slot persistence across dump/load. Requires the postgres-pglite change that exports the symbols pgoutput.so imports from the main module (electric-sql/postgres-pglite side of this fix). Fixes electric-sql#880 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4aaa87118e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| '-c', | ||
| 'max_parallel_maintenance_workers=0', | ||
| '-c', | ||
| 'wal_level=logical', // enable logical decoding / replication slots |
There was a problem hiding this comment.
Bump postgres-pglite for pgoutput exports
This enables wal_level=logical and documents logical slot support, but the commit does not include the required wasm-side symbol export fix: git ls-tree shows both this commit and its parent still pin postgres-pglite to 7b4ee5086055dc5e54ae1e13e487888249438e68, and the root wasm:build script builds the release wasm from that submodule. In a clean build of this tree, pgoutput still lacks the exported core symbols described in the commit message, so calls like pg_logical_slot_get_binary_changes(..., 'pgoutput') will still hit the missing-symbol crash instead of delivering changes; please include the submodule bump before advertising/enabling this feature.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Good catch — addressed in 6d55f6c: the submodule now points at the head of electric-sql/postgres-pglite#79 (04f1bea), which contains the pgoutput symbol-export fix. That commit is served by the upstream repo via refs/pull/79/head, so git submodule update can fetch it by SHA (verified with a git fetch --depth 1 <upstream> 04f1bea from a fresh clone). A clean wasm:build of this tree now produces a main module that exports all 46 symbols pgoutput.so imports, and the new replication-slot tests pass against it. Once #79 merges, this pointer should be bumped to the merge commit.
Points at the head of electric-sql/postgres-pglite#79 (served by the upstream repo via refs/pull/79/head, so submodule checkout by SHA works). Should be updated to the merge commit once that PR lands. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
hey @tdrz i would love your input on that, i'm really looking for logical replication in pglite for a very specific case where i need to replicate in many regions. |
Problem
Fixes #880 — PGlite cannot use logical replication slots.
Two separate things were broken:
wal_leveldefaults toreplica, sopg_create_logical_replication_slot()fails withlogical decoding requires "wal_level" >= "logical".wal_level=logical(via customstartParams), consuming a slot withpgoutputcrashed the wasm instance: the-sMAIN_MODULE=2build dead-code-eliminates the 46 core symbols that onlypgoutput.soimports (defGetStreamingMode,logicalrep_write_*,GetRelationPublications, …), becausepgoutputis built fromsrc/backendand — unlike pgxs extensions — never got a.importsfile feeding the exported-functions list. The first decode call hit an undefined table entry (TypeError: r is not a function), andexecProtocolRawSyncsilently swallowed it, leaving the session returning empty results for every subsequent query until Postgres PANICs withERRORDATA_STACK_SIZE exceeded.Changes
defaultStartParams: add-c wal_level=logicalso replication slots work out of the box. (WAL volume impact is negligible for an embedded single-user database.)execProtocolRawSync: rethrow exceptions that are not an emscriptenExitStatusinstead of swallowing them. A trap or missing-symbol call means the wasm instance state can no longer be trusted; surfacing the error beats silent corruption. Known exit statuses (including thePOSTGRES_MAIN_LONGJMPerror-recovery path) behave exactly as before.tests/replication-slots.test.ts): slot create/list/drop, decoding inserts/updates/deletes throughpg_logical_slot_peek_binary_changes/get_binary_changeswithpgoutput, clean error recovery when decoding options are invalid, and slot persistence across adumpDataDir/loadDataDircycle.startParamssection ofapi.md.Companion PR
The symbol-export fix lives in the submodule: electric-sql/postgres-pglite#79. The submodule pointer in this PR points at the head of that PR (fetchable from the upstream repo via
refs/pull/79/head), so a cleanwasm:buildof this tree includes the fix; it should be re-bumped to the merge commit once #79 lands.Verification
With the rebuilt wasm, all 5 new tests pass and the full existing suite passes (267 tests, no regressions):
🤖 Generated with Claude Code