Skip to content

Enable logical replication slots (wal_level=logical + pgoutput fixes)#1047

Open
riderx wants to merge 2 commits into
electric-sql:mainfrom
riderx:feat/logical-replication-slots
Open

Enable logical replication slots (wal_level=logical + pgoutput fixes)#1047
riderx wants to merge 2 commits into
electric-sql:mainfrom
riderx:feat/logical-replication-slots

Conversation

@riderx

@riderx riderx commented Jul 7, 2026

Copy link
Copy Markdown

Problem

Fixes #880 — PGlite cannot use logical replication slots.

Two separate things were broken:

  1. wal_level defaults to replica, so pg_create_logical_replication_slot() fails with logical decoding requires "wal_level" >= "logical".
  2. Even with wal_level=logical (via custom startParams), consuming a slot with pgoutput crashed the wasm instance: the -sMAIN_MODULE=2 build dead-code-eliminates the 46 core symbols that only pgoutput.so imports (defGetStreamingMode, logicalrep_write_*, GetRelationPublications, …), because pgoutput is built from src/backend and — unlike pgxs extensions — never got a .imports file feeding the exported-functions list. The first decode call hit an undefined table entry (TypeError: r is not a function), and execProtocolRawSync silently swallowed it, leaving the session returning empty results for every subsequent query until Postgres PANICs with ERRORDATA_STACK_SIZE exceeded.

Changes

  • defaultStartParams: add -c wal_level=logical so replication slots work out of the box. (WAL volume impact is negligible for an embedded single-user database.)
  • execProtocolRawSync: rethrow exceptions that are not an emscripten ExitStatus instead of swallowing them. A trap or missing-symbol call means the wasm instance state can no longer be trusted; surfacing the error beats silent corruption. Known exit statuses (including the POSTGRES_MAIN_LONGJMP error-recovery path) behave exactly as before.
  • Tests (tests/replication-slots.test.ts): slot create/list/drop, decoding inserts/updates/deletes through pg_logical_slot_peek_binary_changes/get_binary_changes with pgoutput, clean error recovery when decoding options are invalid, and slot persistence across a dumpDataDir/loadDataDir cycle.
  • Docs: note the new default in the startParams section of api.md.

Companion PR

The symbol-export fix lives in the submodule: electric-sql/postgres-pglite#79. The submodule pointer in this PR points at the head of that PR (fetchable from the upstream repo via refs/pull/79/head), so a clean wasm:build of this tree includes the fix; it should be re-bumped to the merge commit once #79 lands.

Verification

With the rebuilt wasm, all 5 new tests pass and the full existing suite passes (267 tests, no regressions):

✓ tests/replication-slots.test.ts (5 tests) 4613ms
Test Files  1 passed (1)
     Tests  5 passed (5)

🤖 Generated with Claude Code

- Start Postgres with wal_level=logical in defaultStartParams so
  pg_create_logical_replication_slot works out of the box.
- Surface unexpected WASM exceptions from PostgresMainLoopOnce instead of
  swallowing them: previously a crash (e.g. a call into a missing symbol)
  was silently ignored, leaving the session returning empty results until
  it hit 'ERRORDATA_STACK_SIZE exceeded'.
- Add tests covering slot creation/listing/drop, decoding changes with
  pgoutput, error recovery, and slot persistence across dump/load.

Requires the postgres-pglite change that exports the symbols pgoutput.so
imports from the main module (electric-sql/postgres-pglite side of this fix).

Fixes electric-sql#880

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4aaa87118e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

'-c',
'max_parallel_maintenance_workers=0',
'-c',
'wal_level=logical', // enable logical decoding / replication slots

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Bump postgres-pglite for pgoutput exports

This enables wal_level=logical and documents logical slot support, but the commit does not include the required wasm-side symbol export fix: git ls-tree shows both this commit and its parent still pin postgres-pglite to 7b4ee5086055dc5e54ae1e13e487888249438e68, and the root wasm:build script builds the release wasm from that submodule. In a clean build of this tree, pgoutput still lacks the exported core symbols described in the commit message, so calls like pg_logical_slot_get_binary_changes(..., 'pgoutput') will still hit the missing-symbol crash instead of delivering changes; please include the submodule bump before advertising/enabling this feature.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — addressed in 6d55f6c: the submodule now points at the head of electric-sql/postgres-pglite#79 (04f1bea), which contains the pgoutput symbol-export fix. That commit is served by the upstream repo via refs/pull/79/head, so git submodule update can fetch it by SHA (verified with a git fetch --depth 1 <upstream> 04f1bea from a fresh clone). A clean wasm:build of this tree now produces a main module that exports all 46 symbols pgoutput.so imports, and the new replication-slot tests pass against it. Once #79 merges, this pointer should be bumped to the merge commit.

Points at the head of electric-sql/postgres-pglite#79 (served by the
upstream repo via refs/pull/79/head, so submodule checkout by SHA works).
Should be updated to the merge commit once that PR lands.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@riderx

riderx commented Jul 10, 2026

Copy link
Copy Markdown
Author

hey @tdrz i would love your input on that, i'm really looking for logical replication in pglite for a very specific case where i need to replicate in many regions.
If you see anything i could inprove i'm up for the job, also if you can allow CI to run it's nice

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PGlite replication slot

1 participant