feat(server): pin a session to one worker with duckgres.session_affinity - #1044
Draft
posthog[bot] wants to merge 1 commit into
Draft
feat(server): pin a session to one worker with duckgres.session_affinity#1044posthog[bot] wants to merge 1 commit into
posthog[bot] wants to merge 1 commit into
Conversation
A `-c duckgres.session_affinity=true` startup option keeps a connection off the exploratory small-worker tier, so the control plane acquires the standard worker eagerly and never destroys and re-creates the session on another worker. The exploratory tier only replays catalog init and persistent user secrets onto a new worker. An ad-hoc `ATTACH ... (TYPE postgres)` catalog and an exported transaction snapshot are not replayed, so a mid-session migration surfaced "Catalog db does not exist" and "SET TRANSACTION SNAPSHOT does not exist". The opt-in removes the migration for sessions that need this state. The option needs no size GUC and no AllowClientWorkerProfile gate: it only reproduces the pre-tier default of one standard worker per connection. Generated-By: PostHog Code Task-Id: 051216d4-3aea-4ad9-95f0-28d40ee2d43d
Test Impact PlanDeterministic summary of how this PR changes tests, CI runners, and coverage-risk signals. Summary
Signals
Coverage risk: neutral or increased No coverage-reduction warnings detected. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
escalateWorker(server/conn_tier.go) destroys the current session and creates a new one on another worker. Only catalog init (controlplane/control.go) and persistent user secrets (controlplane/session_mgr.go) replay onto it.ATTACH ... (TYPE postgres)catalog and an exported transaction snapshot do not replay, so statements that land on a different worker fail withCatalog db does not existandSET TRANSACTION SNAPSHOT does not exist.pg_pool_max_connections=1) trades those errors for out-of-memory failures, so there is no safe workaround.Change
-c duckgres.session_affinity=truetakes the connection off the exploratory tier, so it acquires its standard worker eagerly and keeps it for the whole session — no destroy-and-recreate, no migration.useExploratoryTiergains asessionAffinityexclusion, alongside the existing passthrough andduckgres.worker_*exclusions.control.goparses and validates the option before a worker is spawned; a non-boolean value is rejectedFATAL 22023, like the otherduckgres.*startup options.AllowClientWorkerProfile: it only reproduces the pre-tier default of one standard worker per connection.Note on the second reported symptom
ATTACH,DETACH,SET TRANSACTION SNAPSHOT, andBEGIN ... ISOLATION LEVELalready classify astierPinningtoday (parse-failure or default → pin), verified empirically againsttier_classify.go. So the complementary "treat ATTACH as pinning" idea is already in place for a single connection; the remaining gap this PR closes is a whole-session opt-out for sessions whose state must never migrate.Tests
TestParseSessionAffinityOption— boolean parsing and rejection of non-boolean values.TestUseExploratoryTierExclusions— a session-affinity request bypasses the tier.go build ./...,go vet, and the affected package tests pass. (TestCheckSocketDirWritablefails on a clean checkout too — unrelated, it asserts a permission the root test user always has.)Why
The customer's migration is blocked and the dedicated instance they are on is being deprecated, so the failure is on a deadline. The errors are structural — one unguarded code path, not load — so they recur on every attempt.
Created with PostHog Desktop from this inbox report.