fix: use CONCURRENTLY only for non-empty schemas to prevent deadlock#493
Closed
cdbartholomew wants to merge 1 commit intomainfrom
Closed
fix: use CONCURRENTLY only for non-empty schemas to prevent deadlock#493cdbartholomew wants to merge 1 commit intomainfrom
cdbartholomew wants to merge 1 commit intomainfrom
Conversation
CREATE INDEX CONCURRENTLY waits for ALL open transactions database-wide before the index becomes valid. When a new tenant schema is provisioned at runtime, the worker task poller holds an idle-in-transaction connection from its polling loop. CONCURRENTLY sees this and waits forever — deadlocking the server. This happens whether the worker runs in-process (dev/tests) or in a separate pod (production), since they share the same database. Fix: check if the target table has data before choosing the index creation strategy. Empty tables (new tenant provisioning) use regular CREATE INDEX which is instant and has no deadlock risk. Tables with data (existing schemas during rolling deployments) continue to use CONCURRENTLY to avoid blocking writes on large tables.
Collaborator
|
the real problem is that the worker keeps the transaction open for the entire consolidation - I'm gonna fix that and close this one - we need to be able to create indices with CONCURRENTLY or they will be super slow |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CREATE INDEX CONCURRENTLYin migration files hangs the server when provisioning new tenant schemas while the worker is runningCONCURRENTLYwaits for ALL open transactions database-wide. The worker's polling loop always has a transaction open, soCONCURRENTLYwaits foreverFix
Migrations now check whether the target table has data before choosing the index creation strategy:
CREATE INDEX— instant, no deadlock riskCREATE INDEX CONCURRENTLY— avoids blocking writes on large tablesTest plan