fix(service): don't report READY before the endpoint accepts connections - #184
Draft
gonzaloserrano wants to merge 1 commit into
Draft
fix(service): don't report READY before the endpoint accepts connections#184gonzaloserrano wants to merge 1 commit into
gonzaloserrano wants to merge 1 commit into
Conversation
service create/start/fork waited only on the control-plane status field, which flips to READY before Postgres binds its port. ready.go already documents READY as "accepting connections", so callers that connect immediately raced it and got ECONNREFUSED. Add WaitForConnectable, which probes the endpoint until it answers, and run it after the status wait. Any Postgres protocol error counts as serving so the probe works without credentials; 57P03 keeps waiting. Best-effort: an unverified endpoint warns rather than failing, since a VPC-only or allowlisted service is legitimately unreachable from the CLI host while being healthy.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
service createwaits on the control-planestatusfield, which flips to READY before Postgres binds its port.internal/common/ready.goalready documents READY as "accepting connections". Callers connecting immediately getECONNREFUSED.Prod repro in
timescale/migration-smoke-test(run 30977861902): READY 20.1s after create, refused 91ms later. Twice in 40 runs.startandforkshare the pattern.WaitForConnectablepolls the endpoint after the status wait, capped at 2 minutes. Any Postgres protocol error counts as serving:28P01proves the server is up, just refusing our credentials, so no password is needed.57P03is the exception and keeps waiting.Best-effort: unverifiable endpoints warn rather than fail; a VPC-only service is healthy but unreachable from here.
For review: do
start/forkbelong here, and should the budget be a flag? Call sites covered only by compilation (create tests use--no-wait).