Implement PortalSuspended for Execute's max_rows cap - #1037
Draft
posthog[bot] wants to merge 2 commits into
Draft
Conversation
A client Execute with a non-zero max_rows (any BI tool/driver that sets a fetch size over the extended protocol) hit the cap in streamSelectRows and was reported to the client as a complete CommandComplete SELECT <n> instead of PortalSuspended — silent truncation with no error. The row already fetched from the DB when the cap tripped was also dropped, since the loop called rows.Next() before checking the cap. Fix: check the cap before advancing the RowSet (so no row is lost), send PortalSuspended when it trips, and keep the portal's RowSet open so the next Execute resumes the same query instead of restarting it (which would have duplicated rows). Branch: posthog-self-driving/fixserver-implement-portalsuspended-so-a3b3dc Generated-By: PostHog Code Task-Id: b830fd36-c3cf-4d21-81e1-235cfc4afd40
Test Impact PlanDeterministic summary of how this PR changes tests, CI runners, and coverage-risk signals. Summary
Signals
Coverage risk: neutral or increased No coverage-reduction warnings detected. |
…truncation behavior CI caught that TestExtendedExecuteMaxRowsUnchanged asserted the pre-fix bug (CommandComplete on a capped Execute) as expected behavior. Replaced with TestExtendedExecuteMaxRowsSuspendsAndResumes, which asserts PortalSuspended is sent, the RowSet stays open with no row dropped at the cap boundary, and a follow-up Execute on the same portal resumes and delivers the remaining rows exactly once. Generated-By: PostHog Code Task-Id: b830fd36-c3cf-4d21-81e1-235cfc4afd40
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
handleExecutereads Execute'smax_rowsand capsstreamSelectRowsat that count, then reportsCommandComplete SELECT <n>instead of PostgreSQL'sPortalSuspended—PortalSuspendedwas absent from the repo entirely.rows.Next()(advancing past the cap) before checking whether the cap had been hit, so that row was consumed from the cursor and thrown away.max_rows = 0and never hit this path, which is why the bug went unnoticed there.Fix
streamSelectRows(server/conn_query_exec.go) now checks themax_rowscap before callingrows.Next(), so a capped stream never advances past (and drops) a row it didn't send. The result set is left open and correctly positioned to continue.handleExecute(server/conn_extended_query.go) sends the newwire.WritePortalSuspendedmessage when the cap trips, and keeps the portal'sRowSetopen (portal.openRows/openCols/openColTypes/openTypeOIDs/openRowsSent) instead of closing it — the nextExecuteon the same portal resumes the same query rather than re-running it from scratch (which would have duplicated every row already sent).RowSets are cleaned up onClose('P'), onBindre-using a portal name, and on connection teardown (closeAllOpenPortalRows, mirroring the existing cursor cleanup), so nothing leaks.wire.MsgPortalSuspended/wire.WritePortalSuspended(PostgreSQL's's'backend message) toserver/wire/protocol.go.Testing
server/conn_extended_query_portal_suspend_test.godriveshandleExecutedirectly at the pgwire-message level: a 3-row result withmax_rows=1sends exactly 1DataRow+PortalSuspended(noCommandComplete), leaves theRowSetopen with no row skipped, and a secondExecuteresumes to deliver exactly the remaining 2 rows with a correct cumulativeSELECT 3tag — asserting no dropped and no duplicated rows across the boundary.docs/postgres-compatibility.mdto document the previously-undocumented gap and the fix.Agent context
This is a core wire-protocol/query-execution fix (reproducible in every run mode, not specific to the multitenant K8s activation pipeline), so coverage was added as a targeted unit test exercising the exact code path rather than in
tests/e2e-mw-dev/harness.sh, whose assertions are scoped to the K8s worker-activation pipeline (spawn/attach/DuckLake wiring) this change doesn't touch.Created with PostHog Desktop from this inbox report.