Skip to content

Implement PortalSuspended for Execute's max_rows cap - #1037

Draft
posthog[bot] wants to merge 2 commits into
mainfrom
posthog-self-driving/fixserver-implement-portalsuspended-so-a3b3dc
Draft

Implement PortalSuspended for Execute's max_rows cap#1037
posthog[bot] wants to merge 2 commits into
mainfrom
posthog-self-driving/fixserver-implement-portalsuspended-so-a3b3dc

Conversation

@posthog

@posthog posthog Bot commented Aug 5, 2026

Copy link
Copy Markdown

Problem

  • Managed Warehouse (and any pgwire client that sets a non-zero fetch size over the extended protocol — Hex, JDBC drivers, psycopg2 named cursors) silently receives a truncated result set labeled as complete.
  • handleExecute reads Execute's max_rows and caps streamSelectRows at that count, then reports CommandComplete SELECT <n> instead of PostgreSQL's PortalSuspendedPortalSuspended was absent from the repo entirely.
  • The row already fetched when the cap tripped was also silently dropped: the stream loop called rows.Next() (advancing past the cap) before checking whether the cap had been hit, so that row was consumed from the cursor and thrown away.
  • psql and the PostHog SQL editor use max_rows = 0 and never hit this path, which is why the bug went unnoticed there.

Fix

  • streamSelectRows (server/conn_query_exec.go) now checks the max_rows cap before calling rows.Next(), so a capped stream never advances past (and drops) a row it didn't send. The result set is left open and correctly positioned to continue.
  • handleExecute (server/conn_extended_query.go) sends the new wire.WritePortalSuspended message when the cap trips, and keeps the portal's RowSet open (portal.openRows/openCols/openColTypes/openTypeOIDs/openRowsSent) instead of closing it — the next Execute on the same portal resumes the same query rather than re-running it from scratch (which would have duplicated every row already sent).
  • Suspended RowSets are cleaned up on Close ('P'), on Bind re-using a portal name, and on connection teardown (closeAllOpenPortalRows, mirroring the existing cursor cleanup), so nothing leaks.
  • Added wire.MsgPortalSuspended / wire.WritePortalSuspended (PostgreSQL's 's' backend message) to server/wire/protocol.go.

Testing

  • server/conn_extended_query_portal_suspend_test.go drives handleExecute directly at the pgwire-message level: a 3-row result with max_rows=1 sends exactly 1 DataRow + PortalSuspended (no CommandComplete), leaves the RowSet open with no row skipped, and a second Execute resumes to deliver exactly the remaining 2 rows with a correct cumulative SELECT 3 tag — asserting no dropped and no duplicated rows across the boundary.
  • Updated docs/postgres-compatibility.md to document the previously-undocumented gap and the fix.

Agent context

This is a core wire-protocol/query-execution fix (reproducible in every run mode, not specific to the multitenant K8s activation pipeline), so coverage was added as a targeted unit test exercising the exact code path rather than in tests/e2e-mw-dev/harness.sh, whose assertions are scoped to the K8s worker-activation pipeline (spawn/attach/DuckLake wiring) this change doesn't touch.


Created with PostHog Desktop from this inbox report.

A client Execute with a non-zero max_rows (any BI tool/driver that sets a
fetch size over the extended protocol) hit the cap in streamSelectRows and
was reported to the client as a complete CommandComplete SELECT <n> instead
of PortalSuspended — silent truncation with no error. The row already
fetched from the DB when the cap tripped was also dropped, since the loop
called rows.Next() before checking the cap.

Fix: check the cap before advancing the RowSet (so no row is lost), send
PortalSuspended when it trips, and keep the portal's RowSet open so the next
Execute resumes the same query instead of restarting it (which would have
duplicated rows).

Branch: posthog-self-driving/fixserver-implement-portalsuspended-so-a3b3dc

Generated-By: PostHog Code
Task-Id: b830fd36-c3cf-4d21-81e1-235cfc4afd40
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown

Test Impact Plan

Deterministic summary of how this PR changes tests, CI runners, and coverage-risk signals.

Summary

Area Added Changed Deleted
Test files 1 1 0
E2E/journey files 0 0 0
Workflow files 0 0 0

Signals

  • Test cases: +2 / -1
  • Assertions: +26 / -1
  • Skips or known failures added: 0
  • Workflow continue-on-error added: 0
  • Workflow path filters added: 0
  • Test commands removed from justfile: 0
  • E2E/journey retry lines added: 0

Coverage risk: neutral or increased

No coverage-reduction warnings detected.

…truncation behavior

CI caught that TestExtendedExecuteMaxRowsUnchanged asserted the pre-fix bug
(CommandComplete on a capped Execute) as expected behavior. Replaced with
TestExtendedExecuteMaxRowsSuspendsAndResumes, which asserts PortalSuspended is
sent, the RowSet stays open with no row dropped at the cap boundary, and a
follow-up Execute on the same portal resumes and delivers the remaining rows
exactly once.

Generated-By: PostHog Code
Task-Id: b830fd36-c3cf-4d21-81e1-235cfc4afd40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants