docs: draft delayed-delivery rotation blueprint#237
Draft
NikolayS wants to merge 2 commits into
Draft
Conversation
Two-table TRUNCATE rotation for pgque.delayed_events modeled on PgQ's event_N pattern. Removes DELETE/UPDATE/VACUUM from the delayed-delivery hot path so send_at() can serve as a primary write path without dead-tuple churn or catalog growth.
cancel_scheduled() is the single exception to the no-DELETE invariant. Dead tuples it produces are bounded above by one rotation interval and TRUNCATEd away on the next rotation that targets the affected table. The bloat-free property of the design is therefore conditional on a low cancellation rate. That caveat is required to appear: - in the SQL comment on cancel_scheduled() - in docs/reference.md next to the function entry - in tutorials / examples that introduce cancellation - in the send_at() function comment For very-high-cancel workloads, docs must steer users to the application-level "skip on consume" pattern instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Draft blueprint for replacing the current single-heap
pgque.delayed_eventsdesign with a two-table TRUNCATE rotation, modeled on PgQ's
event_Npattern. The goal is to make
pgque.send_at()viable as a primary writepath without the dead-tuple churn, index bloat, and
VACUUMdependencethe current experimental design would incur at scale.
Key properties of the proposed design:
DELETEand noUPDATEon delayed-event rows. Reclamation isTRUNCATE-only.VACUUMdependence on the hot path.CREATE/DROPchurn.floor.
1 + rotations_crossedper row) forlong-horizon scheduled messages.
Includes a comparison against the alternative (declarative range
partition + drop), explicit invariants, a TDD-shaped implementation
plan, and acceptance criteria phrased against
pg_stat_user_tablesandpg_classso the no-bloat property is testable.Open questions in the draft
These are flagged in the blueprint and want a call before code lands:
default trades catalog stability for write amplification on
long-horizon rows.
pgque.delayed_eventsexposed as a view over the two physical tables,vs. renaming the public surface.
pgque.cancel_scheduled) is deferred to a follow-upblueprint; locking in a tombstone-set design now would be cheap insurance
if cancel is likely-yes for 0.3.
drop-and-recreate (current draft assumes the latter).
Test plan
Blueprint-only change; no code or test diffs. Once approved, the
implementation slice will:
Add a failing
pg_stat_user_tables.n_dead_tup == 0acceptance check totests/acceptance/us4_delayed_delivery.sql.Make it green by replacing the single heap with the rotation scheme.
Capture before/after dead-tuple and catalog-row counts in the
implementation PR description.
review blueprint design and tradeoffs
decide open questions (rotation interval, view vs. rename, cancellation scope)
convert to implementation PR with failing test + rotation code
https://claude.ai/code/session_01STJ9TCTr7ykbwG23t9ZxJv
Generated by Claude Code