Skip to content

refactor(rivetkit): centralize internal sqlite storage#5446

Merged
NathanFlurry merged 8 commits into
mainfrom
stack/refactor-rivetkit-centralize-internal-sqlite-storage-vtyvyqrn
Jul 20, 2026
Merged

refactor(rivetkit): centralize internal sqlite storage#5446
NathanFlurry merged 8 commits into
mainfrom
stack/refactor-rivetkit-centralize-internal-sqlite-storage-vtyvyqrn

Conversation

@NathanFlurry

Copy link
Copy Markdown
Member

No description provided.

@claude

claude Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Review: refactor(rivetkit): centralize internal sqlite storage

Overall this is a clean, well-tested refactor. The SQL-constant consolidation into internal_storage/queries.rs removes duplicate literals (e.g. READ_SCHEMA_VERSION_SQL and LOAD_META_TEXT_SQL were identical strings, now one constant), and the test-schema fix in schema.rs::initialize_test_schema is a genuine improvement: it replaces a hand-maintained TEST_INTERNAL_SCHEMA_SQL copy (plus a drift-guard test comparing it against production) with a function that runs the real MIGRATIONS array directly, so schema drift between test and production is now structurally impossible rather than just tested against.

Potential issue: loosened remote SQLite batch validation (engine/packages/pegboard-envoy/src/ws_to_tunnel_task.rs) -- handle_remote_sqlite_execute_batch previously enforced two batch-level limits: request.statements.len() > MAX_REMOTE_SQL_BATCH_STATEMENTS (256, reject), and aggregate bind-param bytes across all statements in the batch greater than MAX_REMOTE_SQL_BIND_BYTES (128 KiB, reject). The new validate_remote_sqlite_batch only calls validate_remote_sqlite_params per statement, i.e. it caps each individual statement's bind bytes at 128 KiB but drops both the statement-count cap and the aggregate-bytes cap entirely. The new test (validate_remote_sqlite_batch_preserves_per_statement_limits) confirms this is intentional behavior rather than an oversight in the test itself, but I want to flag it because: the connection between envoy and pegboard-envoy is explicitly called out as an untrusted boundary in CLAUDE.md ("Validate and authorize all envoy-originated data at pegboard-envoy before it reaches trusted internal systems"); the only remaining upstream bound is the generic WebSocket max_message_size, which defaults to 32 MiB (DEFAULT_WEBSOCKET_MAX_MESSAGE_SIZE in engine/packages/config/src/config/guard.rs), roughly two hundred times larger than the old 128 KiB aggregate cap, and it doesn't bound statement count at all -- a single request can now carry many thousands of near-empty statements, or tens of MiB of aggregate bind params via many under-128KiB statements, none of which the old checks would have allowed; and no compensating limit was found downstream in depot-client. If this loosening is intentional (e.g. per-statement checks are now considered sufficient, or the aggregate/count caps no longer matter for some reason), a one-line comment explaining why would help future readers. Otherwise, consider keeping an aggregate bind-byte cap and/or statement-count cap in validate_remote_sqlite_batch alongside the per-statement check.

Nice catch: schedule history prune-counter fix (rivetkit-rust/packages/rivetkit-core/src/actor/schedule.rs) -- Splitting should_prune_global_history (now a pure .load() check) from record_schedule_history_inserted (the .fetch_add, now only called after execute_batch succeeds) fixes a real bug: previously the counter was incremented as a side effect of the should_prune_global_history check itself, before the SQL batch was known to succeed, so a failed history insert would still advance the pruning cadence. The new failed_history_insert_keeps_global_pruning_due test in tests/schedule.rs covers this directly. Good fix, well tested.

Minor nit: rivetkit-rust/packages/rivetkit-core/tests/context.rs has a leftover double blank line where the now-unused use crate::sqlite::ColumnValue; import was removed (around line 24-25). Purely cosmetic.

Test coverage: The Rust-side changes are well covered (new batch-validation test, new prune-counter regression test, sql_efficiency catalog updated for the renamed/relocated query constants). No RivetKit TypeScript/NAPI surface touched here, so no parity concerns.

@NathanFlurry
NathanFlurry changed the base branch from stack/test-rivetkit-simplify-sqlite-efficiency-coverage-spmlntnp to main July 20, 2026 08:37
@NathanFlurry
NathanFlurry merged commit c1e6f0f into main Jul 20, 2026
4 of 6 checks passed
@NathanFlurry
NathanFlurry deleted the stack/refactor-rivetkit-centralize-internal-sqlite-storage-vtyvyqrn branch July 20, 2026 08:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant