Skip to content

storage: realize replica-scoped dyncfg overrides in storage workers - #38065

Open
DAlperin wants to merge 1 commit into
MaterializeInc:mainfrom
DAlperin:dov/storage-replica-dyncfg-overrides
Open

storage: realize replica-scoped dyncfg overrides in storage workers#38065
DAlperin wants to merge 1 commit into
MaterializeInc:mainfrom
DAlperin:dov/storage-replica-dyncfg-overrides

Conversation

@DAlperin

@DAlperin DAlperin commented Aug 5, 2026

Copy link
Copy Markdown
Member

Motivation

Replica-scoped system parameters are realized as per-replica dyncfg overrides, but only the compute controller carries a per-replica dyncfg layer. On clusterd the compute worker applies pushed updates to its own worker ConfigSet and to the shared persist-client ConfigSet, which covers persist-backed and process-global configs. A dyncfg that the storage worker reads from its own ConfigSet (in its UpdateConfiguration handler) never sees a replica-scoped override: the storage protocol delivers only environment-wide values, so scoping such a flag to a replica is silently inert.

Description

Mirror the compute controller's per-replica dyncfg layer in the storage controller:

  • The storage Instance stores per-replica ConfigUpdates overrides and specializes UpdateConfiguration commands per replica, both at send time and when replaying history to new or reconnecting replicas. The history records base commands, so overrides are re-applied at replay rather than baked into the shared history.
  • New StorageController::update_replica_dyncfg_overrides, with the same replace-and-clear semantics as the compute controller's: instances absent from the pushed map have their overrides cleared.
  • The coordinator's push_replica_dyncfg_overrides now pushes the same override map to both controllers and re-pushes the environment-wide storage config afterward, so existing replicas observe changed overrides and removed overrides revert to environment-wide values (storage_config renders the full dyncfg set).

This unblocks scoping storage-realized flags (for example the upsert-v2 spill gate) to individual replicas.

Verification

Adds a unit test for the storage command specialization: an overridden replica's UpdateConfiguration carries the merged override, while replicas without an override receive the base command. This mirrors the existing compute-side specialization tests.

@DAlperin
DAlperin requested review from a team as code owners August 5, 2026 19:49
@DAlperin
DAlperin requested a review from antiguru August 5, 2026 19:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants