Skip to content

DRIVER-153: negotiate and implement SCYLLA_USE_METADATA_ID extension#770

Draft
nikagra wants to merge 1 commit intoscylladb:masterfrom
nikagra:driver-153-scylla-use-metadata-id
Draft

DRIVER-153: negotiate and implement SCYLLA_USE_METADATA_ID extension#770
nikagra wants to merge 1 commit intoscylladb:masterfrom
nikagra:driver-153-scylla-use-metadata-id

Conversation

@nikagra
Copy link
Copy Markdown

@nikagra nikagra commented Mar 26, 2026

Summary

Implements the SCYLLA_USE_METADATA_ID Scylla CQL protocol extension (DRIVER-153), which backports the prepared-statement metadata-ID mechanism from CQL v5 to earlier protocol versions.

When the extension is negotiated:

  • The server includes a result metadata hash in the PREPARE response
  • The driver sends that hash back with every EXECUTE request, allowing the server to skip sending full result metadata on every response (skip_meta=True)
  • If the result schema has changed, the server sets the METADATA_CHANGED flag and includes the new metadata ID + new column metadata in the response — the driver picks this up and updates its cached metadata automatically

Changes

cassandra/protocol_features.py

  • Add USE_METADATA_ID = "SCYLLA_USE_METADATA_ID" constant and use_metadata_id field to ProtocolFeatures
  • Parse the extension from the SUPPORTED frame; include it in STARTUP when present

cassandra/protocol.py

  • Bug fix: _write_query_params now actually writes _SKIP_METADATA_FLAG on the wire — it was stored on _QueryMessage but never sent (effectively dead code)
  • recv_results_prepared: read result_metadata_id for Scylla extension (pre-v5) in addition to standard CQL v5+
  • ExecuteMessage.send_body: send result_metadata_id for Scylla extension (pre-v5) when set

cassandra/cluster.py

  • skip_meta is now True only when safe: CQL v5 is used or SCYLLA_USE_METADATA_ID was negotiated (proxied by a non-None result_metadata_id on the prepared statement). Otherwise False — always fetch full metadata (safest option).
  • _set_result: when the EXECUTE response contains a new result_metadata_id (METADATA_CHANGED), update prepared_statement.result_metadata and result_metadata_id to keep the cached metadata in sync

Test plan

  • Unit tests written first (TDD) — 7 new tests covering feature negotiation, STARTUP options, skip_meta flag encoding, Scylla metadata_id in ExecuteMessage (v4), and PREPARE response decoding with/without extension
  • Full unit test suite passes (627 passed, 97 skipped)
  • Integration tests against a Scylla node with the extension: verify that schema changes after PREPARE are detected and metadata is updated without re-preparation

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Implements negotiation and support for Scylla’s SCYLLA_USE_METADATA_ID protocol extension to enable metadata-id based skip_meta behavior (backporting CQL v5 prepared-statement metadata-id semantics to earlier protocol versions).

Changes:

  • Adds SCYLLA_USE_METADATA_ID parsing from SUPPORTED and includes it in STARTUP when negotiated.
  • Extends protocol encode/decode to read/write result_metadata_id for PREPARE/EXECUTE on pre-v5 when the extension is used, and fixes on-wire encoding of _SKIP_METADATA_FLAG.
  • Updates execution/result handling to conditionally use skip_meta and to refresh cached prepared metadata when the server reports metadata changes.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
cassandra/protocol_features.py Adds the SCYLLA_USE_METADATA_ID feature flag and includes it in negotiated STARTUP options.
cassandra/protocol.py Writes _SKIP_METADATA_FLAG in query params; adds pre-v5 extension handling for result_metadata_id in PREPARE/EXECUTE.
cassandra/cluster.py Adjusts when skip_meta is enabled and updates cached prepared metadata/id on METADATA_CHANGED responses.
tests/unit/test_protocol_features.py Adds unit tests for feature parsing and STARTUP option inclusion.
tests/unit/test_protocol.py Adds unit tests for skip-meta flag encoding and metadata-id handling in pre-v5 PREPARE/EXECUTE paths.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Scylla's SCYLLA_USE_METADATA_ID protocol extension (backport of CQL v5
prepared-statement metadata IDs to earlier protocol versions) allows the
driver to skip sending full result metadata on every EXECUTE request.
The server notifies the driver via the METADATA_CHANGED flag whenever
the result schema changes, at which point the driver updates its cached
metadata before deserialising the response.

Changes:
- protocol_features.py: parse SCYLLA_USE_METADATA_ID from SUPPORTED and
  include it in the STARTUP frame when negotiated
- protocol.py:
  * fix _write_query_params to actually write _SKIP_METADATA_FLAG on the
    wire (it was stored on the message but never sent — dead code before)
  * recv_results_prepared: read result_metadata_id for Scylla extension
    (pre-v5) in addition to standard protocol v5+
  * ExecuteMessage.send_body: send result_metadata_id for Scylla
    extension (pre-v5) when it is set
- cluster.py:
  * ExecuteMessage is built with safe defaults (skip_meta=False,
    result_metadata_id=None); both are set in _query() after borrowing
    the connection, gated on connection.features.use_metadata_id and on
    the prepared statement actually having a result_metadata_id (so a
    statement prepared before the extension was available, or on a node
    that doesn't support it, never gets skip_meta=True with no id)
  * _set_result: update prepared_statement.result_metadata and
    result_metadata_id when the server signals METADATA_CHANGED in an
    EXECUTE response, keeping the driver's cached metadata in sync;
    uses getattr to safely handle FastResultMessage (Cython decoder)
@nikagra nikagra force-pushed the driver-153-scylla-use-metadata-id branch from ade35d8 to f42e225 Compare March 27, 2026 12:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants