Skip to content

Antalya-26.3: Resolve currentDatabase() in Hybrid segment metadata #1995

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mkmkme wants to merge 4 commits into
base: antalya-26.3
Choose a base branch
from
Open

Antalya-26.3: Resolve currentDatabase() in Hybrid segment metadata #1995

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
7ba1a9d
Resolve currentDatabase() in Hybrid segment metadata
mkmkme Jul 1, 2026
081738e
reduce code duplication, fix the original call instead
mkmkme Jul 1, 2026
03fb9e9
replace integration test with a stateless one
mkmkme Jul 1, 2026
1037c7d
fix 03645_hybrid_watermarks after the changes
mkmkme Jul 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions src/Storages/StorageDistributed.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3003,10 +3003,10 @@ void registerStorageHybrid(StorageFactory & factory)

// Normalize arguments (evaluate `currentDatabase()`, expand named collections, etc.).
// TableFunctionFactory::get mutates the AST in-place inside TableFunctionRemote::parseArguments.
ASTPtr normalized_table_function_ast = table_function_ast->clone();
replaceCurrentDatabaseFunction(engine_args[i], local_context);

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 1, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Resolve currentDatabase aliases in persisted segments

This new metadata rewrite only catches AST functions named exactly currentDatabase, but the same function is exposed through accepted aliases such as DATABASE, SCHEMA, and current_database. For an additional segment like remote('localhost:9000', DATABASE(), 'local_cold'), TableFunctionRemote::parseArguments still normalizes the clone used in memory, while the original engine_args[i] keeps the unresolved alias in the stored definition, so reattach/startup from a different current database can still point the segment at the wrong database and raise the same UNKNOWN_TABLE exception this patch is fixing. Please normalize the actual remote/cluster database argument with the existing database-name constant evaluator instead of relying on this exact-name tree walk.

Useful? React with 👍 / 👎.

ASTPtr normalized_table_function_ast = engine_args[i]->clone();
auto additional_table_function = TableFunctionFactory::instance().get(normalized_table_function_ast, local_context);
ColumnsDescription segment_columns = additional_table_function->getActualTableStructure(local_context, true);
replaceCurrentDatabaseFunction(normalized_table_function_ast, local_context);

validate_segment_schema(segment_columns, normalized_table_function_ast->formatForLogging());

Expand Down
12 changes: 6 additions & 6 deletions tests/queries/0_stateless/03645_hybrid_watermarks.reference
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,32 +1,32 @@
--- Test 1: CREATE with watermarks
--- Test 2: SHOW CREATE TABLE
CREATE TABLE default.t\n(\n `ts` DateTime,\n `value` UInt64\n)\nENGINE = Hybrid(remote(\'localhost:9000\', \'default\', \'local_hot\'), ts > hybridParam(\'hybrid_watermark_hot\', \'DateTime\'), remote(\'localhost:9000\', currentDatabase(), \'local_cold\'), ts <= hybridParam(\'hybrid_watermark_hot\', \'DateTime\'))\nSETTINGS hybrid_watermark_hot = \'2025-09-01\'
CREATE TABLE default.t\n(\n `ts` DateTime,\n `value` UInt64\n)\nENGINE = Hybrid(remote(\'localhost:9000\', \'default\', \'local_hot\'), ts > hybridParam(\'hybrid_watermark_hot\', \'DateTime\'), remote(\'localhost:9000\', \'default\', \'local_cold\'), ts <= hybridParam(\'hybrid_watermark_hot\', \'DateTime\'))\nSETTINGS hybrid_watermark_hot = \'2025-09-01\'
--- Test 3: First query after CREATE
1
1
--- Test 4: ALTER watermark
--- Test 5: Query with updated boundary
1
--- Test 6: SHOW CREATE after ALTER
CREATE TABLE default.t\n(\n `ts` DateTime,\n `value` UInt64\n)\nENGINE = Hybrid(remote(\'localhost:9000\', \'default\', \'local_hot\'), ts > hybridParam(\'hybrid_watermark_hot\', \'DateTime\'), remote(\'localhost:9000\', currentDatabase(), \'local_cold\'), ts <= hybridParam(\'hybrid_watermark_hot\', \'DateTime\'))\nSETTINGS hybrid_watermark_hot = \'2025-10-01\'
CREATE TABLE default.t\n(\n `ts` DateTime,\n `value` UInt64\n)\nENGINE = Hybrid(remote(\'localhost:9000\', \'default\', \'local_hot\'), ts > hybridParam(\'hybrid_watermark_hot\', \'DateTime\'), remote(\'localhost:9000\', \'default\', \'local_cold\'), ts <= hybridParam(\'hybrid_watermark_hot\', \'DateTime\'))\nSETTINGS hybrid_watermark_hot = \'2025-10-01\'
--- Test 7: DETACH/ATTACH persistence
CREATE TABLE default.t\n(\n `ts` DateTime,\n `value` UInt64\n)\nENGINE = Hybrid(remote(\'localhost:9000\', \'default\', \'local_hot\'), ts > hybridParam(\'hybrid_watermark_hot\', \'DateTime\'), remote(\'localhost:9000\', currentDatabase(), \'local_cold\'), ts <= hybridParam(\'hybrid_watermark_hot\', \'DateTime\'))\nSETTINGS hybrid_watermark_hot = \'2025-10-01\'
CREATE TABLE default.t\n(\n `ts` DateTime,\n `value` UInt64\n)\nENGINE = Hybrid(remote(\'localhost:9000\', \'default\', \'local_hot\'), ts > hybridParam(\'hybrid_watermark_hot\', \'DateTime\'), remote(\'localhost:9000\', \'default\', \'local_cold\'), ts <= hybridParam(\'hybrid_watermark_hot\', \'DateTime\'))\nSETTINGS hybrid_watermark_hot = \'2025-10-01\'
--- Test 8: Three segments, two watermarks
CREATE TABLE default.t3\n(\n `ts` DateTime,\n `value` UInt64\n)\nENGINE = Hybrid(remote(\'localhost:9000\', \'default\', \'local_hot\'), ts > hybridParam(\'hybrid_watermark_hot\', \'DateTime\'), remote(\'localhost:9000\', currentDatabase(), \'local_warm\'), (ts <= hybridParam(\'hybrid_watermark_hot\', \'DateTime\')) AND (ts > hybridParam(\'hybrid_watermark_cold\', \'DateTime\')), remote(\'localhost:9000\', currentDatabase(), \'local_cold\'), ts <= hybridParam(\'hybrid_watermark_cold\', \'DateTime\'))\nSETTINGS hybrid_watermark_cold = \'2025-07-01\', hybrid_watermark_hot = \'2025-10-01\'
CREATE TABLE default.t3\n(\n `ts` DateTime,\n `value` UInt64\n)\nENGINE = Hybrid(remote(\'localhost:9000\', \'default\', \'local_hot\'), ts > hybridParam(\'hybrid_watermark_hot\', \'DateTime\'), remote(\'localhost:9000\', \'default\', \'local_warm\'), (ts <= hybridParam(\'hybrid_watermark_hot\', \'DateTime\')) AND (ts > hybridParam(\'hybrid_watermark_cold\', \'DateTime\')), remote(\'localhost:9000\', \'default\', \'local_cold\'), ts <= hybridParam(\'hybrid_watermark_cold\', \'DateTime\'))\nSETTINGS hybrid_watermark_cold = \'2025-07-01\', hybrid_watermark_hot = \'2025-10-01\'
--- Test 9: Reject non-watermark parameter
--- Test 10: Missing watermark SETTINGS rejected at CREATE
--- Test 11: Invalid typed value
--- Test 12: Reject non-watermark MODIFY SETTING
--- Test 13: Reject RESET SETTING on Hybrid
--- Test 14: Alter one preserves the other
CREATE TABLE default.t3\n(\n `ts` DateTime,\n `value` UInt64\n)\nENGINE = Hybrid(remote(\'localhost:9000\', \'default\', \'local_hot\'), ts > hybridParam(\'hybrid_watermark_hot\', \'DateTime\'), remote(\'localhost:9000\', currentDatabase(), \'local_warm\'), (ts <= hybridParam(\'hybrid_watermark_hot\', \'DateTime\')) AND (ts > hybridParam(\'hybrid_watermark_cold\', \'DateTime\')), remote(\'localhost:9000\', currentDatabase(), \'local_cold\'), ts <= hybridParam(\'hybrid_watermark_cold\', \'DateTime\'))\nSETTINGS hybrid_watermark_cold = \'2025-08-01\', hybrid_watermark_hot = \'2025-12-01\'
CREATE TABLE default.t3\n(\n `ts` DateTime,\n `value` UInt64\n)\nENGINE = Hybrid(remote(\'localhost:9000\', \'default\', \'local_hot\'), ts > hybridParam(\'hybrid_watermark_hot\', \'DateTime\'), remote(\'localhost:9000\', \'default\', \'local_warm\'), (ts <= hybridParam(\'hybrid_watermark_hot\', \'DateTime\')) AND (ts > hybridParam(\'hybrid_watermark_cold\', \'DateTime\')), remote(\'localhost:9000\', \'default\', \'local_cold\'), ts <= hybridParam(\'hybrid_watermark_cold\', \'DateTime\'))\nSETTINGS hybrid_watermark_cold = \'2025-08-01\', hybrid_watermark_hot = \'2025-12-01\'
1
--- Test 15: Reject DistributedSettings at CREATE
--- Test 16: Plain Distributed unaffected
--- Test 17: Value via SETTINGS
1
1
CREATE TABLE default.t_settings_only\n(\n `ts` DateTime,\n `value` UInt64\n)\nENGINE = Hybrid(remote(\'localhost:9000\', \'default\', \'local_hot\'), ts > hybridParam(\'hybrid_watermark_hot\', \'DateTime\'), remote(\'localhost:9000\', currentDatabase(), \'local_cold\'), ts <= hybridParam(\'hybrid_watermark_hot\', \'DateTime\'))\nSETTINGS hybrid_watermark_hot = \'2025-09-01\'
CREATE TABLE default.t_settings_only\n(\n `ts` DateTime,\n `value` UInt64\n)\nENGINE = Hybrid(remote(\'localhost:9000\', \'default\', \'local_hot\'), ts > hybridParam(\'hybrid_watermark_hot\', \'DateTime\'), remote(\'localhost:9000\', \'default\', \'local_cold\'), ts <= hybridParam(\'hybrid_watermark_hot\', \'DateTime\'))\nSETTINGS hybrid_watermark_hot = \'2025-09-01\'
--- Test 18: Conflicting types rejected
--- Test 19: Invalid SETTINGS value rejected at CREATE
--- Test 20: Typo in CREATE SETTINGS rejected
Expand Down
2 changes: 2 additions & 0 deletions tests/queries/0_stateless/04302_hybrid_current_database_reattach.reference
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
4
4
38 changes: 38 additions & 0 deletions tests/queries/0_stateless/04302_hybrid_current_database_reattach.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
-- Reproduce the Hybrid currentDatabase() metadata bug without a server restart.
-- Detaching the table and re-attaching it while a different database is current
-- mimics the missing session-database context that startup ATTACH has. Before the
-- fix the stored metadata keeps currentDatabase(), which then resolves to the wrong
-- database and the segment fails to attach with UNKNOWN_TABLE.

SET allow_experimental_hybrid_table = 1;

CREATE TABLE {CLICKHOUSE_DATABASE:Identifier}.local_hot (ts DateTime, value UInt64) ENGINE = MergeTree ORDER BY ts;
CREATE TABLE {CLICKHOUSE_DATABASE:Identifier}.local_cold (ts DateTime, value UInt64) ENGINE = MergeTree ORDER BY ts;
INSERT INTO {CLICKHOUSE_DATABASE:Identifier}.local_hot VALUES ('2025-10-15', 1), ('2025-11-01', 2);
INSERT INTO {CLICKHOUSE_DATABASE:Identifier}.local_cold VALUES ('2025-08-01', 3), ('2025-06-15', 4);

-- Create with the test database as current so currentDatabase() resolves to it.
USE {CLICKHOUSE_DATABASE:Identifier};
CREATE TABLE {CLICKHOUSE_DATABASE:Identifier}.hybrid_t (ts DateTime, value UInt64)
ENGINE = Hybrid(
remote('localhost:9000', {CLICKHOUSE_DATABASE:String}, 'local_hot'),
ts > hybridParam('hybrid_watermark_hot', 'DateTime'),
remote('localhost:9000', currentDatabase(), 'local_cold'),
ts <= hybridParam('hybrid_watermark_hot', 'DateTime')
)
SETTINGS hybrid_watermark_hot = '2025-09-01';

SELECT count() FROM {CLICKHOUSE_DATABASE:Identifier}.hybrid_t;

-- Detach, switch the current database, then re-attach. currentDatabase() in the
-- stored metadata now resolves against the other database, whose local_cold does
-- not exist. With the fix the metadata holds the resolved name, so this succeeds.
DETACH TABLE {CLICKHOUSE_DATABASE:Identifier}.hybrid_t;
CREATE DATABASE IF NOT EXISTS {CLICKHOUSE_DATABASE_1:Identifier};
USE {CLICKHOUSE_DATABASE_1:Identifier};
ATTACH TABLE {CLICKHOUSE_DATABASE:Identifier}.hybrid_t;

SELECT count() FROM {CLICKHOUSE_DATABASE:Identifier}.hybrid_t;

USE {CLICKHOUSE_DATABASE:Identifier};
DROP DATABASE {CLICKHOUSE_DATABASE_1:Identifier};
Loading