Skip to content

Fix lookup join returning 0 rows when a dimension primary-key component is a literal [fj4WqyCCw3C5ShR1RfB7MoBPTpkRrBFYP1uT35g3MvT] - #19197

Open
waterWang wants to merge 1 commit into
apache:masterfrom
waterWang:fix/lookup-join-literal-key
Open

Fix lookup join returning 0 rows when a dimension primary-key component is a literal [fj4WqyCCw3C5ShR1RfB7MoBPTpkRrBFYP1uT35g3MvT]#19197
waterWang wants to merge 1 commit into
apache:masterfrom
waterWang:fix/lookup-join-literal-key

Conversation

@waterWang

Copy link
Copy Markdown

Description

When a lookup join condition includes a literal on a dimension table primary key column (e.g. dim_tbl.currency = 'gbp'), Calcite's analyzeCondition() classifies it as a non-equi condition rather than an equi-join key. The LookupJoinOperator builds the lookup key only from leftKeys (equi-join column-column pairs), so the literal component is missing from the key. The key becomes shorter than the dimension table's primary key, causing the lookup to always return null — producing 0 result rows.

Root Cause

The condition dim_tbl.currency = 'gbp' AND dim_tbl.rate_start_date = fact_tbl.rate_start_date is split by Calcite as:

  • Equi-join key: dim_tbl.rate_start_date = fact_tbl.rate_start_dateleftKeys/ rightKeys
  • Non-equi condition: dim_tbl.currency = 'gbp' (column vs literal)

The operator builds the key from leftKeys only, producing a 1-component key. But the dimension table's primary key is [currency, rate_start_date] (2 components), so the lookup always fails.

Fix

Build the lookup key in the dimension table's primary key column order, filling each position from either:

  • The corresponding left column (via equi-join leftKeys/ rightKeys mapping)
  • A literal value extracted from non-equi conditions of the form dim_col = literal

Testing

Added lookup_join_literal_key test case to LookupJoin.json with:

  • Dim table PK: [currency, rate_start_date]
  • Join: dim_tbl.currency = 'gbp' AND dim_tbl.rate_start_date = fact_tbl.rate_start_date
  • Expected: 1 result row ['gbp', 125]

Closes #19188

…nt is a literal

When a lookup join condition includes a literal on a dimension table primary key
column (e.g. dim_tbl.currency = 'gbp'), Calcite's analyzeCondition() classifies
it as a non-equi condition rather than an equi-join key. The LookupJoinOperator
builds the lookup key only from leftKeys (equi-join column-column pairs), so the
literal component is missing from the key, making it shorter than the dimension
table's primary key. The lookup always returns null, producing 0 result rows.

Fix: Build the lookup key in the dimension table's primary key column order.
For each primary key column, determine the value source: either the corresponding
left column (equi-join via leftKeys/rightKeys) or a literal value extracted from
the non-equi condition.

Closes apache#19188
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Lookup Join returns 0 rows when given a literal value

1 participant