Skip to content

fix(optimizer): prevent pruning columns referenced by GROUP BY / HAVING / QUALIFY - #8121

Open
fivetran-kwoodbeck wants to merge 3 commits into
mainfrom
optimizer/fix-pushdown-prunes-shadowed-clause-columns
Open

fix(optimizer): prevent pruning columns referenced by GROUP BY / HAVING / QUALIFY#8121
fivetran-kwoodbeck wants to merge 3 commits into
mainfrom
optimizer/fix-pushdown-prunes-shadowed-clause-columns

Conversation

@fivetran-kwoodbeck

@fivetran-kwoodbeck fivetran-kwoodbeck commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

pushdown_projections decides which projections to keep by scanning Column nodes and grouping them by source table. For BigQuery (PROJECTION_ALIASES_SHADOW_SOURCE_NAMES), the alias-shadow workaround rewrites GROUP BY / HAVING / QUALIFY references into bare Identifiers when a projection alias collides with a source name, and its early return can also leave later references as unqualified Columns. Both forms were invisible to the scan, so the columns they reference were pruned from the inner scope, producing invalid SQL.

Sample Input

SELECT t.n
FROM (
  SELECT a, ARRAY_AGG(b) AS agg, COUNT(*) AS n
  FROM (SELECT a, b FROM x) AS agg
  GROUP BY a
  HAVING a >= 1
) AS t

Previous Output (invalid: a is pruned but still referenced by GROUP BY / HAVING)

SELECT t.n AS n
FROM (
  SELECT COUNT(*) AS n
  FROM (SELECT 1 AS _ FROM x AS x) AS agg
  GROUP BY a
  HAVING a >= 1
) AS t

Expected Output

SELECT t.n AS n
FROM (
  SELECT COUNT(*) AS n
  FROM (SELECT x.a AS a FROM x AS x) AS agg
  GROUP BY a
  HAVING a >= 1
) AS t

The fix scans the GROUP BY / HAVING / QUALIFY clauses for bare Identifiers and unqualified Columns, resolves them with Resolver, and adds them to the kept-column set.

@github-actions

Copy link
Copy Markdown
Contributor

SQLGlot Integration Test Results

✅ All tests passed

Comparing:

  • this branch (sqlglot:optimizer/fix-pushdown-prunes-shadowed-clause-columns @ sqlglot 7100d0d)
  • baseline (main @ sqlglot 7810755)

Overall

main: 182937 total, 160861 passed (pass rate: 87.9%)

sqlglot:optimizer/fix-pushdown-prunes-shadowed-clause-columns: 170743 total, 149709 passed (pass rate: 87.7%)

Transitions:
No change

Dialect pair changes: 0 previous results not found, 3 current results not found

✅ All tests passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant