Skip to content

[FLINK-23555] Support common sub-expression elimination in code generation#27750

Open
featzhang wants to merge 2 commits intoapache:masterfrom
featzhang:FLINK-23555-CSE
Open

[FLINK-23555] Support common sub-expression elimination in code generation#27750
featzhang wants to merge 2 commits intoapache:masterfrom
featzhang:FLINK-23555-CSE

Conversation

@featzhang
Copy link
Member

@featzhang featzhang commented Mar 9, 2026

Implemented CSE optimization for Flink SQL code generation. Key changes:

New files:

  • CseUtils.scala - Utility to scan RexNode trees and identify duplicate sub-expressions
  • CseITCase.scala - Integration tests for CSE (correctness, call count, null handling, nested)
  • CseTestFunctions.java - Test UDFs with AtomicInteger call counters
  • CseUtilsTest.scala - Unit tests for the CSE analyzer

Modified files:

  • ExprCodeGenerator.scala - Added cseEnabled, cseExprCache, cseCandidates fields and CSE logic in generateExpression(). Also changed operand visiting in visitCall to go through generateExpression (not accept(this)) so nested sub-expressions are also cached.
  • CalcCodeGenerator.scala - Added CSE analysis before projection code generation using CseUtils.findDuplicateSubExpressions()

Approach: RexNode digest string is used as expression identity key. Only RexCall nodes appearing more than once are cached. First encounter generates code + stores in local variable. Subsequent encounters return NO_CODE reference to cached variable.

Test results (all passing):

  • CseJavaITCase: 5/5 ✅ (testCseCorrectness, testCseCallCount, testNoCseCandidates, testCseWithNullValues, testCseWithNestedCommonSubExpressions)
  • CseUtilsTest: 6/6 ✅

@flinkbot
Copy link
Collaborator

flinkbot commented Mar 9, 2026

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@featzhang featzhang closed this Mar 9, 2026
@featzhang featzhang deleted the FLINK-23555-CSE branch March 9, 2026 22:38
@featzhang featzhang restored the FLINK-23555-CSE branch March 10, 2026 03:37
@featzhang featzhang deleted the FLINK-23555-CSE branch March 10, 2026 03:37
@featzhang featzhang restored the FLINK-23555-CSE branch March 10, 2026 03:37
@featzhang featzhang reopened this Mar 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants