[refactor](rec_cte)recursive-CTE refactor #59872
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
starocean999
force-pushed
the
recursive_cte
branch
from
b51cdc7 to
9f75bd8
Compare
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 31820 ms |
TPC-DS: Total hot run time: 173839 ms |
ClickBench: Total hot run time: 26.76 s |
Contributor
FE UT Coverage ReportIncrement line coverage |
There was a problem hiding this comment.
Pull request overview
This PR refactors the recursive CTE implementation in Apache Doris's Frontend by replacing the legacy RecursiveCTE model with a clearer RecursiveUnion model. The refactor separates anchor (initial rows) and producer (recursive body) responsibilities and makes work-table references explicit.
Changes:
- Introduced new RecursiveUnion model with anchor/producer/work-table reference sentinel nodes
- Removed legacy RecursiveCte/RecursiveCteScan classes and replaced with new implementation
- Updated type equality checks for recursive CTE compatibility
Reviewed changes
Copilot reviewed 56 out of 56 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| PhysicalWorkTableReference.java | New physical node for work table reference |
| LogicalWorkTableReference.java | New logical node for work table reference |
| PhysicalRecursiveUnion*.java | New physical nodes for recursive union, anchor, and producer |
| LogicalRecursiveUnion*.java | New logical nodes for recursive union, anchor, and producer |
| RecursiveCteScanNode.java | Simplified scan node, removed ScanNode inheritance |
| ThriftPlansBuilder.java | Enhanced documentation for recursive CTE handling |
| StatsCalculator.java | Updated statistics calculation for new nodes |
| ColumnPruning.java | Removed legacy recursive CTE pruning logic |
| Test files | Updated to use new API |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| return false; | ||
| } | ||
| PhysicalWorkTableReference that = (PhysicalWorkTableReference) o; | ||
| return cteId.equals(that.cteId) && nameParts.equals(that.nameParts) && outputs.equals(outputs); |
There was a problem hiding this comment.
In the equals method, outputs.equals(outputs) compares outputs to itself instead of comparing to that.outputs. This should be outputs.equals(that.outputs).
| return false; | ||
| } | ||
| LogicalWorkTableReference that = (LogicalWorkTableReference) o; | ||
| return cteId.equals(that.cteId) && nameParts.equals(that.nameParts) && outputs.equals(outputs); |
There was a problem hiding this comment.
In the equals method, outputs.equals(outputs) compares outputs to itself instead of comparing to that.outputs. This should be outputs.equals(that.outputs).
|
|
||
| @Override | ||
| public String toString() { | ||
| return Utils.toSqlStringSkipNull("PhysicalRecursiveCteRecursiveChild", |
There was a problem hiding this comment.
The toString() method uses an outdated class name 'PhysicalRecursiveCteRecursiveChild' instead of the current class name 'PhysicalRecursiveUnionAnchor'.
| public static double DEFAULT_AGGREGATE_RATIO = 1 / 3.0; | ||
| public static double AGGREGATE_COLUMN_CORRELATION_COEFFICIENT = 0.75; | ||
| public static double DEFAULT_COLUMN_NDV_RATIO = 0.5; | ||
| public static double RECURSIVE_CTE_EXPAND_RATION = 5.0; |
There was a problem hiding this comment.
Corrected spelling of 'RATION' to 'RATIO'.
Suggested change
| public static double RECURSIVE_CTE_EXPAND_RATION = 5.0; | |
| public static double RECURSIVE_CTE_EXPAND_RATIO = 5.0; | |
| @Deprecated | |
| public static double RECURSIVE_CTE_EXPAND_RATION = RECURSIVE_CTE_EXPAND_RATIO; |
| */ | ||
| public Statistics computeRecursiveCte(RecursiveCte recursiveCte, List<Statistics> childStats) { | ||
| // TODO: refactor this for one row relation | ||
| // simliar as computeUnion |
There was a problem hiding this comment.
Corrected spelling of 'simliar' to 'similar'.
Suggested change
| // simliar as computeUnion | |
| // similar as computeUnion |
Contributor
Author
|
run buildall |
Contributor
FE UT Coverage ReportIncrement line coverage |
TPC-H: Total hot run time: 31750 ms |
TPC-DS: Total hot run time: 174446 ms |
ClickBench: Total hot run time: 26.66 s |
Contributor
FE Regression Coverage ReportIncrement line coverage |
Contributor
FE Regression Coverage ReportIncrement line coverage |
morrySnow
approved these changes
Jan 19, 2026
Contributor
|
PR approved by at least one committer and no changes requested. |
github-actions
bot
added
approved
Contributor
|
PR approved by anyone and no changes requested. |
feiniaofeiafei
approved these changes
Jan 20, 2026
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Refactor recursive-CTE fe part.
Related PR: #58916
Summary
High-Level Design Changes
Logical / Physical Nodes Added
Removed / Renamed
LogicalRecursiveCteScan,PhysicalRecursiveCteScan,UnassignedRecursiveCteScanJobwere deleted.Visitor / Copier / Planner Updates
Rewrite / Optimization Rules
Distribution / Job Builder
Statistics / Cost
Type Equality
Analyzer / CTE handling
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)