Skip to content

[fix](fe) Fix silent semi/anti join drop caused by constrained side check skip in leading hint#65467

Draft
starocean999 wants to merge 3 commits into
apache:masterfrom
starocean999:master_0710
Draft

[fix](fe) Fix silent semi/anti join drop caused by constrained side check skip in leading hint#65467
starocean999 wants to merge 3 commits into
apache:masterfrom
starocean999:master_0710

Conversation

@starocean999

Copy link
Copy Markdown
Contributor

Related PR: #65205

Problem Summary:

PR #65205 refactored the semi/anti join constraint matching in
LeadingHint#getJoinConstraint() to support both semi and anti joins and
handle the constrained side correctly. However, it introduced two bugs that
together cause semi/anti joins to be silently downgraded to INNER/CROSS JOIN:

  1. When the constrained side (probe side for left semi/anti, or left side for
    right semi/anti) partially overlaps a join child but that child contains
    extra tables, the new code does continue — skipping the constraint
    instead of marking the leading hint as UNUSED. Since matchedJoinConstraint
    remains null, computeJoinType() falls through to INNER_JOIN or CROSS_JOIN,
    silently dropping the semi/anti semantics.

    Example SQL triggering this:

    SELECT /*+ leading(a2 a5 { a3 a1 }) */ COUNT(*)
    FROM a1 JOIN a2 ON a2.c1 = a2.c1
    LEFT SEMI JOIN a3 ON a2.c2 = a2.c1
    JOIN a5 ON a2.c2 = a2.c1;
  2. The conditionJoinType map uses HashMap<Expression, JoinType> with
    put(), so when the same Expression appears in both a semi/anti join
    condition and an inner join condition, the later inner join overwrites
    the semi/anti join type. This disables the isConditionJoinTypeMatched()
    safety net that should have caught the fallback from bug 1.

Root cause:

For bug 1: continue on constrained-side violation should be
return Pair.of(null, false) to indicate the constraint is broken, not
silently skipped.
For bug 2: a single-value-per-key map cannot represent that one expression
belongs to multiple join types. Changed to Map<Expression, Set>
using computeIfAbsent(...).add().

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@starocean999

Copy link
Copy Markdown
Contributor Author

/review

@github-actions

Copy link
Copy Markdown
Contributor

Codex automated review failed and did not complete.

Error: You've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at 10:57 AM.
Workflow run: https://github.com/apache/doris/actions/runs/29086558592

Please inspect the workflow logs and rerun the review after the underlying issue is resolved.

@starocean999

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review found one correctness issue in the leading-hint rewrite. The changed code is focused and does not add concurrency, lifecycle, config, persistence, or FE/BE protocol surface, but the current Set-based condition provenance can move a repeated predicate across a semi/anti boundary and change rows. The helper tests do not cover this real analyzer path, so the fix should include SQL/analyzer-level coverage for the repeated-condition case.

No existing inline review threads or user focus items were present. Validation was static only: the review prompt forbids builds, and this checkout is missing thirdparty/installed and thirdparty/installed/bin/protoc. Subagents A, B, and R completed the final convergence pass with NO_NEW_VALUABLE_FINDINGS on the final comment set.

Comment thread fe/fe-core/src/main/java/org/apache/doris/nereids/hint/LeadingHint.java Outdated
@starocean999

Copy link
Copy Markdown
Contributor Author

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review found one correctness issue in the leading-hint rewrite.

Checkpoint conclusions: the PR goal is clear and the change is mostly localized to Nereids leading-hint condition collection and constraint matching, but the new incompatible-filter put-back path can still produce a successful hinted plan after dropping a required predicate. No concurrency, lifecycle, configuration, persistence, or FE-BE protocol surface is introduced. Parallel hash/other conjunct collection paths were checked; the remaining problem is final filter consumption/status handling in LeadingHint.generateLeadingJoinPlan(). The added helper tests cover pieces of the behavior, but the accepted issue needs end-to-end leading-hint coverage for an invalid final leftover predicate case.

User focus: no additional user-provided review focus was supplied.

Validation: static review only. Per the review prompt I did not build or run tests; the checkout is shallow and the base SHA is not available for local ancestry-based diff checks.

@starocean999

Copy link
Copy Markdown
Contributor Author

/review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant