Skip to content

[fix](fe) Revert semi join constraint matching fix#65546

Draft
shuke987 wants to merge 1 commit into
apache:masterfrom
shuke987:codex/revert-65205
Draft

[fix](fe) Revert semi join constraint matching fix#65546
shuke987 wants to merge 1 commit into
apache:masterfrom
shuke987:codex/revert-65205

Conversation

@shuke987

Copy link
Copy Markdown
Collaborator

What problem does this PR solve?

Issue Number: N/A

Related PR: #65205

Problem Summary:

PR #65205 introduced a correctness issue in leading hint planning by changing how semi/anti join constraints are matched. This PR reverts merge commit cbccdafa41adfff73e683ac1dfcae46638c16d48 in full, restoring the previous planner behavior while a correct fix is prepared.

Release note

None

Check List (For Author)

  • Test
    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

This is an exact revert of #65205. The reverted paths match the parent of the original merge commit, and git diff --check passed before commit.

  • Behavior changed:
    • No.
    • Yes.

Restores the leading hint behavior from before #65205.

  • Does this need documentation?
    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

### What problem does this PR solve?

Issue Number: N/A

Related PR: apache#65205

Problem Summary: PR apache#65205 introduced a correctness issue in leading hint planning. Revert its semi/anti join constraint matching change and the unit tests added with it to restore the previous planner behavior while a correct fix is prepared.

### Release note

None

### Check List (For Author)

- Test: No need to test
    - Exact revert verified against the parent of apache#65205; targeted FE UT was stopped to publish the revert immediately.
- Behavior changed: Yes. Restore the leading hint behavior before apache#65205.
- Does this need documentation: No
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@shuke987

Copy link
Copy Markdown
Collaborator Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 29355 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 27ab6bd85eebe636d1f96e10c7ca1cc7f2fcff0e, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17661	4083	4116	4083
q2	2012	324	198	198
q3	10303	1423	841	841
q4	4681	472	337	337
q5	7544	856	578	578
q6	177	173	135	135
q7	772	871	639	639
q8	9316	1466	1500	1466
q9	5603	4409	4418	4409
q10	6762	1794	1511	1511
q11	502	330	315	315
q12	714	540	420	420
q13	18131	3431	2791	2791
q14	270	269	252	252
q15	q16	795	772	706	706
q17	1013	1031	951	951
q18	7018	5734	5446	5446
q19	1307	1284	969	969
q20	754	629	570	570
q21	5903	2520	2444	2444
q22	428	360	294	294
Total cold run time: 101666 ms
Total hot run time: 29355 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4323	4476	4240	4240
q2	293	310	218	218
q3	4573	5017	4391	4391
q4	2040	2169	1351	1351
q5	4379	4288	4332	4288
q6	222	175	130	130
q7	1730	1700	1965	1700
q8	2577	2175	2210	2175
q9	8098	8252	7808	7808
q10	4792	4710	4289	4289
q11	590	409	408	408
q12	811	767	563	563
q13	3276	3651	2927	2927
q14	302	298	276	276
q15	q16	740	740	630	630
q17	1326	1323	1438	1323
q18	7812	7214	7200	7200
q19	1125	1056	1095	1056
q20	2202	2214	1978	1978
q21	5184	4542	4369	4369
q22	512	472	390	390
Total cold run time: 56907 ms
Total hot run time: 51710 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 180585 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 27ab6bd85eebe636d1f96e10c7ca1cc7f2fcff0e, data reload: false

query5	4334	628	486	486
query6	467	230	207	207
query7	4878	602	350	350
query8	338	191	165	165
query9	8755	4051	4056	4051
query10	467	336	288	288
query11	5973	2348	2134	2134
query12	163	103	99	99
query13	1259	605	415	415
query14	6303	5310	4955	4955
query14_1	4308	4295	4292	4292
query15	219	206	187	187
query16	1032	482	449	449
query17	949	745	605	605
query18	2475	492	355	355
query19	214	205	164	164
query20	113	109	109	109
query21	228	162	134	134
query22	13700	13522	13372	13372
query23	17473	16592	16226	16226
query23_1	16278	16219	16208	16208
query24	7517	1794	1303	1303
query24_1	1310	1296	1339	1296
query25	581	474	395	395
query26	1349	332	213	213
query27	2662	567	368	368
query28	4463	2043	1996	1996
query29	1118	656	500	500
query30	339	262	225	225
query31	1120	1089	980	980
query32	120	62	68	62
query33	516	318	262	262
query34	1191	1117	652	652
query35	791	799	669	669
query36	1399	1399	1193	1193
query37	156	113	99	99
query38	1886	1800	1635	1635
query39	925	939	891	891
query39_1	873	874	873	873
query40	245	160	134	134
query41	72	77	64	64
query42	95	87	90	87
query43	314	318	279	279
query44	1446	790	768	768
query45	200	193	180	180
query46	1047	1259	760	760
query47	2381	2291	2223	2223
query48	403	421	301	301
query49	585	423	335	335
query50	1024	415	347	347
query51	10788	10854	10463	10463
query52	85	88	77	77
query53	266	293	206	206
query54	274	239	211	211
query55	74	70	70	70
query56	286	304	292	292
query57	1441	1417	1325	1325
query58	285	270	263	263
query59	1554	1635	1451	1451
query60	308	271	247	247
query61	154	158	150	150
query62	688	649	586	586
query63	249	206	200	200
query64	2812	1071	835	835
query65	4881	4811	4780	4780
query66	1809	518	433	433
query67	29511	29433	29267	29267
query68	3228	1516	993	993
query69	401	309	269	269
query70	1095	955	931	931
query71	357	325	318	318
query72	3113	2768	2411	2411
query73	858	780	424	424
query74	5095	4949	4791	4791
query75	2620	2572	2217	2217
query76	2345	1183	796	796
query77	352	386	289	289
query78	12449	12357	11755	11755
query79	1416	1148	800	800
query80	1296	549	456	456
query81	539	324	277	277
query82	928	159	124	124
query83	394	321	290	290
query84	330	168	129	129
query85	966	602	526	526
query86	440	294	296	294
query87	1825	1819	1754	1754
query88	3700	2764	2788	2764
query89	448	401	360	360
query90	1934	202	185	185
query91	202	198	160	160
query92	65	61	58	58
query93	1663	1466	969	969
query94	744	369	308	308
query95	792	485	551	485
query96	1131	876	309	309
query97	2700	2701	2562	2562
query98	223	214	221	214
query99	1146	1181	1036	1036
Total cold run time: 266633 ms
Total hot run time: 180585 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 24.98 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 27ab6bd85eebe636d1f96e10c7ca1cc7f2fcff0e, data reload: false

query1	0.00	0.00	0.00
query2	0.10	0.05	0.05
query3	0.26	0.14	0.13
query4	1.62	0.15	0.13
query5	0.26	0.22	0.22
query6	1.23	1.07	1.11
query7	0.03	0.01	0.00
query8	0.05	0.04	0.04
query9	0.40	0.30	0.30
query10	0.55	0.57	0.55
query11	0.20	0.15	0.15
query12	0.18	0.14	0.15
query13	0.48	0.49	0.47
query14	1.00	1.02	1.02
query15	0.61	0.59	0.60
query16	0.31	0.34	0.33
query17	1.09	1.11	1.07
query18	0.22	0.21	0.20
query19	2.02	1.91	1.93
query20	0.02	0.01	0.01
query21	15.45	0.22	0.14
query22	4.88	0.05	0.05
query23	16.12	0.32	0.12
query24	2.92	0.44	0.31
query25	0.11	0.04	0.04
query26	0.76	0.21	0.15
query27	0.05	0.04	0.04
query28	3.54	0.90	0.52
query29	12.47	4.13	3.28
query30	0.28	0.15	0.15
query31	2.76	0.58	0.31
query32	3.21	0.59	0.49
query33	3.19	3.25	3.13
query34	15.67	4.20	3.54
query35	3.49	3.55	3.54
query36	0.56	0.44	0.42
query37	0.08	0.06	0.07
query38	0.05	0.04	0.04
query39	0.03	0.03	0.03
query40	0.17	0.17	0.16
query41	0.09	0.03	0.02
query42	0.04	0.03	0.03
query43	0.05	0.03	0.03
Total cold run time: 96.6 s
Total hot run time: 24.98 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants