Skip to content

[fix](fe) Reject mismatched project outputs in PullUpJoinFromUnionAll#65472

Open
yujun777 wants to merge 1 commit into
apache:masterfrom
yujun777:fix-pullup-join-from-unionall
Open

[fix](fe) Reject mismatched project outputs in PullUpJoinFromUnionAll#65472
yujun777 wants to merge 1 commit into
apache:masterfrom
yujun777:fix-pullup-join-from-unionall

Conversation

@yujun777

@yujun777 yujun777 commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Avoid comparing and remapping join inputs as equivalent when the common-side projects expose different output arity. This keeps the union pull-up rewrite aligned with the actual plan shape and adds focused coverage for the comparator and rewrite guard.

Key changes:

  • Add an output-size guard before comparing plan nodes in PullUpJoinFromUnionAll.
  • Add a FE unit test that covers both the comparator and the rewrite path.

Unit Test:

  • PullUpJoinFromUnionAllTest

Avoid comparing and remapping join inputs as equivalent when the common-side projects expose different output arity. This keeps the union pull-up rewrite aligned with the actual plan shape and adds focused coverage for the comparator and rewrite guard.

Key changes:
- Add an output-size guard before comparing plan nodes in PullUpJoinFromUnionAll.
- Add a FE unit test that covers both the comparator and the rewrite path.

Unit Test:
- PullUpJoinFromUnionAllTest
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@yujun777

Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 29097 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f21bdd45fce24c7a0e681633ebede16af35899be, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17653	3973	3951	3951
q2	2035	311	189	189
q3	10263	1329	796	796
q4	4683	460	344	344
q5	7484	843	585	585
q6	201	166	137	137
q7	744	825	620	620
q8	9800	1556	1500	1500
q9	6150	4366	4359	4359
q10	6791	1779	1514	1514
q11	500	340	306	306
q12	757	536	433	433
q13	18123	3301	2724	2724
q14	258	256	244	244
q15	q16	794	780	713	713
q17	910	901	924	901
q18	6987	5777	5700	5700
q19	1593	1221	1018	1018
q20	760	637	501	501
q21	5909	2643	2272	2272
q22	424	355	290	290
Total cold run time: 102819 ms
Total hot run time: 29097 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4385	4286	4315	4286
q2	279	317	216	216
q3	4546	4936	4358	4358
q4	2057	2134	1354	1354
q5	4389	4290	4324	4290
q6	221	187	125	125
q7	1752	1850	1776	1776
q8	2466	2156	2055	2055
q9	7932	7806	7828	7806
q10	4704	4700	4246	4246
q11	575	417	364	364
q12	735	765	557	557
q13	3507	3625	2980	2980
q14	320	314	293	293
q15	q16	733	725	699	699
q17	1339	1311	1304	1304
q18	8012	7339	7164	7164
q19	1085	1008	1054	1008
q20	2201	2200	1920	1920
q21	5209	4526	4455	4455
q22	518	473	385	385
Total cold run time: 56965 ms
Total hot run time: 51641 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 180174 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f21bdd45fce24c7a0e681633ebede16af35899be, data reload: false

query5	4357	633	472	472
query6	462	218	198	198
query7	4859	596	348	348
query8	336	182	166	166
query9	8768	4003	3993	3993
query10	489	342	300	300
query11	5901	2322	2126	2126
query12	153	101	99	99
query13	1242	658	456	456
query14	6239	5253	4899	4899
query14_1	4253	4247	4254	4247
query15	221	213	178	178
query16	1015	460	432	432
query17	1093	707	556	556
query18	2432	465	330	330
query19	200	183	158	158
query20	112	107	102	102
query21	226	152	134	134
query22	13554	13564	13472	13472
query23	17172	16521	16149	16149
query23_1	16285	16233	16230	16230
query24	7614	1743	1334	1334
query24_1	1296	1316	1316	1316
query25	570	464	401	401
query26	1341	353	210	210
query27	2637	592	387	387
query28	4481	2029	1995	1995
query29	1113	636	514	514
query30	352	265	240	240
query31	1143	1111	987	987
query32	107	66	62	62
query33	515	324	261	261
query34	1179	1144	687	687
query35	789	799	683	683
query36	1448	1456	1370	1370
query37	160	121	100	100
query38	1927	1747	1676	1676
query39	959	953	945	945
query39_1	924	927	908	908
query40	248	171	150	150
query41	70	70	69	69
query42	94	95	91	91
query43	320	324	277	277
query44	1453	783	776	776
query45	219	192	180	180
query46	1093	1209	765	765
query47	2417	2353	2211	2211
query48	406	404	298	298
query49	587	430	318	318
query50	1023	417	352	352
query51	11116	11119	10922	10922
query52	88	96	76	76
query53	266	274	211	211
query54	325	250	234	234
query55	78	77	67	67
query56	308	290	309	290
query57	1445	1432	1356	1356
query58	313	284	263	263
query59	1586	1646	1448	1448
query60	326	284	268	268
query61	180	170	172	170
query62	725	645	589	589
query63	255	248	204	204
query64	2827	1045	850	850
query65	4851	4751	4779	4751
query66	1834	515	384	384
query67	29407	29266	29305	29266
query68	3248	1572	1044	1044
query69	395	300	273	273
query70	1072	953	901	901
query71	341	311	295	295
query72	2987	2741	2444	2444
query73	833	710	434	434
query74	5101	4919	4732	4732
query75	2610	2570	2213	2213
query76	2347	1165	780	780
query77	348	378	286	286
query78	12303	12369	11652	11652
query79	1516	1190	761	761
query80	1263	513	452	452
query81	527	325	272	272
query82	615	161	121	121
query83	359	311	287	287
query84	282	160	127	127
query85	957	610	507	507
query86	427	298	269	269
query87	1813	1808	1726	1726
query88	3728	2810	2763	2763
query89	474	398	347	347
query90	1877	202	183	183
query91	195	188	159	159
query92	63	58	54	54
query93	1677	1588	965	965
query94	719	330	309	309
query95	833	583	471	471
query96	1017	776	336	336
query97	2666	2689	2545	2545
query98	217	218	203	203
query99	1232	1175	1029	1029
Total cold run time: 266222 ms
Total hot run time: 180174 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 24.99 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f21bdd45fce24c7a0e681633ebede16af35899be, data reload: false

query1	0.01	0.01	0.00
query2	0.10	0.05	0.05
query3	0.25	0.13	0.13
query4	1.61	0.14	0.13
query5	0.28	0.22	0.23
query6	1.21	1.09	1.08
query7	0.04	0.00	0.00
query8	0.06	0.04	0.03
query9	0.38	0.33	0.31
query10	0.56	0.55	0.56
query11	0.20	0.14	0.14
query12	0.17	0.14	0.14
query13	0.45	0.47	0.49
query14	1.00	0.99	1.00
query15	0.62	0.58	0.60
query16	0.30	0.31	0.32
query17	1.11	1.12	1.10
query18	0.22	0.21	0.21
query19	2.06	2.00	2.00
query20	0.02	0.01	0.01
query21	15.44	0.23	0.14
query22	4.81	0.06	0.05
query23	16.09	0.31	0.12
query24	3.01	0.44	0.32
query25	0.11	0.07	0.04
query26	0.76	0.21	0.15
query27	0.04	0.05	0.03
query28	3.46	0.90	0.51
query29	12.54	4.10	3.24
query30	0.27	0.16	0.15
query31	2.77	0.61	0.31
query32	3.23	0.59	0.48
query33	3.24	3.20	3.20
query34	15.73	4.22	3.51
query35	3.53	3.56	3.51
query36	0.56	0.42	0.42
query37	0.08	0.06	0.06
query38	0.07	0.04	0.04
query39	0.04	0.03	0.03
query40	0.18	0.16	0.15
query41	0.09	0.04	0.03
query42	0.04	0.02	0.02
query43	0.05	0.03	0.03
Total cold run time: 96.79 s
Total hot run time: 24.99 s

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants