Skip to content

[feature](scan) isolate slow scan tasks#61731

Open
Gabriel39 wants to merge 3 commits intoapache:masterfrom
Gabriel39:dev_0325
Open

[feature](scan) isolate slow scan tasks#61731
Gabriel39 wants to merge 3 commits intoapache:masterfrom
Gabriel39:dev_0325

Conversation

@Gabriel39
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Mar 25, 2026

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Gabriel39
Copy link
Contributor Author

run buildall

@hello-stephen
Copy link
Contributor

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 78.67% (1796/2283)
Line Coverage 64.39% (32276/50128)
Region Coverage 65.28% (16164/24761)
Branch Coverage 55.72% (8611/15454)

@doris-robot
Copy link

TPC-H: Total hot run time: 26963 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8df13ad652fbc6e47d517065ab70a1483a9eb567, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17672	4461	4444	4444
q2	q3	10739	785	527	527
q4	4743	355	258	258
q5	8193	1241	1025	1025
q6	241	173	147	147
q7	810	843	674	674
q8	10810	1501	1411	1411
q9	6772	4935	4695	4695
q10	6359	1932	1670	1670
q11	473	248	250	248
q12	768	577	469	469
q13	18064	2723	1962	1962
q14	231	239	218	218
q15	q16	722	728	664	664
q17	747	823	485	485
q18	6013	5375	5271	5271
q19	1114	1000	617	617
q20	548	499	379	379
q21	4734	2037	1501	1501
q22	374	345	298	298
Total cold run time: 100127 ms
Total hot run time: 26963 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4818	4616	4596	4596
q2	q3	3976	4363	3836	3836
q4	887	1215	763	763
q5	4082	4407	4390	4390
q6	185	177	157	157
q7	1791	1720	1535	1535
q8	2494	2756	2655	2655
q9	7802	7508	7512	7508
q10	3932	4018	3638	3638
q11	523	444	426	426
q12	540	677	454	454
q13	2850	2958	2096	2096
q14	288	318	305	305
q15	q16	762	762	751	751
q17	1182	1355	1304	1304
q18	7361	6850	6694	6694
q19	894	893	909	893
q20	2173	2206	2018	2018
q21	3997	3484	3297	3297
q22	467	425	374	374
Total cold run time: 51004 ms
Total hot run time: 47690 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 169188 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8df13ad652fbc6e47d517065ab70a1483a9eb567, data reload: false

query5	4324	652	508	508
query6	340	231	211	211
query7	4209	477	265	265
query8	343	247	233	233
query9	8705	2729	2731	2729
query10	525	388	333	333
query11	7032	5069	4856	4856
query12	186	129	126	126
query13	1265	467	341	341
query14	5766	3735	3466	3466
query14_1	2886	2794	2828	2794
query15	206	201	176	176
query16	984	454	444	444
query17	877	711	612	612
query18	2430	434	346	346
query19	209	226	181	181
query20	132	128	130	128
query21	211	138	108	108
query22	13210	13898	14525	13898
query23	16604	16671	16164	16164
query23_1	16267	15829	15746	15746
query24	7255	1610	1227	1227
query24_1	1223	1251	1246	1246
query25	576	506	441	441
query26	1246	275	154	154
query27	2766	484	302	302
query28	4520	1871	1871	1871
query29	876	614	473	473
query30	302	223	190	190
query31	1034	947	862	862
query32	83	68	73	68
query33	519	338	284	284
query34	906	886	530	530
query35	638	679	614	614
query36	1097	1128	990	990
query37	144	100	83	83
query38	2926	2908	2866	2866
query39	877	833	803	803
query39_1	802	787	807	787
query40	234	154	136	136
query41	63	58	59	58
query42	254	260	252	252
query43	237	245	224	224
query44	
query45	199	186	182	182
query46	878	979	615	615
query47	2116	2094	2065	2065
query48	330	325	238	238
query49	639	467	376	376
query50	690	285	208	208
query51	4080	4051	3986	3986
query52	268	263	257	257
query53	293	336	284	284
query54	309	286	270	270
query55	90	93	86	86
query56	326	331	357	331
query57	1925	1797	1665	1665
query58	291	279	280	279
query59	2790	2971	2722	2722
query60	355	342	331	331
query61	176	158	159	158
query62	641	607	561	561
query63	314	281	281	281
query64	5054	1284	1011	1011
query65	
query66	1467	458	379	379
query67	24279	24346	24236	24236
query68	
query69	403	311	287	287
query70	989	989	973	973
query71	366	341	307	307
query72	2786	2733	2475	2475
query73	549	552	319	319
query74	9677	9634	9432	9432
query75	2884	2781	2509	2509
query76	2307	1034	677	677
query77	376	410	335	335
query78	10940	11145	10452	10452
query79	1149	801	569	569
query80	1364	673	591	591
query81	561	263	230	230
query82	1035	160	120	120
query83	390	275	260	260
query84	312	131	109	109
query85	957	498	461	461
query86	437	321	291	291
query87	3158	3164	3043	3043
query88	3596	2647	2641	2641
query89	433	367	344	344
query90	1969	186	190	186
query91	185	169	139	139
query92	82	78	69	69
query93	960	866	492	492
query94	647	336	291	291
query95	596	339	319	319
query96	639	519	227	227
query97	2475	2470	2415	2415
query98	235	224	224	224
query99	1031	985	925	925
Total cold run time: 249576 ms
Total hot run time: 169188 ms

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.81% (19869/37623)
Line Coverage 36.30% (185575/511168)
Region Coverage 32.56% (143765/441569)
Branch Coverage 33.76% (62928/186410)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.70% (26413/36840)
Line Coverage 54.57% (278100/509605)
Region Coverage 51.64% (230170/445695)
Branch Coverage 53.18% (99429/186974)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants