Skip to content

branch-3.0: [enhancement](cloud) make file cache version upgrade faster #50726 #51194

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 28, 2025

Conversation

github-actions[bot]
Copy link
Contributor

Cherry-picked from #50726

parallel the upgrade for each directory to shorten be restart time
@github-actions github-actions bot requested a review from dataroaring as a code owner May 23, 2025 09:52
@Thearas
Copy link
Contributor

Thearas commented May 23, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this May 23, 2025
@Thearas
Copy link
Contributor

Thearas commented May 23, 2025

run buildall

@hello-stephen
Copy link
Contributor

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 83.13% (1094/1316)
Line Coverage 66.22% (18170/27438)
Region Coverage 65.96% (8998/13641)
Branch Coverage 55.74% (4862/8722)

@doris-robot
Copy link

TPC-H: Total hot run time: 39752 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f725755d71e8f9a5053f0bb7ced87a832202d784, data reload: false

------ Round 1 ----------------------------------
q1	17675	7084	6558	6558
q2	2081	173	168	168
q3	10638	1086	1170	1086
q4	10260	711	791	711
q5	7732	2833	2780	2780
q6	212	132	129	129
q7	967	627	608	608
q8	9371	1957	1987	1957
q9	6626	6396	6468	6396
q10	7059	2274	2289	2274
q11	470	264	261	261
q12	399	224	210	210
q13	17800	2991	2962	2962
q14	238	203	203	203
q15	521	465	462	462
q16	482	369	371	369
q17	994	629	635	629
q18	7380	6595	6642	6595
q19	1399	1100	1042	1042
q20	469	209	203	203
q21	3987	3204	3159	3159
q22	1071	993	990	990
Total cold run time: 107831 ms
Total hot run time: 39752 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6577	6528	6558	6528
q2	336	230	222	222
q3	2906	2783	2832	2783
q4	2043	1816	1749	1749
q5	5791	5772	5700	5700
q6	211	125	122	122
q7	2227	1786	1815	1786
q8	3370	3610	3544	3544
q9	8972	8702	8892	8702
q10	3564	3498	3543	3498
q11	577	499	500	499
q12	787	617	653	617
q13	9156	3102	3226	3102
q14	300	255	269	255
q15	521	472	469	469
q16	476	450	454	450
q17	1850	1623	1609	1609
q18	8310	7864	7788	7788
q19	1693	1507	1585	1507
q20	2057	1787	1796	1787
q21	5532	5451	5281	5281
q22	1100	1076	1012	1012
Total cold run time: 68356 ms
Total hot run time: 59010 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196765 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f725755d71e8f9a5053f0bb7ced87a832202d784, data reload: false

query1	1300	909	892	892
query2	6242	2021	1931	1931
query3	10852	4205	4251	4205
query4	61263	29106	23566	23566
query5	5227	460	450	450
query6	404	182	177	177
query7	5550	312	304	304
query8	324	229	226	226
query9	8806	2592	2587	2587
query10	459	257	252	252
query11	17648	15170	15961	15170
query12	162	107	106	106
query13	1460	459	425	425
query14	10561	6899	6603	6603
query15	204	187	180	180
query16	7199	503	520	503
query17	1169	589	612	589
query18	1906	333	325	325
query19	211	166	168	166
query20	127	115	118	115
query21	209	101	106	101
query22	4851	4677	4612	4612
query23	34389	34182	34216	34182
query24	6178	2935	2877	2877
query25	543	454	428	428
query26	662	174	163	163
query27	2011	351	374	351
query28	4179	2191	2128	2128
query29	748	454	434	434
query30	250	166	156	156
query31	987	829	812	812
query32	65	54	55	54
query33	435	273	286	273
query34	941	528	508	508
query35	854	766	711	711
query36	1112	935	932	932
query37	115	75	75	75
query38	4056	3927	3945	3927
query39	1529	1448	1449	1448
query40	201	97	97	97
query41	48	50	47	47
query42	120	102	103	102
query43	550	498	493	493
query44	1185	812	807	807
query45	183	166	166	166
query46	1144	722	720	720
query47	2041	1926	1915	1915
query48	487	375	398	375
query49	737	408	418	408
query50	841	420	438	420
query51	7361	7279	7249	7249
query52	104	92	94	92
query53	263	185	184	184
query54	575	484	479	479
query55	78	82	81	81
query56	280	239	257	239
query57	1269	1174	1122	1122
query58	229	216	224	216
query59	3233	3128	3004	3004
query60	273	267	259	259
query61	111	115	110	110
query62	823	718	703	703
query63	217	200	199	199
query64	1410	693	661	661
query65	3282	3154	3182	3154
query66	699	289	303	289
query67	16014	15513	15578	15513
query68	4206	563	565	563
query69	457	259	264	259
query70	1154	1129	1070	1070
query71	350	254	252	252
query72	6425	4049	4082	4049
query73	736	338	351	338
query74	10477	9314	9308	9308
query75	3365	2638	2653	2638
query76	1998	1118	1075	1075
query77	454	269	266	266
query78	10645	9583	9526	9526
query79	1267	592	580	580
query80	780	431	438	431
query81	504	221	218	218
query82	344	85	86	85
query83	160	148	146	146
query84	287	80	83	80
query85	865	307	296	296
query86	320	310	303	303
query87	4394	4223	4179	4179
query88	3849	2391	2372	2372
query89	412	300	294	294
query90	2020	187	189	187
query91	185	147	147	147
query92	60	50	51	50
query93	1532	556	553	553
query94	738	303	312	303
query95	366	260	255	255
query96	599	276	287	276
query97	3304	3107	3178	3107
query98	217	211	201	201
query99	1603	1310	1320	1310
Total cold run time: 313019 ms
Total hot run time: 196765 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.97 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f725755d71e8f9a5053f0bb7ced87a832202d784, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.05	0.04
query3	0.23	0.06	0.04
query4	1.66	0.08	0.08
query5	0.51	0.51	0.53
query6	1.13	0.74	0.73
query7	0.02	0.01	0.01
query8	0.06	0.05	0.05
query9	0.56	0.50	0.51
query10	0.56	0.56	0.58
query11	0.16	0.12	0.12
query12	0.16	0.13	0.12
query13	0.61	0.60	0.59
query14	0.78	0.81	0.81
query15	0.84	0.83	0.83
query16	0.39	0.37	0.37
query17	1.01	0.97	1.06
query18	0.18	0.18	0.18
query19	1.93	1.77	1.84
query20	0.02	0.01	0.01
query21	15.41	0.65	0.65
query22	3.75	7.63	2.02
query23	18.28	1.41	1.34
query24	2.17	0.26	0.25
query25	0.16	0.08	0.09
query26	0.28	0.18	0.18
query27	0.08	0.08	0.07
query28	13.28	0.60	0.56
query29	12.66	3.40	3.42
query30	0.25	0.06	0.06
query31	2.86	0.41	0.40
query32	3.23	0.48	0.49
query33	2.98	3.03	2.99
query34	17.22	4.53	4.53
query35	4.55	4.57	4.55
query36	0.65	0.47	0.47
query37	0.20	0.16	0.17
query38	0.16	0.16	0.15
query39	0.04	0.04	0.04
query40	0.17	0.13	0.14
query41	0.10	0.06	0.05
query42	0.07	0.05	0.05
query43	0.05	0.05	0.04
Total cold run time: 109.53 s
Total hot run time: 30.97 s

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 76.37% (139/182) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 41.08% (10863/26444)
Line Coverage 31.91% (92852/290983)
Region Coverage 31.01% (47908/154505)
Branch Coverage 27.49% (24544/89286)

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit b43fb25 into branch-3.0 May 28, 2025
22 of 24 checks passed
@github-actions github-actions bot deleted the auto-pick-50726-branch-3.0 branch May 28, 2025 06:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants