Skip to content

[opt][fe/be.conf]optimise system environment replacing rule in conf file #51199

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 23 commits into
base: master
Choose a base branch
from
Open

[opt][fe/be.conf]optimise system environment replacing rule in conf file #51199

wants to merge 23 commits into from

Conversation

zhaorongsheng
Copy link
Contributor

@zhaorongsheng zhaorongsheng commented May 23, 2025 •
edited
Loading

What problem does this PR solve?

Issue Number: close #51200

Related PR: #xxx

Problem Summary:

It does not support $xx format in conf file. If we set ssl_certificate_path=$DORIS_HOME/conf/cert.pem the value of the key ssl_certificate_path is $DORIS_HOME/conf/cert.pem. But in fact the value of $DORIS_HOME must be replaced by the system environment.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented May 23, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zhaorongsheng
Copy link
Contributor Author

run buildall

@zhaorongsheng zhaorongsheng changed the title optimise system environment replacing rule in conf file [opt][be]optimise system environment replacing rule in conf file May 24, 2025
@zhaorongsheng zhaorongsheng changed the title [opt][be]optimise system environment replacing rule in conf file [opt][be.conf]optimise system environment replacing rule in conf file May 24, 2025
@doris-robot
Copy link

Cloud UT Coverage Report

Increment line coverage 22.45% (11/49) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 83.25% (1113/1337)
Line Coverage 66.06% (18666/28255)
Region Coverage 65.67% (9267/14111)
Branch Coverage 55.42% (4983/8992)

@doris-robot
Copy link

TPC-H: Total hot run time: 33837 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 986128a9e5d7cef633d4749da1f94ec4abfbab36, data reload: false

------ Round 1 ----------------------------------
q1	26529	5103	5006	5006
q2	2078	279	182	182
q3	10404	1230	694	694
q4	10225	981	542	542
q5	7515	2318	2288	2288
q6	183	168	130	130
q7	867	732	611	611
q8	9337	1249	1008	1008
q9	6838	5049	5128	5049
q10	6833	2322	1903	1903
q11	493	296	281	281
q12	340	358	222	222
q13	18269	3689	3125	3125
q14	241	233	215	215
q15	547	480	493	480
q16	436	438	377	377
q17	622	850	385	385
q18	8295	7312	7111	7111
q19	1234	940	559	559
q20	343	343	239	239
q21	3900	3209	2393	2393
q22	1055	1075	1037	1037
Total cold run time: 116584 ms
Total hot run time: 33837 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5197	5103	5186	5103
q2	259	335	238	238
q3	2266	2768	2547	2547
q4	1525	1939	1504	1504
q5	4586	4592	4554	4554
q6	230	167	129	129
q7	2076	2020	1856	1856
q8	2670	2719	2620	2620
q9	7329	7164	7190	7164
q10	2966	3249	2828	2828
q11	602	519	516	516
q12	685	781	592	592
q13	3525	3909	3292	3292
q14	276	299	285	285
q15	535	474	476	474
q16	461	501	461	461
q17	1153	1577	1381	1381
q18	7888	7678	7413	7413
q19	807	833	991	833
q20	2035	2049	1942	1942
q21	4762	4411	4369	4369
q22	1079	1068	1002	1002
Total cold run time: 52912 ms
Total hot run time: 51103 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192801 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 986128a9e5d7cef633d4749da1f94ec4abfbab36, data reload: false

query1	1432	1095	1081	1081
query2	6516	1834	1826	1826
query3	11030	4412	4439	4412
query4	54727	25054	23412	23412
query5	5027	549	470	470
query6	375	219	211	211
query7	4971	535	294	294
query8	338	263	243	243
query9	5829	2643	2665	2643
query10	445	347	288	288
query11	15138	15000	14847	14847
query12	163	117	114	114
query13	1098	539	425	425
query14	10166	6367	6288	6288
query15	203	212	176	176
query16	7040	676	505	505
query17	1059	736	575	575
query18	1595	403	309	309
query19	196	190	162	162
query20	131	128	128	128
query21	206	126	110	110
query22	4367	4324	4307	4307
query23	34293	33400	33622	33400
query24	7148	2417	2429	2417
query25	496	515	418	418
query26	673	300	155	155
query27	2205	533	346	346
query28	3238	2158	2156	2156
query29	617	601	463	463
query30	281	221	198	198
query31	900	882	838	838
query32	79	69	73	69
query33	470	381	321	321
query34	782	890	571	571
query35	788	854	770	770
query36	943	1003	906	906
query37	147	108	78	78
query38	4305	4372	4245	4245
query39	1606	1456	1453	1453
query40	212	121	110	110
query41	81	55	52	52
query42	127	115	109	109
query43	535	523	481	481
query44	1363	852	860	852
query45	178	177	174	174
query46	875	1038	661	661
query47	1868	1882	1792	1792
query48	398	443	341	341
query49	722	521	453	453
query50	699	724	407	407
query51	4271	4437	4152	4152
query52	106	113	101	101
query53	226	266	202	202
query54	594	585	524	524
query55	92	88	89	88
query56	319	314	293	293
query57	1184	1202	1135	1135
query58	267	263	269	263
query59	2744	2846	2745	2745
query60	340	344	322	322
query61	146	130	132	130
query62	714	706	643	643
query63	224	182	190	182
query64	1789	1091	695	695
query65	4263	4205	4204	4204
query66	728	411	306	306
query67	16302	15859	15456	15456
query68	7767	898	531	531
query69	543	318	276	276
query70	1232	1133	1069	1069
query71	515	321	309	309
query72	5963	4756	4942	4756
query73	1373	640	355	355
query74	8936	8891	8662	8662
query75	3821	3208	2731	2731
query76	4276	1201	760	760
query77	609	358	341	341
query78	10046	10319	9322	9322
query79	2397	801	590	590
query80	643	526	465	465
query81	474	254	220	220
query82	293	131	97	97
query83	256	255	240	240
query84	301	114	85	85
query85	811	361	320	320
query86	397	299	290	290
query87	4397	4477	4321	4321
query88	3780	2321	2315	2315
query89	420	318	298	298
query90	1794	208	209	208
query91	143	147	117	117
query92	76	67	55	55
query93	2034	935	582	582
query94	633	436	297	297
query95	381	291	294	291
query96	502	580	285	285
query97	2803	2787	2636	2636
query98	247	210	207	207
query99	1353	1405	1267	1267
Total cold run time: 301606 ms
Total hot run time: 192801 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.23 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 986128a9e5d7cef633d4749da1f94ec4abfbab36, data reload: false

query1	0.04	0.04	0.03
query2	0.14	0.10	0.12
query3	0.26	0.19	0.20
query4	1.61	0.21	0.19
query5	0.46	0.45	0.43
query6	1.15	0.67	0.65
query7	0.02	0.02	0.01
query8	0.04	0.04	0.04
query9	0.59	0.51	0.54
query10	0.58	0.58	0.58
query11	0.16	0.12	0.11
query12	0.15	0.11	0.12
query13	0.62	0.60	0.60
query14	0.78	0.81	0.81
query15	0.89	0.88	0.87
query16	0.39	0.40	0.38
query17	1.04	1.05	1.07
query18	0.23	0.22	0.21
query19	1.94	1.78	1.82
query20	0.01	0.01	0.02
query21	15.40	0.90	0.54
query22	0.76	1.15	0.73
query23	14.91	1.38	0.65
query24	7.19	0.91	0.65
query25	0.47	0.14	0.13
query26	0.69	0.17	0.15
query27	0.06	0.05	0.05
query28	9.18	0.91	0.44
query29	12.53	4.14	3.41
query30	0.25	0.10	0.06
query31	2.81	0.61	0.39
query32	3.23	0.55	0.48
query33	3.03	3.10	3.05
query34	15.86	5.16	4.49
query35	4.60	4.55	4.53
query36	0.68	0.51	0.49
query37	0.08	0.07	0.07
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.14	0.13
query41	0.08	0.02	0.02
query42	0.04	0.03	0.02
query43	0.04	0.04	0.03
Total cold run time: 103.24 s
Total hot run time: 29.23 s

yiguolei
yiguolei reviewed May 24, 2025
View reviewed changes
yiguolei
yiguolei reviewed May 24, 2025
View reviewed changes
yiguolei
yiguolei reviewed May 24, 2025
View reviewed changes
yiguolei
yiguolei reviewed May 24, 2025
View reviewed changes
@zhaorongsheng
Copy link
Contributor Author

run buildall

yiguolei
yiguolei previously approved these changes May 26, 2025
View reviewed changes
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 26, 2025
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@zhaorongsheng zhaorongsheng changed the title [opt][be.conf]optimise system environment replacing rule in conf file [opt][fe/be.conf]optimise system environment replacing rule in conf file May 26, 2025
@doris-robot
Copy link

Cloud UT Coverage Report

Increment line coverage 22.45% (11/49) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 83.25% (1113/1337)
Line Coverage 66.11% (18679/28255)
Region Coverage 65.67% (9266/14111)
Branch Coverage 55.46% (4987/8992)

w41ter
w41ter reviewed May 26, 2025
View reviewed changes
@yiguolei yiguolei force-pushed the feature_support_conf_env_replace branch from 1be1ff2 to 708c068 Compare May 30, 2025 01:37
@yiguolei
Copy link
Contributor

run buildall

@hello-stephen
Copy link
Contributor

Cloud UT Coverage Report

Increment line coverage 22.45% (11/49) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 83.26% (1114/1338)
Line Coverage 66.43% (18813/28318)
Region Coverage 66.00% (9326/14130)
Branch Coverage 55.97% (5038/9002)

@doris-robot
Copy link

TPC-H: Total hot run time: 33415 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 708c068f3e54bb66e72ecb7513fc8c294eadd95c, data reload: false

------ Round 1 ----------------------------------
q1	26122	4961	4989	4961
q2	1953	272	170	170
q3	10306	1223	686	686
q4	10218	996	515	515
q5	7538	2343	2345	2343
q6	183	173	134	134
q7	890	726	612	612
q8	9316	1266	1015	1015
q9	6899	5154	5018	5018
q10	6801	2315	1909	1909
q11	483	290	273	273
q12	350	349	213	213
q13	17756	3642	3089	3089
q14	243	239	218	218
q15	574	480	499	480
q16	422	427	382	382
q17	588	858	342	342
q18	7467	7146	7027	7027
q19	1235	955	558	558
q20	346	328	220	220
q21	3736	3160	2305	2305
q22	1054	1019	945	945
Total cold run time: 114480 ms
Total hot run time: 33415 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5052	5081	5060	5060
q2	232	314	224	224
q3	2116	2621	2306	2306
q4	1377	1761	1354	1354
q5	4315	4349	4350	4349
q6	217	171	125	125
q7	1989	1934	1745	1745
q8	2556	2507	2585	2507
q9	7178	7238	7039	7039
q10	3008	3187	2762	2762
q11	581	518	485	485
q12	681	755	608	608
q13	3516	3890	3245	3245
q14	289	292	266	266
q15	512	480	478	478
q16	466	506	435	435
q17	1132	1557	1378	1378
q18	7736	7588	7518	7518
q19	789	800	958	800
q20	2002	1963	1872	1872
q21	4748	4535	4342	4342
q22	1110	1076	1022	1022
Total cold run time: 51602 ms
Total hot run time: 49920 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192337 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 708c068f3e54bb66e72ecb7513fc8c294eadd95c, data reload: false

query1	1437	1123	1085	1085
query2	6044	1811	1829	1811
query3	11143	4624	4662	4624
query4	55629	25356	23081	23081
query5	5008	524	454	454
query6	337	195	187	187
query7	4877	508	277	277
query8	277	216	208	208
query9	5355	2668	2690	2668
query10	433	323	277	277
query11	14997	14986	14824	14824
query12	158	104	108	104
query13	1037	536	428	428
query14	10245	6198	6334	6198
query15	205	207	179	179
query16	7151	607	548	548
query17	1099	737	603	603
query18	1547	421	315	315
query19	217	201	180	180
query20	132	128	130	128
query21	220	129	109	109
query22	4390	4337	4253	4253
query23	34174	33497	33539	33497
query24	6899	2405	2423	2405
query25	528	460	427	427
query26	681	273	151	151
query27	2151	533	342	342
query28	2987	2218	2205	2205
query29	572	565	439	439
query30	272	218	193	193
query31	868	881	782	782
query32	77	65	72	65
query33	449	361	347	347
query34	794	869	541	541
query35	776	835	775	775
query36	938	1008	902	902
query37	119	101	80	80
query38	4268	4314	4330	4314
query39	1514	1464	1458	1458
query40	212	122	113	113
query41	68	56	55	55
query42	135	128	107	107
query43	509	515	482	482
query44	1305	837	840	837
query45	179	178	176	176
query46	845	1039	682	682
query47	1855	1921	1767	1767
query48	421	427	317	317
query49	647	502	405	405
query50	662	706	424	424
query51	4330	4235	4220	4220
query52	112	108	100	100
query53	222	265	189	189
query54	582	591	526	526
query55	84	95	85	85
query56	320	309	290	290
query57	1162	1192	1089	1089
query58	271	261	253	253
query59	2651	2887	2670	2670
query60	333	312	299	299
query61	130	131	131	131
query62	751	753	663	663
query63	227	192	191	191
query64	1480	1047	698	698
query65	4313	4154	4153	4153
query66	700	414	299	299
query67	15761	15673	15352	15352
query68	7921	882	536	536
query69	553	304	262	262
query70	1230	1136	1083	1083
query71	502	318	305	305
query72	5521	4860	4785	4785
query73	1490	616	358	358
query74	9219	9091	8699	8699
query75	3889	3187	2696	2696
query76	4236	1194	752	752
query77	629	366	290	290
query78	10065	10038	9303	9303
query79	3482	793	574	574
query80	819	503	486	486
query81	492	256	219	219
query82	502	128	97	97
query83	351	254	233	233
query84	296	107	88	88
query85	796	364	321	321
query86	385	269	274	269
query87	4506	4359	4347	4347
query88	3689	2311	2275	2275
query89	403	316	284	284
query90	1801	207	213	207
query91	141	141	112	112
query92	75	58	57	57
query93	2292	942	590	590
query94	663	422	315	315
query95	375	293	285	285
query96	495	568	281	281
query97	2691	2847	2679	2679
query98	232	209	208	208
query99	1415	1404	1304	1304
Total cold run time: 301428 ms
Total hot run time: 192337 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.63 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 708c068f3e54bb66e72ecb7513fc8c294eadd95c, data reload: false

query1	0.04	0.03	0.03
query2	0.12	0.10	0.10
query3	0.24	0.20	0.20
query4	1.60	0.19	0.19
query5	0.44	0.44	0.44
query6	1.19	0.67	0.66
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.60	0.51	0.53
query10	0.56	0.58	0.58
query11	0.16	0.11	0.10
query12	0.15	0.11	0.12
query13	0.62	0.59	0.60
query14	0.79	0.80	0.82
query15	0.89	0.88	0.86
query16	0.37	0.39	0.37
query17	1.04	1.07	1.05
query18	0.24	0.21	0.22
query19	2.00	1.90	1.90
query20	0.01	0.01	0.01
query21	15.39	0.90	0.55
query22	0.76	1.24	0.67
query23	14.86	1.37	0.64
query24	7.42	1.42	0.97
query25	0.44	0.09	0.09
query26	0.63	0.17	0.14
query27	0.05	0.05	0.06
query28	9.54	0.89	0.45
query29	12.86	4.10	3.46
query30	0.26	0.09	0.07
query31	2.81	0.60	0.38
query32	3.24	0.55	0.48
query33	3.07	3.17	3.06
query34	15.75	5.15	4.51
query35	4.54	4.58	4.56
query36	0.68	0.50	0.48
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.03
query40	0.16	0.13	0.11
query41	0.08	0.02	0.02
query42	0.03	0.02	0.03
query43	0.04	0.03	0.03
Total cold run time: 103.91 s
Total hot run time: 29.63 s

zhaorongsheng added 2 commits May 30, 2025 14:11
support format in fe/be conf
7a32185
Merge remote-tracking branch 'origin/feature_support_conf_env_replace…
e581a36 
…' into feature_support_conf_env_replace

# Conflicts:
#	be/src/common/config.cpp
#	cloud/src/common/configbase.cpp
@zhaorongsheng
Copy link
Contributor Author

run buildall

@hello-stephen
Copy link
Contributor

Cloud UT Coverage Report

Increment line coverage 22.45% (11/49) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 83.31% (1118/1342)
Line Coverage 66.69% (19135/28694)
Region Coverage 66.21% (9464/14293)
Branch Coverage 56.24% (5136/9132)

@doris-robot
Copy link

TPC-H: Total hot run time: 33730 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e581a360d8ec891224685949ebdc3912dace0514, data reload: false

------ Round 1 ----------------------------------
q1	26333	5069	5004	5004
q2	1960	280	181	181
q3	10286	1234	699	699
q4	10238	982	524	524
q5	7537	2433	2318	2318
q6	174	163	129	129
q7	900	729	612	612
q8	9308	1260	1121	1121
q9	6896	5033	5102	5033
q10	6806	2322	1890	1890
q11	492	282	279	279
q12	342	351	220	220
q13	17753	3684	3080	3080
q14	225	238	207	207
q15	548	484	498	484
q16	420	436	372	372
q17	591	856	341	341
q18	7545	7249	7173	7173
q19	1211	947	554	554
q20	347	336	231	231
q21	3732	3217	2310	2310
q22	1063	1037	968	968
Total cold run time: 114707 ms
Total hot run time: 33730 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5122	5019	5211	5019
q2	238	329	231	231
q3	2193	2696	2325	2325
q4	1376	1793	1374	1374
q5	4505	4442	4386	4386
q6	219	173	124	124
q7	1986	1894	1797	1797
q8	2578	2518	2415	2415
q9	7268	7141	7048	7048
q10	2988	3190	2722	2722
q11	572	499	513	499
q12	660	801	601	601
q13	3472	3824	3318	3318
q14	280	284	267	267
q15	521	487	485	485
q16	442	481	461	461
q17	1216	1537	1396	1396
q18	7784	7516	7446	7446
q19	797	802	805	802
q20	1946	2002	1890	1890
q21	4853	4466	4326	4326
q22	1076	1079	985	985
Total cold run time: 52092 ms
Total hot run time: 49917 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185657 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e581a360d8ec891224685949ebdc3912dace0514, data reload: false

query1	994	492	508	492
query2	6594	1921	1914	1914
query3	6746	226	224	224
query4	26440	23730	23007	23007
query5	4301	619	451	451
query6	310	205	192	192
query7	4621	488	288	288
query8	261	223	209	209
query9	8626	2614	2635	2614
query10	481	339	280	280
query11	15639	15053	14803	14803
query12	174	113	107	107
query13	1664	532	405	405
query14	10395	6284	6204	6204
query15	202	198	167	167
query16	7255	647	508	508
query17	1202	746	580	580
query18	1970	397	326	326
query19	186	177	168	168
query20	119	120	120	120
query21	211	128	109	109
query22	4080	4202	3959	3959
query23	34203	33024	33300	33024
query24	7777	2362	2404	2362
query25	539	456	385	385
query26	1231	271	149	149
query27	2639	467	337	337
query28	4352	2118	2093	2093
query29	775	550	455	455
query30	279	217	191	191
query31	950	881	754	754
query32	78	65	65	65
query33	555	368	297	297
query34	839	836	524	524
query35	771	819	725	725
query36	921	1010	913	913
query37	116	103	78	78
query38	4104	4072	4035	4035
query39	1475	1452	1407	1407
query40	208	115	102	102
query41	65	60	55	55
query42	119	107	102	102
query43	500	519	481	481
query44	1308	813	824	813
query45	177	175	175	175
query46	827	1041	619	619
query47	1772	1776	1737	1737
query48	391	422	330	330
query49	736	477	435	435
query50	644	681	395	395
query51	4128	4166	4173	4166
query52	115	103	95	95
query53	218	244	186	186
query54	580	574	500	500
query55	84	81	80	80
query56	297	301	295	295
query57	1110	1166	1068	1068
query58	257	260	259	259
query59	2695	2783	2652	2652
query60	340	316	300	300
query61	158	151	149	149
query62	814	734	680	680
query63	223	195	191	191
query64	4434	1050	664	664
query65	4234	4121	4123	4121
query66	1135	402	362	362
query67	16177	15885	15546	15546
query68	8232	881	513	513
query69	467	316	270	270
query70	1168	1111	1113	1111
query71	462	333	302	302
query72	5308	4686	4635	4635
query73	707	583	354	354
query74	8833	9182	8932	8932
query75	3840	3173	2671	2671
query76	3669	1176	755	755
query77	812	371	293	293
query78	10053	10053	9298	9298
query79	2537	818	571	571
query80	593	511	439	439
query81	486	316	218	218
query82	694	129	92	92
query83	249	248	237	237
query84	248	120	94	94
query85	809	349	326	326
query86	393	319	285	285
query87	4378	4493	4322	4322
query88	3773	2262	2304	2262
query89	394	327	284	284
query90	1911	206	204	204
query91	141	146	112	112
query92	77	61	57	57
query93	1644	942	567	567
query94	678	408	312	312
query95	376	296	287	287
query96	486	570	279	279
query97	2759	2794	2630	2630
query98	240	215	206	206
query99	1429	1390	1260	1260
Total cold run time: 275667 ms
Total hot run time: 185657 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.02 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e581a360d8ec891224685949ebdc3912dace0514, data reload: false

query1	0.05	0.03	0.03
query2	0.11	0.11	0.10
query3	0.26	0.19	0.19
query4	1.60	0.18	0.19
query5	0.45	0.46	0.44
query6	1.19	0.66	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.04
query9	0.59	0.52	0.52
query10	0.57	0.56	0.56
query11	0.16	0.11	0.10
query12	0.14	0.12	0.12
query13	0.60	0.61	0.59
query14	0.79	0.80	0.81
query15	0.88	0.84	0.85
query16	0.40	0.38	0.38
query17	1.05	1.03	1.05
query18	0.23	0.21	0.20
query19	1.89	1.81	1.80
query20	0.01	0.01	0.01
query21	15.44	0.90	0.55
query22	0.76	1.15	0.92
query23	14.72	1.38	0.60
query24	7.62	1.02	0.35
query25	0.47	0.18	0.14
query26	0.61	0.15	0.13
query27	0.05	0.05	0.06
query28	9.62	0.88	0.44
query29	12.55	4.03	3.35
query30	0.25	0.09	0.06
query31	3.00	0.59	0.40
query32	3.51	0.56	0.46
query33	3.17	3.12	3.09
query34	15.78	5.12	4.53
query35	4.55	4.56	4.58
query36	0.67	0.50	0.48
query37	0.09	0.06	0.07
query38	0.06	0.05	0.04
query39	0.02	0.02	0.02
query40	0.16	0.13	0.13
query41	0.08	0.02	0.02
query42	0.03	0.03	0.02
query43	0.04	0.04	0.03
Total cold run time: 104.28 s
Total hot run time: 29.02 s

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 93.88% (46/49) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 56.13% (15029/26774)
Line Coverage 45.01% (134191/298160)
Region Coverage 44.13% (67551/153088)
Branch Coverage 38.69% (34609/89446)

@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 31, 2025
@zhaorongsheng
Copy link
Contributor Author

@w41ter Would you mind reviewing the code once again? Thanks~

@zhaorongsheng
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

Cloud UT Coverage Report

Increment line coverage 22.45% (11/49) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 83.31% (1118/1342)
Line Coverage 66.72% (19145/28694)
Region Coverage 66.29% (9475/14293)
Branch Coverage 56.26% (5138/9132)

@doris-robot
Copy link

TPC-H: Total hot run time: 33945 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2b6eef57b2653dbe79fd55dad281ee77f7995ac7, data reload: false

------ Round 1 ----------------------------------
q1	25609	5015	5045	5015
q2	1962	280	184	184
q3	10269	1221	707	707
q4	10216	1004	552	552
q5	7534	2348	2294	2294
q6	176	165	134	134
q7	904	710	632	632
q8	9318	1245	1091	1091
q9	6835	5083	5104	5083
q10	6846	2288	1886	1886
q11	515	296	292	292
q12	342	355	217	217
q13	17796	3696	3078	3078
q14	229	228	212	212
q15	559	488	472	472
q16	422	429	370	370
q17	631	862	387	387
q18	7538	7239	7145	7145
q19	1208	942	555	555
q20	356	360	240	240
q21	3794	2629	2425	2425
q22	1070	994	974	974
Total cold run time: 114129 ms
Total hot run time: 33945 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5141	5092	5150	5092
q2	255	319	229	229
q3	2196	2672	2348	2348
q4	1408	1822	1486	1486
q5	4440	4470	4483	4470
q6	235	177	133	133
q7	2069	1983	1789	1789
q8	2608	2542	2621	2542
q9	7256	7172	6899	6899
q10	3044	3182	2736	2736
q11	607	512	505	505
q12	665	766	611	611
q13	3527	3958	3303	3303
q14	290	290	296	290
q15	531	478	490	478
q16	454	495	436	436
q17	1146	1559	1409	1409
q18	7754	7603	7577	7577
q19	841	854	988	854
q20	2050	1968	1873	1873
q21	4811	4460	4429	4429
q22	1071	1037	1024	1024
Total cold run time: 52399 ms
Total hot run time: 50513 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191969 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2b6eef57b2653dbe79fd55dad281ee77f7995ac7, data reload: false

query1	1428	1086	1054	1054
query2	6224	1823	1829	1823
query3	11012	4450	4459	4450
query4	55250	25068	23056	23056
query5	5023	550	464	464
query6	346	218	200	200
query7	4883	506	289	289
query8	291	235	216	216
query9	5375	2659	2672	2659
query10	435	320	277	277
query11	15062	15358	14787	14787
query12	160	106	106	106
query13	1022	534	418	418
query14	10119	6360	6257	6257
query15	211	198	174	174
query16	7088	630	499	499
query17	1073	740	568	568
query18	1591	405	305	305
query19	200	185	159	159
query20	130	129	121	121
query21	206	129	105	105
query22	4415	4345	4148	4148
query23	34264	33543	33587	33543
query24	6770	2421	2488	2421
query25	479	480	413	413
query26	682	275	154	154
query27	2269	514	336	336
query28	2983	2171	2161	2161
query29	580	584	464	464
query30	277	227	186	186
query31	843	840	797	797
query32	76	66	70	66
query33	456	361	330	330
query34	781	871	546	546
query35	843	861	803	803
query36	936	1001	885	885
query37	118	103	81	81
query38	4177	4326	4202	4202
query39	1533	1517	1449	1449
query40	207	129	124	124
query41	62	59	58	58
query42	137	109	113	109
query43	501	505	490	490
query44	1335	859	857	857
query45	190	186	175	175
query46	877	1046	647	647
query47	1842	1842	1833	1833
query48	410	444	328	328
query49	661	499	404	404
query50	682	711	417	417
query51	4311	4280	4176	4176
query52	109	107	105	105
query53	235	259	181	181
query54	603	578	526	526
query55	82	82	90	82
query56	299	301	280	280
query57	1145	1198	1104	1104
query58	268	261	264	261
query59	2820	2824	2633	2633
query60	338	333	305	305
query61	132	127	146	127
query62	776	764	668	668
query63	229	197	181	181
query64	1766	1037	684	684
query65	4259	4124	4135	4124
query66	725	397	300	300
query67	15954	15803	15410	15410
query68	7200	901	528	528
query69	534	316	268	268
query70	1203	1110	1075	1075
query71	508	328	300	300
query72	6001	4925	4971	4925
query73	1485	703	352	352
query74	9146	9053	8866	8866
query75	3985	3185	2690	2690
query76	4289	1188	762	762
query77	659	374	294	294
query78	10191	10267	9322	9322
query79	2231	820	583	583
query80	570	502	442	442
query81	470	255	215	215
query82	444	126	100	100
query83	243	245	235	235
query84	292	114	94	94
query85	778	361	309	309
query86	412	304	293	293
query87	4322	4434	4240	4240
query88	3438	2274	2326	2274
query89	406	307	278	278
query90	1833	209	208	208
query91	143	147	114	114
query92	73	58	56	56
query93	1747	941	588	588
query94	650	464	307	307
query95	373	295	293	293
query96	499	571	284	284
query97	2729	2782	2630	2630
query98	235	206	206	206
query99	1337	1380	1277	1277
Total cold run time: 298992 ms
Total hot run time: 191969 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.12 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2b6eef57b2653dbe79fd55dad281ee77f7995ac7, data reload: false

query1	0.04	0.04	0.03
query2	0.12	0.11	0.11
query3	0.25	0.19	0.20
query4	1.59	0.20	0.20
query5	0.46	0.44	0.46
query6	1.18	0.67	0.65
query7	0.03	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.51	0.53
query10	0.58	0.57	0.56
query11	0.16	0.11	0.11
query12	0.15	0.11	0.12
query13	0.61	0.59	0.60
query14	0.79	0.81	0.81
query15	0.88	0.86	0.88
query16	0.39	0.37	0.37
query17	1.06	1.06	1.04
query18	0.22	0.20	0.22
query19	1.90	1.81	1.85
query20	0.01	0.02	0.02
query21	15.40	0.87	0.56
query22	0.76	1.29	0.78
query23	14.74	1.38	0.64
query24	7.17	1.05	0.70
query25	0.48	0.32	0.09
query26	0.51	0.16	0.14
query27	0.05	0.06	0.05
query28	9.86	0.94	0.45
query29	12.86	3.94	3.30
query30	0.25	0.09	0.06
query31	2.82	0.60	0.39
query32	3.23	0.55	0.47
query33	3.02	3.03	3.04
query34	15.92	5.06	4.49
query35	4.52	4.51	4.51
query36	0.66	0.52	0.48
query37	0.09	0.07	0.06
query38	0.06	0.04	0.04
query39	0.03	0.02	0.03
query40	0.16	0.13	0.13
query41	0.07	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.78 s
Total hot run time: 29.12 s

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 93.88% (46/49) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 56.06% (15029/26809)
Line Coverage 44.95% (134189/298531)
Region Coverage 44.08% (67545/153219)
Branch Coverage 38.64% (34604/89554)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@w41ter w41ter w41ter left review comments

@yiguolei yiguolei yiguolei approved these changes

@gavinchou gavinchou Awaiting requested review from gavinchou gavinchou is a code owner

@dataroaring dataroaring Awaiting requested review from dataroaring dataroaring is a code owner

@CalvinKirs CalvinKirs Awaiting requested review from CalvinKirs CalvinKirs is a code owner

@morningman morningman Awaiting requested review from morningman morningman is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Bug] can not set $DORIS_HOME in be conf file
6 participants
@zhaorongsheng @Thearas @doris-robot @hello-stephen @yiguolei @w41ter