Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve](mow) make delete bitmap lock default expiration on metaservice more reasonable #39429

Merged
merged 1 commit into from
Aug 16, 2024

Conversation

hust-hhb
Copy link
Contributor

Now the lock expiration is too small, be will get lock expired very frequently when load task on mow is heavy.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@hust-hhb
Copy link
Contributor Author

run buildall

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 15, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 38214 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fa142e06cd68ac6ae8225892eaf6be8484f6000b, data reload: false

------ Round 1 ----------------------------------
q1	17874	4437	4272	4272
q2	2046	222	220	220
q3	10411	1270	1108	1108
q4	10170	838	801	801
q5	7790	2818	2776	2776
q6	263	158	161	158
q7	1000	649	640	640
q8	9394	2117	2065	2065
q9	7255	6555	6533	6533
q10	7083	2188	2208	2188
q11	495	272	282	272
q12	436	263	266	263
q13	17783	3014	3038	3014
q14	298	273	257	257
q15	574	541	554	541
q16	529	416	426	416
q17	979	665	628	628
q18	7733	6986	6763	6763
q19	5730	1079	1037	1037
q20	701	355	366	355
q21	3824	2885	3006	2885
q22	1111	1062	1022	1022
Total cold run time: 113479 ms
Total hot run time: 38214 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4614	4238	4279	4238
q2	416	326	309	309
q3	2832	2673	2642	2642
q4	1936	1610	1602	1602
q5	5397	5430	5399	5399
q6	243	152	154	152
q7	2087	1689	1743	1689
q8	3211	3366	3322	3322
q9	8433	8387	8402	8387
q10	3407	3150	3146	3146
q11	600	522	521	521
q12	833	635	613	613
q13	16882	3007	3025	3007
q14	312	288	298	288
q15	557	537	503	503
q16	496	443	429	429
q17	1805	1504	1489	1489
q18	7755	7565	7403	7403
q19	5049	1610	1517	1517
q20	2078	1823	1806	1806
q21	11875	5081	5137	5081
q22	1141	1033	1056	1033
Total cold run time: 81959 ms
Total hot run time: 54576 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 189667 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fa142e06cd68ac6ae8225892eaf6be8484f6000b, data reload: false

query1	1019	412	388	388
query2	6715	2019	1948	1948
query3	6690	223	241	223
query4	29307	23902	23258	23258
query5	4382	624	627	624
query6	289	198	184	184
query7	4616	311	307	307
query8	417	353	354	353
query9	8766	2491	2473	2473
query10	469	343	310	310
query11	16112	15076	14956	14956
query12	166	127	127	127
query13	1707	410	408	408
query14	10192	7115	7285	7115
query15	279	183	189	183
query16	7901	524	550	524
query17	1623	591	582	582
query18	1718	326	328	326
query19	222	162	164	162
query20	138	131	133	131
query21	252	147	138	138
query22	4459	4164	4121	4121
query23	34019	33263	33296	33263
query24	11618	2896	2909	2896
query25	641	414	412	412
query26	1194	176	179	176
query27	2781	297	306	297
query28	7582	2121	2119	2119
query29	821	440	467	440
query30	339	187	179	179
query31	1092	820	832	820
query32	115	77	79	77
query33	814	342	340	340
query34	904	497	500	497
query35	889	744	761	744
query36	1107	930	973	930
query37	160	106	103	103
query38	3960	3820	3792	3792
query39	1516	1463	1454	1454
query40	233	154	153	153
query41	137	136	135	135
query42	138	111	124	111
query43	543	499	510	499
query44	1239	773	771	771
query45	219	194	198	194
query46	1156	790	759	759
query47	1916	1813	1804	1804
query48	409	334	332	332
query49	1230	574	581	574
query50	853	450	453	450
query51	6852	6769	6685	6685
query52	115	110	113	110
query53	294	225	221	221
query54	1001	531	510	510
query55	89	85	90	85
query56	320	306	304	304
query57	1248	1105	1116	1105
query58	320	298	302	298
query59	2962	2709	2844	2709
query60	346	345	325	325
query61	148	141	143	141
query62	865	694	698	694
query63	259	224	233	224
query64	5377	2307	1853	1853
query65	3453	3197	3201	3197
query66	1271	661	650	650
query67	15973	14882	14928	14882
query68	6324	572	594	572
query69	779	413	339	339
query70	1261	1063	1182	1063
query71	555	309	306	306
query72	7556	2312	2032	2032
query73	865	359	352	352
query74	9382	8726	8844	8726
query75	5010	2691	2816	2691
query76	4935	1049	1006	1006
query77	929	435	444	435
query78	9791	9178	8996	8996
query79	5977	556	562	556
query80	2382	594	602	594
query81	627	263	254	254
query82	839	156	151	151
query83	341	213	210	210
query84	286	99	99	99
query85	1462	350	346	346
query86	449	318	316	316
query87	4444	4238	4200	4200
query88	4864	2492	2454	2454
query89	473	327	329	327
query90	2018	227	227	227
query91	151	124	126	124
query92	87	76	73	73
query93	5121	556	561	556
query94	899	333	369	333
query95	401	298	294	294
query96	645	282	283	282
query97	3261	3075	3130	3075
query98	249	229	226	226
query99	1642	1308	1297	1297
Total cold run time: 309962 ms
Total hot run time: 189667 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.91 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fa142e06cd68ac6ae8225892eaf6be8484f6000b, data reload: false

query1	0.05	0.05	0.04
query2	0.07	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.09	0.10
query5	0.51	0.48	0.47
query6	1.13	0.74	0.72
query7	0.02	0.01	0.02
query8	0.06	0.05	0.05
query9	0.56	0.48	0.48
query10	0.55	0.54	0.53
query11	0.16	0.14	0.13
query12	0.16	0.14	0.13
query13	0.61	0.60	0.59
query14	0.77	0.79	0.78
query15	0.86	0.83	0.83
query16	0.36	0.38	0.37
query17	0.98	1.03	0.98
query18	0.25	0.23	0.23
query19	1.84	1.68	1.88
query20	0.02	0.01	0.01
query21	15.41	0.84	0.64
query22	4.44	6.95	1.71
query23	18.25	1.43	1.26
query24	2.11	0.23	0.21
query25	0.16	0.09	0.08
query26	0.29	0.22	0.22
query27	0.45	0.23	0.24
query28	13.27	1.05	1.02
query29	12.62	3.35	3.31
query30	0.38	0.20	0.25
query31	2.81	0.40	0.40
query32	3.23	0.49	0.50
query33	2.94	2.94	2.94
query34	17.01	4.37	4.34
query35	4.42	4.41	4.41
query36	0.66	0.50	0.51
query37	0.21	0.19	0.18
query38	0.18	0.16	0.18
query39	0.06	0.06	0.06
query40	0.18	0.16	0.15
query41	0.11	0.08	0.07
query42	0.08	0.07	0.06
query43	0.07	0.07	0.06
Total cold run time: 110.2 s
Total hot run time: 30.91 s

Copy link
Contributor

@zhannngchen zhannngchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit b3d20e7 into apache:master Aug 16, 2024
31 of 33 checks passed
eldenmoon pushed a commit to eldenmoon/incubator-doris that referenced this pull request Nov 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.1-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants