Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[opt](memory) Purge all unused Jemalloc dirty pages for all arenas when exceed limit #37164

Merged
merged 4 commits into from
Jul 5, 2024
Merged

[opt](memory) Purge all unused Jemalloc dirty pages for all arenas when exceed limit #37164

merged 4 commits into from
Jul 5, 2024

Conversation

xinyiZzz
Copy link
Contributor

@xinyiZzz xinyiZzz commented Jul 2, 2024 •
edited
Loading

add BE conf je_dirty_pages_mem_limit_percent, default 5% * mem_limit, purge all unused Jemalloc dirty pages for all arenas when exceed je_dirty_pages_mem_limit and process exceed soft limit.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@xinyiZzz xinyiZzz force-pushed the 20240702_fix_free_jemalloc_cache branch from 4681ab8 to 4ab0248 Compare July 2, 2024 12:13
@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Jul 2, 2024

run buildall

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 2, 2024

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 2, 2024

clang-tidy review says "All clean, LGTM! 👍"

@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Jul 2, 2024

run buildall

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 2, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39818 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8d429b8598233fe75f4049b9766f9e83bb27e9f1, data reload: false

------ Round 1 ----------------------------------
q1	17680	4440	4252	4252
q2	2029	197	189	189
q3	10552	1194	1102	1102
q4	10215	774	728	728
q5	7510	2677	2628	2628
q6	227	140	141	140
q7	957	612	614	612
q8	9231	2084	2067	2067
q9	9068	6527	6459	6459
q10	8954	3702	3741	3702
q11	452	234	246	234
q12	544	240	237	237
q13	18810	2972	2993	2972
q14	273	234	212	212
q15	519	480	504	480
q16	526	372	372	372
q17	974	721	694	694
q18	8034	7431	7517	7431
q19	6396	1447	1494	1447
q20	682	321	335	321
q21	5024	3207	3964	3207
q22	395	346	332	332
Total cold run time: 119052 ms
Total hot run time: 39818 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4364	4272	4283	4272
q2	373	269	268	268
q3	3059	2912	3031	2912
q4	1994	1769	1748	1748
q5	5550	5531	5430	5430
q6	221	140	135	135
q7	2297	1875	1834	1834
q8	3261	3428	3399	3399
q9	8704	8784	8759	8759
q10	4157	3716	3834	3716
q11	617	502	494	494
q12	838	650	632	632
q13	17017	3143	3146	3143
q14	311	272	299	272
q15	532	482	483	482
q16	486	453	438	438
q17	1788	1506	1509	1506
q18	8292	8031	7827	7827
q19	2855	1576	1551	1551
q20	2182	1877	1868	1868
q21	5168	4967	4831	4831
q22	665	580	557	557
Total cold run time: 74731 ms
Total hot run time: 56074 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174861 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8d429b8598233fe75f4049b9766f9e83bb27e9f1, data reload: false

query1	911	384	370	370
query2	6483	2346	2427	2346
query3	6648	209	214	209
query4	21209	17615	17365	17365
query5	3688	479	503	479
query6	261	165	165	165
query7	4593	301	304	301
query8	338	300	273	273
query9	8438	2380	2372	2372
query10	576	285	279	279
query11	10574	10043	9944	9944
query12	115	92	78	78
query13	1640	387	367	367
query14	9541	7719	7750	7719
query15	234	187	183	183
query16	7873	274	265	265
query17	1892	561	542	542
query18	2002	267	264	264
query19	198	151	151	151
query20	90	87	83	83
query21	228	152	131	131
query22	4448	4074	3915	3915
query23	33962	33606	33582	33582
query24	10667	2853	2802	2802
query25	590	380	374	374
query26	709	155	159	155
query27	2251	326	320	320
query28	6051	2151	2139	2139
query29	897	667	639	639
query30	256	159	162	159
query31	976	775	763	763
query32	98	55	57	55
query33	689	297	292	292
query34	871	486	496	486
query35	752	690	659	659
query36	1113	1001	964	964
query37	141	76	77	76
query38	2934	2873	2839	2839
query39	882	845	863	845
query40	210	126	129	126
query41	53	54	55	54
query42	115	110	103	103
query43	578	529	549	529
query44	1106	743	741	741
query45	194	160	162	160
query46	1078	743	737	737
query47	1869	1768	1766	1766
query48	372	318	321	318
query49	855	432	437	432
query50	759	393	395	393
query51	6943	6888	6817	6817
query52	111	92	98	92
query53	361	290	304	290
query54	886	463	457	457
query55	78	75	77	75
query56	306	286	279	279
query57	1132	1066	1070	1066
query58	267	253	261	253
query59	3350	3226	3199	3199
query60	337	305	295	295
query61	125	111	116	111
query62	619	441	439	439
query63	324	306	296	296
query64	8682	2357	1836	1836
query65	3139	3102	3102	3102
query66	743	323	320	320
query67	15611	15055	14821	14821
query68	6206	550	542	542
query69	717	435	328	328
query70	1239	1144	1156	1144
query71	522	286	279	279
query72	8803	5546	5200	5200
query73	827	328	327	327
query74	5995	5523	5589	5523
query75	4675	2696	2716	2696
query76	4205	922	944	922
query77	734	302	301	301
query78	10470	9767	9763	9763
query79	4245	523	517	517
query80	1067	467	474	467
query81	588	223	214	214
query82	741	105	112	105
query83	282	173	175	173
query84	268	86	91	86
query85	1211	279	269	269
query86	455	332	317	317
query87	3308	3130	3112	3112
query88	4174	2456	2447	2447
query89	495	383	376	376
query90	1913	193	193	193
query91	129	98	100	98
query92	64	49	48	48
query93	5107	523	514	514
query94	1184	188	191	188
query95	411	321	389	321
query96	610	273	277	273
query97	3235	3052	3031	3031
query98	228	201	202	201
query99	1283	829	845	829
Total cold run time: 281413 ms
Total hot run time: 174861 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.41 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8d429b8598233fe75f4049b9766f9e83bb27e9f1, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.04	0.04
query3	0.22	0.06	0.05
query4	1.67	0.08	0.07
query5	0.52	0.49	0.48
query6	1.14	0.74	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.55	0.48	0.47
query10	0.54	0.54	0.55
query11	0.15	0.11	0.10
query12	0.14	0.12	0.12
query13	0.60	0.58	0.58
query14	0.76	0.77	0.78
query15	0.85	0.82	0.85
query16	0.38	0.37	0.35
query17	0.97	0.95	1.03
query18	0.24	0.23	0.24
query19	1.86	1.73	1.77
query20	0.02	0.01	0.01
query21	15.45	0.85	0.75
query22	4.32	5.69	2.64
query23	18.28	1.30	1.32
query24	2.12	0.26	0.22
query25	0.17	0.09	0.08
query26	0.26	0.17	0.17
query27	0.09	0.08	0.08
query28	13.18	1.03	1.00
query29	12.56	3.31	3.25
query30	0.24	0.05	0.06
query31	2.90	0.39	0.39
query32	3.24	0.47	0.46
query33	2.90	2.90	2.93
query34	17.19	4.48	4.43
query35	4.48	4.54	4.50
query36	0.65	0.49	0.48
query37	0.18	0.15	0.16
query38	0.16	0.16	0.15
query39	0.04	0.03	0.04
query40	0.17	0.15	0.14
query41	0.09	0.04	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.03
Total cold run time: 109.56 s
Total hot run time: 31.41 s

@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Jul 3, 2024

run buildall

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 3, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 40733 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4211565d8fcc6e45d53441db41e2daddde9a5768, data reload: false

------ Round 1 ----------------------------------
q1	17676	4440	4364	4364
q2	2552	200	198	198
q3	10929	1242	1179	1179
q4	10963	810	791	791
q5	7731	2717	2675	2675
q6	223	138	137	137
q7	970	623	626	623
q8	9608	2108	2067	2067
q9	9056	6479	6481	6479
q10	8968	3697	3746	3697
q11	456	241	237	237
q12	446	234	230	230
q13	17801	2995	2979	2979
q14	280	228	228	228
q15	520	486	475	475
q16	502	375	383	375
q17	986	665	759	665
q18	7956	7513	7323	7323
q19	3703	1500	1539	1500
q20	641	319	324	319
q21	4895	3852	4029	3852
q22	405	340	341	340
Total cold run time: 117267 ms
Total hot run time: 40733 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4366	4263	4189	4189
q2	382	264	275	264
q3	2983	2728	2730	2728
q4	1844	1573	1597	1573
q5	5280	5284	5291	5284
q6	214	130	128	128
q7	2173	1751	1701	1701
q8	3183	3355	3296	3296
q9	8393	8344	8312	8312
q10	3852	3676	3632	3632
q11	608	480	482	480
q12	763	609	594	594
q13	16377	2993	3009	2993
q14	288	271	262	262
q15	524	494	474	474
q16	456	413	421	413
q17	1783	1501	1443	1443
q18	7600	7589	7425	7425
q19	2600	1578	1532	1532
q20	1981	1776	1765	1765
q21	4889	4747	4829	4747
q22	642	560	562	560
Total cold run time: 71181 ms
Total hot run time: 53795 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173889 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4211565d8fcc6e45d53441db41e2daddde9a5768, data reload: false

query1	925	376	368	368
query2	6458	2514	2421	2421
query3	6659	206	210	206
query4	19917	17297	17108	17108
query5	4220	484	504	484
query6	259	178	187	178
query7	4596	298	291	291
query8	315	305	274	274
query9	8510	2524	2496	2496
query10	624	307	285	285
query11	10598	10073	10076	10073
query12	134	92	86	86
query13	1649	370	375	370
query14	10446	7875	7643	7643
query15	243	195	204	195
query16	8029	279	269	269
query17	1897	597	521	521
query18	2036	272	267	267
query19	196	151	149	149
query20	95	80	81	80
query21	209	127	124	124
query22	4457	4032	4000	4000
query23	33876	33149	33079	33079
query24	11101	2816	2807	2807
query25	610	366	370	366
query26	1367	155	152	152
query27	2952	318	317	317
query28	7298	2149	2130	2130
query29	889	613	614	613
query30	280	150	150	150
query31	967	732	748	732
query32	98	51	54	51
query33	772	293	278	278
query34	959	462	469	462
query35	733	631	641	631
query36	1053	957	931	931
query37	165	74	76	74
query38	2893	2709	2703	2703
query39	869	815	792	792
query40	204	128	125	125
query41	53	53	52	52
query42	117	98	104	98
query43	607	567	539	539
query44	1203	721	737	721
query45	193	163	166	163
query46	1078	721	699	699
query47	1831	1750	1794	1750
query48	359	295	295	295
query49	1092	405	406	405
query50	763	383	386	383
query51	6937	6868	6800	6800
query52	104	93	95	93
query53	368	292	303	292
query54	882	459	449	449
query55	74	77	76	76
query56	289	297	275	275
query57	1125	1066	1027	1027
query58	262	230	257	230
query59	3540	3436	3392	3392
query60	309	285	277	277
query61	95	90	91	90
query62	671	439	461	439
query63	328	295	304	295
query64	9069	2237	1728	1728
query65	3180	3118	3119	3118
query66	1385	348	334	334
query67	15475	14950	14941	14941
query68	4960	548	548	548
query69	634	400	315	315
query70	1187	1153	1138	1138
query71	430	288	272	272
query72	8038	5374	5306	5306
query73	763	327	329	327
query74	5859	5519	5465	5465
query75	3607	2642	2648	2642
query76	3499	926	907	907
query77	671	305	307	305
query78	10292	9857	9710	9710
query79	2266	515	512	512
query80	875	469	467	467
query81	604	221	218	218
query82	903	107	104	104
query83	265	168	163	163
query84	238	87	84	84
query85	1611	287	274	274
query86	488	317	319	317
query87	3267	3104	3092	3092
query88	4245	2451	2516	2451
query89	478	387	375	375
query90	1823	188	193	188
query91	129	98	98	98
query92	68	48	52	48
query93	2288	513	511	511
query94	1243	193	193	193
query95	411	323	328	323
query96	588	272	270	270
query97	3255	3000	3060	3000
query98	218	199	188	188
query99	1109	867	836	836
Total cold run time: 277259 ms
Total hot run time: 173889 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.41 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4211565d8fcc6e45d53441db41e2daddde9a5768, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.04	0.04
query4	1.68	0.09	0.09
query5	0.47	0.50	0.48
query6	1.13	0.72	0.74
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.54	0.50	0.50
query10	0.54	0.55	0.53
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.58	0.59	0.59
query14	0.80	0.76	0.78
query15	0.83	0.82	0.81
query16	0.34	0.36	0.37
query17	1.04	0.94	1.01
query18	0.24	0.25	0.25
query19	1.87	1.71	1.66
query20	0.02	0.01	0.01
query21	15.44	0.78	0.66
query22	4.24	7.43	1.89
query23	18.30	1.38	1.26
query24	2.14	0.22	0.22
query25	0.16	0.09	0.08
query26	0.26	0.17	0.18
query27	0.08	0.07	0.08
query28	13.19	1.02	1.00
query29	12.64	3.31	3.28
query30	0.25	0.06	0.05
query31	2.87	0.39	0.38
query32	3.29	0.47	0.45
query33	2.88	2.90	2.97
query34	17.12	4.39	4.42
query35	4.49	4.46	4.48
query36	0.65	0.47	0.48
query37	0.18	0.15	0.16
query38	0.15	0.14	0.14
query39	0.04	0.03	0.03
query40	0.18	0.14	0.17
query41	0.10	0.04	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.56 s
Total hot run time: 30.41 s

@xinyiZzz
Copy link
Contributor Author

xinyiZzz commented Jul 3, 2024

run buildall

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 3, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39991 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 57e523ebdcc98b9d30aa77ed86f44159366219b7, data reload: false

------ Round 1 ----------------------------------
q1	17611	4449	4346	4346
q2	2029	199	195	195
q3	10470	1215	1144	1144
q4	10190	741	862	741
q5	7499	2781	2621	2621
q6	224	140	138	138
q7	961	606	620	606
q8	9241	2088	2080	2080
q9	8462	6493	6506	6493
q10	9076	3742	3712	3712
q11	442	244	248	244
q12	479	231	230	230
q13	18079	2977	2983	2977
q14	283	235	219	219
q15	528	481	475	475
q16	532	377	381	377
q17	981	708	662	662
q18	8093	7512	7378	7378
q19	5642	1479	1588	1479
q20	634	318	331	318
q21	5002	3218	3236	3218
q22	400	338	349	338
Total cold run time: 116858 ms
Total hot run time: 39991 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4396	4218	4259	4218
q2	369	265	255	255
q3	3007	2859	2881	2859
q4	2038	1661	1677	1661
q5	5667	5478	5548	5478
q6	219	133	131	131
q7	2222	1798	1905	1798
q8	3242	3427	3427	3427
q9	8750	8716	8896	8716
q10	4108	3936	3706	3706
q11	596	483	494	483
q12	805	648	682	648
q13	16408	3159	3192	3159
q14	321	276	281	276
q15	537	485	492	485
q16	467	433	421	421
q17	1835	1545	1543	1543
q18	8278	7992	7810	7810
q19	1771	1587	1682	1587
q20	2147	1887	1864	1864
q21	5106	5033	4916	4916
q22	627	565	591	565
Total cold run time: 72916 ms
Total hot run time: 56006 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174356 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 57e523ebdcc98b9d30aa77ed86f44159366219b7, data reload: false

query1	905	380	372	372
query2	6443	2465	2365	2365
query3	6638	204	212	204
query4	19653	17327	17421	17327
query5	3671	497	480	480
query6	268	170	166	166
query7	4596	306	296	296
query8	312	288	297	288
query9	8600	2500	2479	2479
query10	570	320	288	288
query11	10530	10124	10028	10028
query12	116	93	85	85
query13	1646	368	374	368
query14	9443	6965	6966	6965
query15	235	189	187	187
query16	7462	277	278	277
query17	1391	571	542	542
query18	1936	288	278	278
query19	200	159	157	157
query20	89	81	86	81
query21	205	132	127	127
query22	4171	4117	4164	4117
query23	33996	33724	33667	33667
query24	10964	2932	3006	2932
query25	638	404	394	394
query26	712	162	157	157
query27	2212	332	336	332
query28	6021	2252	2208	2208
query29	921	746	622	622
query30	254	161	159	159
query31	961	763	745	745
query32	89	55	55	55
query33	660	302	300	300
query34	878	480	496	480
query35	755	661	646	646
query36	1129	979	956	956
query37	135	79	79	79
query38	2954	2846	2808	2808
query39	870	846	816	816
query40	207	126	121	121
query41	54	50	55	50
query42	114	102	96	96
query43	587	575	549	549
query44	1081	745	721	721
query45	197	161	172	161
query46	1057	723	721	721
query47	1854	1764	1767	1764
query48	364	318	297	297
query49	829	422	423	422
query50	763	385	389	385
query51	6887	6884	6671	6671
query52	98	90	89	89
query53	357	285	284	284
query54	885	440	443	440
query55	77	75	74	74
query56	285	275	271	271
query57	1134	1043	1045	1043
query58	250	251	255	251
query59	3344	3282	3215	3215
query60	299	266	288	266
query61	131	92	95	92
query62	596	438	428	428
query63	311	288	286	286
query64	8440	2233	1786	1786
query65	3185	3106	3103	3103
query66	768	325	329	325
query67	15680	14957	15239	14957
query68	8492	558	574	558
query69	686	416	324	324
query70	1423	1105	1136	1105
query71	465	280	275	275
query72	7783	5437	5199	5199
query73	2214	331	330	330
query74	5890	5509	5471	5471
query75	4065	2719	2745	2719
query76	4419	1001	903	903
query77	660	307	306	306
query78	10553	9943	9783	9783
query79	5907	518	510	510
query80	1092	475	469	469
query81	572	214	216	214
query82	569	110	107	107
query83	202	171	170	170
query84	266	87	90	87
query85	897	298	268	268
query86	345	299	312	299
query87	3326	3077	3087	3077
query88	4265	2462	2484	2462
query89	487	375	376	375
query90	1833	187	182	182
query91	125	102	99	99
query92	110	47	48	47
query93	5010	531	514	514
query94	1094	188	187	187
query95	418	316	321	316
query96	596	273	268	268
query97	3154	3061	3022	3022
query98	211	197	191	191
query99	1159	844	818	818
Total cold run time: 281351 ms
Total hot run time: 174356 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.2 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 57e523ebdcc98b9d30aa77ed86f44159366219b7, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.06	0.06
query4	1.67	0.07	0.09
query5	0.49	0.49	0.49
query6	1.14	0.73	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.54	0.49	0.48
query10	0.54	0.54	0.54
query11	0.16	0.11	0.11
query12	0.16	0.12	0.13
query13	0.58	0.58	0.59
query14	0.76	0.77	0.78
query15	0.85	0.81	0.83
query16	0.36	0.37	0.36
query17	0.96	0.95	0.98
query18	0.21	0.25	0.23
query19	1.75	1.66	1.73
query20	0.01	0.01	0.01
query21	15.43	0.78	0.66
query22	4.26	7.77	1.61
query23	18.30	1.38	1.24
query24	2.07	0.23	0.22
query25	0.14	0.10	0.09
query26	0.27	0.17	0.18
query27	0.08	0.08	0.08
query28	13.29	1.02	1.01
query29	12.65	3.28	3.32
query30	0.26	0.06	0.06
query31	2.89	0.40	0.39
query32	3.24	0.47	0.47
query33	2.89	2.91	2.88
query34	17.25	4.42	4.46
query35	4.48	4.47	4.53
query36	0.66	0.46	0.47
query37	0.19	0.16	0.15
query38	0.15	0.15	0.15
query39	0.04	0.03	0.03
query40	0.16	0.14	0.14
query41	0.09	0.05	0.06
query42	0.07	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.51 s
Total hot run time: 30.2 s

HappenLee
HappenLee approved these changes Jul 5, 2024
View reviewed changes
Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 5, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 5, 2024

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 5, 2024

PR approved by anyone and no changes requested.

wangbo
wangbo approved these changes Jul 5, 2024
View reviewed changes
Copy link
Contributor

@wangbo wangbo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@xinyiZzz xinyiZzz merged commit ba410b5 into apache:master Jul 5, 2024
25 of 29 checks passed
xinyiZzz added a commit to xinyiZzz/incubator-doris that referenced this pull request Jul 11, 2024
@xinyiZzz
[opt](memory) Purge all unused Jemalloc dirty pages for all arenas wh…
2fb7f15 
…en exceed limit (apache#37164)

add BE conf je_dirty_pages_mem_limit_percent, default 5% * mem_limit,
purge all unused Jemalloc dirty pages for all arenas when exceed
je_dirty_pages_mem_limit and process exceed soft limit.
@xinyiZzz xinyiZzz mentioned this pull request Jul 11, 2024
Merged
xinyiZzz added a commit to xinyiZzz/incubator-doris that referenced this pull request Jul 12, 2024
@xinyiZzz
[opt](memory) Purge all unused Jemalloc dirty pages for all arenas wh…
8ae1d9f 
…en exceed limit (apache#37164)

add BE conf je_dirty_pages_mem_limit_percent, default 5% * mem_limit,
purge all unused Jemalloc dirty pages for all arenas when exceed
je_dirty_pages_mem_limit and process exceed soft limit.
@xinyiZzz xinyiZzz mentioned this pull request Jul 12, 2024
Merged
xinyiZzz added a commit that referenced this pull request Jul 14, 2024
@xinyiZzz
[branch-2.1](memory) Pick some memory GC patch (#37725)
7471722 
pick
#36768
#37164
#37174
#37525
dataroaring pushed a commit that referenced this pull request Jul 17, 2024
@xinyiZzz @dataroaring
[opt](memory) Purge all unused Jemalloc dirty pages for all arenas wh…
3f78fd8 
…en exceed limit (#37164)

add BE conf je_dirty_pages_mem_limit_percent, default 5% * mem_limit,
purge all unused Jemalloc dirty pages for all arenas when exceed
je_dirty_pages_mem_limit and process exceed soft limit.
@gavinchou gavinchou mentioned this pull request Aug 19, 2024
Open
@yiguolei yiguolei mentioned this pull request Sep 5, 2024
Open
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wangbo wangbo wangbo approved these changes

@HappenLee HappenLee HappenLee approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.5-merged dev/3.0.1-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@xinyiZzz @doris-robot @HappenLee @wangbo @dataroaring