Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[feature](cloud) Support cache warmup for any remote filesystem #34191

Merged
merged 1 commit into from
Apr 30, 2024
Merged

[feature](cloud) Support cache warmup for any remote filesystem #34191

merged 1 commit into from
Apr 30, 2024

Conversation

platoneko
Copy link
Contributor

Proposed changes

Support cache warmup for any remote filesystem

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@platoneko
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@platoneko
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-DS: Total hot run time: 183204 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b90f6f48cfdbdd4ccf40e9ce32927e2388f5c467, data reload: false

query1	902	364	349	349
query2	6255	2436	2310	2310
query3	6652	213	210	210
query4	23259	22075	21948	21948
query5	3859	428	443	428
query6	272	187	182	182
query7	4795	301	298	298
query8	247	198	188	188
query9	8421	2441	2417	2417
query10	421	251	245	245
query11	12056	11128	11630	11128
query12	122	88	85	85
query13	1726	371	354	354
query14	10667	6757	7373	6757
query15	227	173	171	171
query16	8052	258	266	258
query17	1701	552	545	545
query18	2043	273	269	269
query19	202	146	140	140
query20	87	84	82	82
query21	198	122	123	122
query22	5040	4864	4827	4827
query23	33639	33166	33371	33166
query24	11415	2923	2938	2923
query25	659	388	384	384
query26	1662	157	151	151
query27	2994	317	322	317
query28	7681	2049	2033	2033
query29	1002	618	616	616
query30	255	153	150	150
query31	960	718	727	718
query32	95	53	53	53
query33	745	250	252	250
query34	1060	464	475	464
query35	832	668	668	668
query36	1099	903	890	890
query37	138	66	72	66
query38	3149	3002	3063	3002
query39	1569	1537	1534	1534
query40	244	126	132	126
query41	45	40	48	40
query42	104	96	95	95
query43	562	573	543	543
query44	1224	722	736	722
query45	289	265	264	264
query46	1086	749	754	749
query47	1947	1865	1831	1831
query48	370	307	291	291
query49	972	413	404	404
query50	763	384	393	384
query51	6709	6695	6601	6601
query52	102	88	94	88
query53	351	278	281	278
query54	327	243	234	234
query55	76	72	72	72
query56	311	223	223	223
query57	1195	1148	1101	1101
query58	222	197	195	195
query59	3660	3137	3194	3137
query60	266	238	234	234
query61	90	90	101	90
query62	634	442	450	442
query63	307	272	279	272
query64	9387	7169	7170	7169
query65	3109	3042	3031	3031
query66	1363	337	331	331
query67	16148	14941	15144	14941
query68	9148	568	561	561
query69	544	314	313	313
query70	1244	1118	1064	1064
query71	508	268	266	266
query72	7874	2639	2435	2435
query73	970	327	327	327
query74	6744	6057	6196	6057
query75	4589	2677	2663	2663
query76	5419	1075	968	968
query77	694	266	261	261
query78	11109	10274	10221	10221
query79	12955	519	528	519
query80	2197	485	430	430
query81	497	219	224	219
query82	708	93	89	89
query83	211	164	163	163
query84	266	84	85	84
query85	1472	268	264	264
query86	406	303	291	291
query87	3312	3077	3077	3077
query88	5579	2426	2427	2426
query89	535	369	367	367
query90	1995	185	185	185
query91	128	97	97	97
query92	63	45	46	45
query93	7574	525	508	508
query94	1209	182	177	177
query95	399	295	307	295
query96	600	274	264	264
query97	3157	2960	2926	2926
query98	234	210	209	209
query99	1256	847	869	847
Total cold run time: 311617 ms
Total hot run time: 183204 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.53% (8925/25117)
Line Coverage: 27.16% (73459/270448)
Region Coverage: 26.34% (37947/144063)
Branch Coverage: 23.13% (19328/83576)
Coverage Report: http://coverage.selectdb-in.cc/coverage/b90f6f48cfdbdd4ccf40e9ce32927e2388f5c467_b90f6f48cfdbdd4ccf40e9ce32927e2388f5c467/report/index.html

dataroaring
dataroaring approved these changes Apr 30, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 30, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@dataroaring dataroaring merged commit 66bf4d9 into apache:master Apr 30, 2024
24 of 28 checks passed
ByteYue pushed a commit to ByteYue/doris that referenced this pull request May 15, 2024
@platoneko @ByteYue
[feature](cloud) Support cache warmup for any remote filesystem (apac…
35b2847 
…he#34191)
dataroaring added a commit to dataroaring/incubator-doris that referenced this pull request Aug 6, 2024
@dataroaring
[fix](cloud-balance) use right file size instead of block size
4b7e5ce 
introduced by apache#34191.
@dataroaring dataroaring mentioned this pull request Aug 6, 2024
Merged
gavinchou pushed a commit that referenced this pull request Aug 7, 2024
@dataroaring
[fix](cloud-balance) use right file size instead of block size (#38939)
d33f580 
introduced by #34191
dataroaring added a commit that referenced this pull request Aug 7, 2024
@dataroaring
[fix](cloud-balance) use right file size instead of block size (#38939)
51ca8ac 
introduced by #34191
wyxxxcat pushed a commit to wyxxxcat/doris that referenced this pull request Aug 14, 2024
@dataroaring @wyxxxcat
[fix](cloud-balance) use right file size instead of block size (apach…
459170f 
…e#38939)

introduced by apache#34191
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dataroaring dataroaring dataroaring approved these changes

@gavinchou gavinchou gavinchou approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@platoneko @doris-robot @gavinchou @dataroaring