Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[bugfix](paimon)adding dependencies for clang #37512

Merged
merged 1 commit into from
Jul 9, 2024
Merged

[bugfix](paimon)adding dependencies for clang #37512

merged 1 commit into from
Jul 9, 2024

Conversation

wuwenchi
Copy link
Contributor

@wuwenchi wuwenchi commented Jul 9, 2024

Proposed changes

When paimon uses the hms type catalog and reads data in jni mode, it needs to use the org.apache.commons.lang.StringUtils class.
(This problem is not tested in the pipeline environment because the pipeline environment automatically generates the java-udf-case-jar-with-dependencies.jar for testing, which contains the lang package.)

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@wuwenchi
Copy link
Contributor Author

wuwenchi commented Jul 9, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39900 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2df5cdc7220cc91cb12ee1b91180770dd0cfaedb, data reload: false

------ Round 1 ----------------------------------
q1	17616	4291	4249	4249
q2	2024	210	201	201
q3	10424	1214	1231	1214
q4	10183	765	798	765
q5	7504	2695	2621	2621
q6	226	138	137	137
q7	966	603	615	603
q8	9246	2083	2064	2064
q9	9052	6549	6548	6548
q10	8948	3708	3675	3675
q11	457	246	244	244
q12	523	237	225	225
q13	17762	2991	2998	2991
q14	265	223	221	221
q15	527	478	494	478
q16	504	372	384	372
q17	961	645	708	645
q18	7953	7374	7490	7374
q19	3772	1550	1373	1373
q20	688	345	349	345
q21	4980	3221	3901	3221
q22	394	338	334	334
Total cold run time: 114975 ms
Total hot run time: 39900 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4362	4248	4237	4237
q2	368	284	259	259
q3	2947	2772	2863	2772
q4	1984	1685	1775	1685
q5	5638	5567	5514	5514
q6	221	137	130	130
q7	2163	1837	1900	1837
q8	3272	3426	3456	3426
q9	8677	8668	8759	8668
q10	4062	3908	3870	3870
q11	589	498	490	490
q12	801	631	644	631
q13	16957	3172	3123	3123
q14	320	280	287	280
q15	551	485	484	484
q16	498	430	443	430
q17	1808	1520	1512	1512
q18	8061	8028	7836	7836
q19	3126	1641	1593	1593
q20	2200	1938	1853	1853
q21	5149	4958	4957	4957
q22	647	540	584	540
Total cold run time: 74401 ms
Total hot run time: 56127 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173799 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2df5cdc7220cc91cb12ee1b91180770dd0cfaedb, data reload: false

query1	915	370	369	369
query2	6469	2422	2339	2339
query3	6636	207	230	207
query4	28518	17614	17340	17340
query5	3639	481	477	477
query6	277	182	178	178
query7	4589	289	290	289
query8	339	304	297	297
query9	8544	2417	2386	2386
query10	592	315	310	310
query11	11578	10111	10064	10064
query12	117	83	86	83
query13	1656	375	394	375
query14	10024	6812	7610	6812
query15	237	244	210	210
query16	7725	315	312	312
query17	1720	535	513	513
query18	1856	274	271	271
query19	191	147	148	147
query20	90	82	87	82
query21	233	149	140	140
query22	4173	4025	3995	3995
query23	34212	33888	33912	33888
query24	10967	2933	2884	2884
query25	608	391	416	391
query26	1030	149	143	143
query27	2291	270	275	270
query28	6855	2094	2094	2094
query29	894	633	622	622
query30	257	153	160	153
query31	957	770	757	757
query32	86	53	54	53
query33	754	308	291	291
query34	1039	481	500	481
query35	687	589	553	553
query36	1177	958	992	958
query37	155	90	82	82
query38	2940	2825	2807	2807
query39	899	855	862	855
query40	204	133	125	125
query41	57	59	57	57
query42	110	104	102	102
query43	587	562	556	556
query44	1247	716	716	716
query45	195	159	159	159
query46	1089	713	699	699
query47	1832	1772	1784	1772
query48	377	289	287	287
query49	854	406	409	406
query50	777	394	381	381
query51	6812	6847	6879	6847
query52	104	94	92	92
query53	355	286	286	286
query54	882	441	434	434
query55	77	77	72	72
query56	284	267	270	267
query57	1129	1087	1050	1050
query58	244	251	258	251
query59	3397	3277	3336	3277
query60	295	277	276	276
query61	96	93	129	93
query62	803	657	643	643
query63	321	290	289	289
query64	9477	2158	1675	1675
query65	3110	3111	3098	3098
query66	749	329	322	322
query67	15471	15022	14927	14927
query68	4800	524	521	521
query69	630	442	349	349
query70	1178	1132	1155	1132
query71	444	279	281	279
query72	7498	5344	5312	5312
query73	760	319	312	312
query74	6086	5447	5481	5447
query75	3553	2681	2665	2665
query76	3123	948	958	948
query77	632	301	287	287
query78	9416	8978	8922	8922
query79	2436	516	514	514
query80	1533	473	484	473
query81	587	216	218	216
query82	1356	135	133	133
query83	262	177	171	171
query84	241	87	92	87
query85	1400	313	299	299
query86	472	294	309	294
query87	3282	3144	3068	3068
query88	3618	2320	2324	2320
query89	471	385	373	373
query90	1735	189	187	187
query91	135	100	109	100
query92	59	48	48	48
query93	2302	497	499	497
query94	1099	204	203	203
query95	404	391	304	304
query96	586	270	260	260
query97	3220	3000	3054	3000
query98	233	198	193	193
query99	1568	1270	1248	1248
Total cold run time: 281980 ms
Total hot run time: 173799 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.87 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2df5cdc7220cc91cb12ee1b91180770dd0cfaedb, data reload: false

query1	0.05	0.03	0.03
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.66	0.10	0.09
query5	0.51	0.49	0.51
query6	1.12	0.72	0.73
query7	0.01	0.01	0.01
query8	0.04	0.04	0.04
query9	0.56	0.50	0.49
query10	0.55	0.56	0.55
query11	0.14	0.11	0.12
query12	0.15	0.11	0.12
query13	0.60	0.58	0.58
query14	0.76	0.79	0.77
query15	0.86	0.81	0.82
query16	0.38	0.38	0.37
query17	1.01	0.99	1.01
query18	0.22	0.21	0.22
query19	1.85	1.73	1.74
query20	0.01	0.00	0.01
query21	15.39	0.80	0.67
query22	4.28	6.35	2.21
query23	18.28	1.33	1.24
query24	2.08	0.24	0.22
query25	0.16	0.09	0.09
query26	0.29	0.22	0.20
query27	0.45	0.23	0.22
query28	13.22	1.02	0.99
query29	12.62	3.32	3.25
query30	0.25	0.06	0.07
query31	2.84	0.40	0.38
query32	3.27	0.49	0.47
query33	2.87	2.90	2.94
query34	17.04	4.35	4.34
query35	4.42	4.38	4.40
query36	0.65	0.46	0.47
query37	0.18	0.15	0.16
query38	0.15	0.14	0.15
query39	0.04	0.04	0.04
query40	0.16	0.13	0.14
query41	0.10	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.62 s
Total hot run time: 30.87 s

morningman
morningman approved these changes Jul 9, 2024
View reviewed changes
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 9, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 9, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 9, 2024

PR approved by anyone and no changes requested.

@morningman morningman merged commit d325b43 into apache:master Jul 9, 2024
27 of 30 checks passed
dataroaring pushed a commit that referenced this pull request Jul 9, 2024
@wuwenchi @dataroaring
[bugfix](paimon)adding dependencies for clang (#37512)
fdd71ae 
## Proposed changes

When paimon uses the `hms` type catalog and reads data in jni mode, it
needs to use the `org.apache.commons.lang.StringUtils` class.
(This problem is not tested in the pipeline environment because the
pipeline environment automatically generates the
`java-udf-case-jar-with-dependencies.jar` for testing, which contains
the `lang` package.)
@morningman morningman mentioned this pull request Jul 12, 2024
Merged
morningman pushed a commit to morningman/doris that referenced this pull request Jul 12, 2024
@wuwenchi @morningman
[bugfix](paimon)adding dependencies for clang (apache#37512)
ec14d16 
## Proposed changes

When paimon uses the `hms` type catalog and reads data in jni mode, it
needs to use the `org.apache.commons.lang.StringUtils` class.
(This problem is not tested in the pipeline environment because the
pipeline environment automatically generates the
`java-udf-case-jar-with-dependencies.jar` for testing, which contains
the `lang` package.)
morningman added a commit that referenced this pull request Jul 13, 2024
@morningman @wuwenchi
[bugfix](paimon)adding dependencies for clang #37512 (#37737)
d91376c 
cherry pick from #37512

Co-authored-by: wuwenchi <wuwenchihdu@hotmail.com>
@yiguolei yiguolei mentioned this pull request Jul 19, 2024
Open
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@CalvinKirs CalvinKirs CalvinKirs approved these changes

@morningman morningman morningman approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.5-merged dev/3.0.0-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@wuwenchi @doris-robot @morningman @CalvinKirs @dataroaring