Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Fix](function) Fix unix_timestamp core for string input #32871

Merged
merged 2 commits into from
Mar 27, 2024
Merged

[Fix](function) Fix unix_timestamp core for string input #32871

merged 2 commits into from
Mar 27, 2024

Conversation

zclllyybb
Copy link
Contributor

@zclllyybb zclllyybb commented Mar 26, 2024 •
edited
Loading

Proposed changes

Issue Number: close #xxx

before when we got unix_timestamp(string_col, literal), BE will core. now fixed.

no need to pick to branch 2.0

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zclllyybb
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.26% (8739/24782)
Line Coverage: 27.04% (71537/264578)
Region Coverage: 26.28% (37117/141247)
Branch Coverage: 23.17% (18977/81896)
Coverage Report: http://coverage.selectdb-in.cc/coverage/d32e11787b988b7d8dcc43f81336334420b6f96d_d32e11787b988b7d8dcc43f81336334420b6f96d/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 37761 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d32e11787b988b7d8dcc43f81336334420b6f96d, data reload: false

------ Round 1 ----------------------------------
q1	17626	4166	4082	4082
q2	2101	166	149	149
q3	10577	1140	1173	1140
q4	10228	778	781	778
q5	7417	3004	2952	2952
q6	202	122	121	121
q7	1021	585	562	562
q8	9318	1971	1995	1971
q9	7142	6599	6571	6571
q10	8437	3450	3532	3450
q11	445	222	230	222
q12	386	199	195	195
q13	17793	2856	2849	2849
q14	228	193	206	193
q15	529	453	465	453
q16	510	375	369	369
q17	949	542	592	542
q18	7084	6426	6334	6334
q19	1550	1471	1412	1412
q20	549	251	242	242
q21	3678	2918	2877	2877
q22	341	297	303	297
Total cold run time: 108111 ms
Total hot run time: 37761 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4137	4076	4086	4076
q2	325	229	229	229
q3	2968	2808	2855	2808
q4	1825	1537	1593	1537
q5	5334	5346	5388	5346
q6	196	115	113	113
q7	2255	1868	1837	1837
q8	3148	3304	3280	3280
q9	8772	8787	8782	8782
q10	3811	3817	3791	3791
q11	546	447	437	437
q12	735	555	595	555
q13	16921	2879	3040	2879
q14	314	265	297	265
q15	498	471	459	459
q16	491	435	433	433
q17	1754	1513	1492	1492
q18	7952	7652	7547	7547
q19	3793	1519	1498	1498
q20	1926	1752	1717	1717
q21	4998	4804	4598	4598
q22	522	464	440	440
Total cold run time: 73221 ms
Total hot run time: 54119 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 181845 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d32e11787b988b7d8dcc43f81336334420b6f96d, data reload: false

query1	936	363	363	363
query2	6539	1981	1855	1855
query3	6713	236	216	216
query4	31476	21324	21271	21271
query5	4307	392	412	392
query6	266	180	173	173
query7	4634	303	292	292
query8	224	181	172	172
query9	9253	2289	2305	2289
query10	567	246	252	246
query11	17197	14242	14370	14242
query12	142	91	90	90
query13	1650	421	441	421
query14	9204	7744	7717	7717
query15	256	190	200	190
query16	8056	267	271	267
query17	1937	592	552	552
query18	1919	301	282	282
query19	281	154	160	154
query20	95	94	90	90
query21	202	125	146	125
query22	5052	4873	4872	4872
query23	33564	32707	32803	32707
query24	10368	2872	2811	2811
query25	574	388	388	388
query26	719	162	158	158
query27	2234	352	356	352
query28	5903	1916	1889	1889
query29	883	649	619	619
query30	300	151	149	149
query31	987	710	735	710
query32	94	58	58	58
query33	653	260	249	249
query34	858	489	479	479
query35	820	622	586	586
query36	1022	874	898	874
query37	96	68	65	65
query38	3508	3508	3418	3418
query39	1482	1441	1422	1422
query40	210	121	111	111
query41	53	48	51	48
query42	108	100	95	95
query43	488	446	439	439
query44	1078	734	753	734
query45	280	289	263	263
query46	1113	709	703	703
query47	1940	1858	1875	1858
query48	462	373	360	360
query49	1100	353	339	339
query50	775	370	371	370
query51	6725	6685	6721	6685
query52	107	93	92	92
query53	349	276	278	276
query54	308	244	237	237
query55	87	86	82	82
query56	257	218	223	218
query57	1195	1145	1116	1116
query58	228	201	207	201
query59	2794	2535	2534	2534
query60	263	238	246	238
query61	97	99	93	93
query62	665	455	459	455
query63	305	272	272	272
query64	5466	4051	3997	3997
query65	3083	3046	3023	3023
query66	859	361	350	350
query67	15233	15090	14741	14741
query68	5691	542	550	542
query69	592	369	376	369
query70	1247	1154	1176	1154
query71	433	267	254	254
query72	6655	2753	2563	2563
query73	711	334	339	334
query74	7861	6437	6405	6405
query75	3040	2201	2224	2201
query76	4250	863	883	863
query77	615	251	259	251
query78	10967	10216	10268	10216
query79	7806	534	526	526
query80	2026	376	389	376
query81	562	224	218	218
query82	1622	84	88	84
query83	341	153	157	153
query84	295	84	82	82
query85	1526	343	314	314
query86	495	291	302	291
query87	3701	3552	3556	3552
query88	4942	2385	2369	2369
query89	474	380	374	374
query90	1942	174	175	174
query91	172	136	142	136
query92	62	49	47	47
query93	5845	509	496	496
query94	1208	174	173	173
query95	431	330	323	323
query96	602	280	271	271
query97	2671	2463	2528	2463
query98	234	215	223	215
query99	1185	924	921	921
Total cold run time: 300803 ms
Total hot run time: 181845 ms

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit d32e11787b988b7d8dcc43f81336334420b6f96d with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       14.5 seconds inserted 10000000 Rows, about 689K ops/s

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

yiguolei
yiguolei approved these changes Mar 27, 2024
View reviewed changes
Copy link
Contributor

@yiguolei yiguolei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yiguolei
Copy link
Contributor

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 27, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@wm1581066 wm1581066 requested a review from HappenLee March 27, 2024 01:33
@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.25% (8737/24787)
Line Coverage: 27.03% (71516/264627)
Region Coverage: 26.27% (37109/141264)
Branch Coverage: 23.17% (18975/81906)
Coverage Report: http://coverage.selectdb-in.cc/coverage/054a037728d58991db0d0de4ac5e814f504d9429_054a037728d58991db0d0de4ac5e814f504d9429/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 37899 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 054a037728d58991db0d0de4ac5e814f504d9429, data reload: false

------ Round 1 ----------------------------------
q1	17606	4447	4112	4112
q2	2115	158	151	151
q3	10594	1162	1173	1162
q4	10437	755	803	755
q5	7469	3029	2983	2983
q6	203	127	122	122
q7	1012	594	573	573
q8	9343	1991	1968	1968
q9	7131	6620	6561	6561
q10	8477	3490	3582	3490
q11	428	228	223	223
q12	415	198	192	192
q13	17800	2861	2845	2845
q14	250	213	217	213
q15	510	463	482	463
q16	505	379	369	369
q17	953	615	575	575
q18	7151	6483	6390	6390
q19	1536	1414	1455	1414
q20	546	254	252	252
q21	3624	2877	2793	2793
q22	352	293	294	293
Total cold run time: 108457 ms
Total hot run time: 37899 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4152	4112	4102	4102
q2	326	237	241	237
q3	2993	2841	2897	2841
q4	1841	1582	1555	1555
q5	5340	5346	5393	5346
q6	192	115	117	115
q7	2249	1908	1894	1894
q8	3157	3312	3266	3266
q9	8694	8683	8710	8683
q10	3735	3802	3752	3752
q11	542	457	450	450
q12	710	545	539	539
q13	16933	2892	2880	2880
q14	276	251	257	251
q15	499	472	457	457
q16	486	416	429	416
q17	1722	1488	1510	1488
q18	7486	7196	7081	7081
q19	1581	1432	1572	1432
q20	1894	1748	1728	1728
q21	4875	4883	4672	4672
q22	524	461	453	453
Total cold run time: 70207 ms
Total hot run time: 53638 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 182272 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 054a037728d58991db0d0de4ac5e814f504d9429, data reload: false

query1	930	367	357	357
query2	6549	2033	1993	1993
query3	6708	226	219	219
query4	31868	21307	21434	21307
query5	4351	407	399	399
query6	267	177	174	174
query7	4624	297	289	289
query8	226	178	170	170
query9	9035	2280	2274	2274
query10	569	263	267	263
query11	14782	14218	14220	14218
query12	141	93	88	88
query13	1626	447	429	429
query14	11422	7641	7679	7641
query15	267	215	198	198
query16	7968	267	266	266
query17	1934	582	542	542
query18	1412	291	282	282
query19	258	154	154	154
query20	102	85	85	85
query21	205	135	127	127
query22	5019	4784	4792	4784
query23	33401	32963	32639	32639
query24	10779	2918	2922	2918
query25	597	368	368	368
query26	1175	156	152	152
query27	2789	351	363	351
query28	7494	1875	1865	1865
query29	908	632	604	604
query30	294	148	148	148
query31	993	733	731	731
query32	95	56	54	54
query33	760	253	244	244
query34	1093	489	484	484
query35	844	620	623	620
query36	1074	884	905	884
query37	124	64	64	64
query38	3544	3497	3445	3445
query39	1495	1463	1428	1428
query40	216	114	112	112
query41	46	44	46	44
query42	108	95	98	95
query43	489	463	440	440
query44	1187	733	752	733
query45	280	261	266	261
query46	1222	722	751	722
query47	1946	1824	1816	1816
query48	452	367	354	354
query49	1134	352	344	344
query50	838	379	374	374
query51	6719	6661	6531	6531
query52	110	96	99	96
query53	393	280	282	280
query54	302	243	247	243
query55	82	83	78	78
query56	251	228	215	215
query57	1244	1130	1139	1130
query58	227	207	205	205
query59	2747	2554	2611	2554
query60	275	239	242	239
query61	94	96	108	96
query62	662	445	473	445
query63	313	278	273	273
query64	5510	4293	4323	4293
query65	3659	3073	3040	3040
query66	1313	371	378	371
query67	15249	14761	14735	14735
query68	5182	533	544	533
query69	557	389	386	386
query70	1274	1157	1197	1157
query71	418	274	271	271
query72	6446	2889	2714	2714
query73	773	323	331	323
query74	6919	6310	6347	6310
query75	3072	2253	2207	2207
query76	3553	981	979	979
query77	594	280	265	265
query78	10847	10109	10123	10109
query79	8254	541	537	537
query80	1421	402	422	402
query81	520	225	221	221
query82	878	90	90	90
query83	228	153	158	153
query84	290	82	82	82
query85	1509	377	372	372
query86	453	307	320	307
query87	3755	3539	3545	3539
query88	5242	2390	2384	2384
query89	512	381	375	375
query90	1982	184	177	177
query91	173	146	137	137
query92	58	49	48	48
query93	6131	511	492	492
query94	1169	178	177	177
query95	438	335	332	332
query96	680	270	271	270
query97	2685	2487	2463	2463
query98	236	217	204	204
query99	1269	895	879	879
Total cold run time: 302308 ms
Total hot run time: 182272 ms

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 054a037728d58991db0d0de4ac5e814f504d9429 with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       13.8 seconds inserted 10000000 Rows, about 724K ops/s

HappenLee
HappenLee approved these changes Mar 27, 2024
View reviewed changes
Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yiguolei yiguolei merged commit 6c24a0a into apache:master Mar 27, 2024
26 of 32 checks passed
@zclllyybb zclllyybb deleted the fix_unixtimestamp branch March 27, 2024 03:23
yiguolei pushed a commit that referenced this pull request Apr 1, 2024
@zclllyybb @Doris-Extras
[Fix](function) Fix unix_timestamp core for string input (#32871)
7e9a146
yiguolei pushed a commit that referenced this pull request Apr 9, 2024
@zclllyybb @Doris-Extras
[Fix](function) Fix unix_timestamp core for string input (#32871)
2a0644f
@yiguolei yiguolei mentioned this pull request Apr 9, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@HappenLee HappenLee HappenLee approved these changes

@yiguolei yiguolei yiguolei approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.2-merged p0_c reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@zclllyybb @doris-robot @yiguolei @HappenLee @xiaokang @zhangstar333