Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](agg_state) adjust nullable should apply on agg_state inner type too #37489

Merged
merged 1 commit into from
Jul 10, 2024
Merged

[fix](agg_state) adjust nullable should apply on agg_state inner type too #37489

merged 1 commit into from
Jul 10, 2024

Conversation

morrySnow
Copy link
Contributor

after adjust nullable, some children nullable has changed. so, we need to update agg_state type inner type nullable too.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@morrySnow
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39728 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c1ece093e1e9d4049ad3a68610b4fe7dbb4206c6, data reload: false

------ Round 1 ----------------------------------
q1	17597	4342	4263	4263
q2	2008	198	198	198
q3	10429	1224	1086	1086
q4	10191	862	737	737
q5	7509	2734	2702	2702
q6	219	132	132	132
q7	933	597	609	597
q8	9220	2078	2038	2038
q9	8925	6494	6483	6483
q10	9008	3702	3722	3702
q11	465	237	232	232
q12	443	223	229	223
q13	17770	2981	2990	2981
q14	255	227	219	219
q15	524	495	491	491
q16	506	370	374	370
q17	961	632	642	632
q18	7986	7400	7368	7368
q19	5363	1474	1505	1474
q20	689	325	347	325
q21	4857	3135	3819	3135
q22	395	345	340	340
Total cold run time: 116253 ms
Total hot run time: 39728 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4325	4234	4228	4228
q2	387	264	280	264
q3	2992	2769	2890	2769
q4	1966	1718	1690	1690
q5	5602	5511	5441	5441
q6	222	133	139	133
q7	2207	1826	1863	1826
q8	3295	3398	3403	3398
q9	8663	8634	8803	8634
q10	4039	3863	3682	3682
q11	590	472	473	472
q12	810	680	624	624
q13	16361	3161	3152	3152
q14	316	284	283	283
q15	547	479	488	479
q16	480	422	444	422
q17	1803	1517	1499	1499
q18	8188	8064	7888	7888
q19	2099	1442	1640	1442
q20	2185	1847	1865	1847
q21	5013	4903	4878	4878
q22	817	568	528	528
Total cold run time: 72907 ms
Total hot run time: 55579 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174497 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c1ece093e1e9d4049ad3a68610b4fe7dbb4206c6, data reload: false

query1	916	372	365	365
query2	6466	2373	2347	2347
query3	6637	203	213	203
query4	27749	17399	17279	17279
query5	3876	481	503	481
query6	258	171	168	168
query7	4601	282	272	272
query8	319	306	294	294
query9	8576	2405	2392	2392
query10	594	303	283	283
query11	11649	10050	10076	10050
query12	119	84	82	82
query13	1651	375	367	367
query14	9708	7540	7670	7540
query15	240	185	191	185
query16	7627	306	300	300
query17	1538	559	525	525
query18	1917	275	266	266
query19	209	148	158	148
query20	88	82	82	82
query21	204	132	129	129
query22	4302	4251	4140	4140
query23	34067	33843	33674	33674
query24	11277	2886	2834	2834
query25	591	366	393	366
query26	699	151	146	146
query27	2250	271	264	264
query28	5901	2112	2105	2105
query29	894	636	630	630
query30	250	165	153	153
query31	963	784	797	784
query32	100	79	56	56
query33	762	314	320	314
query34	948	478	498	478
query35	684	593	610	593
query36	1141	982	972	972
query37	142	83	84	83
query38	3013	2865	2797	2797
query39	887	846	831	831
query40	204	120	118	118
query41	52	50	56	50
query42	113	99	99	99
query43	592	530	533	530
query44	1223	730	722	722
query45	188	160	159	159
query46	1087	721	704	704
query47	1860	1813	1771	1771
query48	365	286	316	286
query49	823	405	406	405
query50	761	386	376	376
query51	6959	6814	6808	6808
query52	99	94	90	90
query53	354	282	279	279
query54	884	449	438	438
query55	71	72	71	71
query56	283	251	259	251
query57	1160	1052	1045	1045
query58	258	232	242	232
query59	3456	3072	3359	3072
query60	307	269	270	269
query61	117	101	95	95
query62	827	653	649	649
query63	319	284	287	284
query64	9147	2143	1634	1634
query65	3146	3112	3402	3112
query66	745	321	329	321
query67	15336	15191	15018	15018
query68	4490	517	527	517
query69	506	366	345	345
query70	1132	1133	1147	1133
query71	410	278	275	275
query72	6988	5745	5597	5597
query73	742	321	314	314
query74	5914	5532	5534	5532
query75	3393	2664	2656	2656
query76	2268	956	936	936
query77	449	295	296	295
query78	9447	9022	9896	9022
query79	1961	510	521	510
query80	2580	463	532	463
query81	585	215	216	215
query82	563	132	132	132
query83	291	163	163	163
query84	259	87	85	85
query85	1758	323	291	291
query86	476	325	318	318
query87	3247	3096	3083	3083
query88	3565	2334	2329	2329
query89	468	369	375	369
query90	1746	193	184	184
query91	130	101	101	101
query92	62	51	47	47
query93	1608	493	488	488
query94	1112	206	205	205
query95	414	317	307	307
query96	578	268	264	264
query97	3244	3064	3021	3021
query98	223	206	197	197
query99	1525	1237	1246	1237
Total cold run time: 276683 ms
Total hot run time: 174497 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.26 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c1ece093e1e9d4049ad3a68610b4fe7dbb4206c6, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.04	0.04
query3	0.23	0.06	0.05
query4	1.67	0.10	0.09
query5	0.50	0.47	0.50
query6	1.14	0.72	0.72
query7	0.02	0.02	0.02
query8	0.06	0.04	0.05
query9	0.55	0.50	0.48
query10	0.55	0.55	0.53
query11	0.15	0.11	0.11
query12	0.14	0.12	0.13
query13	0.59	0.59	0.58
query14	0.78	0.77	0.79
query15	0.84	0.81	0.81
query16	0.35	0.39	0.37
query17	1.03	0.98	1.02
query18	0.22	0.21	0.20
query19	1.73	1.70	1.72
query20	0.01	0.01	0.02
query21	15.42	0.76	0.65
query22	3.99	8.00	1.72
query23	18.26	1.36	1.24
query24	2.11	0.23	0.22
query25	0.15	0.09	0.08
query26	0.30	0.21	0.21
query27	0.45	0.23	0.22
query28	13.19	1.02	0.98
query29	12.58	3.22	3.23
query30	0.25	0.06	0.06
query31	2.86	0.40	0.38
query32	3.29	0.48	0.47
query33	2.90	2.91	2.92
query34	16.93	4.34	4.36
query35	4.43	4.41	4.41
query36	0.66	0.47	0.47
query37	0.18	0.15	0.15
query38	0.15	0.14	0.14
query39	0.04	0.03	0.04
query40	0.15	0.13	0.13
query41	0.10	0.05	0.04
query42	0.05	0.05	0.04
query43	0.05	0.04	0.03
Total cold run time: 109.16 s
Total hot run time: 30.26 s

@morrySnow morrySnow force-pushed the fix_agg_state_nullable branch 2 times, most recently from 1a3b237 to e0b4835 Compare July 9, 2024 09:35
@morrySnow
Copy link
Contributor Author

run buildall

@morrySnow
[fix](agg_state) adjust nullable should apply on agg_state inner type…
d6e1e7b 
… too

after adjust nullable, some children nullable has changed.
so, we need to update agg_state type inner type nullable too.
@morrySnow morrySnow force-pushed the fix_agg_state_nullable branch from e0b4835 to d6e1e7b Compare July 10, 2024 05:56
@morrySnow
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40014 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d6e1e7ba8dd35a0bdbadb10d2463f7177b8335f0, data reload: false

------ Round 1 ----------------------------------
q1	18272	4419	4359	4359
q2	2018	191	187	187
q3	10477	1186	1105	1105
q4	10197	808	852	808
q5	7485	2668	2651	2651
q6	218	138	137	137
q7	969	598	612	598
q8	9316	2094	2072	2072
q9	8795	6510	6516	6510
q10	8845	3743	3725	3725
q11	447	237	236	236
q12	408	227	234	227
q13	17912	2974	2958	2958
q14	271	233	242	233
q15	533	485	494	485
q16	483	389	373	373
q17	967	694	681	681
q18	8079	7558	7435	7435
q19	1688	1380	1474	1380
q20	673	338	329	329
q21	4958	3177	3944	3177
q22	409	351	348	348
Total cold run time: 113420 ms
Total hot run time: 40014 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4367	4255	4285	4255
q2	377	262	268	262
q3	2965	2741	2731	2731
q4	1899	1590	1612	1590
q5	5262	5251	5295	5251
q6	223	131	131	131
q7	2139	1732	1710	1710
q8	3201	3391	3321	3321
q9	8343	8345	8336	8336
q10	3884	3672	3613	3613
q11	567	490	515	490
q12	795	610	604	604
q13	16283	2970	2986	2970
q14	282	257	256	256
q15	513	482	472	472
q16	462	407	416	407
q17	1763	1485	1457	1457
q18	7870	7645	7292	7292
q19	1703	1461	1513	1461
q20	1982	1782	1778	1778
q21	4801	4851	4611	4611
q22	624	569	559	559
Total cold run time: 70305 ms
Total hot run time: 53557 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173751 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d6e1e7ba8dd35a0bdbadb10d2463f7177b8335f0, data reload: false

query1	911	375	361	361
query2	6461	2358	2463	2358
query3	6672	206	213	206
query4	26053	17123	17343	17123
query5	4197	488	479	479
query6	281	162	166	162
query7	4591	289	287	287
query8	312	287	294	287
query9	8703	2399	2370	2370
query10	458	279	263	263
query11	11448	10119	10025	10025
query12	138	86	81	81
query13	1643	383	380	380
query14	10101	7639	7690	7639
query15	262	176	182	176
query16	7987	353	318	318
query17	1773	572	546	546
query18	1997	282	286	282
query19	207	156	159	156
query20	95	84	81	81
query21	205	124	126	124
query22	4184	4045	3993	3993
query23	33873	33236	33168	33168
query24	12058	2889	2825	2825
query25	678	393	394	393
query26	1768	154	155	154
query27	2911	273	276	273
query28	7420	2055	2063	2055
query29	1063	639	643	639
query30	274	151	147	147
query31	969	753	747	747
query32	96	54	62	54
query33	782	308	307	307
query34	961	488	497	488
query35	688	570	594	570
query36	1108	961	937	937
query37	280	85	83	83
query38	2881	2739	2715	2715
query39	875	813	800	800
query40	282	127	121	121
query41	55	53	52	52
query42	120	98	105	98
query43	593	570	544	544
query44	1187	837	755	755
query45	188	160	159	159
query46	1079	707	747	707
query47	1868	1763	1809	1763
query48	367	294	302	294
query49	1208	425	417	417
query50	774	409	416	409
query51	6900	6771	6678	6678
query52	107	91	94	91
query53	364	301	291	291
query54	984	462	452	452
query55	74	74	73	73
query56	290	265	291	265
query57	1145	1062	1055	1055
query58	263	255	249	249
query59	3411	3319	3180	3180
query60	305	286	295	286
query61	102	95	99	95
query62	850	629	654	629
query63	332	291	289	289
query64	10473	2214	1666	1666
query65	3188	3084	3092	3084
query66	1373	359	333	333
query67	15721	14907	14911	14907
query68	8840	565	561	561
query69	704	484	370	370
query70	1357	1127	1135	1127
query71	536	283	282	282
query72	9034	5479	5333	5333
query73	2232	335	327	327
query74	5852	5495	5458	5458
query75	5325	2698	2646	2646
query76	5329	993	992	992
query77	776	303	304	303
query78	9570	9176	13641	9176
query79	5849	518	510	510
query80	799	492	474	474
query81	576	217	224	217
query82	286	133	132	132
query83	348	174	164	164
query84	277	93	85	85
query85	998	307	361	307
query86	353	319	311	311
query87	3277	3077	3122	3077
query88	3595	2464	2452	2452
query89	468	367	379	367
query90	2118	196	198	196
query91	133	103	105	103
query92	67	50	52	50
query93	1037	513	511	511
query94	1339	209	217	209
query95	399	311	311	311
query96	576	280	273	273
query97	3170	2996	3031	2996
query98	211	205	198	198
query99	1474	1259	1239	1239
Total cold run time: 297872 ms
Total hot run time: 173751 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.6 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d6e1e7ba8dd35a0bdbadb10d2463f7177b8335f0, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.03	0.04
query3	0.22	0.04	0.04
query4	1.69	0.08	0.09
query5	0.51	0.47	0.48
query6	1.14	0.72	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.57	0.49	0.49
query10	0.55	0.56	0.54
query11	0.15	0.12	0.11
query12	0.14	0.12	0.12
query13	0.59	0.59	0.58
query14	0.76	0.78	0.77
query15	0.85	0.82	0.81
query16	0.34	0.36	0.36
query17	0.95	0.96	1.00
query18	0.23	0.23	0.22
query19	1.81	1.85	1.70
query20	0.01	0.01	0.01
query21	15.39	0.81	0.66
query22	4.28	7.11	2.04
query23	18.25	1.28	1.24
query24	2.10	0.22	0.22
query25	0.15	0.10	0.09
query26	0.30	0.20	0.21
query27	0.45	0.24	0.22
query28	13.28	1.02	0.99
query29	12.63	3.33	3.29
query30	0.25	0.06	0.06
query31	2.86	0.38	0.40
query32	3.26	0.46	0.47
query33	2.86	2.90	2.94
query34	17.01	4.33	4.33
query35	4.37	4.42	4.44
query36	0.64	0.45	0.49
query37	0.19	0.16	0.15
query38	0.16	0.16	0.15
query39	0.04	0.04	0.03
query40	0.14	0.12	0.12
query41	0.09	0.05	0.04
query42	0.06	0.04	0.05
query43	0.04	0.03	0.04
Total cold run time: 109.5 s
Total hot run time: 30.6 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 10, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@morrySnow morrySnow merged commit 2ec7903 into apache:master Jul 10, 2024
26 of 29 checks passed
@morrySnow morrySnow deleted the fix_agg_state_nullable branch July 10, 2024 09:13
dataroaring pushed a commit that referenced this pull request Jul 17, 2024
@morrySnow @dataroaring
[fix](agg_state) adjust nullable should apply on agg_state inner type…
a1e0b04 
… too (#37489)

after adjust nullable, some children nullable has changed. so, we need
to update agg_state type inner type nullable too.
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request Jul 24, 2024
@morrySnow
[fix](agg_state) adjust nullable should apply on agg_state inner type…
74ea260 
… too (apache#37489)

pick from master apache#37489

after adjust nullable, some children nullable has changed. so, we need
to update agg_state type inner type nullable too.
@morrySnow morrySnow mentioned this pull request Jul 24, 2024
Merged
morrySnow added a commit that referenced this pull request Jul 24, 2024
@morrySnow
[fix](agg_state) adjust nullable should apply on agg_state inner type…
17c3366 
… too (#37489) (#38281)

pick from master #37489

after adjust nullable, some children nullable has changed. so, we need
to update agg_state type inner type nullable too.
@gavinchou gavinchou mentioned this pull request Aug 19, 2024
Open
@yiguolei yiguolei mentioned this pull request Sep 5, 2024
Open
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@starocean999 starocean999 starocean999 approved these changes

@keanji-x keanji-x keanji-x approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.1-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@morrySnow @doris-robot @keanji-x @starocean999 @dataroaring