Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](coldheat) fix missing partition's storage policy in create_table_like stmt #47009

Merged
merged 3 commits into from
Jan 17, 2025
Merged

[fix](coldheat) fix missing partition's storage policy in create_table_like stmt #47009

merged 3 commits into from
Jan 17, 2025
+61 −1

Conversation

Yulei-Yang
Copy link
Contributor

What problem does this PR solve?

storage policy of partition is missing for a table created by create table like stmt

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Yulei-Yang
Copy link
Contributor Author

run buildall

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@doris-robot
Copy link

TPC-H: Total hot run time: 32582 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d8da356adbbea4052ffd37c56d94e07258305a83, data reload: false

------ Round 1 ----------------------------------
q1	17600	6210	6026	6026
q2	2042	322	174	174
q3	10411	1275	718	718
q4	10204	870	440	440
q5	7513	2180	1985	1985
q6	212	182	153	153
q7	882	753	610	610
q8	9246	1354	1165	1165
q9	5287	4863	4906	4863
q10	6729	2308	1862	1862
q11	469	277	263	263
q12	339	354	224	224
q13	17758	3699	3048	3048
q14	229	234	212	212
q15	580	523	521	521
q16	638	619	585	585
q17	555	864	334	334
q18	7299	6494	6347	6347
q19	1208	966	556	556
q20	320	334	211	211
q21	2854	2259	1968	1968
q22	368	336	317	317
Total cold run time: 102743 ms
Total hot run time: 32582 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6257	6236	6270	6236
q2	239	336	237	237
q3	2292	2629	2301	2301
q4	1405	1841	1365	1365
q5	4306	4797	4725	4725
q6	188	181	146	146
q7	2092	1966	1896	1896
q8	2631	2761	2685	2685
q9	7272	7345	7303	7303
q10	3056	3331	2834	2834
q11	596	545	505	505
q12	661	755	656	656
q13	3440	3890	3272	3272
q14	278	297	278	278
q15	576	530	521	521
q16	661	683	633	633
q17	1195	1741	1262	1262
q18	7730	7419	7399	7399
q19	789	1132	1034	1034
q20	2070	2022	1870	1870
q21	5703	5240	4776	4776
q22	614	565	587	565
Total cold run time: 54051 ms
Total hot run time: 52499 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 188944 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d8da356adbbea4052ffd37c56d94e07258305a83, data reload: false

query1	989	388	378	378
query2	6534	2160	2103	2103
query3	6711	209	208	208
query4	33767	23669	23443	23443
query5	4323	626	491	491
query6	276	195	176	176
query7	4606	486	303	303
query8	278	232	232	232
query9	9522	2640	2640	2640
query10	473	320	256	256
query11	17956	15458	15366	15366
query12	162	111	109	109
query13	1662	514	388	388
query14	9146	7425	7306	7306
query15	254	193	192	192
query16	8654	651	459	459
query17	1658	738	586	586
query18	2117	407	303	303
query19	219	186	158	158
query20	119	123	118	118
query21	215	126	103	103
query22	4327	4342	4275	4275
query23	35040	33321	33096	33096
query24	6649	2187	2241	2187
query25	457	448	413	413
query26	1097	255	154	154
query27	2091	456	331	331
query28	5123	2441	2410	2410
query29	541	518	413	413
query30	232	186	156	156
query31	947	852	775	775
query32	93	64	67	64
query33	517	345	308	308
query34	746	854	504	504
query35	783	844	722	722
query36	1009	1050	945	945
query37	117	96	83	83
query38	4117	4119	4054	4054
query39	1477	1401	1405	1401
query40	201	119	100	100
query41	51	51	51	51
query42	124	104	103	103
query43	523	541	515	515
query44	1357	803	798	798
query45	183	167	167	167
query46	849	1056	637	637
query47	1836	1848	1798	1798
query48	384	404	313	313
query49	777	482	411	411
query50	610	646	386	386
query51	6846	6902	6641	6641
query52	105	101	91	91
query53	223	261	179	179
query54	469	498	401	401
query55	83	79	78	78
query56	254	266	246	246
query57	1144	1197	1104	1104
query58	245	232	246	232
query59	2931	3240	3036	3036
query60	268	287	253	253
query61	119	120	115	115
query62	851	800	700	700
query63	231	198	191	191
query64	4333	1037	629	629
query65	3267	3182	3183	3182
query66	1098	429	322	322
query67	15986	15430	15478	15430
query68	5532	822	532	532
query69	473	296	267	267
query70	1205	1159	1087	1087
query71	367	282	265	265
query72	5847	3824	3818	3818
query73	673	760	362	362
query74	10073	9241	8653	8653
query75	3178	3178	2686	2686
query76	3179	1135	784	784
query77	473	382	281	281
query78	9975	10026	9333	9333
query79	3047	810	591	591
query80	728	527	458	458
query81	516	271	243	243
query82	445	159	130	130
query83	181	164	157	157
query84	263	93	71	71
query85	786	340	299	299
query86	405	318	297	297
query87	4370	4575	4393	4393
query88	5024	2203	2230	2203
query89	400	339	302	302
query90	1827	187	194	187
query91	138	134	103	103
query92	68	56	54	54
query93	2482	887	527	527
query94	670	400	291	291
query95	345	270	253	253
query96	491	608	279	279
query97	2808	2883	2743	2743
query98	217	217	192	192
query99	1433	1490	1385	1385
Total cold run time: 287971 ms
Total hot run time: 188944 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.37 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d8da356adbbea4052ffd37c56d94e07258305a83, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.04
query3	0.23	0.07	0.06
query4	1.63	0.10	0.11
query5	0.43	0.42	0.41
query6	1.15	0.66	0.66
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.59	0.49	0.52
query10	0.56	0.56	0.55
query11	0.14	0.11	0.10
query12	0.14	0.11	0.11
query13	0.61	0.61	0.60
query14	2.70	2.89	2.85
query15	0.89	0.83	0.82
query16	0.40	0.39	0.37
query17	1.01	1.01	1.06
query18	0.22	0.20	0.21
query19	1.85	1.80	2.00
query20	0.01	0.01	0.01
query21	15.35	0.96	0.59
query22	0.77	0.79	0.59
query23	15.33	1.43	0.59
query24	2.92	1.20	0.97
query25	0.20	0.05	0.11
query26	0.28	0.17	0.15
query27	0.08	0.04	0.04
query28	13.98	1.54	1.05
query29	12.57	3.97	3.24
query30	0.25	0.08	0.06
query31	2.82	0.61	0.38
query32	3.24	0.56	0.48
query33	2.94	3.02	3.10
query34	16.71	5.10	4.56
query35	4.62	4.58	4.58
query36	0.64	0.48	0.47
query37	0.10	0.06	0.06
query38	0.04	0.03	0.04
query39	0.04	0.02	0.02
query40	0.16	0.13	0.13
query41	0.09	0.03	0.02
query42	0.03	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 105.92 s
Total hot run time: 31.37 s

@Yulei-Yang
Copy link
Contributor Author

run buildall

@Yulei-Yang
Copy link
Contributor Author

run feut

@Yulei-Yang
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32399 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 069d25c3df227ae30713be789f1a59ee2c39bad5, data reload: false

------ Round 1 ----------------------------------
q1	17582	5511	5391	5391
q2	2042	331	171	171
q3	10396	1261	712	712
q4	10218	969	531	531
q5	7555	2450	2188	2188
q6	194	163	135	135
q7	914	770	604	604
q8	9250	1389	1165	1165
q9	5355	4929	4891	4891
q10	6817	2326	1903	1903
q11	506	294	268	268
q12	346	363	223	223
q13	17768	3715	3112	3112
q14	227	241	213	213
q15	508	476	471	471
q16	633	607	601	601
q17	545	852	318	318
q18	7039	6613	6409	6409
q19	1211	951	551	551
q20	320	337	197	197
q21	2872	2265	2032	2032
q22	362	336	313	313
Total cold run time: 102660 ms
Total hot run time: 32399 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5472	5483	5462	5462
q2	237	339	242	242
q3	2272	2706	2308	2308
q4	1422	1829	1362	1362
q5	4409	4775	4717	4717
q6	166	156	129	129
q7	2083	2040	1847	1847
q8	2675	2860	2734	2734
q9	7237	7129	7205	7129
q10	2999	3298	2810	2810
q11	587	524	510	510
q12	712	786	608	608
q13	3604	3966	3352	3352
q14	288	296	276	276
q15	510	455	467	455
q16	648	682	651	651
q17	1234	1736	1260	1260
q18	7690	7397	7371	7371
q19	853	1199	1122	1122
q20	2028	2038	1890	1890
q21	5857	5408	5267	5267
q22	657	641	579	579
Total cold run time: 53640 ms
Total hot run time: 52081 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 188442 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 069d25c3df227ae30713be789f1a59ee2c39bad5, data reload: false

query1	980	391	374	374
query2	6523	2114	2057	2057
query3	6810	217	214	214
query4	33532	23272	23344	23272
query5	4448	650	464	464
query6	286	190	182	182
query7	4599	507	308	308
query8	288	233	229	229
query9	9720	2706	2702	2702
query10	499	318	256	256
query11	17865	15316	14995	14995
query12	154	108	105	105
query13	1651	532	402	402
query14	10321	6638	7453	6638
query15	234	198	203	198
query16	8111	649	455	455
query17	1604	757	542	542
query18	2118	387	294	294
query19	230	180	159	159
query20	128	115	110	110
query21	211	118	100	100
query22	4107	4209	4058	4058
query23	34058	33295	33051	33051
query24	6573	2323	2390	2323
query25	528	446	388	388
query26	1202	282	156	156
query27	1993	483	335	335
query28	5346	2458	2418	2418
query29	747	553	427	427
query30	236	184	159	159
query31	955	869	810	810
query32	96	60	63	60
query33	543	360	298	298
query34	783	861	522	522
query35	816	831	740	740
query36	1007	1033	899	899
query37	122	106	82	82
query38	4208	4152	4141	4141
query39	1440	1405	1411	1405
query40	215	121	104	104
query41	54	54	52	52
query42	121	103	103	103
query43	531	573	504	504
query44	1330	803	797	797
query45	188	172	177	172
query46	909	1086	642	642
query47	1795	1845	1779	1779
query48	378	403	325	325
query49	776	494	410	410
query50	662	686	399	399
query51	6819	6823	6909	6823
query52	103	104	94	94
query53	233	267	197	197
query54	487	509	430	430
query55	83	82	80	80
query56	263	283	251	251
query57	1189	1181	1084	1084
query58	256	253	240	240
query59	2945	3345	2862	2862
query60	275	271	256	256
query61	126	116	116	116
query62	790	740	663	663
query63	224	201	195	195
query64	4622	1018	652	652
query65	3276	3153	3182	3153
query66	1074	436	324	324
query67	15857	15885	15628	15628
query68	2274	861	570	570
query69	413	301	276	276
query70	1221	1165	1092	1092
query71	336	316	279	279
query72	5277	3869	3761	3761
query73	662	794	370	370
query74	9670	9403	9083	9083
query75	3165	3207	2687	2687
query76	2186	1165	793	793
query77	354	381	285	285
query78	10175	10238	9345	9345
query79	3095	832	590	590
query80	1848	562	461	461
query81	559	277	237	237
query82	856	155	115	115
query83	280	186	158	158
query84	237	96	75	75
query85	798	355	313	313
query86	474	317	316	316
query87	4456	4463	4521	4463
query88	3507	2230	2176	2176
query89	418	324	294	294
query90	1810	195	195	195
query91	147	139	110	110
query92	62	58	56	56
query93	1215	884	549	549
query94	689	409	301	301
query95	340	270	274	270
query96	487	625	292	292
query97	2781	2864	2787	2787
query98	223	206	195	195
query99	1321	1344	1285	1285
Total cold run time: 281989 ms
Total hot run time: 188442 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.17 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 069d25c3df227ae30713be789f1a59ee2c39bad5, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.04	0.04
query3	0.24	0.07	0.06
query4	1.61	0.11	0.11
query5	0.43	0.39	0.41
query6	1.14	0.65	0.65
query7	0.02	0.01	0.02
query8	0.04	0.04	0.03
query9	0.58	0.52	0.50
query10	0.55	0.57	0.55
query11	0.14	0.10	0.10
query12	0.13	0.12	0.11
query13	0.61	0.61	0.59
query14	2.82	2.74	2.73
query15	0.91	0.83	0.85
query16	0.36	0.39	0.38
query17	0.98	1.06	1.04
query18	0.23	0.20	0.21
query19	1.97	1.98	1.83
query20	0.01	0.02	0.01
query21	15.38	0.94	0.60
query22	0.74	0.90	0.70
query23	15.12	1.45	0.59
query24	3.37	1.53	1.30
query25	0.16	0.18	0.17
query26	0.26	0.14	0.13
query27	0.07	0.05	0.06
query28	14.35	0.99	0.43
query29	12.57	3.96	3.24
query30	0.25	0.09	0.07
query31	2.84	0.61	0.39
query32	3.23	0.56	0.47
query33	2.97	3.00	2.99
query34	16.56	5.13	4.54
query35	4.50	4.52	4.49
query36	0.63	0.49	0.51
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.03
query40	0.18	0.13	0.14
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.39 s
Total hot run time: 31.17 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 16, 2025
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@lide-reed lide-reed merged commit 3a37f9f into apache:master Jan 17, 2025
25 of 26 checks passed
@Yulei-Yang Yulei-Yang deleted the fix_miss_policy_in_create_table_like branch January 17, 2025 03:06
@Yulei-Yang Yulei-Yang mentioned this pull request Jan 17, 2025
Open
16 tasks
Yulei-Yang added a commit to Yulei-Yang/doris that referenced this pull request Jan 17, 2025
@Yulei-Yang
[fix](coldheat) fix missing partition's storage policy in create_tabl…
4a4f991 
…e_like stmt (apache#47009)
@Yulei-Yang Yulei-Yang mentioned this pull request Jan 17, 2025
Open
16 tasks
github-actions bot pushed a commit that referenced this pull request Jan 17, 2025
@Yulei-Yang
[fix](coldheat) fix missing partition's storage policy in create_tabl…
e207d49 
…e_like stmt (#47009)
@github-actions github-actions bot mentioned this pull request Jan 17, 2025
Open
github-actions bot pushed a commit that referenced this pull request Jan 17, 2025
@Yulei-Yang
[fix](coldheat) fix missing partition's storage policy in create_tabl…
f7430d5 
…e_like stmt (#47009)
@github-actions github-actions bot mentioned this pull request Jan 17, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@liutang123 liutang123 liutang123 approved these changes

@cambyzju cambyzju cambyzju approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.x dev/2.1.x dev/3.0.x reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@Yulei-Yang @hello-stephen @doris-robot @cambyzju @liutang123 @lide-reed