Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[opt](ctas) add a variable to control varchar length in ctas #37069

Merged
merged 1 commit into from
Jul 2, 2024
Merged

[opt](ctas) add a variable to control varchar length in ctas #37069

merged 1 commit into from
Jul 2, 2024

Conversation

morrySnow
Copy link
Contributor

add a new session variable: use_max_length_of_varchar_in_ctas

In CTAS (Create Table As Select), if CHAR/VARCHAR columns do not originate from the source table, whether to set the length of such a column to MAX, which is 65533. The default is true.

@morrySnow
[opt](ctas) add a variable to control varchar length in ctas
741ac15 
add a new session variable: use_max_length_of_varchar_in_ctas

In CTAS (Create Table As Select), if CHAR/VARCHAR columns do not
originate from the source table, whether to set the length of
such a column to MAX, which is 65533. The default is true.
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@morrySnow
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40633 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 741ac155e869dbd72761e2755b671cbebcd674f5, data reload: false

------ Round 1 ----------------------------------
q1	17609	5110	4284	4284
q2	2023	191	194	191
q3	10478	1241	1154	1154
q4	10189	754	841	754
q5	7486	2672	2644	2644
q6	223	141	141	141
q7	954	598	625	598
q8	9231	2097	2094	2094
q9	9062	6521	6514	6514
q10	9001	3707	3673	3673
q11	447	224	235	224
q12	475	236	231	231
q13	17791	2998	2978	2978
q14	252	227	223	223
q15	535	476	479	476
q16	494	379	381	379
q17	978	617	663	617
q18	8089	7463	7378	7378
q19	6577	1534	1510	1510
q20	669	323	327	323
q21	4910	3907	3905	3905
q22	395	342	344	342
Total cold run time: 117868 ms
Total hot run time: 40633 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4377	4249	4256	4249
q2	380	278	264	264
q3	3045	2850	2936	2850
q4	1992	1786	1725	1725
q5	5467	5527	5444	5444
q6	234	131	133	131
q7	2269	1856	1833	1833
q8	3291	3491	3436	3436
q9	8684	8733	8844	8733
q10	4188	3745	3821	3745
q11	607	504	487	487
q12	818	632	644	632
q13	16985	3211	3147	3147
q14	305	293	287	287
q15	541	499	479	479
q16	509	432	432	432
q17	1869	1553	1515	1515
q18	8105	8002	7775	7775
q19	4723	1697	1678	1678
q20	2199	1883	1853	1853
q21	5081	4912	4730	4730
q22	649	567	584	567
Total cold run time: 76318 ms
Total hot run time: 55992 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173031 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 741ac155e869dbd72761e2755b671cbebcd674f5, data reload: false

query1	934	387	382	382
query2	6432	2463	2313	2313
query3	6651	214	237	214
query4	19528	17496	17220	17220
query5	3613	487	489	487
query6	268	167	165	165
query7	4593	303	315	303
query8	316	292	287	287
query9	8441	2370	2348	2348
query10	568	293	280	280
query11	10597	10073	9895	9895
query12	115	82	77	77
query13	1646	359	344	344
query14	10105	6884	7029	6884
query15	236	186	189	186
query16	7732	272	265	265
query17	1899	605	516	516
query18	1940	281	273	273
query19	192	154	148	148
query20	89	82	84	82
query21	210	130	126	126
query22	4239	4033	3961	3961
query23	33969	33419	33497	33419
query24	10325	2870	2771	2771
query25	608	376	392	376
query26	709	157	153	153
query27	2286	342	333	333
query28	5854	2131	2142	2131
query29	887	658	622	622
query30	242	153	158	153
query31	1013	792	766	766
query32	94	53	56	53
query33	691	304	325	304
query34	893	489	501	489
query35	756	655	634	634
query36	1132	971	971	971
query37	151	91	82	82
query38	2943	2830	2866	2830
query39	889	859	819	819
query40	211	174	127	127
query41	52	51	54	51
query42	108	102	100	100
query43	598	568	547	547
query44	1058	715	726	715
query45	189	163	158	158
query46	1072	731	708	708
query47	1858	1798	1767	1767
query48	373	310	290	290
query49	836	410	422	410
query50	776	376	376	376
query51	6821	6721	6723	6721
query52	109	91	97	91
query53	362	283	280	280
query54	877	448	436	436
query55	76	73	71	71
query56	288	264	266	264
query57	1171	1053	1084	1053
query58	234	246	253	246
query59	3515	3377	3125	3125
query60	325	284	293	284
query61	97	91	92	91
query62	587	441	447	441
query63	313	283	287	283
query64	8508	2261	1744	1744
query65	3163	3089	3089	3089
query66	747	324	332	324
query67	15211	14979	15041	14979
query68	6052	532	541	532
query69	667	456	354	354
query70	1219	1140	1172	1140
query71	430	277	270	270
query72	7385	5445	5235	5235
query73	776	323	326	323
query74	5892	5514	5548	5514
query75	3649	2707	2660	2660
query76	3164	920	928	920
query77	616	305	300	300
query78	10457	9907	9783	9783
query79	2115	513	508	508
query80	2148	473	478	473
query81	554	218	215	215
query82	1424	112	108	108
query83	266	171	163	163
query84	247	89	84	84
query85	1288	284	272	272
query86	473	313	308	308
query87	3283	3140	3091	3091
query88	3354	2337	2335	2335
query89	474	364	376	364
query90	1761	185	184	184
query91	124	98	101	98
query92	61	49	48	48
query93	2259	492	509	492
query94	1108	194	261	194
query95	415	308	311	308
query96	594	274	262	262
query97	3200	3054	3074	3054
query98	213	197	200	197
query99	1089	873	809	809
Total cold run time: 270078 ms
Total hot run time: 173031 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.57 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 741ac155e869dbd72761e2755b671cbebcd674f5, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.69	0.09	0.08
query5	0.51	0.49	0.49
query6	1.14	0.73	0.73
query7	0.02	0.02	0.02
query8	0.05	0.05	0.04
query9	0.56	0.49	0.49
query10	0.54	0.54	0.55
query11	0.15	0.12	0.12
query12	0.14	0.12	0.12
query13	0.58	0.59	0.58
query14	0.77	0.79	0.79
query15	0.85	0.79	0.82
query16	0.36	0.35	0.38
query17	0.99	0.95	0.97
query18	0.20	0.24	0.23
query19	1.77	1.72	1.68
query20	0.02	0.01	0.01
query21	15.44	0.76	0.65
query22	4.22	7.35	1.98
query23	18.26	1.39	1.27
query24	2.15	0.23	0.22
query25	0.15	0.09	0.08
query26	0.27	0.18	0.18
query27	0.09	0.08	0.07
query28	13.25	1.01	1.00
query29	12.62	3.25	3.30
query30	0.25	0.06	0.06
query31	2.87	0.39	0.39
query32	3.28	0.46	0.46
query33	2.86	2.95	2.91
query34	17.27	4.41	4.41
query35	4.46	4.45	4.54
query36	0.65	0.46	0.48
query37	0.18	0.16	0.15
query38	0.16	0.15	0.15
query39	0.04	0.03	0.04
query40	0.18	0.14	0.14
query41	0.09	0.05	0.04
query42	0.05	0.04	0.04
query43	0.05	0.04	0.04
Total cold run time: 109.53 s
Total hot run time: 30.57 s

@morrySnow
Copy link
Contributor Author

run feut

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 2, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Jul 2, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 2, 2024

PR approved by anyone and no changes requested.

@morrySnow morrySnow merged commit 2e63fef into apache:master Jul 2, 2024
27 of 30 checks passed
@morrySnow morrySnow deleted the varchar_length branch July 2, 2024 08:20
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request Jul 4, 2024
@morrySnow
[opt](ctas) add a variable to control varchar length in ctas (apache#…
3313221 
…37069)

pick from master apache#37069

add a new session variable: use_max_length_of_varchar_in_ctas

In CTAS (Create Table As Select), if CHAR/VARCHAR columns do not
originate from the source table, whether to set the length of such a
column to MAX, which is 65533. The default is true.
@morrySnow morrySnow mentioned this pull request Jul 4, 2024
Merged
morrySnow added a commit that referenced this pull request Jul 4, 2024
@morrySnow
[opt](ctas) add a variable to control varchar length in ctas (#37069) (…
8373610 
…#37284)

pick from master #37069

add a new session variable: use_max_length_of_varchar_in_ctas

In CTAS (Create Table As Select), if CHAR/VARCHAR columns do not
originate from the source table, whether to set the length of such a
column to MAX, which is 65533. The default is true.
dataroaring pushed a commit that referenced this pull request Jul 17, 2024
@morrySnow @dataroaring
[opt](ctas) add a variable to control varchar length in ctas (#37069)
59b87de 
add a new session variable: use_max_length_of_varchar_in_ctas

In CTAS (Create Table As Select), if CHAR/VARCHAR columns do not
originate from the source table, whether to set the length of such a
column to MAX, which is 65533. The default is true.
@gavinchou gavinchou mentioned this pull request Aug 19, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@feiniaofeiafei feiniaofeiafei feiniaofeiafei approved these changes

@starocean999 starocean999 starocean999 approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.5-merged dev/3.0.1-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@morrySnow @doris-robot @starocean999 @feiniaofeiafei @dataroaring