Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[test](build index)Remove index_meta in tablet schema when the index is dropped. (#37646) #37902

Merged
merged 1 commit into from
Jul 16, 2024
Merged

[test](build index)Remove index_meta in tablet schema when the index is dropped. (#37646) #37902

merged 1 commit into from
Jul 16, 2024

Conversation

qidaye
Copy link
Contributor

@qidaye qidaye commented Jul 16, 2024

Proposed changes

branch-2.0 dose not have the bug. Only back port the test.
backport #37646

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@qidaye
Copy link
Contributor Author

qidaye commented Jul 16, 2024

run buildall

@qidaye qidaye force-pushed the pick_fix_drop_index_2.0 branch from 6e7fc5b to 9f84418 Compare July 16, 2024 07:36
@qidaye
Copy link
Contributor Author

qidaye commented Jul 16, 2024

run buildall

@qidaye qidaye changed the title [fix](build index)Remove index_meta in tablet schema when the index is dropped. (#37646) [test](build index)Remove index_meta in tablet schema when the index is dropped. (#37646) Jul 16, 2024
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.82% (8119/21466)
Line Coverage: 29.49% (66558/225688)
Region Coverage: 28.96% (34306/118442)
Branch Coverage: 24.85% (17629/70950)
Coverage Report: http://coverage.selectdb-in.cc/coverage/9f84418101655b019dd4981bd017ff82690ffaa8_9f84418101655b019dd4981bd017ff82690ffaa8/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 49798 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9f84418101655b019dd4981bd017ff82690ffaa8, data reload: false

------ Round 1 ----------------------------------
q1	17906	4431	4364	4364
q2	2032	163	147	147
q3	10263	1882	1959	1882
q4	10328	1246	1335	1246
q5	8442	3901	3922	3901
q6	233	128	128	128
q7	2081	1589	1603	1589
q8	9517	2753	2714	2714
q9	13826	10378	10244	10244
q10	8623	3568	3481	3481
q11	402	250	242	242
q12	473	311	318	311
q13	18338	3944	4002	3944
q14	356	322	335	322
q15	536	459	469	459
q16	671	571	571	571
q17	1139	944	914	914
q18	7151	6919	6918	6918
q19	1786	1668	1622	1622
q20	539	297	304	297
q21	4362	4152	4060	4060
q22	525	444	442	442
Total cold run time: 119529 ms
Total hot run time: 49798 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4343	4324	4320	4320
q2	325	226	222	222
q3	4171	4152	4145	4145
q4	2764	2757	2756	2756
q5	7115	7104	7064	7064
q6	237	121	122	121
q7	3189	2864	2755	2755
q8	4298	4426	4493	4426
q9	16821	16780	16827	16780
q10	4215	4246	4278	4246
q11	788	690	663	663
q12	1009	852	847	847
q13	6912	3736	3760	3736
q14	454	418	424	418
q15	497	467	459	459
q16	743	679	666	666
q17	3765	3885	3810	3810
q18	8808	8789	8862	8789
q19	1730	1680	1682	1680
q20	2391	2118	2102	2102
q21	8406	8365	8408	8365
q22	1070	1003	964	964
Total cold run time: 84051 ms
Total hot run time: 79334 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 202960 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9f84418101655b019dd4981bd017ff82690ffaa8, data reload: false

query1	948	423	381	381
query2	6522	2886	2854	2854
query3	6913	205	203	203
query4	21458	18059	18039	18039
query5	19741	6533	6545	6533
query6	295	219	236	219
query7	4156	309	307	307
query8	400	430	443	430
query9	3103	2665	2611	2611
query10	417	307	292	292
query11	11260	10697	10578	10578
query12	125	79	73	73
query13	5605	699	681	681
query14	17901	13862	13013	13013
query15	362	269	249	249
query16	6475	287	262	262
query17	1711	1426	891	891
query18	2320	415	416	415
query19	208	155	152	152
query20	84	78	80	78
query21	196	95	100	95
query22	5285	5064	4987	4987
query23	32795	32092	32043	32043
query24	7020	6552	6591	6552
query25	532	425	432	425
query26	526	165	162	162
query27	1864	301	302	301
query28	6173	2362	2316	2316
query29	2915	2723	2793	2723
query30	244	166	168	166
query31	906	725	753	725
query32	71	63	63	63
query33	409	253	248	248
query34	840	487	486	486
query35	1100	915	891	891
query36	1189	1146	1185	1146
query37	92	59	62	59
query38	3074	2907	2907	2907
query39	1389	1315	1324	1315
query40	206	99	96	96
query41	47	46	44	44
query42	92	86	81	81
query43	848	702	722	702
query44	1165	719	718	718
query45	249	238	239	238
query46	1234	941	982	941
query47	1883	1691	1656	1656
query48	1019	731	704	704
query49	630	370	378	370
query50	859	624	591	591
query51	4782	4687	4704	4687
query52	86	81	87	81
query53	455	328	316	316
query54	2652	2455	2425	2425
query55	85	76	80	76
query56	262	219	216	216
query57	1152	1083	1011	1011
query58	213	213	197	197
query59	4058	4189	3956	3956
query60	216	231	214	214
query61	98	96	99	96
query62	856	482	438	438
query63	487	340	341	340
query64	2399	1508	1433	1433
query65	3632	3572	3572	3572
query66	831	370	387	370
query67	15447	17704	14845	14845
query68	8642	660	664	660
query69	573	363	353	353
query70	1586	1385	1393	1385
query71	410	306	321	306
query72	6519	3508	3507	3507
query73	733	321	326	321
query74	6279	5834	5787	5787
query75	5232	3675	3698	3675
query76	5400	1115	1195	1115
query77	928	271	281	271
query78	12751	11579	11531	11531
query79	9276	658	668	658
query80	1581	410	402	402
query81	503	234	234	234
query82	1680	98	99	98
query83	176	134	130	130
query84	255	70	74	70
query85	900	339	333	333
query86	325	294	288	288
query87	3219	3001	2984	2984
query88	5090	2329	2346	2329
query89	390	298	289	289
query90	1923	204	213	204
query91	189	156	143	143
query92	67	53	56	53
query93	6395	568	585	568
query94	679	205	209	205
query95	1120	1045	1060	1045
query96	649	329	331	329
query97	6466	6383	6434	6383
query98	193	170	171	170
query99	2913	849	849	849
Total cold run time: 315609 ms
Total hot run time: 202960 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.88 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9f84418101655b019dd4981bd017ff82690ffaa8, data reload: false

query1	0.02	0.02	0.03
query2	0.07	0.03	0.02
query3	0.25	0.05	0.06
query4	1.79	0.06	0.06
query5	0.54	0.52	0.52
query6	1.24	0.62	0.61
query7	0.02	0.01	0.01
query8	0.03	0.03	0.02
query9	0.52	0.49	0.48
query10	0.54	0.54	0.53
query11	0.12	0.09	0.10
query12	0.11	0.09	0.09
query13	0.63	0.61	0.61
query14	0.78	0.78	0.80
query15	0.77	0.75	0.77
query16	0.37	0.36	0.35
query17	0.99	0.98	1.03
query18	0.22	0.24	0.24
query19	1.92	1.84	1.89
query20	0.01	0.02	0.01
query21	15.45	0.55	0.54
query22	1.94	2.66	2.07
query23	17.20	1.08	0.93
query24	5.10	0.91	1.05
query25	0.36	0.06	0.08
query26	0.59	0.17	0.16
query27	0.04	0.05	0.04
query28	8.20	0.72	0.71
query29	12.64	2.18	2.08
query30	0.62	0.53	0.52
query31	2.82	0.40	0.37
query32	3.37	0.50	0.49
query33	3.06	3.06	3.10
query34	15.28	4.79	4.78
query35	4.86	4.84	4.84
query36	1.07	1.01	1.02
query37	0.06	0.04	0.05
query38	0.03	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.15	0.14
query41	0.07	0.01	0.01
query42	0.02	0.01	0.02
query43	0.02	0.02	0.02
Total cold run time: 103.92 s
Total hot run time: 30.88 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 9f84418101655b019dd4981bd017ff82690ffaa8 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.9 seconds inserted 10000000 Rows, about 456K ops/s

@qidaye
Copy link
Contributor Author

qidaye commented Jul 16, 2024

run external

1 similar comment
@qidaye
Copy link
Contributor Author

qidaye commented Jul 16, 2024

run external

@xiaokang xiaokang merged commit 5f5ec1c into apache:branch-2.0 Jul 16, 2024
23 of 25 checks passed
@qidaye qidaye deleted the pick_fix_drop_index_2.0 branch July 16, 2024 12:33
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@qidaye
[test](build index)Remove index_meta in tablet schema when the index …
b2fc219 
…is dropped. (apache#37646) (apache#37902)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
kind/test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@qidaye @doris-robot @xiaokang