Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](index compaction)Remove return error in merged rows check #38732

Merged
merged 1 commit into from
Aug 2, 2024
Merged

[fix](index compaction)Remove return error in merged rows check #38732

merged 1 commit into from
Aug 2, 2024

Conversation

qidaye
Copy link
Contributor

@qidaye qidaye commented Aug 1, 2024

Proposed changes

Log here just for debugging, do not return error.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@qidaye
Copy link
Contributor Author

qidaye commented Aug 1, 2024

run buildall

@github-actions github-actions bot added the doing label Aug 1, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 1, 2024

clang-tidy review says "All clean, LGTM! 👍"

zhannngchen
zhannngchen approved these changes Aug 1, 2024
View reviewed changes
Copy link
Contributor

@zhannngchen zhannngchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 1, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 1, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 1, 2024

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 41435 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 91367051a3838e91114916b00ae82f4ed0d26ac6, data reload: false

------ Round 1 ----------------------------------
q1	18830	4196	4086	4086
q2	2021	199	206	199
q3	10462	1271	1293	1271
q4	10158	809	877	809
q5	7635	2973	2935	2935
q6	219	142	137	137
q7	1023	609	620	609
q8	9434	1826	1897	1826
q9	8378	6598	6621	6598
q10	8706	3824	3838	3824
q11	427	250	243	243
q12	407	222	225	222
q13	17756	2974	2931	2931
q14	268	234	242	234
q15	525	497	487	487
q16	475	381	385	381
q17	943	896	891	891
q18	8005	7259	7206	7206
q19	1400	1205	1210	1205
q20	563	337	368	337
q21	5232	4719	4728	4719
q22	355	289	285	285
Total cold run time: 113222 ms
Total hot run time: 41435 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4004	3982	4005	3982
q2	329	216	223	216
q3	2974	3049	2946	2946
q4	1887	1850	1865	1850
q5	5257	5203	5206	5203
q6	215	134	131	131
q7	2064	1707	1699	1699
q8	3159	3246	3273	3246
q9	8249	8236	8222	8222
q10	3743	3810	3799	3799
q11	534	456	466	456
q12	733	553	553	553
q13	16279	2936	2916	2916
q14	283	252	252	252
q15	516	477	470	470
q16	442	400	397	397
q17	1720	1689	1695	1689
q18	7785	7388	7127	7127
q19	1645	1660	1672	1660
q20	1954	1766	1754	1754
q21	5355	5180	5113	5113
q22	509	464	466	464
Total cold run time: 69636 ms
Total hot run time: 54145 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 167354 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 91367051a3838e91114916b00ae82f4ed0d26ac6, data reload: false

query1	928	368	364	364
query2	6471	1692	1680	1680
query3	6674	218	226	218
query4	19949	17507	17113	17113
query5	4328	526	519	519
query6	280	164	167	164
query7	4610	294	291	291
query8	244	193	194	193
query9	8540	2385	2355	2355
query10	429	276	261	261
query11	10447	10023	9809	9809
query12	143	89	92	89
query13	1637	385	370	370
query14	9196	6324	7262	6324
query15	224	167	165	165
query16	7106	444	460	444
query17	970	589	561	561
query18	1737	290	284	284
query19	196	148	146	146
query20	94	87	85	85
query21	209	100	96	96
query22	4098	4032	4081	4032
query23	33809	32978	32950	32950
query24	10451	3083	3075	3075
query25	723	402	407	402
query26	1845	155	151	151
query27	2931	275	278	275
query28	7001	1968	1956	1956
query29	1273	434	443	434
query30	286	151	151	151
query31	953	804	762	762
query32	102	58	58	58
query33	779	314	315	314
query34	896	477	464	464
query35	837	707	697	697
query36	1031	880	882	880
query37	307	77	79	77
query38	2906	2800	2748	2748
query39	887	796	821	796
query40	283	111	111	111
query41	47	50	46	46
query42	119	95	101	95
query43	463	430	414	414
query44	1181	726	731	726
query45	206	178	177	177
query46	1075	831	828	828
query47	1797	1688	1735	1688
query48	356	291	288	288
query49	1190	423	414	414
query50	883	430	433	430
query51	6754	6733	6663	6663
query52	106	85	91	85
query53	258	183	177	177
query54	643	437	466	437
query55	76	77	75	75
query56	272	270	251	251
query57	1141	1011	1048	1011
query58	275	266	268	266
query59	2551	2572	2375	2375
query60	295	262	274	262
query61	97	95	110	95
query62	931	671	669	669
query63	231	189	185	185
query64	5795	1897	1882	1882
query65	3154	3119	3102	3102
query66	1411	341	334	334
query67	15190	14711	14845	14711
query68	4493	568	586	568
query69	453	309	290	290
query70	1136	1091	1097	1091
query71	428	274	275	274
query72	7165	2702	2469	2469
query73	750	330	329	329
query74	6100	5616	5592	5592
query75	3406	2739	2705	2705
query76	2561	1189	1261	1189
query77	440	305	309	305
query78	9375	8891	8765	8765
query79	2749	531	541	531
query80	1200	497	498	497
query81	551	224	227	224
query82	849	134	135	134
query83	240	167	172	167
query84	267	78	79	78
query85	1527	368	299	299
query86	487	290	302	290
query87	3242	3100	3092	3092
query88	3821	2384	2372	2372
query89	375	286	282	282
query90	1893	194	191	191
query91	127	103	100	100
query92	65	51	53	51
query93	2210	623	626	623
query94	963	283	263	263
query95	377	267	265	265
query96	612	278	278	278
query97	3222	3092	3023	3023
query98	217	199	193	193
query99	1651	1274	1312	1274
Total cold run time: 265259 ms
Total hot run time: 167354 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.01 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 91367051a3838e91114916b00ae82f4ed0d26ac6, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.05
query3	0.23	0.05	0.05
query4	1.67	0.07	0.07
query5	0.48	0.48	0.49
query6	1.14	0.71	0.71
query7	0.02	0.02	0.01
query8	0.06	0.04	0.05
query9	0.56	0.50	0.52
query10	0.57	0.56	0.57
query11	0.15	0.12	0.11
query12	0.14	0.13	0.12
query13	0.61	0.60	0.59
query14	0.78	0.80	0.79
query15	0.88	0.87	0.86
query16	0.36	0.36	0.36
query17	0.99	0.98	0.98
query18	0.22	0.21	0.22
query19	1.88	1.74	1.73
query20	0.02	0.01	0.01
query21	15.41	0.74	0.68
query22	3.76	7.87	1.44
query23	17.83	1.26	1.29
query24	2.26	0.23	0.21
query25	0.19	0.08	0.08
query26	0.32	0.21	0.21
query27	0.45	0.23	0.24
query28	13.19	1.01	0.96
query29	12.51	3.30	3.25
query30	0.26	0.06	0.05
query31	2.90	0.41	0.41
query32	3.21	0.49	0.50
query33	2.93	2.98	2.91
query34	15.43	4.25	4.23
query35	4.31	4.27	4.30
query36	0.68	0.49	0.49
query37	0.19	0.16	0.16
query38	0.16	0.15	0.14
query39	0.04	0.03	0.03
query40	0.16	0.13	0.13
query41	0.10	0.04	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 107.27 s
Total hot run time: 30.01 s

airborne12
airborne12 approved these changes Aug 2, 2024
View reviewed changes
Copy link
Member

@airborne12 airborne12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@qidaye qidaye merged commit 1ada810 into apache:master Aug 2, 2024
29 of 31 checks passed
@qidaye qidaye deleted the fix_index_compaction_error branch August 2, 2024 03:37
@qidaye qidaye removed the dev/2.1.x label Aug 2, 2024
qidaye added a commit to qidaye/incubator-doris that referenced this pull request Aug 2, 2024
@qidaye
[fix](index compaction)Remove return error in merged rows check apach…
a9c94b3 
…e#38732
@qidaye qidaye mentioned this pull request Aug 2, 2024
Merged
xiaokang pushed a commit that referenced this pull request Aug 2, 2024
@qidaye
[fix](index compaction)Remove return error in merged rows check #38732 (
af7a285 
#38757)
dataroaring pushed a commit that referenced this pull request Aug 5, 2024
@qidaye @dataroaring
[fix](index compaction)Remove return error in merged rows check (#38732)
370d09b 
Log here just for debugging, do not return error.
@qidaye qidaye removed the doing label Aug 7, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@qidaye @Baymine
[fix](index compaction)Remove return error in merged rows check apach…
1be2c2f 
…e#38732 (apache#38757)
@gavinchou gavinchou mentioned this pull request Aug 19, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zhannngchen zhannngchen zhannngchen approved these changes

@airborne12 airborne12 airborne12 approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.14-merged dev/3.0.1-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@qidaye @doris-robot @airborne12 @zhannngchen @xiaokang @dataroaring