[fix](inverted index) Modify Error Handling for File Open Failure #44551

zzzxl1993 · 2024-11-25T09:06:58Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

doris-robot · 2024-11-25T09:07:03Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

zzzxl1993 · 2024-11-25T09:07:11Z

run buildall

github-actions · 2024-11-25T09:11:35Z

clang-tidy review says "All clean, LGTM! 👍"

qidaye · 2024-11-25T09:44:52Z

regression-test/suites/fault_injection_p0/test_skip_index_compaction_fault_injection.groovy

+      // first
+      trigger_full_compaction_on_tablets.call(tablets)
+      wait_full_compaction_done.call(tablets)
+    } finally {


The first time trigger full compaction will fail.
Need check rowset count to confirm.

doris-robot · 2024-11-25T09:55:25Z

TPC-H: Total hot run time: 39894 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6994db8430799154ce14d7aac1e49f60a0c1575a, data reload: false

------ Round 1 ----------------------------------
q1	17608	8010	7291	7291
q2	2038	169	164	164
q3	10719	1114	1119	1114
q4	10571	781	734	734
q5	7630	2754	2699	2699
q6	240	150	151	150
q7	982	637	610	610
q8	9252	1857	1932	1857
q9	6626	6398	6392	6392
q10	6986	2302	2332	2302
q11	459	263	264	263
q12	442	222	215	215
q13	17768	3036	2991	2991
q14	257	211	209	209
q15	564	529	520	520
q16	683	585	591	585
q17	976	499	597	499
q18	7407	6804	6792	6792
q19	1330	1046	993	993
q20	462	181	175	175
q21	3947	3027	3074	3027
q22	370	325	312	312
Total cold run time: 107317 ms
Total hot run time: 39894 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7231	7206	7223	7206
q2	333	226	233	226
q3	2918	2785	2948	2785
q4	2075	1838	1761	1761
q5	5662	5697	5705	5697
q6	225	145	147	145
q7	2265	1844	1813	1813
q8	3430	3589	3531	3531
q9	8879	8977	8943	8943
q10	3606	3574	3576	3574
q11	607	515	517	515
q12	840	607	631	607
q13	11038	3335	3289	3289
q14	289	264	285	264
q15	575	508	518	508
q16	682	653	654	653
q17	1835	1631	1602	1602
q18	8480	7874	7735	7735
q19	1764	1494	1598	1494
q20	2113	1889	1864	1864
q21	5559	5525	5300	5300
q22	627	600	581	581
Total cold run time: 71033 ms
Total hot run time: 60093 ms

doris-robot · 2024-11-25T10:07:50Z

TeamCity be ut coverage result:
Function Coverage: 38.33% (9978/26032)
Line Coverage: 29.43% (83515/283776)
Region Coverage: 28.59% (42979/150320)
Branch Coverage: 25.18% (21836/86730)
Coverage Report: http://coverage.selectdb-in.cc/coverage/6994db8430799154ce14d7aac1e49f60a0c1575a_6994db8430799154ce14d7aac1e49f60a0c1575a/report/index.html

doris-robot · 2024-11-25T10:08:02Z

TPC-DS: Total hot run time: 196847 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6994db8430799154ce14d7aac1e49f60a0c1575a, data reload: false

query1	1241	940	960	940
query2	6240	2132	1981	1981
query3	10813	4037	3961	3961
query4	67274	28633	23713	23713
query5	4972	451	439	439
query6	380	175	170	170
query7	5457	305	304	304
query8	317	227	223	223
query9	8611	2706	2682	2682
query10	444	254	248	248
query11	17258	15232	15925	15232
query12	153	104	104	104
query13	1456	454	428	428
query14	10879	7517	7515	7515
query15	210	186	198	186
query16	7225	482	492	482
query17	1057	577	594	577
query18	1891	319	305	305
query19	212	203	155	155
query20	135	108	109	108
query21	197	98	103	98
query22	4865	4480	4429	4429
query23	34652	34302	34339	34302
query24	5372	2502	2536	2502
query25	477	391	393	391
query26	659	150	153	150
query27	1932	278	273	273
query28	4583	2462	2473	2462
query29	674	419	408	408
query30	230	151	156	151
query31	1028	831	818	818
query32	69	55	51	51
query33	428	288	276	276
query34	942	502	513	502
query35	891	757	734	734
query36	1084	954	985	954
query37	119	73	81	73
query38	4517	4400	4292	4292
query39	1518	1486	1472	1472
query40	205	111	103	103
query41	44	45	43	43
query42	103	125	107	107
query43	518	482	502	482
query44	1201	814	830	814
query45	184	169	163	163
query46	1122	686	709	686
query47	2018	1905	1944	1905
query48	438	341	316	316
query49	725	403	376	376
query50	832	392	398	392
query51	7395	7266	7236	7236
query52	101	84	84	84
query53	253	179	176	176
query54	498	404	393	393
query55	77	73	75	73
query56	243	230	235	230
query57	1299	1195	1145	1145
query58	217	216	210	210
query59	3169	3178	3073	3073
query60	272	240	237	237
query61	106	102	108	102
query62	804	654	680	654
query63	208	181	184	181
query64	1380	658	639	639
query65	3258	3192	3205	3192
query66	708	316	314	314
query67	16088	15524	15683	15524
query68	3748	578	543	543
query69	452	254	258	254
query70	1205	1108	1149	1108
query71	350	244	247	244
query72	6393	4076	4095	4076
query73	744	352	359	352
query74	10185	8961	8961	8961
query75	3404	2708	2631	2631
query76	1843	1098	1108	1098
query77	476	273	264	264
query78	10555	9422	9419	9419
query79	1596	589	581	581
query80	875	430	426	426
query81	542	240	236	236
query82	1269	120	114	114
query83	341	153	150	150
query84	277	65	66	65
query85	890	293	288	288
query86	354	308	273	273
query87	4675	4621	4523	4523
query88	3766	2227	2184	2184
query89	418	290	298	290
query90	2042	190	186	186
query91	136	104	102	102
query92	71	48	57	48
query93	1965	536	535	535
query94	794	304	286	286
query95	348	243	245	243
query96	628	270	284	270
query97	2841	2711	2665	2665
query98	216	197	197	197
query99	1638	1294	1315	1294
Total cold run time: 318965 ms
Total hot run time: 196847 ms

doris-robot · 2024-11-25T10:13:32Z

ClickBench: Total hot run time: 33.17 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6994db8430799154ce14d7aac1e49f60a0c1575a, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.06
query3	0.23	0.08	0.06
query4	1.62	0.10	0.10
query5	0.42	0.43	0.41
query6	1.14	0.66	0.65
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.57	0.53	0.52
query10	0.55	0.57	0.57
query11	0.14	0.11	0.09
query12	0.14	0.11	0.12
query13	0.60	0.61	0.60
query14	2.71	2.88	2.76
query15	0.92	0.82	0.82
query16	0.40	0.40	0.38
query17	1.06	1.03	1.04
query18	0.22	0.21	0.20
query19	1.96	1.75	2.06
query20	0.02	0.01	0.01
query21	15.37	0.61	0.58
query22	2.55	2.05	2.47
query23	16.90	0.97	0.96
query24	3.13	1.26	1.49
query25	0.26	0.11	0.15
query26	0.46	0.13	0.14
query27	0.04	0.04	0.03
query28	10.10	1.10	1.07
query29	12.63	3.20	3.23
query30	0.25	0.07	0.07
query31	2.86	0.39	0.39
query32	3.27	0.44	0.48
query33	3.00	3.01	3.06
query34	17.12	4.46	4.50
query35	4.47	4.50	4.46
query36	0.65	0.49	0.48
query37	0.09	0.06	0.07
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.16	0.13	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 106.4 s
Total hot run time: 33.17 s

airborne12

LGTM

github-actions · 2024-11-26T01:58:08Z

PR approved by at least one committer and no changes requested.

github-actions · 2024-11-26T01:58:10Z

PR approved by anyone and no changes requested.

zzzxl1993 · 2024-11-26T02:30:00Z

run buildall

doris-robot · 2024-11-26T03:01:01Z

TPC-H: Total hot run time: 40092 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 64a3a1e31eec73285cad880bfab0165c2996a5e2, data reload: false

------ Round 1 ----------------------------------
q1	17588	7499	7473	7473
q2	2038	177	167	167
q3	10641	1096	1155	1096
q4	10546	731	725	725
q5	7604	2739	2649	2649
q6	239	149	149	149
q7	992	652	632	632
q8	9255	1864	1905	1864
q9	6545	6349	6363	6349
q10	7016	2286	2302	2286
q11	478	264	254	254
q12	456	223	239	223
q13	17773	3034	3099	3034
q14	248	212	214	212
q15	579	527	516	516
q16	674	602	578	578
q17	995	559	523	523
q18	7180	6721	6669	6669
q19	1341	1081	999	999
q20	472	187	195	187
q21	4132	3198	3285	3198
q22	380	309	314	309
Total cold run time: 107172 ms
Total hot run time: 40092 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7257	7258	7271	7258
q2	324	230	227	227
q3	2961	2847	2906	2847
q4	2099	1874	1900	1874
q5	5649	5679	5682	5679
q6	229	147	146	146
q7	2284	1858	1778	1778
q8	3443	3596	3536	3536
q9	8856	8910	8820	8820
q10	3608	3518	3532	3518
q11	589	514	530	514
q12	850	612	632	612
q13	10269	3260	3242	3242
q14	311	280	291	280
q15	554	520	527	520
q16	676	636	651	636
q17	1895	1657	1631	1631
q18	8342	7764	7741	7741
q19	1687	1597	1442	1442
q20	2116	1876	1930	1876
q21	5635	5395	5464	5395
q22	632	548	588	548
Total cold run time: 70266 ms
Total hot run time: 60120 ms

doris-robot · 2024-11-26T03:11:55Z

TeamCity be ut coverage result:
Function Coverage: 38.33% (9978/26032)
Line Coverage: 29.42% (83491/283780)
Region Coverage: 28.60% (42986/150307)
Branch Coverage: 25.17% (21828/86728)
Coverage Report: http://coverage.selectdb-in.cc/coverage/64a3a1e31eec73285cad880bfab0165c2996a5e2_64a3a1e31eec73285cad880bfab0165c2996a5e2/report/index.html

doris-robot · 2024-11-26T03:13:41Z

TPC-DS: Total hot run time: 196538 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 64a3a1e31eec73285cad880bfab0165c2996a5e2, data reload: false

query1	1260	951	947	947
query2	6221	2075	2016	2016
query3	10779	4113	3801	3801
query4	66985	29421	23621	23621
query5	4973	441	442	441
query6	391	172	176	172
query7	5627	291	282	282
query8	298	230	222	222
query9	9266	2737	2724	2724
query10	437	249	238	238
query11	17627	15180	15981	15180
query12	154	107	103	103
query13	1571	433	438	433
query14	10896	7330	7480	7330
query15	216	178	191	178
query16	7069	478	447	447
query17	1126	563	577	563
query18	1818	309	312	309
query19	205	183	166	166
query20	129	124	123	123
query21	209	103	105	103
query22	4751	4478	4342	4342
query23	34923	34427	34230	34230
query24	5480	2495	2536	2495
query25	554	390	389	389
query26	642	145	149	145
query27	1853	281	285	281
query28	4715	2506	2500	2500
query29	729	426	416	416
query30	218	151	154	151
query31	1003	832	829	829
query32	66	56	54	54
query33	401	288	284	284
query34	935	505	518	505
query35	868	758	744	744
query36	1081	992	977	977
query37	124	76	83	76
query38	4479	4558	4427	4427
query39	1566	1479	1474	1474
query40	207	95	108	95
query41	46	45	42	42
query42	107	97	95	95
query43	529	494	505	494
query44	1197	834	848	834
query45	198	175	174	174
query46	1153	713	704	704
query47	2009	1925	1918	1918
query48	422	335	330	330
query49	743	403	420	403
query50	859	396	409	396
query51	7425	7251	7269	7251
query52	93	87	88	87
query53	251	179	179	179
query54	503	398	399	398
query55	81	76	75	75
query56	260	238	243	238
query57	1267	1194	1156	1156
query58	217	210	217	210
query59	3198	2920	2861	2861
query60	259	280	248	248
query61	108	105	105	105
query62	796	670	684	670
query63	209	187	203	187
query64	1369	655	630	630
query65	3315	3237	3220	3220
query66	694	301	308	301
query67	15814	15604	15887	15604
query68	3999	570	550	550
query69	452	258	252	252
query70	1187	1166	1035	1035
query71	351	247	258	247
query72	6460	4063	3988	3988
query73	761	360	357	357
query74	10119	9068	9004	9004
query75	3391	2664	2815	2664
query76	1715	1050	1088	1050
query77	466	274	279	274
query78	10696	9436	9411	9411
query79	1667	606	607	606
query80	875	422	434	422
query81	509	234	226	226
query82	1230	121	115	115
query83	274	146	149	146
query84	288	70	76	70
query85	902	301	294	294
query86	343	287	296	287
query87	4761	4540	4674	4540
query88	3672	2239	2203	2203
query89	419	296	295	295
query90	2010	191	184	184
query91	134	105	102	102
query92	68	48	51	48
query93	1898	547	541	541
query94	823	279	281	279
query95	347	246	254	246
query96	616	277	285	277
query97	2893	2724	2709	2709
query98	215	199	196	196
query99	2088	1323	1297	1297
Total cold run time: 320603 ms
Total hot run time: 196538 ms

doris-robot · 2024-11-26T03:19:10Z

ClickBench: Total hot run time: 32.76 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 64a3a1e31eec73285cad880bfab0165c2996a5e2, data reload: false

query1	0.03	0.02	0.04
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.61	0.10	0.11
query5	0.42	0.42	0.41
query6	1.13	0.67	0.66
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.59	0.52	0.50
query10	0.56	0.56	0.56
query11	0.15	0.11	0.12
query12	0.14	0.11	0.11
query13	0.62	0.61	0.59
query14	2.77	2.70	2.86
query15	0.92	0.83	0.83
query16	0.38	0.39	0.37
query17	1.03	1.06	1.05
query18	0.23	0.21	0.21
query19	1.89	1.90	2.04
query20	0.01	0.01	0.02
query21	15.36	0.59	0.60
query22	2.82	1.91	1.61
query23	17.27	0.89	0.85
query24	2.92	1.54	1.33
query25	0.30	0.14	0.07
query26	0.58	0.14	0.13
query27	0.06	0.04	0.05
query28	10.54	1.11	1.09
query29	12.57	3.19	3.21
query30	0.25	0.06	0.06
query31	2.86	0.38	0.38
query32	3.26	0.47	0.48
query33	3.08	3.02	3.01
query34	16.93	4.41	4.43
query35	4.59	4.45	4.48
query36	0.66	0.49	0.49
query37	0.10	0.06	0.07
query38	0.05	0.04	0.04
query39	0.04	0.02	0.02
query40	0.16	0.12	0.13
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 107.4 s
Total hot run time: 32.76 s

qidaye reviewed Nov 25, 2024

View reviewed changes

airborne12 previously approved these changes Nov 26, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 26, 2024

github-actions bot added the reviewed label Nov 26, 2024

[fix](inverted index) Modify Error Handling for File Open Failure

64a3a1e

zzzxl1993 dismissed airborne12’s stale review via 64a3a1e November 26, 2024 02:27

zzzxl1993 force-pushed the 202411251052 branch from 6994db8 to 64a3a1e Compare November 26, 2024 02:27

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Nov 26, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[fix](inverted index) Modify Error Handling for File Open Failure #44551

[fix](inverted index) Modify Error Handling for File Open Failure #44551

zzzxl1993 commented Nov 25, 2024 •

edited

Loading

doris-robot commented Nov 25, 2024

zzzxl1993 commented Nov 25, 2024

github-actions bot commented Nov 25, 2024

qidaye Nov 25, 2024

doris-robot commented Nov 25, 2024

doris-robot commented Nov 25, 2024

doris-robot commented Nov 25, 2024

doris-robot commented Nov 25, 2024

airborne12 left a comment

github-actions bot commented Nov 26, 2024

github-actions bot commented Nov 26, 2024

zzzxl1993 commented Nov 26, 2024

doris-robot commented Nov 26, 2024

doris-robot commented Nov 26, 2024

doris-robot commented Nov 26, 2024

doris-robot commented Nov 26, 2024

[fix](inverted index) Modify Error Handling for File Open Failure #44551

Are you sure you want to change the base?

[fix](inverted index) Modify Error Handling for File Open Failure #44551

Conversation

zzzxl1993 commented Nov 25, 2024 • edited Loading

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

doris-robot commented Nov 25, 2024

zzzxl1993 commented Nov 25, 2024

github-actions bot commented Nov 25, 2024

qidaye Nov 25, 2024

Choose a reason for hiding this comment

doris-robot commented Nov 25, 2024

doris-robot commented Nov 25, 2024

doris-robot commented Nov 25, 2024

doris-robot commented Nov 25, 2024

airborne12 left a comment

Choose a reason for hiding this comment

github-actions bot commented Nov 26, 2024

github-actions bot commented Nov 26, 2024

zzzxl1993 commented Nov 26, 2024

doris-robot commented Nov 26, 2024

doris-robot commented Nov 26, 2024

doris-robot commented Nov 26, 2024

doris-robot commented Nov 26, 2024

zzzxl1993 commented Nov 25, 2024 •

edited

Loading