Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](fe) Fix removeDroppedMember use incorrect nodeName #37987

Merged
merged 1 commit into from
Jul 17, 2024

Conversation

SWJTU-ZhangLei
Copy link
Contributor

com.sleepycat.je.rep.MasterStateException: (JE 18.3.12) Cannot remove an active master
    at com.sleepycat.je.rep.util.ReplicationGroupAdmin.checkMember(ReplicationGroupAdmin.java:591)
    at com.sleepycat.je.rep.util.ReplicationGroupAdmin.removeMember(ReplicationGroupAdmin.java:315)
    at org.apache.doris.ha.BDBHA.removeDroppedMember(BDBHA.java:262)
    at org.apache.doris.catalog.Env.removeDroppedFrontends(Env.java:3077)
    at org.apache.doris.catalog.Env.transferToMaster(Env.java:1520)
    at org.apache.doris.catalog.Env.access$1300(Env.java:338)
    at org.apache.doris.catalog.Env$5.runOneCycle(Env.java:2792)
    at org.apache.doris.common.util.Daemon.run(Daemon.java:116)

Proposed changes

Issue Number: close #xxx

```
com.sleepycat.je.rep.MasterStateException: (JE 18.3.12) Cannot remove an active master
    at com.sleepycat.je.rep.util.ReplicationGroupAdmin.checkMember(ReplicationGroupAdmin.java:591)
    at com.sleepycat.je.rep.util.ReplicationGroupAdmin.removeMember(ReplicationGroupAdmin.java:315)
    at org.apache.doris.ha.BDBHA.removeDroppedMember(BDBHA.java:262)
    at org.apache.doris.catalog.Env.removeDroppedFrontends(Env.java:3077)
    at org.apache.doris.catalog.Env.transferToMaster(Env.java:1520)
    at org.apache.doris.catalog.Env.access$1300(Env.java:338)
    at org.apache.doris.catalog.Env$5.runOneCycle(Env.java:2792)
    at org.apache.doris.common.util.Daemon.run(Daemon.java:116)
```

* the bug is introduced by apache#35203
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@SWJTU-ZhangLei
Copy link
Contributor Author

run buildall

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 17, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 39789 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 467bac43b4d8ac4934f3743e7863643fc696f54e, data reload: false

------ Round 1 ----------------------------------
q1	17620	4325	4226	4226
q2	2006	192	192	192
q3	10448	1200	1053	1053
q4	10200	791	777	777
q5	7633	2792	2671	2671
q6	216	136	137	136
q7	957	599	591	591
q8	9215	2058	2058	2058
q9	8698	6577	6546	6546
q10	8782	3782	3741	3741
q11	456	229	225	225
q12	425	225	223	223
q13	17766	2967	3005	2967
q14	267	243	241	241
q15	517	491	502	491
q16	495	375	375	375
q17	953	611	663	611
q18	7990	7529	7481	7481
q19	7906	1421	1376	1376
q20	697	319	309	309
q21	4955	3231	3215	3215
q22	356	286	284	284
Total cold run time: 118558 ms
Total hot run time: 39789 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4341	4251	4234	4234
q2	385	254	255	254
q3	3012	2958	2898	2898
q4	2029	1696	1782	1696
q5	5600	5543	5502	5502
q6	230	130	134	130
q7	2261	1890	1882	1882
q8	3258	3432	3412	3412
q9	8745	8841	8835	8835
q10	4119	3801	3935	3801
q11	605	516	486	486
q12	819	661	607	607
q13	17296	3139	3134	3134
q14	310	293	286	286
q15	544	488	497	488
q16	483	423	433	423
q17	1796	1511	1489	1489
q18	8168	7984	7878	7878
q19	1709	1521	1489	1489
q20	2083	1883	1867	1867
q21	5135	4782	4928	4782
q22	583	518	518	518
Total cold run time: 73511 ms
Total hot run time: 56091 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 175138 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 467bac43b4d8ac4934f3743e7863643fc696f54e, data reload: false

query1	903	370	362	362
query2	6431	1920	1916	1916
query3	6630	214	212	212
query4	28231	17502	17456	17456
query5	3821	480	482	480
query6	255	170	173	170
query7	4588	292	282	282
query8	242	194	182	182
query9	8515	2389	2344	2344
query10	431	280	276	276
query11	10887	10096	10069	10069
query12	111	91	81	81
query13	1632	359	364	359
query14	10238	7647	7617	7617
query15	222	164	172	164
query16	7648	316	311	311
query17	1828	552	525	525
query18	1872	285	292	285
query19	190	147	148	147
query20	91	79	82	79
query21	204	128	121	121
query22	4478	4191	4209	4191
query23	33895	33927	33888	33888
query24	11141	2901	2943	2901
query25	613	421	393	393
query26	1205	147	149	147
query27	2403	279	282	279
query28	7212	2078	2067	2067
query29	900	623	622	622
query30	254	159	150	150
query31	977	786	784	784
query32	102	80	55	55
query33	769	301	306	301
query34	1017	506	495	495
query35	683	582	578	578
query36	1121	983	1000	983
query37	150	80	81	80
query38	3054	2842	2839	2839
query39	907	840	862	840
query40	201	120	122	120
query41	46	45	43	43
query42	121	108	97	97
query43	504	476	463	463
query44	1132	727	717	717
query45	202	164	167	164
query46	1110	772	736	736
query47	1898	1811	1823	1811
query48	372	303	296	296
query49	856	425	434	425
query50	771	391	391	391
query51	6863	6801	6765	6765
query52	108	92	93	92
query53	367	299	280	280
query54	903	455	460	455
query55	78	75	76	75
query56	306	285	284	284
query57	1122	1067	1085	1067
query58	259	271	291	271
query59	2866	2785	2749	2749
query60	327	288	293	288
query61	121	170	97	97
query62	818	647	630	630
query63	318	287	281	281
query64	9428	2188	4533	2188
query65	3179	3092	3119	3092
query66	733	325	330	325
query67	15291	15085	14900	14900
query68	4467	545	541	541
query69	634	421	345	345
query70	1163	1136	1144	1136
query71	387	283	277	277
query72	7158	5973	5622	5622
query73	738	324	326	324
query74	6217	5770	5827	5770
query75	3374	2703	2714	2703
query76	2413	952	892	892
query77	462	304	304	304
query78	9675	9091	8896	8896
query79	2995	522	527	522
query80	2319	474	469	469
query81	597	218	224	218
query82	949	138	136	136
query83	289	166	168	166
query84	265	93	86	86
query85	1981	311	296	296
query86	480	308	319	308
query87	3370	3134	3155	3134
query88	4196	2369	2381	2369
query89	468	387	381	381
query90	1770	190	184	184
query91	131	105	170	105
query92	57	49	51	49
query93	3508	502	496	496
query94	1130	208	206	206
query95	400	311	322	311
query96	595	275	270	270
query97	3165	3053	3025	3025
query98	211	203	200	200
query99	1483	1264	1282	1264
Total cold run time: 283198 ms
Total hot run time: 175138 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.55 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 467bac43b4d8ac4934f3743e7863643fc696f54e, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.04	0.04
query3	0.22	0.04	0.04
query4	1.67	0.07	0.07
query5	0.52	0.49	0.50
query6	1.14	0.72	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.56	0.49	0.49
query10	0.56	0.55	0.54
query11	0.15	0.11	0.12
query12	0.14	0.12	0.13
query13	0.60	0.59	0.59
query14	0.75	0.78	0.77
query15	0.85	0.81	0.81
query16	0.36	0.37	0.35
query17	0.98	0.95	1.01
query18	0.22	0.22	0.21
query19	1.81	1.65	1.67
query20	0.02	0.01	0.01
query21	15.39	0.75	0.67
query22	4.43	6.83	1.92
query23	18.69	1.38	1.25
query24	2.16	0.22	0.22
query25	0.16	0.09	0.09
query26	0.30	0.21	0.22
query27	0.46	0.23	0.23
query28	13.30	1.01	1.00
query29	12.56	3.36	3.31
query30	0.25	0.06	0.05
query31	2.86	0.39	0.39
query32	3.29	0.47	0.46
query33	2.87	2.89	2.93
query34	17.18	4.38	4.37
query35	4.45	4.43	4.43
query36	0.64	0.46	0.47
query37	0.18	0.16	0.16
query38	0.16	0.16	0.15
query39	0.04	0.03	0.04
query40	0.15	0.12	0.11
query41	0.10	0.04	0.05
query42	0.06	0.04	0.04
query43	0.05	0.04	0.03
Total cold run time: 110.46 s
Total hot run time: 30.55 s

@w41ter w41ter merged commit 721ecbc into apache:master Jul 17, 2024
29 of 31 checks passed
dataroaring pushed a commit that referenced this pull request Jul 17, 2024
```
com.sleepycat.je.rep.MasterStateException: (JE 18.3.12) Cannot remove an active master
    at com.sleepycat.je.rep.util.ReplicationGroupAdmin.checkMember(ReplicationGroupAdmin.java:591)
    at com.sleepycat.je.rep.util.ReplicationGroupAdmin.removeMember(ReplicationGroupAdmin.java:315)
    at org.apache.doris.ha.BDBHA.removeDroppedMember(BDBHA.java:262)
    at org.apache.doris.catalog.Env.removeDroppedFrontends(Env.java:3077)
    at org.apache.doris.catalog.Env.transferToMaster(Env.java:1520)
    at org.apache.doris.catalog.Env.access$1300(Env.java:338)
    at org.apache.doris.catalog.Env$5.runOneCycle(Env.java:2792)
    at org.apache.doris.common.util.Daemon.run(Daemon.java:116)
```

* the bug is introduced by #35203
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.1-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants