Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DLedger mode metrics will display the old master data #36

Closed
maixiaohai opened this issue Oct 21, 2020 · 2 comments
Closed

DLedger mode metrics will display the old master data #36

maixiaohai opened this issue Oct 21, 2020 · 2 comments

Comments

@maixiaohai
Copy link
Contributor

Describe the bug
When restart cluster which deployed on DLedger mode, the master may change.But the broker tps metrics will always show the old one.

To Reproduce
1.restart cluster and the master broker machine change.
2.the exporter service not restart.

Expected behavior
only new metrics data show, or the old data change to 0.

Screenshots
tps dashboard
image

some exporter data:

rocketmq_broker_tps{cluster="c3cloudsrv-common-rocketmq",brokerIP="Amachine",broker="c3cloudsrv-common-rocketmq-raft0",} 33441.97
rocketmq_broker_tps{cluster="c3cloudsrv-common-rocketmq",brokerIP="Bmachine",broker="c3cloudsrv-common-rocketmq-raft1",} 13892.3
rocketmq_broker_tps{cluster="c3cloudsrv-common-rocketmq",brokerIP="",broker="c3cloudsrv-common-rocketmq-raft1",} 0.0
rocketmq_broker_tps{cluster="c3cloudsrv-common-rocketmq",brokerIP="Cmachine",broker="c3cloudsrv-common-rocketmq-raft1",} 0.0
rocketmq_broker_tps{cluster="c3cloudsrv-common-rocketmq",brokerIP="Dmachine",broker="c3cloudsrv-common-rocketmq-raft1",} 12083.87
rocketmq_broker_tps{cluster="c3cloudsrv-common-rocketmq",brokerIP="Emachine",broker="c3cloudsrv-common-rocketmq-raft0",} 12083.2
rocketmq_broker_tps{cluster="c3cloudsrv-common-rocketmq",brokerIP="Fmachine",broker="c3cloudsrv-common-rocketmq-raft0",} 12270.9

there is only two master in the cluster, but the metrics is more than 2.

  • rocketmq version 4.7.0

  • exporter version 0.0.2

@maixiaohai
Copy link
Contributor Author

@francisoliverlee May I submit a pr to fix this?

@maixiaohai
Copy link
Contributor Author

After fix the broker stats always return 2 master data

rocketmq_broker_tps{cluster="c3cloudsrv-common-rocketmq",brokerIP="A",broker="c3cloudsrv-common-rocketmq-raft1",} 11848.72
rocketmq_broker_tps{cluster="c3cloudsrv-common-rocketmq",brokerIP="B",broker="c3cloudsrv-common-rocketmq-raft0",} 11847.95
rocketmq_broker_qps{cluster="c3cloudsrv-common-rocketmq",brokerIP="A",broker="c3cloudsrv-common-rocketmq-raft1",} 23774.55
rocketmq_broker_qps{cluster="c3cloudsrv-common-rocketmq",brokerIP="B",broker="c3cloudsrv-common-rocketmq-raft0",} 23340.07

cserwen pushed a commit to cserwen/rocketmq-exporter that referenced this issue May 17, 2022
Summary:
Ref T16363
issue link : apache#36

Test Plan: e2e

Reviewers: #rocketmq, yongxing, zhangyang21, wangyifan8

Reviewed By: #rocketmq, zhangyang21

Subscribers: cloud-messagequeue

Maniphest Tasks: T16363

Differential Revision: https://phabricator.d.xiaomi.net/D244797
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants