Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Duplicate metrics lodestar_gossip_block_elapsed_time_till_received_bucket #6118

Closed
gathecageorge opened this issue Nov 21, 2023 · 1 comment · Fixed by #6120
Closed

Duplicate metrics lodestar_gossip_block_elapsed_time_till_received_bucket #6118

gathecageorge opened this issue Nov 21, 2023 · 1 comment · Fixed by #6120
Labels
meta-bug Issues that identify a bug and require a fix.

Comments

@gathecageorge
Copy link

Describe the bug

The client is exposing duplicate metrics for the metric lodestar_gossip_block_elapsed_time_till_received_bucket making it result in alot of prometheus errors with duplicate metric when sending to grafana cloud.

A curl of the client metrics url eg curl lodestar:8008/metrics | grep lodestar_gossip_block_elapsed_time_till_received shows duplicate metrics for this as shown below with output

# HELP lodestar_gossip_block_elapsed_time_till_received Time elapsed between block slot time and the time block received via gossip
# TYPE lodestar_gossip_block_elapsed_time_till_received histogram
lodestar_gossip_block_elapsed_time_till_received_bucket{le="0.5"} 668
lodestar_gossip_block_elapsed_time_till_received_bucket{le="1"} 15027
lodestar_gossip_block_elapsed_time_till_received_bucket{le="2"} 63776
lodestar_gossip_block_elapsed_time_till_received_bucket{le="4"} 84614
lodestar_gossip_block_elapsed_time_till_received_bucket{le="6"} 84783
lodestar_gossip_block_elapsed_time_till_received_bucket{le="12"} 84845
lodestar_gossip_block_elapsed_time_till_received_bucket{le="+Inf"} 84863
lodestar_gossip_block_elapsed_time_till_received_sum 137942.57899308205
lodestar_gossip_block_elapsed_time_till_received_count 84863
# HELP lodestar_gossip_block_elapsed_time_till_received Time elapsed between block slot time and the time block received via gossip
# TYPE lodestar_gossip_block_elapsed_time_till_received histogram
lodestar_gossip_block_elapsed_time_till_received_bucket{le="0.5"} 0
lodestar_gossip_block_elapsed_time_till_received_bucket{le="1"} 0
lodestar_gossip_block_elapsed_time_till_received_bucket{le="2"} 0
lodestar_gossip_block_elapsed_time_till_received_bucket{le="4"} 0
lodestar_gossip_block_elapsed_time_till_received_bucket{le="6"} 0
lodestar_gossip_block_elapsed_time_till_received_bucket{le="12"} 0
lodestar_gossip_block_elapsed_time_till_received_bucket{le="+Inf"} 0
lodestar_gossip_block_elapsed_time_till_received_sum 0
lodestar_gossip_block_elapsed_time_till_received_count 0

Expected behavior

Each metric should be unique. Thus should only show one of in the output.

Steps to reproduce

No response

Additional context

No response

Operating system

Linux

Lodestar version or commit hash

v1.12.0/7000473

@gathecageorge gathecageorge added the meta-bug Issues that identify a bug and require a fix. label Nov 21, 2023
@dapplion
Copy link
Contributor

Thanks for reporting @gathecageorge !

@wemeetagain can this be due to manually merging the txt returns of the multiple prom-client instances?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
meta-bug Issues that identify a bug and require a fix.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants