v1.11.8 metric misspelling.

[mark-liu]

Describe the bug
In v1.11.8 the metric avalanche_health_checks_failing has been misspelt avalanche_health_health_checks_failing

To Reproduce
Just curl for the metric.

Expected behavior
It should be named avalanche_health_checks_failing

Screenshots

Logs
If applicable, please include the relevant logs that indicate a problem and/or the log directory of your node. By default, this can be found at ~/.avalanchego/logs/.

Metrics
If applicable, please include any metrics gathered from your node to assist us in diagnosing the problem.

Operating System
Which OS you used to reveal the bug.
Linux amd64

[StephenButtolph]

This wasn't actually completely unintended... There are 3 different types of health checks:

1. health
2. liveness
3. readiness

So, the namespace of these metrics are: avalanche_health and then we add:

1. _health_checks_failing
2. _liveness_checks_failing
3. _readiness_checks_failing

To make:

# HELP avalanche_health_health_checks_failing number of currently failing health checks
# TYPE avalanche_health_health_checks_failing gauge
avalanche_health_health_checks_failing{tag="11111111111111111111111111111111LpoYY"} 3
avalanche_health_health_checks_failing{tag="all"} 3
avalanche_health_health_checks_failing{tag="application"} 2
# HELP avalanche_health_liveness_checks_failing number of currently failing health checks
# TYPE avalanche_health_liveness_checks_failing gauge
avalanche_health_liveness_checks_failing{tag="all"} 0
avalanche_health_liveness_checks_failing{tag="application"} 0
# HELP avalanche_health_readiness_checks_failing number of currently failing health checks
# TYPE avalanche_health_readiness_checks_failing gauge
avalanche_health_readiness_checks_failing{tag="all"} 1
avalanche_health_readiness_checks_failing{tag="application"} 1

Do you (@mark-liu) think it would make more sense to change the inner names to themselves be labels? Which would result in:

# HELP avalanche_health_checks_failing number of currently failing health checks
# TYPE avalanche_health_checks_failing gauge
avalanche_health_checks_failing{check="health",tag="11111111111111111111111111111111LpoYY"} 1
avalanche_health_checks_failing{check="health",tag="all"} 1
avalanche_health_checks_failing{check="health",tag="application"} 1
avalanche_health_checks_failing{check="liveness",tag="all"} 0
avalanche_health_checks_failing{check="liveness",tag="application"} 0
avalanche_health_checks_failing{check="readiness",tag="all"} 1
avalanche_health_checks_failing{check="readiness",tag="application"} 1

[mark-liu]

@StephenButtolph labels is a nice approach, clean and lines up with prom best practice and allows easy aggregation.