Replies: 6 comments
-
Here is the code snippet I am using to provide prometheus handler with gofiber.
|
Beta Was this translation helpful? Give feedback.
-
The problematic metric is a custom GaugeVec registerd by the following code.
getLabels returns the labels supported. I see the metric being collected fine for the first few scrapes and suddenly I start seeing 500 server error on the scrapes. The error thrown is like this.
|
Beta Was this translation helpful? Give feedback.
-
Hi, It looks like your instrumentation collected the same metric twice from the all metrics (e.g. gauges you showed) and custom collectors in your application. Looks like you are using the default registry ( It's unlikely the presented code is responsible for the problem. The When My recommendation is to NEVER use default registry as it causes the problems like here, where we think we know about all the collectors registers, but in fact any package might we import might implicitly register something funky. I suggest creating your own registry import (
“github.com/gofiber/adaptor/v2"
“github.com/prometheus/client_golang/prometheus/promhttp"
)
func MetricsRoutes(api fiber.Router, reg prometheus.Gatherer) {
api.Get(“/metrics”, adaptor.HTTPHandler(promhttp.HandlerFor(reg, nil)))
} func RegisterMetric(metricNames []string, reg prometheus.Gatherer) {
for i, metricName := range metricNames {
metricGauge := promauto.With(reg).NewGaugeVec(prometheus.GaugeOpts{
Name: metricName,
}, getLabels(metricName))
metricsList[i] = metricName
metrics[i] = metricGauge
}
} With this you will know explicitly that it is the only collector handler is using. Otherwise you have to track exactly all collectors and check who is creating TODO(#1101): Document this path |
Beta Was this translation helpful? Give feedback.
-
In this code snippet
I guess instead of passing a Gatherer, it needs to accept a prometheus.Registerer type |
Beta Was this translation helpful? Give feedback.
-
Hi, I made the changes as suggested and still seeing errors from the prometheus client. There are multiple occurrence of such errors.
|
Beta Was this translation helpful? Give feedback.
-
@geet, you are using Fiber, in Fiber http parameters, body unsafe and mutable. Set Immutable: true in fiber.Config. |
Beta Was this translation helpful? Give feedback.
-
I have instrumented custom prometheus metrics from my application. I have configured prometheus to scrape the target. I have started seeing 500 error in the prometheus and due to that the scrape failed. I tried to manually hit the /metrics endpoint in my application and got the error saying
It works initially but after a few minutes I start seeing the error.
I don't think it should be considered a duplicate metric but if the metric with same label and values are added,then it should increase the count.
go client version: github.com/prometheus/client_golang v1.12.2
Beta Was this translation helpful? Give feedback.
All reactions