-
What is the cost of calling the metrics endpoint? Does it calculate the value of the metrics when we call the endpoint or when the metrics are updated? |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments
-
It heavily depends on the collector implementation. HTTP handler calls
|
Beta Was this translation helpful? Give feedback.
-
I think @bwplotka described it quite well. Just as a different perspective: What's really optimized in this library are the calls that happen for instrumentation, like e.g. Calling the metrics endpoint is expected to happen a few times per minute, in rare cases it might happen as often as about once per second. But even then you can see that we are orders of magnitude away from the instrumentation calls above. Copying the metrics value out of the instrumentation primitives (e.g. a counter, a histogram, …) is still quite fast compared to the heavy-lifting that a call to the metrics endpoint implies anyway (network access, HTTP handling and such). The most "needless" work is currently done because this library was designed around protobuf, but most scrapers use the text format. Therefore, the code usually takes the detour of rendering everything as protobuf (which is, on top of everything else, quite expensive because it is using the standard protobuf library rather than one of the more efficient alternative implementation) to then convert the protobuf into the text format. That's where some projects with rather extreme use cases (like https://github.com/kubernetes/kube-state-metrics ) have run into a wall and decided to handcode the metrics exposition in a way perfectly tailored to their use case. Finally, if collecting the metrics themselves is a lot of work, then that's usually the bottleneck, shadowing everything else mentioned so far. @bwplotka has discussed that above already. As a rule of thumb, if you instrument a binary that has a "main job", and you just want to monitor that main job, you will rarely run into the problem that the instrumentation eats a significant portion of your resources. The more your binary focuses on metrics collection itself (like exporters, or binaries that collect metrics from many other sources, e.g. KSM or cadvisor), the tradeoffs change and you might start to worry. |
Beta Was this translation helpful? Give feedback.
It heavily depends on the collector implementation. HTTP handler calls
Gather() ([]*dto.MetricFamily, error)
on yourprometheus.Registry
, which callsCollect(ch chan<- Metric)
on all your registered collectors. You can register many types of collectors, which might have different characteristics:Collect
will then mostly just copy the value to new protobuf object. You update those metrics in separate goroutines usingInc()
,Add()
, observe and so on....Func
family of metrics e.gNewGaugeFunc
are invoking providing function to on every call. Users can provide expensive computation …