Duplicate sample for HPA metrics using multiple external metrics with same metric name #2405
Labels
kind/bug
Categorizes issue or PR as related to a bug.
triage/accepted
Indicates an issue or PR is ready to be actively worked on.
What happened:
Upgraded to prometheus 2.52 which now is more strict about duplicates: prometheus/prometheus#14089 (similar to #2390 )
Have an HPA that looks like this:
and this causes KSM to attempt to produce duplicate metrics because it expects the metric name to be unique across target metrics which is not true (the selectors does that).
debug logs from prometheus:
and this causes the standard mixin alert
PrometheusDuplicateTimestamps
to continuously trigger.What you expected to happen:
No duplicate metrics. I'm guessing the temporary solution is to
drop
action onkube_horizontalpodautoscaler_status_target_metric|kube_horizontalpodautoscaler_spec_target_metric
, but figured it might be worth raising an issue here for others.How to reproduce it (as minimally and precisely as possible):
An HPA like above, and some way to use external metrics in HPAs (prometheus-adapter or keda I guess) and I expectdefault kube-state-metrics scraping of hpa metrics.
Anything else we need to know?:
Environment:
resources flag from ksm:
The text was updated successfully, but these errors were encountered: