Flux custom metrics monitoring broken in 2.12 #2386
Labels
kind/bug
Categorizes issue or PR as related to a bug.
triage/accepted
Indicates an issue or PR is ready to be actively worked on.
We are using Flux CD and have set up custom metrics for monitoring. The config was copied from the flux2-monitoring-examples.
What happened:
After updating to 2.12.0 we are missing several gotk_resource_info metrics.
The remaining gotk_resource_info metrics are exclusively for the custom resource helmreleases.helm.toolkit.fluxcd.io:
What you expected to happen:
We should also see metrics for other custom resources, like this:
That's how it was in 2.11 and downgrading to 2.11.0 restores these metrics.
Workaround & possible cause:
It seems the issue is caused because the kube-state-metrics config want's to compile all CRD metrics into a single metric name (gotk_resource_info) and that is no longer possible with 2.12.
If we use a dedicated metric name for each CRD type, eg. gotk_resource_info for helmreleases.helm.toolkit.fluxcd.io and gotk_resource_info2 for helmcharts.source.toolkit.fluxcd.io, the metrics show up correctly:
Since I don't see any mention of this in the release logs I assume this is not by design. If it was KSM should output an error message if a config with non-unique names is supplied.
Environment:
kubectl version
): v1.29.3-eks-adc7111The text was updated successfully, but these errors were encountered: