-
Notifications
You must be signed in to change notification settings - Fork 212
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Empty external metrics list will fail the client-go CachedDiscoveryClient caching and make helm upgrade resending api call #729
Comments
From grafana/tanka#1076, it might be related to #690 which bumped client-go version to fix vulnerabilities. |
Could you try this image: gcr.io/gke-release/custom-metrics-stackdriver-adapter:v0.14.2-gke.0 ? |
Helm recently upgraded from client-go v0.29 to v0.30 in helm/helm#13021. |
Adding a fake metric here #743 |
Hi, we will release a new image version soon. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The
custom-metrics-stackdriver-adapter
returns an empty list inListAllExternalMetrics
(e.g.,kubectl get --raw '/apis/external.metrics.k8s.io/v1beta1'
). Although this allows the HPA to function, it causes side effects in other Kubernetes ecosystem tools, such as Helm.k8s-stackdriver/custom-metrics-stackdriver-adapter/pkg/adapter/provider/provider.go
Lines 354 to 358 in d0e6643
We encountered an issue where
helm upgrade
sends over 400 API requests. This is because Helm relies on the client-goCachedDiscoveryClient
for validation.CachedDiscoveryClient
caches responses and saves them to a file if the response is not empty. When thecustom-metrics-stackdriver-adapter
returns an empty list forexternal.metrics.k8s.io/v1beta1
, it is never cached, causing theCachedDiscoveryClient
to continuously refresh all API resources (not justexternal.metrics.k8s.io/v1beta1
). This results in a high number of API requests being sent.https://github.com/kubernetes/client-go/blob/c7396197f39ed43fb0369c054360fcb989d65c88/discovery/cached/disk/cached_discovery.go#L82C35-L82C65
https://github.com/kubernetes/client-go/blob/c7396197f39ed43fb0369c054360fcb989d65c88/discovery/cached/memory/memcache.go#L104
The text was updated successfully, but these errors were encountered: