apiserver received an error "exactly one scaled object should match label "

[wiseelf]

Hello!

Actual Behavior

Right after the installation keda-metrics-apiserver shows a lot of errors in the logs.
I haven't configured any ScaledObjects yet.

E1119 06:53:11.302715       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"exactly one scaled object should match label "}
E1119 06:53:17.411988       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"exactly one scaled object should match label "}
E1119 06:53:20.586987       1 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:"exactly one scaled object should match label "}

Why does this happen? Is it critical?

Expected Behavior

No errors in the log

Steps to Reproduce the Problem

1. kubectl apply -f https://github.com/kedacore/keda/releases/download/v2.0.0/keda-2.0.0.yaml
2. kubectl -n keda logs -f keda-metrics-apiserver-65c46b9b8c-gd45c

Specifications

• KEDA Version: 2.0.0
• Platform & Version: AWS EKS 1.17
• Kubernetes Version: 1.17
• Scaler(s): Have no any yet

[zroubalik]

Do you have anything else installed on the cluster that could eventually use External Metrics? (Datadog, k8s-prometheus-adapter etc?)

[wiseelf]

Yes, I have the prometheus-adapter installed.

[zroubalik]

Both projects are using the same external metrics provider, they cannot be installed on the cluster at the same time. With your KEDA installation, you have just replaced the Metrics Server from prometheus-adapter, which is not working currently.

The errors you see are requests from prometheus-adapter, which is polling a wrong Metrics server.

Unfortunately this is a limitiation of kubernetes, see this: kubernetes-sigs/custom-metrics-apiserver#70

[wiseelf]

got it. Thank you.

[CarloColumna]

Hey @wiseelf, did it get resolved for you after removing prometheus-adapter? We are encountering the same errors even after removing prometheus-adapter. We dont have cloudwatch-adapter either.

[wiseelf]

@CarloColumna yes, removing prometheus-adapter helped.

[CarloColumna]

@CarloColumna yes, removing prometheus-adapter helped.

Thanks for that @wiseelf !

[CarloColumna]

If someone else ends up in here, I managed to sort out the issue by deleting all the old HPAs that were created by k8s-cloudwatch-adapter. We used to run cloudwatch-adapter but have since removed in favour of KEDAI missed one hpa which was polling the keda-metrics-server resulting with this issue. I turned on debug mode for the keda-metrics-server pod and managed to pinpoint this hpa that I missed.