Report

When I submit a ScaledObject that includes both CPU utilization triggers and other resource triggers, the KEDA operator may continuously update hpa and never stops.

Expected Behavior

Only one update occurs

Actual Behavior

Continuous HPA updates

Steps to Reproduce the Problem

1. Create a ScaledObject with both CPU utilization triggers and other resource triggers.
2. Ensure the CPU utilization trigger is not the last one in the ScaledObject.
3. Use a Kubernetes cluster with a version below 1.27 (e.g., 1.26).
4. Observe the continuous triggering of "Found difference in the HPA spec according to ScaledObject" by the KEDA operator.

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: uucloud-test-so
  namespace: default
spec:
  scaleTargetRef:
    name: uucloud-test  # your deployment
  minReplicaCount:  1                                     
  maxReplicaCount:  2                                   
  triggers:
    - metadata:
        value: "50"
      metricType: Utilization
      type: cpu
    - metadata:
        value: "50"
      metricType: Utilization
      type: memory

Logs from KEDA operator

No response

KEDA Version

2.12.1

Kubernetes Version

< 1.27

Scaler Details

Resource

Anything else?

This issue is fundamentally the same as the one encountered in kubernetes/kubernetes#74099. The root cause is Kubernetes reordering spec.metrics, the HPA v1 conversion logic causing the position of the CPU utilization HPA metric to be adjusted.

When creating or updating an HPA, the conversion logic in these segments of code link1 and link2 converts the first CPU utilization metric into the HPA v1 metric (if there are multiple CPU utilization triggers, the others are lost), and stores the remaining metrics in annotations. When converting back from HPA v1 to HPA, it appends the CPU utilization metric to the end (link3).

This results in a situation where, if the ScaledObject has multiple resource triggers and one of them is a CPU utilization trigger, the final HPA will always have the CPU utilization trigger at the end. Additionally, if there are multiple CPU utilization triggers, only one will remain (though having multiple CPU utilization triggers in one HPA configuration might seem to have little practical value...). This causes the KEDA operator to continuously detect differences and persistently update the HPA.

In Kubernetes 1.27 and later, this issue is resolved because the autoscaling v1 schema is deprioritized behind v2, meaning it no longer defaults to converting to HPA v1. The relevant change is shown below:

diff --git a/pkg/apis/autoscaling/install/install.go b/pkg/apis/autoscaling/install/install.go
index 3740aee3155..424fc5ce85d 100644
--- a/pkg/apis/autoscaling/install/install.go
+++ b/pkg/apis/autoscaling/install/install.go
@@ -40,6 +40,5 @@ func Install(scheme *runtime.Scheme) {
        utilruntime.Must(v2.AddToScheme(scheme))
        utilruntime.Must(v2beta1.AddToScheme(scheme))
        utilruntime.Must(v1.AddToScheme(scheme))
-       // TODO: move v2 to the front of the list in 1.24
-       utilruntime.Must(scheme.SetVersionPriority(v1.SchemeGroupVersion, v2.SchemeGroupVersion, v2beta1.SchemeGroupVersion, v2beta2.SchemeGroupVersion))
+       utilruntime.Must(scheme.SetVersionPriority(v2.SchemeGroupVersion, v1.SchemeGroupVersion, v2beta1.SchemeGroupVersion, v2beta2.SchemeGroupVersion))