[Helm] Make HPA behavior configurable

[amirschw]

What this PR does / why we need it:

Allow configuring the controller HPA behavior. This is already possible in the KEDA advanced settings.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Which issue/s this PR fixes

How Has This Been Tested?

1. Install the chart with the default settings:

helm upgrade --install ingress-nginx charts/ingress-nginx \
	--set controller.autoscaling.enabled=true

2. Make sure that the HPA is created without behavior:

kubectl neat get -- hpa.v2beta2.autoscaling ingress-nginx-controller -oyaml

output:

apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
  annotations:
    meta.helm.sh/release-name: ingress-nginx
    meta.helm.sh/release-namespace: default
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/version: 0.46.0
    helm.sh/chart: ingress-nginx-3.33.0
  name: ingress-nginx-controller
  namespace: default
spec:
  maxReplicas: 11
  metrics:
  - resource:
      name: memory
      target:
        averageUtilization: 50
        type: Utilization
    type: Resource
  - resource:
      name: cpu
      target:
        averageUtilization: 50
        type: Utilization
    type: Resource
  minReplicas: 1
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: ingress-nginx-controller

3. Upgrade the release with behavior:

helm upgrade --install ingress-nginx charts/ingress-nginx \
	--set controller.autoscaling.enabled=true \
	--set controller.autoscaling.behavior.scaleDown.stabilizationWindowSeconds=300

4. Make sure that the HPA now has behavior:

kubectl neat get -- hpa.v2beta2.autoscaling ingress-nginx-controller -oyaml

output:

apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
  annotations:
    meta.helm.sh/release-name: ingress-nginx
    meta.helm.sh/release-namespace: default
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/version: 0.46.0
    helm.sh/chart: ingress-nginx-3.33.0
  name: ingress-nginx-controller
  namespace: default
spec:
  behavior:
    scaleDown:
      policies:
      - periodSeconds: 15
        type: Percent
        value: 100
      selectPolicy: Max
      stabilizationWindowSeconds: 300
    scaleUp:
      policies:
      - periodSeconds: 15
        type: Pods
        value: 4
      - periodSeconds: 15
        type: Percent
        value: 100
      selectPolicy: Max
      stabilizationWindowSeconds: 0
  maxReplicas: 11
  metrics:
  - resource:
      name: memory
      target:
        averageUtilization: 50
        type: Utilization
    type: Resource
  - resource:
      name: cpu
      target:
        averageUtilization: 50
        type: Utilization
    type: Resource
  minReplicas: 1
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: ingress-nginx-controller

Checklist:

• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I've read the CONTRIBUTION guide
• I have added tests to cover my changes.
• All new and existing tests passed.

[k8s-ci-robot]

Hi @amirschw. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[rikatz]

/area helm

[rikatz]

/ok-to-test

[tao12345666333]

/assign

[rikatz]

Hello and thanks for your PR. 2 comments here:

• Rebase seems problematic here (with Merge commit message), can you please do a proper rebase?
• Please add some CI tests for this (ci directory) with values for this case

Thanks!

[k8s-ci-robot]

@amirschw: This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[amirschw]

@rikatz done, thank you

[rikatz]

/lgtm
/approve
Thanks!

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: amirschw, rikatz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• charts/ingress-nginx/OWNERS [rikatz]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment