Implementation for KEP-4831: Control VPA eviction behavior based on scaling direction and resource

[voelzmo]

What type of PR is this?

/kind feature

What this PR does / why we need it:

• Implements ScalingDirectionPodEvictionAdmission, which controls Pod admission to eviction based on EvictionRequirements defined in a VPA's .spec.updatePolicy.evictionRequirements
• Makes Updater use the new ScalingDirectionPodEvictionAdmission instead of DefaultPodEvictionAdmission
• Change e2e tests to use VPA builder that's already used in the unit tests
  • Changes to VPA builder to allow for multiple containers and more detailed settings per container

Which issue(s) this PR fixes:

Implements #4831
API PR done in #5176

Special notes for your reviewer:

Depends on the API changes in #5176

Does this PR introduce a user-facing change?

Added `.updatePolicy.evictionRequirements` to the VPA spec allowing to constrain Pod eviction based on scaling direction (scaling up or scaling down) and resource (cpu or memory).

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

- [AEP]: https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler/enhancements/4831-control-eviction-behavior

[voelzmo]

@jbartosik Can you please take a look? Thanks!

[voelzmo]

@jbartosik Can you try to give this a review, please? Thanks!

[voelzmo]

Force-pushed an update:

• rebasing on most recent state of API for KEP-4831: Control VPA eviction behavior based on scaling direction and resource #5176 and adapting the test cases to Replace BuildTestContainer with use of builder #5896
• removing the OrEqual ChangeRequirements and test cases as discussed in Adjust Eviction Control AEP for in-place updates #5754 (comment)

[jbartosik]

I'm not reviewing this here; I'm instead looking at version in #5176 and assuming we'll sync this to not make ny changes from there

[jbartosik]

I'm reviewing this in #5176 and presume we'll not change this file in this PR

[voelzmo]

force-pushed an update:

• rebased on current master that includes the merged API PR
• refactorings to expose VPA builder in e2e tests
• modifications to VPA builder such that it can replace the previous SetupVPAForNHamstersWithMinReplicas functions

Open points:

• one TODO in the VPA builder
• Simplifications/inlining for the SetupVPAForNHamstersWithMinReplicas functions

I can get to that next week

[voelzmo]

Next set of updates, containing the first inline of the test setup helper functions.
Still todo:

• inline SetupVPA helper function and have tests directly call the builder to create VPA objects

[voelzmo]

@jbartosik rebased on master, please take another look. Unfortunately, the bot removes LGTM labels on new changes ;(

[voelzmo]

@jbartosik can we merge this before the next rebase is necessary? I’m kind of afraid having to go through a lot of files again when other changes get merged 😱
Thanks!

[jbartosik]

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jbartosik, voelzmo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• vertical-pod-autoscaler/OWNERS [jbartosik]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment