Add LiveMigrateIfPossible to cluster level EvictionStrategy (#2423)

Add `LiveMigrateIfPossible` as a valid option for
cluster level EvictionStrategy.
`LiveMigrateIfPossible` will cause a VM to be live migrated,
if possible, otherwise it will be directly evicted in order to complete the node drain with no manual actions.

Since in general we privilege business continuity
of our workload over simplicity of platform upgrades,
`LiveMigrate` will continue being our default value.
This will be consistent with the default value on /spec/workloadUpdateStrategy/workloadUpdateMethods
where we currently have `{"LiveMigrate"}` as our default
and not `{"LiveMigrate", "Evict"}` exactly in order to prevent
unexpected implicit shutdown due to our upgrades.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2219785

Signed-off-by: Simone Tiraboschi <stirabos@redhat.com>

api/v1beta1/hyperconverged_types.go

@@ -186,8 +186,14 @@ type HyperConvergedSpec struct {
 
 	// EvictionStrategy defines at the cluster level if the VirtualMachineInstance should be
 	// migrated instead of shut-off in case of a node drain. If the VirtualMachineInstance specific
-	// field is set it overrides the cluster level one. Defaults to LiveMigrate with multiple worker nodes, None on single worker clusters.
-	// +kubebuilder:validation:Enum=None;LiveMigrate;External
+	// field is set it overrides the cluster level one.
+	// Allowed values:
+	// - `None` no eviction strategy at cluster level.
+	// - `LiveMigrate` migrate the VM on eviction; a not live migratable VM with no specific strategy will block the drain of the node util manually evicted.
+	// - `LiveMigrateIfPossible` migrate the VM on eviction if live migration is possible, otherwise directly evict.
+	// - `External` block the drain, track eviction and notify an external controller.
+	// Defaults to LiveMigrate with multiple worker nodes, None on single worker clusters.
+	// +kubebuilder:validation:Enum=None;LiveMigrate;LiveMigrateIfPossible;External
 	// +optional
 	EvictionStrategy *v1.EvictionStrategy `json:"evictionStrategy,omitempty"`
 

config/crd/bases/hco.kubevirt.io_hyperconvergeds.yaml

@@ -1040,14 +1040,21 @@ spec:
                   are eventually going to consume the new default RuntimeClass.
                 type: string
               evictionStrategy:
-                description: EvictionStrategy defines at the cluster level if the
+                description: 'EvictionStrategy defines at the cluster level if the
                   VirtualMachineInstance should be migrated instead of shut-off in
                   case of a node drain. If the VirtualMachineInstance specific field
-                  is set it overrides the cluster level one. Defaults to LiveMigrate
-                  with multiple worker nodes, None on single worker clusters.
+                  is set it overrides the cluster level one. Allowed values: - `None`
+                  no eviction strategy at cluster level. - `LiveMigrate` migrate the
+                  VM on eviction; a not live migratable VM with no specific strategy
+                  will block the drain of the node util manually evicted. - `LiveMigrateIfPossible`
+                  migrate the VM on eviction if live migration is possible, otherwise
+                  directly evict. - `External` block the drain, track eviction and
+                  notify an external controller. Defaults to LiveMigrate with multiple
+                  worker nodes, None on single worker clusters.'
                 enum:
                 - None
                 - LiveMigrate
+                - LiveMigrateIfPossible
                 - External
                 type: string
               featureGates:

controllers/operands/kubevirt_test.go

@@ -2738,7 +2738,7 @@ Version: 1.2.3`)
 				Expect(err).ToNot(HaveOccurred())
 
 				By("Modify HCO's eviction strategy configuration")
-				evictionStrategyExternal := kubevirtcorev1.EvictionStrategyExternal
+				evictionStrategyExternal := kubevirtcorev1.EvictionStrategyLiveMigrateIfPossible
 				hco.Spec.EvictionStrategy = &evictionStrategyExternal
 
 				cl := commontestutils.InitClient([]client.Object{hco, existingResource})
@@ -2761,7 +2761,7 @@ Version: 1.2.3`)
 
 				Expect(foundResource.Spec.Configuration.EvictionStrategy).ToNot(BeNil())
 				foundEvictionStrategy := *foundResource.Spec.Configuration.EvictionStrategy
-				Expect(foundEvictionStrategy).To(Equal(kubevirtcorev1.EvictionStrategyExternal))
+				Expect(foundEvictionStrategy).To(Equal(kubevirtcorev1.EvictionStrategyLiveMigrateIfPossible))
 
 				Expect(req.Conditions).To(BeEmpty())
 			})

deploy/crds/hco00.crd.yaml

@@ -1040,14 +1040,21 @@ spec:
                   are eventually going to consume the new default RuntimeClass.
                 type: string
               evictionStrategy:
-                description: EvictionStrategy defines at the cluster level if the
+                description: 'EvictionStrategy defines at the cluster level if the
                   VirtualMachineInstance should be migrated instead of shut-off in
                   case of a node drain. If the VirtualMachineInstance specific field
-                  is set it overrides the cluster level one. Defaults to LiveMigrate
-                  with multiple worker nodes, None on single worker clusters.
+                  is set it overrides the cluster level one. Allowed values: - `None`
+                  no eviction strategy at cluster level. - `LiveMigrate` migrate the
+                  VM on eviction; a not live migratable VM with no specific strategy
+                  will block the drain of the node util manually evicted. - `LiveMigrateIfPossible`
+                  migrate the VM on eviction if live migration is possible, otherwise
+                  directly evict. - `External` block the drain, track eviction and
+                  notify an external controller. Defaults to LiveMigrate with multiple
+                  worker nodes, None on single worker clusters.'
                 enum:
                 - None
                 - LiveMigrate
+                - LiveMigrateIfPossible
                 - External
                 type: string
               featureGates:

deploy/index-image/community-kubevirt-hyperconverged/1.10.0/manifests/hco00.crd.yaml

@@ -1040,14 +1040,21 @@ spec:
                   are eventually going to consume the new default RuntimeClass.
                 type: string
               evictionStrategy:
-                description: EvictionStrategy defines at the cluster level if the
+                description: 'EvictionStrategy defines at the cluster level if the
                   VirtualMachineInstance should be migrated instead of shut-off in
                   case of a node drain. If the VirtualMachineInstance specific field
-                  is set it overrides the cluster level one. Defaults to LiveMigrate
-                  with multiple worker nodes, None on single worker clusters.
+                  is set it overrides the cluster level one. Allowed values: - `None`
+                  no eviction strategy at cluster level. - `LiveMigrate` migrate the
+                  VM on eviction; a not live migratable VM with no specific strategy
+                  will block the drain of the node util manually evicted. - `LiveMigrateIfPossible`
+                  migrate the VM on eviction if live migration is possible, otherwise
+                  directly evict. - `External` block the drain, track eviction and
+                  notify an external controller. Defaults to LiveMigrate with multiple
+                  worker nodes, None on single worker clusters.'
                 enum:
                 - None
                 - LiveMigrate
+                - LiveMigrateIfPossible
                 - External
                 type: string
               featureGates:

deploy/olm-catalog/community-kubevirt-hyperconverged/1.10.0/manifests/hco00.crd.yaml

@@ -1040,14 +1040,21 @@ spec:
                   are eventually going to consume the new default RuntimeClass.
                 type: string
               evictionStrategy:
-                description: EvictionStrategy defines at the cluster level if the
+                description: 'EvictionStrategy defines at the cluster level if the
                   VirtualMachineInstance should be migrated instead of shut-off in
                   case of a node drain. If the VirtualMachineInstance specific field
-                  is set it overrides the cluster level one. Defaults to LiveMigrate
-                  with multiple worker nodes, None on single worker clusters.
+                  is set it overrides the cluster level one. Allowed values: - `None`
+                  no eviction strategy at cluster level. - `LiveMigrate` migrate the
+                  VM on eviction; a not live migratable VM with no specific strategy
+                  will block the drain of the node util manually evicted. - `LiveMigrateIfPossible`
+                  migrate the VM on eviction if live migration is possible, otherwise
+                  directly evict. - `External` block the drain, track eviction and
+                  notify an external controller. Defaults to LiveMigrate with multiple
+                  worker nodes, None on single worker clusters.'
                 enum:
                 - None
                 - LiveMigrate
+                - LiveMigrateIfPossible
                 - External
                 type: string
               featureGates:

docs/api.md

@@ -191,7 +191,7 @@ HyperConvergedSpec defines the desired state of HyperConverged
 | tektonPipelinesNamespace | TektonPipelinesNamespace defines namespace in which example pipelines will be deployed. If unset, then the default value is the operator namespace. | *string |  | false |
 | tektonTasksNamespace | TektonTasksNamespace defines namespace in which tekton tasks will be deployed. If unset, then the default value is the operator namespace. | *string |  | false |
 | kubeSecondaryDNSNameServerIP | KubeSecondaryDNSNameServerIP defines name server IP used by KubeSecondaryDNS | *string |  | false |
-| evictionStrategy | EvictionStrategy defines at the cluster level if the VirtualMachineInstance should be migrated instead of shut-off in case of a node drain. If the VirtualMachineInstance specific field is set it overrides the cluster level one. Defaults to LiveMigrate with multiple worker nodes, None on single worker clusters. | *v1.EvictionStrategy |  | false |
+| evictionStrategy | EvictionStrategy defines at the cluster level if the VirtualMachineInstance should be migrated instead of shut-off in case of a node drain. If the VirtualMachineInstance specific field is set it overrides the cluster level one. Allowed values: - `None` no eviction strategy at cluster level. - `LiveMigrate` migrate the VM on eviction; a not live migratable VM with no specific strategy will block the drain of the node util manually evicted. - `LiveMigrateIfPossible` migrate the VM on eviction if live migration is possible, otherwise directly evict. - `External` block the drain, track eviction and notify an external controller. Defaults to LiveMigrate with multiple worker nodes, None on single worker clusters. | *v1.EvictionStrategy |  | false |
 | vmStateStorageClass | VMStateStorageClass is the name of the storage class to use for the PVCs created to preserve VM state, like TPM. The storage class must support RWX in filesystem mode. | *string |  | false |
 | virtualMachineOptions | VirtualMachineOptions holds the cluster level information regarding the virtual machine. | *[VirtualMachineOptions](#virtualmachineoptions) | {"disableFreePageReporting": true} | false |
 

docs/cluster-configuration.md

@@ -842,8 +842,9 @@ field is set it overrides the cluster level one.
 Possible values:
 
 - `None` no eviction strategy at cluster level.
-- `LiveMigrate` migrate the VM on eviction.
-- `External` block eviction and notify an external controller.
+- `LiveMigrate` migrate the VM on eviction; a not live migratable VM with no specific strategy will block the drain of the node util manually evicted.
+- `LiveMigrateIfPossible` migrate the VM on eviction if live migration is possible, otherwise directly evict.
+- `External` block the drain, track the eviction and notify an external controller.
 
 `LiveMigrate` is the default behaviour with multiple worker nodes, `None` on single worker clusters.