Import populator (#2690)

* Create populators package to be used for all populators

This commit introduces the basic reconciler for
populators with common function that can be used
by the different populators.

Signed-off-by: Shelly Kagan <skagan@redhat.com>

* unite getcontenttype func across code

Signed-off-by: Shelly Kagan <skagan@redhat.com>

* Add VolumeImportSource CRD for import populator

This commit adds the VolumeImportSource CRD into CDI.

CRs created from this CRD will be referenced in the dataSourceRef field to populate PVCs with the import populator.

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Refactor common populator code to be shared among all populators

This commit introduces and modifies several functions so we can reuse common code between all populators.

Other than having a common reconcile function, a new populatorController interface has been introduced so we are able to call populator-specific methods from the populator-base reconciler.

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Create Import Populator

The import populator is a controller that handles the import of data in PVCs without the need of DataVolumes while still taking advantage of the import-controller flow.

This controller creates an additional PVC' with import annotations. After the import process succeeds, the controller rebinds the PV to the original target PVc and deletes the PVC prime.

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Add functional tests to cover the import populator flow

This commit updates the import tests to cover the new import populator flow.

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Add unit tests for import populator

Signed-off-by: Alvaro Romero <alromero@redhat.com>

* Minor fixes and enhancements in import/common populator code

* Modify indexes and other related code to support namespaced dataSourceRefs. Cross-namespace population is still not supported as it depends on alpha feature gates.
* Add functional test to cover static binding.
* Fix selected node annotation bug in scratch space PVCs
* Fix linter alerts

Signed-off-by: Alvaro Romero <alromero@redhat.com>

---------

Signed-off-by: Shelly Kagan <skagan@redhat.com>
Signed-off-by: Alvaro Romero <alromero@redhat.com>
Co-authored-by: Shelly Kagan <skagan@redhat.com>

cmd/cdi-controller/BUILD.bazel

@@ -9,6 +9,7 @@ go_library(
         "//pkg/common:go_default_library",
         "//pkg/controller:go_default_library",
         "//pkg/controller/datavolume:go_default_library",
+        "//pkg/controller/populators:go_default_library",
         "//pkg/controller/transfer:go_default_library",
         "//pkg/util:go_default_library",
         "//pkg/util/cert:go_default_library",

cmd/cdi-controller/controller.go

@@ -35,6 +35,7 @@ import (
 	"kubevirt.io/containerized-data-importer/pkg/common"
 	"kubevirt.io/containerized-data-importer/pkg/controller"
 	dvc "kubevirt.io/containerized-data-importer/pkg/controller/datavolume"
+	"kubevirt.io/containerized-data-importer/pkg/controller/populators"
 	"kubevirt.io/containerized-data-importer/pkg/controller/transfer"
 	"kubevirt.io/containerized-data-importer/pkg/util"
 	"kubevirt.io/containerized-data-importer/pkg/util/cert"
@@ -266,6 +267,15 @@ func start() {
 		klog.Errorf("Unable to setup datasource controller: %v", err)
 		os.Exit(1)
 	}
+	// Populator controllers and indexes
+	if err := populators.CreateCommonPopulatorIndexes(mgr); err != nil {
+		klog.Errorf("Unable to create common populator indexes: %v", err)
+		os.Exit(1)
+	}
+	if _, err := populators.NewImportPopulator(ctx, mgr, log, installerLabels); err != nil {
+		klog.Errorf("Unable to setup import populator: %v", err)
+		os.Exit(1)
+	}
 
 	klog.V(1).Infoln("created cdi controllers")
 

pkg/client/clientset/versioned/typed/core/v1beta1/BUILD.bazel

@@ -13,6 +13,7 @@ go_library(
         "generated_expansion.go",
         "objecttransfer.go",
         "storageprofile.go",
+        "volumeimportsource.go",
     ],
     importpath = "kubevirt.io/containerized-data-importer/pkg/client/clientset/versioned/typed/core/v1beta1",
     visibility = ["//visibility:public"],

pkg/client/clientset/versioned/typed/core/v1beta1/fake/BUILD.bazel

@@ -12,6 +12,7 @@ go_library(
         "fake_datavolume.go",
         "fake_objecttransfer.go",
         "fake_storageprofile.go",
+        "fake_volumeimportsource.go",
     ],
     importpath = "kubevirt.io/containerized-data-importer/pkg/client/clientset/versioned/typed/core/v1beta1/fake",
     visibility = ["//visibility:public"],