Upload populator (#2678)

* Create CRD for volumeuploadsource populator

This CRD will be used in the DataSourceRef on PVCs
to trigger population that upload to the volume.
This will be performed by the upload populator
that will be added in future commits.

Signed-off-by: Shelly Kagan <skagan@redhat.com>

* Create upload populator controller

The upload populator controller can be used
standalone without the need of datavolume.
It reconciles pvc with upload dataSourceRef
and uses populators API to populated the pvc
with an upload command.
The controller creates pvc' with upload
annotation. After the upload completes it
rebinds the pv to the original target pvc and
deletes pvc prime.
Eventually we get a bound PVC which is already
populated.

Signed-off-by: Shelly Kagan <skagan@redhat.com>

* Adjust upload-proxy to handle upload population

In case of pvc with datasourceref to upload population
we should create the url to the upload server with the
pvc' name.

Signed-off-by: Shelly Kagan <skagan@redhat.com>

* Add tests for upload population

Signed-off-by: Shelly Kagan <skagan@redhat.com>

* Add unit tests for upload populator

Signed-off-by: Shelly Kagan <skagan@redhat.com>

* Add preallocation to volumeuploadsource crd

Also some other small fixes

Signed-off-by: Shelly Kagan <skagan@redhat.com>

---------

Signed-off-by: Shelly Kagan <skagan@redhat.com>

cmd/cdi-controller/controller.go

@@ -276,6 +276,10 @@ func start() {
 		klog.Errorf("Unable to setup import populator: %v", err)
 		os.Exit(1)
 	}
+	if _, err := populators.NewUploadPopulator(ctx, mgr, log, installerLabels); err != nil {
+		klog.Errorf("Unable to setup upload populator: %v", err)
+		os.Exit(1)
+	}
 
 	klog.V(1).Infoln("created cdi controllers")
 

pkg/client/clientset/versioned/typed/core/v1beta1/BUILD.bazel

@@ -14,6 +14,7 @@ go_library(
         "objecttransfer.go",
         "storageprofile.go",
         "volumeimportsource.go",
+        "volumeuploadsource.go",
     ],
     importpath = "kubevirt.io/containerized-data-importer/pkg/client/clientset/versioned/typed/core/v1beta1",
     visibility = ["//visibility:public"],

pkg/client/clientset/versioned/typed/core/v1beta1/fake/BUILD.bazel

@@ -13,6 +13,7 @@ go_library(
         "fake_objecttransfer.go",
         "fake_storageprofile.go",
         "fake_volumeimportsource.go",
+        "fake_volumeuploadsource.go",
     ],
     importpath = "kubevirt.io/containerized-data-importer/pkg/client/clientset/versioned/typed/core/v1beta1/fake",
     visibility = ["//visibility:public"],