Pass DV labels to PVC #2547

ido106 · 2023-01-19T13:47:22Z

Signed-off-by: Ido Aharon iaharon@redhat.com

What this PR does / why we need it:
Pass all the DataVolume labels to its created PVC. It's fixing the current behavior which passes only specific labels, aligning it with DV annotations passing to the PVC.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #2533

Special notes for your reviewer:

Release note:

Pass all the DataVolume labels to its created PVC

kubevirt-bot · 2023-01-19T13:47:32Z

Hi @ido106. Thanks for your PR.

I'm waiting for a kubevirt member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

awels · 2023-01-19T14:35:57Z

So you essentially want to pass all labels from the DV to the PVC? Any particular reason you want that? We are already passing labels from the DV to the import/upload/clone pods for things like istio and I don't think we want to pass those to the PVC.

akalenyu · 2023-01-19T14:54:08Z

So you essentially want to pass all labels from the DV to the PVC? Any particular reason you want that? We are already passing labels from the DV to the import/upload/clone pods for things like istio and I don't think we want to pass those to the PVC.

So we actually discussed this in the sig-storage meeting and I tried to summarize it in the issue itself;
Basically, if we look at DVs as a PVC creation template, you will want to say which labels you want on your PVC, right?

I would understand the argument that you could just grab the PVC and label it after the DV creation, but that is a bit clunky..
What if you created the DV with generateName? like the issue author did?
Anyway, we were already passing down annotations and could not think why we aren't giving the same treatment to labels.

akalenyu

Awesome! great spot!

Would it be possible to also add a one-liner to some func tests that ensures we pass the labels?
in e.g. import/upload/clone/...

akalenyu · 2023-01-19T15:04:47Z

pkg/controller/datavolume/controller-base.go

@@ -761,7 +746,9 @@ func (r *ReconcilerBase) newPersistentVolumeClaim(dataVolume *cdiv1.DataVolume,
 	if util.ResolveVolumeMode(targetPvcSpec.VolumeMode) == corev1.PersistentVolumeFilesystem {
 		labels[common.KubePersistentVolumeFillingUpSuppressLabelKey] = common.KubePersistentVolumeFillingUpSuppressLabelValue
 	}
-	labels = passDataVolumeInstancetypeLabelstoPVC(dataVolume.GetLabels(), labels)
+	for k, v := range dataVolume.Labels {


Note that not all flows we have are using newPersistentVolumeClaim
One example I could find is smart clone which creates the target PVC in

containerized-data-importer/pkg/controller/datavolume/smart-clone-controller.go

Lines 370 to 374 in b875775

target := &corev1.PersistentVolumeClaim{

ObjectMeta: metav1.ObjectMeta{

Name: snapshot.Name,

Namespace: snapshot.Namespace,

Labels: labels,

Good point!

Hi Alex, there is no DV in that function and the PVC is created from a VolumeSnapshot, so I don't think labels should be passed in this case. wdyt ?

There should be a way to grab/pass the associated DV to this func

containerized-data-importer/pkg/controller/datavolume/smart-clone-controller.go

Line 209 in b875775

dataVolume, err := r.getDataVolume(snapshot)

arnongilboa · 2023-01-19T15:51:15Z

@ido106 please fill in the What this PR does / why we need it and Release note sections in the card description.

mhenriks · 2023-01-19T22:18:25Z

/ok-to-test

awels · 2023-01-24T21:02:29Z

/retest-required

mhenriks · 2023-02-07T01:04:50Z

@ido106 any update on this? I believe we are waiting on a functional test?

arnongilboa · 2023-02-07T06:38:55Z

@ido106 any update on this? I believe we are waiting on a functional test?

Ido is in his exam period, so I guess he'll continue his first PR when he's back.

akalenyu

Looking good, some things to consider, and maybe it's worth to add one liner checks in some of the functional tests?

akalenyu · 2023-02-21T15:20:55Z

pkg/controller/datavolume/smart-clone-controller.go

@@ -352,7 +352,7 @@ func (r *SmartCloneReconciler) getTargetPVC(dataVolume *cdiv1.DataVolume) (*core
 	return pvc, nil
 }

-func newPvcFromSnapshot(name string, snapshot *snapshotv1.VolumeSnapshot, targetPvcSpec *corev1.PersistentVolumeClaimSpec) (*corev1.PersistentVolumeClaim, error) {
+func newPvcFromSnapshot(c client.Client, name string, snapshot *snapshotv1.VolumeSnapshot, targetPvcSpec *corev1.PersistentVolumeClaimSpec) (*corev1.PersistentVolumeClaim, error) {


I think you may even be able to just pass
annotations, labels map[string]string/dv *cdiv1.DataVolume to this func
(the DV should be already available to you when calling newPvcFromSnapshot)

akalenyu · 2023-02-21T15:21:30Z

pkg/controller/datavolume/smart-clone-controller.go

+		return nil, err
+	}
+	if dv != nil {
+		for k, v := range dv.Labels {


It seems like we also don't pass the annotations in this case, we probably should

akalenyu · 2023-02-21T15:28:43Z

pkg/controller/datavolume/controller-base.go

@@ -365,9 +365,9 @@ func (r *ReconcilerBase) getPVC(dv *cdiv1.DataVolume) (*corev1.PersistentVolumeC
 	return pvc, nil
 }

-func (r *ReconcilerBase) getDataVolume(key types.NamespacedName) (*cdiv1.DataVolume, error) {
+func getDataVolume(c client.Client, key types.NamespacedName) (*cdiv1.DataVolume, error) {


Hmm not sure why this one needed to be changed?

I use this function in 2 different places in the code and one of them doesn't have access to the function this way, so I changed the signature to be a regular function and requested the client as a parameter instead.

AFAIK you only use func (r *SmartCloneReconciler) getDataVolume and not this one

Thanks Alex, I tried to fix what you mentioned.
Plz let me know if there is any problem and I'll fix it tomorrow.

Looks good, I just think we should test this in functional tests as well,
since we are now committing that CDI will always pass labels

alromeros · 2023-02-22T08:34:09Z

/test pull-containerized-data-importer-e2e-upg

alromeros

Good job Ido! I think after your latest commit this looks great. I'm only commenting some minor nitpicks in case you end up committing again, no need to change them. Everything else looks good to me so I'll give a preventive lgtm, but let's see what other folks have to say.

/lgtm

alromeros · 2023-02-22T08:53:48Z

pkg/controller/datavolume/smart-clone-controller.go

@@ -369,22 +370,32 @@ func newPvcFromSnapshot(name string, snapshot *snapshotv1.VolumeSnapshot, target
 		common.CDILabelKey:       common.CDILabelValue,
 		common.CDIComponentLabel: common.SmartClonerCDILabel,
 	}
+


This is a massive nitpick but I'd drop this empty line, as done with annotations below

alromeros · 2023-02-22T08:53:51Z

pkg/controller/datavolume/import-controller_test.go

 			dv := NewImportDataVolume("test-dv")
 			dv.SetAnnotations(make(map[string]string))
 			dv.GetAnnotations()["test-ann-1"] = "test-value-1"
 			dv.GetAnnotations()["test-ann-2"] = "test-value-2"
 			dv.GetAnnotations()[AnnSource] = "invalid phase should not copy"
 			dv.GetAnnotations()[AnnPodNetwork] = "data-network"
 			dv.GetAnnotations()[AnnPodSidecarInjection] = "false"
+			dv.Labels = map[string]string{"test": "test-label"}


Nitpick, but I'd use the same convention for Annotations and Labels (Get/Set or not).

Signed-off-by: Ido Aharon <iaharon@redhat.com>

akalenyu · 2023-03-02T22:13:31Z

/lgtm

awesome!

akalenyu · 2023-03-03T09:51:52Z

/hold

I think I see the problem

containerized-data-importer/tests/cloner_test.go

Lines 2878 to 2879 in dbc9997

    
           targetDV.Labels = map[string]string{"test-label-1": "test-label-key-1"} 
        
           targetDV.Annotations = map[string]string{"test-annotation-1": "test-annotation-key-1"}

The annotations are being overwritten so we lose AnnDeleteAfterCompletion
We should probably just append the test annotations/labels to existing ones

Signed-off-by: Ido Aharon <iaharon@redhat.com>

akalenyu · 2023-03-05T09:40:36Z

/hold cancel

But we have an unrelated CI issue following 1.56.0 release related to v1alpha1 manifests

akalenyu · 2023-03-05T09:41:00Z

/lgtm

akalenyu · 2023-03-06T15:07:05Z

/retest

akalenyu · 2023-03-07T10:30:47Z

/retest

akalenyu · 2023-03-07T21:45:38Z

/test pull-containerized-data-importer-e2e-ceph

akalenyu · 2023-03-13T11:33:58Z

/cc @mhenriks

mhenriks · 2023-03-17T17:11:27Z

/approve

kubevirt-bot · 2023-03-17T17:11:37Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mhenriks

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [mhenriks]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

mhenriks · 2023-03-17T17:12:17Z

/cherrypick release-v1.56

kubevirt-bot · 2023-03-17T17:12:18Z

@mhenriks: once the present PR merges, I will cherry-pick it on top of release-v1.56 in a new PR and assign it to you.

In response to this:

/cherrypick release-v1.56

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

awels · 2023-03-17T18:35:35Z

/override pull-containerized-data-importer-e2e-destructive

kubevirt-bot · 2023-03-17T18:35:38Z

@awels: Overrode contexts on behalf of awels: pull-containerized-data-importer-e2e-destructive

In response to this:

/override pull-containerized-data-importer-e2e-destructive

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

kubevirt-bot · 2023-03-17T20:58:18Z

@mhenriks: new pull request created: #2646

In response to this:

/cherrypick release-v1.56

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

kubevirt-bot added release-note-none Denotes a PR that doesn't merit a release note. dco-signoff: yes Indicates the PR's author has DCO signed all their commits. labels Jan 19, 2023

kubevirt-bot requested review from aglitke and mhenriks January 19, 2023 13:47

kubevirt-bot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Jan 19, 2023

kubevirt-bot added the size/S label Jan 19, 2023

akalenyu reviewed Jan 19, 2023

View reviewed changes

kubevirt-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed release-note-none Denotes a PR that doesn't merit a release note. labels Jan 19, 2023

kubevirt-bot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jan 19, 2023

ido106 force-pushed the pass_dv_labels_to_pvc branch from b875775 to ea577b8 Compare February 21, 2023 15:02

kubevirt-bot added size/M and removed size/S labels Feb 21, 2023

akalenyu reviewed Feb 21, 2023

View reviewed changes

alromeros reviewed Feb 22, 2023

View reviewed changes

kubevirt-bot assigned alromeros Feb 22, 2023

kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label Feb 22, 2023

ido106 force-pushed the pass_dv_labels_to_pvc branch from 91c4767 to c02fa12 Compare February 22, 2023 15:48

kubevirt-bot removed the lgtm Indicates that a PR is ready to be merged. label Feb 22, 2023

ido106 force-pushed the pass_dv_labels_to_pvc branch from c02fa12 to 3f19b15 Compare February 22, 2023 16:13

Transferring annotations in smart clone and passing dv instead of client

cd50a2f

Signed-off-by: Ido Aharon <iaharon@redhat.com>

ido106 force-pushed the pass_dv_labels_to_pvc branch from a6f59bf to b058926 Compare March 2, 2023 17:08

ido106 added 2 commits March 2, 2023 20:34

CSI clone and Cloning from snapshot source functional tests

77ceb4d

Signed-off-by: Ido Aharon <iaharon@redhat.com>

CSI clone and Cloning from snapshot source functional tests

dbc9997

Signed-off-by: Ido Aharon <iaharon@redhat.com>

ido106 force-pushed the pass_dv_labels_to_pvc branch from b058926 to dbc9997 Compare March 2, 2023 21:44

kubevirt-bot assigned akalenyu Mar 2, 2023

kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label Mar 2, 2023

kubevirt-bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 3, 2023

uninitialized map case fixed

bd33df8

Signed-off-by: Ido Aharon <iaharon@redhat.com>

kubevirt-bot removed the lgtm Indicates that a PR is ready to be merged. label Mar 5, 2023

kubevirt-bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 5, 2023

kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label Mar 5, 2023

kubevirt-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 17, 2023

kubevirt-bot merged commit 184ee03 into kubevirt:main Mar 17, 2023

kubevirt-bot mentioned this pull request Mar 17, 2023

[release-v1.56] Pass DV labels to PVC #2646

Merged

	target := &corev1.PersistentVolumeClaim{
	ObjectMeta: metav1.ObjectMeta{
	Name: snapshot.Name,
	Namespace: snapshot.Namespace,
	Labels: labels,

Pass DV labels to PVC #2547

Pass DV labels to PVC #2547

Conversation

ido106 commented Jan 19, 2023 • edited Loading

kubevirt-bot commented Jan 19, 2023

awels commented Jan 19, 2023

akalenyu commented Jan 19, 2023

akalenyu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arnongilboa commented Jan 19, 2023

mhenriks commented Jan 19, 2023

awels commented Jan 24, 2023

mhenriks commented Feb 7, 2023

arnongilboa commented Feb 7, 2023

akalenyu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

akalenyu Feb 22, 2023 • edited Loading

Choose a reason for hiding this comment

alromeros commented Feb 22, 2023

alromeros left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

akalenyu commented Mar 2, 2023

akalenyu commented Mar 3, 2023

akalenyu commented Mar 5, 2023

akalenyu commented Mar 5, 2023

akalenyu commented Mar 6, 2023

akalenyu commented Mar 7, 2023

akalenyu commented Mar 7, 2023

akalenyu commented Mar 13, 2023

mhenriks commented Mar 17, 2023

kubevirt-bot commented Mar 17, 2023

mhenriks commented Mar 17, 2023

kubevirt-bot commented Mar 17, 2023

awels commented Mar 17, 2023

kubevirt-bot commented Mar 17, 2023

kubevirt-bot commented Mar 17, 2023

ido106 commented Jan 19, 2023 •

edited

Loading

akalenyu Feb 22, 2023 •

edited

Loading

alromeros left a comment •

edited

Loading