Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documentation for cdi populators #2776

Merged
merged 2 commits into from
Jul 11, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
129 changes: 129 additions & 0 deletions doc/cdi-populators.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
# CDI Volume Populators

## Recommended knowledge

What are [populators](https://kubernetes.io/blog/2022/05/16/volume-populators-beta/)

## Introduction
CDI volume populators are CDI's implementation of importing/uploading/cloning data to PVCs using the new `dataSourceRef` field. New controllers and custom resources for each population method were introduced.

**Benefits of the new API**
* Native synchronization with Kubernetes - this is Kubernetes way of populating PVCs. Once PVC is bound we know it is populated (Before introducing populators, the PVC was bound the moment the datavolume created it, and the population progress was monitored via the datavolume)
* Use PVCs directly and have them populated without the need for datavolumes.
* Can use one population definition for multiple PVCs - create 1 CR defining population source and use it for any PVC.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may make sense to mention that this is limited to a single namespace until https://kubernetes.io/blog/2023/01/02/cross-namespace-data-sources-alpha/ goes beta and is incorporated into CDI.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mhenriks didn't you solve that for now by using the tokens or I miss understood?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cross namespace only works with datavolume integration

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we have to mention this limitation since currently alpha

* Better compatibility with existing backup and restore solutions:
* Standalone PVC should be compatible with all solutions.
* Datavolumes using the populators should be compatible with most solutions, for example Metro DR and [Gitops](https://www.redhat.com/en/topics/devops/what-is-gitops#:~:text=GitOps%20uses%20Git%20repositories%20as,set%20for%20the%20application%20framework.) - when restoring datavolume manifest will be applied, the datavolume will create the PVC that will bind immediately to the PV waiting for it.
* [Kubevirt](https://github.com/kubevirt/kubevirt) VMs with PVCs/datavolumetemplate should also be compatible with most solutions.
* Integration with [Kubevirt](https://github.com/kubevirt/kubevirt) with WaitForFirstConsumer(WFFC) storage class is simpler and does not require a [doppelganger pod](https://github.com/kubevirt/kubevirt/blob/main/docs/localstorage-disks.md#the-problem) for the start of the VM.


## Using the populators

We introduced new controllers for each population method. Each controller supports a matching CRD which provides the needed information for the population.
Example for an instance of VolumeImportSource CRD:

```yaml
apiVersion: cdi.kubevirt.io/v1beta1
kind: VolumeImportSource
metadata:
name: my-import-source
spec:
source:
http:
url: "https://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img"
```

### Using populators with PVCs
User can create a CR and PVCs specifying the CR in the `DataSourceRef` field and those will be handled by the matching populator controller.

#### Import
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should be more specific with each example: We should add an example of a valid CRD and a valid PVC for each populator.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have it for every example except upload which Ill add

Example for PVC which use the VolumeImportSource above that will be handled by the import populator:
```yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: my-pvc
spec:
dataSourceRef:
apiGroup: cdi.kubevirt.io
kind: VolumeImportSource
name: my-import-source
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
```

Above PVC will trigger the reconcile of the import populator controller.
The controller will create a matching temporary PVC with the appropriate annotations, which will get bound and populated.
Once the temporary PVC population is done, the PV will be rebound to the original PVC completing the population process.

#### Upload
Example of VolumeUploadSource and a PVC that will be handled by the upload populator:
```yaml
apiVersion: cdi.kubevirt.io/v1beta1
kind: VolumeUploadSource
metadata:
name: my-upload-source
spec:
contentType: kubevirt
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: my-pvc
spec:
dataSourceRef:
apiGroup: cdi.kubevirt.io
kind: VolumeUploadSource
name: my-upload-source
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
```
After creating the VolumeUploadSource and PVC, you can start the upload to the pvc as describe in the [upload doc](upload.md).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forgive my ignorance but does the user target PVC prime when uploading or the PVC they created?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, the user can upload to the target pvc as always, we take care of the rest under the covers.