Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Datamover CRD design #597

Merged
merged 11 commits into from
May 24, 2022

Conversation

savitharaghunathan
Copy link
Member

@savitharaghunathan savitharaghunathan commented Mar 16, 2022

Adding Data Mover CRD design

@savitharaghunathan savitharaghunathan marked this pull request as draft March 16, 2022 14:49
@openshift-ci
Copy link

openshift-ci bot commented Mar 16, 2022

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 16, 2022

This design supports adding the data mover feature to the OADP operator and facilitates integrating various vendor implemented data movers.

![DataMover CRD](../images/datamovercrd.png)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update this image path.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I create a PR to just merge the image first? I think the path is correct. WDYT?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kaovilai I think so.

docs/design/datamover.md Outdated Show resolved Hide resolved
docs/design/datamover.md Outdated Show resolved Hide resolved
docs/design/datamover.md Outdated Show resolved Hide resolved
@savitharaghunathan savitharaghunathan changed the title WIP: Datamover design Datamover CRD design Mar 17, 2022
@savitharaghunathan savitharaghunathan marked this pull request as ready for review March 17, 2022 19:27
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 17, 2022
spec:
DataMoverClass: <DataMoverClass name>
- type: <VolumeSnapshot|PVC>
sourceClaimRef:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sourceRef: can we model the ref based on that go type and remove the type field in line 99?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated. Please take a look and let me know if it looks good.


### Data Mover Backup

Assuming that the `DataMover Enable` flag is set to true in the DPA config, when a velero backup is created, it triggers DataMover plugin to create the `DataMoverBackup` CR in the app namespace. The plugin looks up for the PVCs in the user namespace mentioned in the velero backup and creates a `DataMoverBackup` CR for every PVC in that namespace.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this just a regular velero BackupItemAction plugin that creates the DataMoverBackup CR without modifying the PVC being acted on?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There may be a possibility of a race condition here -- the thing we just created may not immediately be available in a Get call. I know we've had this issue in the past with ImageStreamTags, but since this hasn't been an issue for the CSI plugin, maybe it's fine here too.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review, Scott. Clarified that the plugin is a BackupItemAction plugin. Should I capture the race condition information anywhere?

docs/design/datamover.md Outdated Show resolved Hide resolved
@openshift-ci
Copy link

openshift-ci bot commented Apr 19, 2022

@savitharaghunathan: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@kaovilai kaovilai added the kind/design Categorizes issue or PR as related to design. label Apr 29, 2022
@dymurray dymurray merged commit bac65af into openshift:master May 24, 2022
kaovilai pushed a commit to kaovilai/oadp-operator that referenced this pull request Jun 1, 2022
* Adding initial datamover design

* Typo in reviewers

* Add summary & user stories

* Fixing nits

* Adding intial feedback

* Adding feedback

* nit: changing the selector field name

* Adding review comments

* Adding plugin clarification

* updating image to reflect the latest design

* Reflect recent velero csi plugin changes
kaovilai added a commit that referenced this pull request Jun 3, 2022
…p (fix azure flakes) (#717)

* revert to b64e96e

* bump probe to 10s

* mimic velero cli loglevel options (#692)

use logrus ParseLevel directly

* Fix OADP 526 (#704)

* fix OADP 526

* add ignore err comments

* fix docs link for build from src (#706)

* Datamover CRD design (#597)

* Adding initial datamover design

* Typo in reviewers

* Add summary & user stories

* Fixing nits

* Adding intial feedback

* Adding feedback

* nit: changing the selector field name

* Adding review comments

* Adding plugin clarification

* updating image to reflect the latest design

* Reflect recent velero csi plugin changes

* use a full vm for arm builds in travis (#713)

* OADP-535 allow for nullable resource allocations (#711)

* OADP-535 allow for nullable resource allocations

* Add missing nullables

* Include nullable on additionalProperties

Co-authored-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>
Co-authored-by: Savitha Raghunathan <sraghuna@redhat.com>
Co-authored-by: Jason Montleon <jmontleo@redhat.com>
Co-authored-by: Dylan Murray <dymurray@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/design Categorizes issue or PR as related to design.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants