feat(REST, CVC): add a support for rest service in CVC Operator #83

mittachaitu · 2020-05-27T15:26:05Z

This PR adds support for rest service in CVC operator and it adds the following functionality:

Add support for handling Create, Get and Delete operations on CStorBackup and CStorRestore APIS.

Sample Backup resource created by backup endpoint:

metadata:
  name: snapshot2-pv-687c169c-4ec5-4144-9bcb-a30b75882c56
  namespace: openebs
  labels:
    cstorpoolinstance.openebs.io/name: cspc-cstor-pool2-hh6c
    cstorpoolinstance.openebs.io/uid: 577c169c-4ec5-4144-9bcb-a30b75882ca4
    openebs.io/backup: backup2
    openebs.io/persistent-volume: pv-687c169c-4ec5-4144-9bcb-a30b75882c56
spec:
  backupName: backup2
  volumeName: pv-687c169c-4ec5-4144-9bcb-a30b75882c56
  snapName: snapshot2
  prevSnapName: ''"
  backupDest: 172.102.29.12:3234
  localSnap: false
status: Pending

Sample Restore resource created by restore endpoint:

metadata:
  name: restore1-577c169c-4ec5-4144-9bcb
  namespace: openebs
  labels:
    cstorpoolinstance.openebs.io/name: cspc-cstor-pool2-hh6c
    cstorpoolinstance.openebs.io/uid: 577c169c-4ec5-4144-9bcb-a30b75882ca4
    openebs.io/restore: backup2
    openebs.io/persistent-volume: pv-687c169c-4ec5-4144-9bcb-a30b75882c56
TODO: add spec details

Note:

This is the initial commit to support CSI volumes backup and restore.

Manual testing:

Deployed CStor-CSI setup and made curl request on CVC-Operator service to interact with REST endpoints.
Able to successfully create the local backup and restore.
Able to successfully create CStorBackup and CStorRestore CR's by making curl request.

TODO:

Unit Test for Restore endpoint will take in separate PR.

Issues:

This PR address the partial part of issues Support for Backup/Restore of CSI based cStor volumes. openebs/openebs#2905, Backup/Restore for CSI based volumes using CSPC pools with V1 APIs #40

Signed-off-by: mittachaitu sai.chaithanya@mayadata.io

deploy/cstor-operator.yaml

pkg/controllers/cstorvolumeconfig/start.go

pkg/server/cstorvolumeconfig/backup_endpoint.go

sonasingh46 · 2020-06-02T12:48:19Z

pkg/server/cstorvolumeconfig/backup_endpoint.go

+
+	switch req.Method {
+	case "POST":
+		return backupOp.create()


Let us log for create request also.

Instead of logging here good to log in create method.

Then why it is logged for Get and Delete

I will log it.

pkg/server/cstorvolumeconfig/restore_endpoint.go

pkg/controllers/cstorvolumeconfig/volume_operations.go

mynktl

looks good 👍

mynktl · 2020-06-03T08:51:45Z

pkg/snapshot/client.go

+
+// constants
+const (
+	VolumeGrpcListenPort = 7777


Please add comment here. Better not to define VolumeGrpcListenPort value here, instead we can use it from API/some package.

Sure I will move it to openebs/api repo in subsequent PRs.

mynktl · 2020-06-03T08:55:59Z

pkg/server/cstorvolumeconfig/backup_endpoint.go

+	case "POST":
+		return backupOp.create()
+	case "GET":
+		klog.Infof("Got backup GET request")


Better to change level to debug.

This is good point but I need to check which log level is best i.e for this entier repo we need to fix specific log levels like 1, 2, 4 for different levels verbose. So I will incorporate this in subsequent PRs.

pkg/server/cstorvolumeconfig/backup_endpoint.go

mynktl · 2020-06-03T09:18:54Z

pkg/server/cstorvolumeconfig/backup_endpoint.go

+
+	// If cspiName is not empty then fetch the CStor pool pod using CSPI name
+	if cspiName == "" {
+		klog.Errorf("failed to find pool manager empty CSPI is provided")


Suggested change

klog.Errorf("failed to find pool manager empty CSPI is provided")

klog.Errorf("failed to find pool manager, empty CSPI is provided")

mynktl · 2020-06-03T09:20:26Z

pkg/server/cstorvolumeconfig/backup_endpoint.go

+
+	// If cspiName is not empty then fetch the CStor pool pod using CSPI name
+	if cspiName == "" {
+		klog.Errorf("failed to find pool manager empty CSPI is provided")


Suggested change

klog.Errorf("failed to find pool manager empty CSPI is provided")

klog.Errorf("failed to find pool manager, empty CSPI is provided")

sonasingh46 · 2020-06-03T10:15:25Z

pkg/server/cstorvolumeconfig/backup_endpoint.go

+		return "", err
+	}
+
+	return cstorvolume.Spec.TargetIP, nil


Is there not a possibility of cstorvolume.Spec.TargetIP be empty?

If it is empty snap creation will fail. So backup will fail

sonasingh46 · 2020-06-03T10:16:37Z

pkg/server/cstorvolumeconfig/backup_endpoint.go

+	volume string) (*cstorapis.CStorVolumeReplica, error) {
+	namespace := getOpenEBSNamespace()
+	listOptions := metav1.ListOptions{
+		LabelSelector: "openebs.io/persistent-volume=" + volume,


This label is hardcoded.

replaced with api constant

sonasingh46 · 2020-06-03T10:18:32Z

pkg/server/cstorvolumeconfig/backup_endpoint.go

+}
+
+// getLastBackupSnap will fetch the last successful backup's snapshot name
+func (bOps *backupAPIOps) getLastBackupSnap(backUp *openebsapis.CStorBackup) (string, error) {


This function is misleading in terms of what it does vs the comments and the function name.
ToDo: Next PR can improve this.

pkg/server/cstorvolumeconfig/restore_endpoint.go

Signed-off-by: mittachaitu <sai.chaithanya@mayadata.io>

pkg/server/cstorvolumeconfig/https_test.go

pkg/server/cstorvolumeconfig/restore_endpoint.go

Signed-off-by: mittachaitu <sai.chaithanya@mayadata.io>

prateekpandey14

lgtm

mittachaitu requested a review from sonasingh46 May 27, 2020 15:26

mittachaitu added the pr/hold-review label May 28, 2020

mittachaitu self-assigned this May 28, 2020

mittachaitu linked an issue May 28, 2020 that may be closed by this pull request

Backup/Restore for CSI based volumes using CSPC pools with V1 APIs #40

Closed

mittachaitu force-pushed the rest_cvc branch from 3a42ebf to d09b401 Compare June 1, 2020 17:37

mittachaitu changed the title ~~[WIP] feat(REST, CVC): add a support for rest service in CVC Operator~~ feat(REST, CVC): add a support for rest service in CVC Operator Jun 1, 2020

mittachaitu requested review from prateekpandey14 and mynktl June 1, 2020 17:38

mittachaitu added needs-review and removed pr/hold-review labels Jun 2, 2020

mittachaitu mentioned this pull request Jun 2, 2020

feat(restore-controller): add suport for restore controller #87

Merged

sonasingh46 reviewed Jun 2, 2020

View reviewed changes

deploy/cstor-operator.yaml Outdated Show resolved Hide resolved

sonasingh46 reviewed Jun 2, 2020

View reviewed changes

deploy/cstor-operator.yaml Show resolved Hide resolved

sonasingh46 reviewed Jun 2, 2020

View reviewed changes

pkg/controllers/cstorvolumeconfig/start.go Outdated Show resolved Hide resolved

sonasingh46 reviewed Jun 2, 2020

View reviewed changes

pkg/server/cstorvolumeconfig/backup_endpoint.go Outdated Show resolved Hide resolved

sonasingh46 reviewed Jun 2, 2020

View reviewed changes

pkg/server/cstorvolumeconfig/restore_endpoint.go Outdated Show resolved Hide resolved

sonasingh46 added the area: cvc-operator label Jun 2, 2020

mittachaitu force-pushed the rest_cvc branch from d09b401 to 17ec490 Compare June 3, 2020 05:13

sonasingh46 reviewed Jun 3, 2020

View reviewed changes

pkg/controllers/cstorvolumeconfig/volume_operations.go Show resolved Hide resolved

mynktl approved these changes Jun 3, 2020

View reviewed changes

sonasingh46 reviewed Jun 3, 2020

View reviewed changes

mittachaitu commented Jun 3, 2020

View reviewed changes

pkg/server/cstorvolumeconfig/restore_endpoint.go Show resolved Hide resolved

sonasingh46 approved these changes Jun 3, 2020

View reviewed changes

mittachaitu commented Jun 3, 2020

View reviewed changes

pkg/server/cstorvolumeconfig/restore_endpoint.go Outdated Show resolved Hide resolved

mittachaitu added 2 commits June 4, 2020 18:34

feat(REST, CVC): add a support for rest service in CVC Operator

c3cce71

Signed-off-by: mittachaitu <sai.chaithanya@mayadata.io>

This commit fixes bug in the code

a5af9eb

Signed-off-by: mittachaitu <sai.chaithanya@mayadata.io>

mittachaitu added 7 commits June 4, 2020 18:34

This commit close the listner

3b173d6

Signed-off-by: mittachaitu <sai.chaithanya@mayadata.io>

This commit adds the test code to verify backup REST endpoint

4550c77

Signed-off-by: mittachaitu <sai.chaithanya@mayadata.io>

This commit adds vendor files

864b4ff

Signed-off-by: mittachaitu <sai.chaithanya@mayadata.io>

This commit incorporate review comments

92eab43

Signed-off-by: mittachaitu <sai.chaithanya@mayadata.io>

This commit incorporates the review comments

8500a7f

Signed-off-by: mittachaitu <sai.chaithanya@mayadata.io>

This commit updates the source volume label on CVC with spec.restoreSrc

96f4dac

Signed-off-by: mittachaitu <sai.chaithanya@mayadata.io>

This commit adds the backup and restore crds

d9984e4

Signed-off-by: mittachaitu <sai.chaithanya@mayadata.io>

mittachaitu force-pushed the rest_cvc branch from 3776e77 to d9984e4 Compare June 4, 2020 13:23

This commit updates the vendor file changes

2b7aad8

Signed-off-by: mittachaitu <sai.chaithanya@mayadata.io>

prateekpandey14 reviewed Jun 8, 2020

View reviewed changes

pkg/server/cstorvolumeconfig/https_test.go Show resolved Hide resolved

prateekpandey14 reviewed Jun 8, 2020

View reviewed changes

pkg/server/cstorvolumeconfig/restore_endpoint.go Outdated Show resolved Hide resolved

This commit incorporates the review comments

3cfcd68

Signed-off-by: mittachaitu <sai.chaithanya@mayadata.io>

prateekpandey14 approved these changes Jun 9, 2020

View reviewed changes

prateekpandey14 merged commit 0177387 into openebs-archive:master Jun 9, 2020

mittachaitu mentioned this pull request Jun 15, 2020

Support for Backup/Restore of CSI based cStor volumes. openebs/openebs#2905

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(REST, CVC): add a support for rest service in CVC Operator #83

feat(REST, CVC): add a support for rest service in CVC Operator #83

mittachaitu commented May 27, 2020 •

edited

Loading

sonasingh46 Jun 2, 2020

mittachaitu Jun 3, 2020

sonasingh46 Jun 3, 2020

mittachaitu Jun 3, 2020

mynktl left a comment

mynktl Jun 3, 2020

mittachaitu Jun 3, 2020

mynktl Jun 3, 2020

mittachaitu Jun 4, 2020

mynktl Jun 3, 2020

mittachaitu Jun 4, 2020

mynktl Jun 3, 2020

mittachaitu Jun 4, 2020

sonasingh46 Jun 3, 2020

mittachaitu Jun 3, 2020

mittachaitu Jun 4, 2020

sonasingh46 Jun 3, 2020

mittachaitu Jun 4, 2020

sonasingh46 Jun 3, 2020

prateekpandey14 left a comment

	klog.Errorf("failed to find pool manager empty CSPI is provided")
	klog.Errorf("failed to find pool manager, empty CSPI is provided")

feat(REST, CVC): add a support for rest service in CVC Operator #83

feat(REST, CVC): add a support for rest service in CVC Operator #83

Conversation

mittachaitu commented May 27, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mynktl left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

prateekpandey14 left a comment

Choose a reason for hiding this comment

mittachaitu commented May 27, 2020 •

edited

Loading