Multi cloud e2e master rebase #568

deepakraj1997 · 2022-02-10T18:12:54Z

https://hackmd.io/Gzmvnu90RC20f-6OYnwskA

openshift-ci · 2022-02-10T18:12:55Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

kaovilai · 2022-02-10T18:26:01Z

/test all

codecov-commenter · 2022-02-11T06:58:42Z

Codecov Report

Merging #568 (9bedca6) into master (19b5dc3) will decrease coverage by 0.71%.
The diff coverage is 13.23%.

@@            Coverage Diff             @@
##           master     #568      +/-   ##
==========================================
- Coverage   38.43%   37.72%   -0.72%     
==========================================
  Files          13       13              
  Lines        2849     2908      +59     
==========================================
+ Hits         1095     1097       +2     
- Misses       1673     1728      +55     
- Partials       81       83       +2

Impacted Files	Coverage Δ
pkg/credentials/credentials.go	`25.54% <0.00%> (-5.03%)`	⬇️
controllers/registry.go	`40.69% <26.66%> (-0.76%)`	⬇️
controllers/bsl.go	`48.82% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 19b5dc3...9bedca6. Read the comment docs.

weshayutin · 2022-02-11T15:10:17Z

Makefile

+	OADP_CRED_FILE = ${OADP_CRED_DIR}/gcp-credentials
+	CREDS_SECRET_REF = cloud-credentials-gcp
+	OADP_BUCKET_FILE = ${OADP_CRED_DIR}/gcp-velero-bucket-name
+else ifeq ($(CLUSTER_TYPE), azure4)


azure4, is that the real cluster type or typo?

Yes, thats the real cluster type @weshayutin. Changing it on the next line to a proper one.

weshayutin

This looks good, one stupid question in my comments... re: azure4.
Thank you Deepak!

dymurray

VISIACK, looks good

weshayutin · 2022-02-12T22:33:50Z

/test 4.7-operator-e2e

mperetzred · 2022-02-13T07:44:27Z

config/manager/kustomization.yaml

@@ -12,5 +12,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 images:
 - name: controller
-  newName: quay.io/konveyor/oadp-operator
-  newTag: latest
+  newName: quay.io/deepakraj1997/oadp-operator


do we want to change that?

mperetzred · 2022-02-13T07:59:00Z

tests/e2e/dpa_deployment_suite_test.go

@@ -563,6 +378,9 @@ var _ = Describe("Configuration testing for DPA Custom Resource", func() {
 			},
 			WantError: false,
 		}, nil),
+	}
+
+	awsTests := []TableEntry{


is it possible to use labels instead? for Example, add in the Makefile:
FILTER_PROVIDER := $(shell echo '! aws-provider && ! gcp-provider && ! azure-provider' | sed -r "s/[&]* [!] $(POVIDER)-provider|[!] $(POVIDER)-provider[&]*//"
And then mark:
Entry("AWS Without Region No S3ForcePathStyle with BackupImages false should succeed", Label("aws-provider"), InstallCase{

mperetzred · 2022-02-13T08:08:53Z

tests/e2e/dpa_deployment_suite_test.go

@@ -661,5 +480,132 @@ var _ = Describe("Configuration testing for DPA Custom Resource", func() {
 			},
 			WantError: true,
 		}, fmt.Errorf("region for AWS backupstoragelocation cannot be empty when s3ForcePathStyle is true or when backing up images")),
+	}
+
+	if provider == "aws" {


same as the comment above

mperetzred · 2022-02-13T08:13:00Z

tests/e2e/dpa_deployment_suite_test.go

+			Expect(err).NotTo(HaveOccurred())
+			if len(installCase.DpaSpec.BackupLocations) > 0 {
+				switch dpaCR.Provider {
+				case "aws":


You can feed this data to the json instead, and then you won't have to switch over the provider type; this way it will be initialized generically inside dpaCr.build function:

Velero: &velero.BackupStorageLocationSpec{ Provider: v.CustomResource.Spec.BackupLocations[0].Velero.Provider, Default: true, Config: v.CustomResource.Spec.BackupLocations[0].Velero.Config, Credential: v.CustomResource.Spec.BackupLocations[0].Velero.Credential,

mperetzred · 2022-02-13T08:16:20Z

tests/e2e/e2e_suite_test.go

 )

 // Common vars obtained from flags passed in ginkgo.
-var namespace, instanceName, settings, cloud, clusterProfile, credSecretRef string
+var credFile, namespace, credSecretRef, instanceName, provider, azure_resource_file, openshift_ci, ci_cred_file, settings, bsl_profile string


is it possible to feed all platform specific vars to the json file instead? it's better to keep the tests platform agnostic as possible

mperetzred · 2022-02-13T08:18:25Z

Makefile

 	-velero_instance_name=$(VELERO_INSTANCE_NAME) \
 	-timeout_multiplier=$(E2E_TIMEOUT_MULTIPLIER) \
-	-cluster_profile=$(CLUSTER_PROFILE) \
-	--ginkgo.label-filter="$(TEST_FILTER)"
+	-cluster_profile=$(CLUSTER_TYPE) \


is it possible to feed all platform specific info to the json file instead? better to keep it platform agnostic...

mperetzred · 2022-02-13T08:22:08Z

tests/e2e/lib/dpa_helpers.go

+		// 	},
+		// }
+	case "azure":
+		dpaInstance.Spec.BackupLocations[0].Velero.Config = map[string]string{


is it possible to feed this info to the json instead? this way the code above already covers this initialization:

BackupLocations: []oadpv1alpha1.BackupLocation{ { Velero: &velero.BackupStorageLocationSpec{ Provider: v.CustomResource.Spec.BackupLocations[0].Velero.Provider, Default: true, Config: v.CustomResource.Spec.BackupLocations[0].Velero.Config, <<<<<<<<<< Credential: v.CustomResource.Spec.BackupLocations[0].Velero.Credential,

mperetzred · 2022-02-13T08:24:56Z

tests/e2e/lib/dpa_helpers.go

+				Key: "cloud",
+			}
+		}
+		dpaInstance.Spec.SnapshotLocations = []oadpv1alpha1.SnapshotLocation{


is it possible to feed this info to the json instead? this way the code above already covers this initialization:

SnapshotLocations: v.CustomResource.Spec.SnapshotLocations, <<<<<<<<<<<<<<<<<<<<< BackupLocations: []oadpv1alpha1.BackupLocation{ { Velero: &velero.BackupStorageLocationSpec{ Provider: v.CustomResource.Spec.BackupLocations[0].Velero.Provider, Default: true, Config: v.CustomResource.Spec.BackupLocations[0].Velero.Config, Credential: v.CustomResource.Spec.BackupLocations[0].Velero.Credential,

mperetzred · 2022-02-13T08:28:06Z

tests/e2e/lib/dpa_helpers.go

@@ -84,6 +98,73 @@ func (v *DpaCustomResource) Build(backupRestoreType BackupRestoreType) error {
 			},
 		},
 	}
+	switch v.Provider {
+	case "aws":


can do instead of this switch:

if v.OpenshiftCi { if dpaInstance.Spec.BackupLocations[0].Velero.Config != nil { dpaInstance.Spec.BackupLocations[0].Velero.Config["credentialsFile"] = fmt.Sprintf(""bsl-cloud-credentials-%v/cloud"+line+"\n", v.Provider) } } else { dpaInstance.Spec.BackupLocations[0].Velero.Credential = &corev1.SecretKeySelector{ LocalObjectReference: corev1.LocalObjectReference{ Name: v.CredSecretRef, }, Key: "cloud", } }

mperetzred · 2022-02-13T08:30:10Z

tests/e2e/scripts/azure_setting.sh

+        {
+          "velero": {
+            "provider": "$PROVIDER",
+            "config": {


is it possible to feed the info from the makefile to here and use it throughout the tests instead of passing it as arguments? this way the code would be more generic

mperetzred · 2022-02-13T08:41:04Z

tests/e2e/e2e_suite_test.go


+			Expect(err).NotTo(HaveOccurred())
+			dpaCR.DpaAzureConfig = DpaAzureConfig{


is it possible to feed this info to the json script?

mperetzred · 2022-02-13T08:43:24Z

tests/e2e/e2e_suite_test.go

+	if openshift_ci_bool == true {
+		switch dpaCR.Provider {
+		case "aws":
+			cloudCredData, err := utils.ReadFile(dpaCR.Credentials)


Instead of switching over each provider, can do:

cloudCredData, err := utils.ReadFile(dpaCR.Credentials) Expect(err).NotTo(HaveOccurred()) err = CreateCredentialsSecret(cloudCredData, namespace, fmt.Sprintf("bsl-cloud-credentials-$v", dpaCr.Provider) Expect(err).NotTo(HaveOccurred()) dpaCR.Credentials = ci_cred_file

mperetzred · 2022-02-13T08:48:00Z

tests/e2e/lib/kube_helpers.go

+	return resourceGroup, err
+}
+
+func WriteFile(credFile string, data []byte) error {


belongs more to common.go

mperetzred · 2022-02-13T08:48:21Z

tests/e2e/lib/kube_helpers.go

@@ -217,3 +219,46 @@ func isCredentialsSecretDeleted(namespace string, credSecretRef string) wait.Con
 		return false, err
 	}
 }
+
+func GetJsonData(path string) (map[string]interface{}, error) {


belongs more to common.go

mperetzred · 2022-02-13T08:49:36Z

tests/e2e/lib/dpa_helpers.go

@@ -37,13 +37,27 @@ const (
 	RESTIC BackupRestoreType = "restic"
 )

+type DpaAzureConfig struct {


can be read from the json instead to keep the code platform agnostic

mperetzred · 2022-02-13T08:49:42Z

tests/e2e/lib/dpa_helpers.go

 type DpaCustomResource struct {
 	Name              string
 	Namespace         string
 	SecretName        string
 	backupRestoreType BackupRestoreType
 	CustomResource    *oadpv1alpha1.DataProtectionApplication
 	Client            client.Client
+	DpaAzureConfig    DpaAzureConfig


can be read from the json instead to keep the code platform agnostic

openshift-ci · 2022-02-13T20:15:19Z

@deepakraj1997: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

weshayutin · 2022-02-14T03:15:06Z

Let's land this and start working through @mperetzred's comments. 3 e2e passes in a row +2 approvals. Commiting the change will free up others e2e patches that are pending.

mperetzred · 2022-02-14T09:59:36Z

@weshayutin sure.
@deepakraj1997, let me know if you need any help, would be happy to help with it.

deepakraj1997 · 2022-02-14T14:27:35Z

Thanks @mperetzred . To summarize my response to your review comments, The reason I have switch is due to azure where the credentials and other informations are split up in the CI environment. I had to join them and put them in a single file for VSL and there are lots of additional parameters required in the instance template. Most of your review comments will be resolved if we can parse the required files in Makefile and put it in JSON templates for different providers.

I will work on this incrementally. Merging this @weshayutin

kaovilai · 2022-04-13T23:31:19Z

Makefile

+# bsl cred file
+OADP_CRED_FILE ?= ${OADP_CRED_DIR}/new-aws-credentials
+# vsl cred file
+CI_CRED_FILE ?= ${CLUSTER_PROFILE_DIR}/.awscred


After much confusion debugging ci.. this probably should've been called VSL_CRED_FILE

* Adding changes from ci-multi-cloud branch * Updating flags in makefile * Updating flags in makefile * Updating flags in makefile * Update defaults * Fix helpers * Cleaning up makefile * Update makefile * Update kustomization * Cleaning up Makefile * Adding changes for AWS CredentialFile * Handling no default backuplocations * Removing comments * Adding config check in helpers * Fix AWS Test case * Fix basic review comments

…n updateRegistrySecret (#679) * Replace carriage return when getting provider secrets for registry + E2E changes (#652) * Replace carriage return when getting provider secrets for registry Co-authored-by: GitHub Co-pilot <idkwhatemailgithubcopilotuses@github.com> * add tests * replaceCarriageReturn unit test * Registry credentials can use other secret names Co-authored-by: GitHub Co-pilot <idkwhatemailgithubcopilotuses@github.com> * Multi cloud E2E Updates (#568) * Adding changes from ci-multi-cloud branch * Updating flags in makefile * Updating flags in makefile * Updating flags in makefile * Update defaults * Fix helpers * Cleaning up makefile * Update makefile * Update kustomization * Cleaning up Makefile * Adding changes for AWS CredentialFile * Handling no default backuplocations * Removing comments * Adding config check in helpers * Fix AWS Test case * Fix basic review comments * genericTests... * Changing Credentials Mount Name for BSL when CredentialsFile key is used in Config - Fixes CI (#637) * Changing Credentials Mount Name for BSL for GCP * Fix DefaultPlugin for credentialsFile * E2E Tests: Fix azure templating name. (#573) * Fixing Azure to run locally * Aligning json * Handling Azure Parameters in CI Env using templates. (#582) * Adding azure parameters * Adding Openshift CI params * Update Test Suite and Helpers * Adding cred ref * Adding cred ref * Changing azure oadp cred dir to tmp * Fixing duplicate error * Removing OpenShift CI * Changing test instance name * Changing the AWS BSL Profile to default * Changing BslMountPath variable in registry controller * Replace carriage return when getting provider secrets for registry + E2E changes (#652) * Replace carriage return when getting provider secrets for registry Co-authored-by: GitHub Co-pilot <idkwhatemailgithubcopilotuses@github.com> * add tests * replaceCarriageReturn unit test * Registry credentials can use other secret names Co-authored-by: GitHub Co-pilot <idkwhatemailgithubcopilotuses@github.com> * fix missing err return for updateRegistrySecret function * go fmt * upgraded ginkgo on tests to ginkgo 2.0 (#560) Co-authored-by: Nitish Srivastava <43022982+nitishSr@users.noreply.github.com> Co-authored-by: GitHub Co-pilot <idkwhatemailgithubcopilotuses@github.com> Co-authored-by: Deepak Raj D S <drajds@redhat.com> Co-authored-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com> Co-authored-by: Maya Peretz <mperetz@redhat.com>

openshift-ci bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Feb 10, 2022

openshift-ci bot requested review from jwmatthews and sseago February 10, 2022 18:13

This was referenced Feb 10, 2022

Changed backup suite to be more generic & changed mssql app to mysql #565

Closed

[WIP] adding negative error testing to e2e test suite #516

Closed

deepakraj1997 added 7 commits February 11, 2022 01:52

Adding changes from ci-multi-cloud branch

71bea25

Updating flags in makefile

3e94c27

Updating flags in makefile

2192676

Updating flags in makefile

056cb42

Update defaults

12cd710

Fix helpers

914ad3c

Cleaning up makefile

38ec7a9

deepakraj1997 force-pushed the multi-cloud-e2e-master-rebase branch from 7a39fd2 to 38ec7a9 Compare February 11, 2022 06:55

openshift-ci bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 11, 2022

deepakraj1997 changed the base branch from master to ci-multi-cloud February 11, 2022 14:40

deepakraj1997 changed the base branch from ci-multi-cloud to master February 11, 2022 14:40

deepakraj1997 added 2 commits February 11, 2022 09:41

Update makefile

3e0f678

Update kustomization

3c21bad

deepakraj1997 marked this pull request as ready for review February 11, 2022 14:51

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 11, 2022

openshift-ci bot requested a review from shubham-pampattiwar February 11, 2022 14:52

weshayutin reviewed Feb 11, 2022

View reviewed changes

kaovilai approved these changes Feb 11, 2022

View reviewed changes

Cleaning up Makefile

113ffec

dymurray reviewed Feb 11, 2022

View reviewed changes

mperetzred reviewed Feb 13, 2022

View reviewed changes

Fix basic review comments

9bedca6

deepakraj1997 changed the title ~~Multi cloud e2e master rebase [WIP]~~ Multi cloud e2e master rebase Feb 13, 2022

deepakraj1997 merged commit 22f0807 into openshift:master Feb 14, 2022

deepakraj1997 mentioned this pull request Feb 17, 2022

E2E Tests: Fix azure templating name. #573

Merged

kaovilai reviewed Apr 13, 2022

View reviewed changes

kaovilai mentioned this pull request May 9, 2022

[OADP-463] cherrypicks 1.0 - carriage return + #678 missing err return updateRegistrySecret #679

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Multi cloud e2e master rebase #568

Multi cloud e2e master rebase #568

deepakraj1997 commented Feb 10, 2022 •

edited

Loading

openshift-ci bot commented Feb 10, 2022

kaovilai commented Feb 10, 2022

codecov-commenter commented Feb 11, 2022 •

edited

Loading

weshayutin Feb 11, 2022

deepakraj1997 Feb 11, 2022 •

edited

Loading

weshayutin left a comment

dymurray left a comment

weshayutin commented Feb 12, 2022

mperetzred Feb 13, 2022

mperetzred Feb 13, 2022

mperetzred Feb 13, 2022

mperetzred Feb 13, 2022

mperetzred Feb 13, 2022

mperetzred Feb 13, 2022

mperetzred Feb 13, 2022 •

edited

Loading

mperetzred Feb 13, 2022 •

edited

Loading

mperetzred Feb 13, 2022

mperetzred Feb 13, 2022

mperetzred Feb 13, 2022

mperetzred Feb 13, 2022

mperetzred Feb 13, 2022

mperetzred Feb 13, 2022

mperetzred Feb 13, 2022

mperetzred Feb 13, 2022

openshift-ci bot commented Feb 13, 2022

weshayutin commented Feb 14, 2022

mperetzred commented Feb 14, 2022 •

edited

Loading

deepakraj1997 commented Feb 14, 2022 •

edited

Loading

kaovilai Apr 13, 2022


		Expect(err).NotTo(HaveOccurred())
		dpaCR.DpaAzureConfig = DpaAzureConfig{

Multi cloud e2e master rebase #568

Multi cloud e2e master rebase #568

Conversation

deepakraj1997 commented Feb 10, 2022 • edited Loading

openshift-ci bot commented Feb 10, 2022

kaovilai commented Feb 10, 2022

codecov-commenter commented Feb 11, 2022 • edited Loading

Codecov Report

Choose a reason for hiding this comment

deepakraj1997 Feb 11, 2022 • edited Loading

Choose a reason for hiding this comment

weshayutin left a comment

Choose a reason for hiding this comment

dymurray left a comment

Choose a reason for hiding this comment

weshayutin commented Feb 12, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mperetzred Feb 13, 2022 • edited Loading

Choose a reason for hiding this comment

mperetzred Feb 13, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

openshift-ci bot commented Feb 13, 2022

weshayutin commented Feb 14, 2022

mperetzred commented Feb 14, 2022 • edited Loading

deepakraj1997 commented Feb 14, 2022 • edited Loading

Choose a reason for hiding this comment

deepakraj1997 commented Feb 10, 2022 •

edited

Loading

codecov-commenter commented Feb 11, 2022 •

edited

Loading

deepakraj1997 Feb 11, 2022 •

edited

Loading

mperetzred Feb 13, 2022 •

edited

Loading

mperetzred Feb 13, 2022 •

edited

Loading

mperetzred commented Feb 14, 2022 •

edited

Loading

deepakraj1997 commented Feb 14, 2022 •

edited

Loading