Set unsupported images via shell in our e2e tests #1541

weshayutin · 2024-10-03T17:56:58Z

Why the changes were made

Generally speaking this will be handy for dev's to be able to easily use custom images in our e2e tests.
This especially is handy as we build out and use openshift/oadp-operator e2e tests across our various repos.

How to test the changes made

Each cloud provider has a json template for the dpa under tests/e2e/scripts. All the unsupported Image Overrides were added to each provider. If the variable is empty oadp will use the default images [1]. The map of strings is patched to the created dpa.

[1]

      podConfig:
        resourceAllocations: {}
  podDnsConfig: {}
  unsupportedOverrides:
    openshiftPluginImageFqin: ''
    gcpPluginImageFqin: ''
    veleroImageFqin: 'quay.io/rhn_engineering_whayutin/velero:testlatest'
    azurePluginImageFqin: ''
    resticRestoreImageFqin: ''
    nonAdminControllerImageFqin: ''
    kubevirtPluginImageFqin: ''
    awsPluginImageFqin: ''
    csiPluginImageFqin: ''

Signed-off-by: Wesley Hayutin <weshayutin@gmail.com>

…ication CSI' Signed-off-by: Wesley Hayutin <weshayutin@gmail.com>

Signed-off-by: Wesley Hayutin <weshayutin@gmail.com>

kaovilai

Wow. Just Amazing.

tests/e2e/lib/dpa_helpers.go

docs/developer/testing/TESTING.md

tests/e2e/e2e_suite_test.go

mateusoliveira43 · 2024-10-03T18:38:49Z

tests/e2e/lib/dpa_helpers.go

@@ -155,13 +150,16 @@ func (v *DpaCustomResource) CreateOrUpdate(c client.Client, spec *oadpv1alpha1.D
 				},
 				Spec: *spec.DeepCopy(),
 			}
-			return v.Create(dpa)


All changes from this line on, I think are unnecessary

the workflow is: UnsupportedOverrides will be in json, tests/e2e/e2e_suite_test.go will load the values, line 96 of this file will add values to DPAs created by test run

changes being done on this line on are duplication of was already done in line 96

You are incorrect. There is duplication in how the tests create the dpa. My addition requires the above. I was also confused about that. DPA creation is handled in two different ways.
Follow the dpa creation from the two points:

make test-e2e GINKGO_ARGS="--ginkgo.focus='MySQL application DATAMOVER'"

make test-e2e GINKGO_ARGS="--ginkgo.focus='DPA reconciled to true'"

Should we perhaps fix that yes, but it's certainly NOT in scope for this change :) 1 scope per PR.

Understood what you want to do, but I do not think this is the correct approach

Since tests here https://github.com/openshift/oadp-operator/blob/d5adc852931f1e23a24c7f0be7f86a27d107ce2d/tests/e2e/dpa_deployment_suite_test.go do not use DpaCustomResource.Build function, you were not seeing UnsupportedOverrides there

But this change breaks this test

oadp-operator/tests/e2e/dpa_deployment_suite_test.go

Lines 362 to 369 in d5adc85

ginkgo.Entry("DPA CR with unsupportedOverrides", ginkgo.Label("aws", "ibmcloud"), InstallCase{

DpaSpec: createTestDPASpec(TestDPASpec{

BSLSecretName: bslSecretName,

UnsupportedOverrides: map[oadpv1alpha1.UnsupportedImageKey]string{

"awsPluginImageFqin": "quay.io/konveyor/velero-plugin-for-aws:oadp-1.4",

},

}),

}),

If I export AWS_PLUGIN_IMAGE, or not, the value defined in json will overwrite value defined in test

The way I would go with this:

remove the test mentioned above (since the idea is to run these tests with values of UnsupportedOverrides in the near future, so test would not be necessary anymore)

remove this

oadp-operator/tests/e2e/dpa_deployment_suite_test.go

Line 35 in d5adc85

UnsupportedOverrides map[oadpv1alpha1.UnsupportedImageKey]string

change testSpec.UnsupportedOverrides here to dpaCR.UnsupportedOverrides

oadp-operator/tests/e2e/dpa_deployment_suite_test.go

Line 69 in d5adc85

UnsupportedOverrides: testSpec.UnsupportedOverrides,

ok.. thank you for the in depth review! I think I have made the changes you requested, I'm going to push an update and take some time to double check the changes and test.

with the current changes, I think you can safely undo changes in lines 156-158 and line 165

the same changes are being applied to line 68 of tests/e2e/dpa_deployment_suite_test.go

openshift-ci · 2024-10-03T20:52:12Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: weshayutin
Once this PR has been reviewed and has the lgtm label, please assign jwmatthews for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci · 2024-10-03T20:52:31Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: weshayutin
Once this PR has been reviewed and has the lgtm label, please assign jwmatthews for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

thanks, addressed all the comments sir!

weshayutin · 2024-10-04T01:36:36Z

/retest

docs/developer/testing/TESTING.md

* removed image override test * remove unsed var declaration * other mindor changes Signed-off-by: Wesley Hayutin <weshayutin@gmail.com>

Signed-off-by: Wesley Hayutin <weshayutin@gmail.com>

openshift-ci · 2024-10-04T20:03:58Z

@weshayutin: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/4.15-e2e-test-aws	`131b389`	link	true	`/test 4.15-e2e-test-aws`
ci/prow/4.16-e2e-test-aws	`131b389`	link	true	`/test 4.16-e2e-test-aws`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

weshayutin added 5 commits October 2, 2024 15:39

not working

9f868a4

Signed-off-by: Wesley Hayutin <weshayutin@gmail.com>

WOOT, working w/ make test-e2e GINKGO_ARGS=--ginkgo.focus='MySQL appl…

6a7812c

…ication CSI' Signed-off-by: Wesley Hayutin <weshayutin@gmail.com>

all dpa creation / tests working

a832745

Signed-off-by: Wesley Hayutin <weshayutin@gmail.com>

update other cloud providers w/ args

0f26958

Signed-off-by: Wesley Hayutin <weshayutin@gmail.com>

add note to docs

b2d9dd2

Signed-off-by: Wesley Hayutin <weshayutin@gmail.com>

openshift-ci bot requested review from hhpatel14 and sseago October 3, 2024 17:57

kaovilai previously approved these changes Oct 3, 2024

View reviewed changes

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 3, 2024

mateusoliveira43 reviewed Oct 3, 2024

View reviewed changes

tests/e2e/lib/dpa_helpers.go Show resolved Hide resolved

mateusoliveira43 reviewed Oct 3, 2024

View reviewed changes

docs/developer/testing/TESTING.md Outdated Show resolved Hide resolved

mateusoliveira43 reviewed Oct 3, 2024

View reviewed changes

tests/e2e/e2e_suite_test.go Outdated Show resolved Hide resolved

mateusoliveira43 reviewed Oct 3, 2024

View reviewed changes

This comment was marked as outdated.

Sign in to view

address comments and fix aws settings

f96662e

weshayutin dismissed kaovilai’s stale review via f96662e October 3, 2024 20:51

openshift-ci bot removed the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 3, 2024

mateusoliveira43 reviewed Oct 4, 2024

View reviewed changes

docs/developer/testing/TESTING.md Show resolved Hide resolved

weshayutin added 2 commits October 4, 2024 09:52

updated for Mateus's comments

80579c4

* removed image override test * remove unsed var declaration * other mindor changes Signed-off-by: Wesley Hayutin <weshayutin@gmail.com>

remove csi plugin override

131b389

Signed-off-by: Wesley Hayutin <weshayutin@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Set unsupported images via shell in our e2e tests #1541

Set unsupported images via shell in our e2e tests #1541

weshayutin commented Oct 3, 2024

kaovilai left a comment

mateusoliveira43 Oct 3, 2024

weshayutin Oct 3, 2024

mateusoliveira43 Oct 4, 2024

mateusoliveira43 Oct 4, 2024

weshayutin Oct 4, 2024

mateusoliveira43 Oct 4, 2024

This comment was marked as outdated.

openshift-ci bot commented Oct 3, 2024

openshift-ci bot commented Oct 3, 2024

weshayutin commented Oct 4, 2024

openshift-ci bot commented Oct 4, 2024

	ginkgo.Entry("DPA CR with unsupportedOverrides", ginkgo.Label("aws", "ibmcloud"), InstallCase{
	DpaSpec: createTestDPASpec(TestDPASpec{
	BSLSecretName: bslSecretName,
	UnsupportedOverrides: map[oadpv1alpha1.UnsupportedImageKey]string{
	"awsPluginImageFqin": "quay.io/konveyor/velero-plugin-for-aws:oadp-1.4",
	},
	}),
	}),

Set unsupported images via shell in our e2e tests #1541

Are you sure you want to change the base?

Set unsupported images via shell in our e2e tests #1541

Conversation

weshayutin commented Oct 3, 2024

Why the changes were made

How to test the changes made

kaovilai left a comment

Choose a reason for hiding this comment

mateusoliveira43 Oct 3, 2024

Choose a reason for hiding this comment

weshayutin Oct 3, 2024

Choose a reason for hiding this comment

mateusoliveira43 Oct 4, 2024

Choose a reason for hiding this comment

mateusoliveira43 Oct 4, 2024

Choose a reason for hiding this comment

weshayutin Oct 4, 2024

Choose a reason for hiding this comment

mateusoliveira43 Oct 4, 2024

Choose a reason for hiding this comment

This comment was marked as outdated.

openshift-ci bot commented Oct 3, 2024

openshift-ci bot commented Oct 3, 2024

weshayutin commented Oct 4, 2024

openshift-ci bot commented Oct 4, 2024