Add e2e scaffolding using Testify Suites

[codegold79]

Per @nrb's review of PR #3133, this PR splits out the e2e Testify Suites framework and supporting code.

Signed-off-by: F. Gold fgold@vmware.com

[zubron]

Thanks for splitting this out into a separate PR, @codegold79!

I think it will be nice to have the option on how to write tests (using testify vs ginkgo) but I'm concerned about the long term maintainability of having two separate build/run processes. I'm not familiar with how ginkgo tests run, but is it possible to have both regular go tests and ginkgo tests in one package? If yes, then I think that ultimately we should move any tests using that approach inside the e2e package rather than creating a new one. Otherwise, it will be very easy for these to get out of sync and increase the maintenance burden. Also, I think that some of the functionality added in this package will be useful for the main e2e package :)

I don't want to block your progress on this though, @codegold79, so I think we can address the overall structure of the e2e packages in a follow up and I don't want to consider it a blocker for this PR. I mostly just want to make sure that we've considered the impact on the rest of the e2e tests moving forward as there is a lot of work happening in that area.

@dsu-igeek What are your thoughts on this?

[zubron]

I'm apprehensive about repeating these flag definitions again. I don't want us to have two separate sets of flags that are passed through to a shared set of utility functions. I know that #3286 is changing some of these flags in the e2e and adding more that will need to handled. I don't want to block your progress on this and #3133 though, as you've already had to wait a long time on that :)

I think there are probably general improvements to the flags in the main e2e package (such as using a struct to contain them) that can be made which would make sharing across these two packages easier. I'll open a follow up issue for this.

[zubron]

You could export getProviderPlugins so that you can use that function.

[zubron]

This can be addressed in #3133, but I think it would be better to specify the namespace where the CRDs are being installed using the ns value passed in rather than having to know the namespace specified in the YAML.

[codegold79]

Thank you, @zubron, for your review. I completely agree that it is difficult to maintain a parallel set of code (i.e. duplicated) and already some of my code is outdated.

I had wanted to provide an example of using Testify Suites for e2e tests so people would have something to compare with Ginkgo in Velero. However, per @nrb in a code review meeting for PR #3133, given the limited resources (including time), we won't be rewriting the Ginkgo tests. So I'll write tests for PR #3133 to work with the existing Ginkgo framework. I have copied over the Testify Suites code to this PR for future reference and will close the PR.