Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rollouts: end to end tests #3879

Merged
merged 6 commits into from
Mar 21, 2023
Merged

Conversation

droot
Copy link
Contributor

@droot droot commented Mar 13, 2023

This PR does the following:

  • Adds support for spinning up kind target clusters (huge shoutout to @ChristopherFry . Code is heavily borrowed from his internal code)
  • Adds e2e tests with one target cluster for rolling out a simple namespace package for a non-progressive and progressive rollouts. It also covers manually advancing through the waves for progressive rollouts.

I think this sets up our e2e test story nicely (except the git part), It should be easy to add new tests now.

cnrm.cloud.google.com/system: "true"
cnrm.cloud.google.com/tf2crd: "true"
core.cnrm.cloud.google.com/configconnector: configconnector.core.cnrm.cloud.google.com
name: containerclusters.container.cnrm.cloud.google.com
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this file come from another Github source? We could consider using the remote URL in the kustomization in that case, instead of copying it over

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to find a better solution for this CRD. This is an external CRD (comes from KCC) and is already part of our release manifests, I will try to reuse that instead of adding a new one here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am reusing the containercluster.yaml from the manifests directory for now. So we have one source of truth for now. Eventually we want to make rollouts to be able to gracefully handle unavailability of this CRD.

@droot droot force-pushed the rollouts-integration-tests branch from 42514ab to 3c7e341 Compare March 17, 2023 23:14
@droot droot marked this pull request as ready for review March 17, 2023 23:20
@droot droot requested a review from a team as a code owner March 17, 2023 23:20
@droot droot requested a review from justinsb March 17, 2023 23:20
@droot droot changed the title rollouts: basic integration test rollouts: end to end tests Mar 19, 2023
Copy link
Contributor

@natasha41575 natasha41575 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pr lgtm! I have small comments

err := testEnv.Stop()
Expect(err).NotTo(HaveOccurred())
_ = testEnv.Stop()
// Expect(err).NotTo(HaveOccurred())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this an intentional change? If so, curious why we are doing this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is an intentional change. Added a note with a TODO explaining why we are ignore this assertions.

Prefix: "e2e-sfoo-",
Count: 1,
Labels: map[string]string{
"city": "sfoo",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why sjcc and sfoo instead of sjc and sfo? Is there some conflict with the clusters setup in the previous test?

Since this is otherwise the same function passed to BeforeEach as the previous test, I was wondering if it would make sense to define this func elsewhere and reuse it across tests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want these two tests to be independent of each other so that we can run them in parallel (though running in parallel requires running with ginkgo CLI, but that's a different story). Since we share the APIServer/Etcd between the two tests, keeping a separate target clusters is required to keep these tests independent.

Also, each tests may require different target cluster environment (for ex. progressive may have more advance test setup here), so keeping them separate gives that flexibility. So duplication here is the feature not a bug :)

targetClusterSetup, err = e2eclusters.GetClusterSetup(tt, k8sClient, targets...)
Expect(err).NotTo(HaveOccurred())

Expect(targetClusterSetup.PrepareAndWait(context.TODO(), 5*time.Minute)).To(Succeed())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we expecting to use 5*time.Minute in every test? If so, maybe we can put it in a const

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So these timeouts really depend on the target cluster setup and that may vary between these tests, so avoiding the reuse.

@droot
Copy link
Contributor Author

droot commented Mar 21, 2023

@natasha41575 responded to your comments, pl. take another look and let me know.

@droot droot merged commit ce0ff27 into kptdev:main Mar 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants