Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test automation: zero-downtime upgrades #1438

Merged
merged 9 commits into from
Jan 8, 2024

Conversation

sjberman
Copy link
Contributor

@sjberman sjberman commented Jan 3, 2024

Problem: Running system tests manually is a time consuming and arduous process, and can lead to inconsistent results.

Solution: Add automation for the zero-downtime upgrades test. This test must be run on GKE. The test deploys the previous released version of NGF, then begins sending http and https traffic while performing an upgrade to NGF. Results are rendered and written.

CSV files are getting too large to commit to the repository, so not including those going forward.

Closes #1364

  • I have read the CONTRIBUTING doc
  • I have added tests that prove my fix is effective or that my feature works
  • I have checked that all unit tests pass after adding my changes
  • I have updated necessary documentation
  • I have rebased my branch onto main
  • I will ensure my PR is targeting the main branch and pulling from my branch from my own fork

@sjberman sjberman requested a review from a team as a code owner January 3, 2024 20:34
@github-actions github-actions bot added documentation Improvements or additions to documentation tests Pull requests that update tests labels Jan 3, 2024
@sjberman sjberman removed the documentation Improvements or additions to documentation label Jan 3, 2024
tests/framework/load.go Fixed Show fixed Hide fixed
tests/framework/ngf.go Show resolved Hide resolved
tests/suite/dataplane_perf_test.go Show resolved Hide resolved
tests/suite/system_suite_test.go Show resolved Hide resolved
tests/framework/resourcemanager.go Outdated Show resolved Hide resolved
tests/suite/system_suite_test.go Show resolved Hide resolved
tests/suite/upgrade_test.go Outdated Show resolved Hide resolved
tests/suite/upgrade_test.go Show resolved Hide resolved
tests/suite/upgrade_test.go Show resolved Hide resolved
tests/suite/upgrade_test.go Outdated Show resolved Hide resolved
tests/suite/upgrade_test.go Outdated Show resolved Hide resolved
@github-actions github-actions bot added the documentation Improvements or additions to documentation label Jan 4, 2024
tests/scripts/vars.env-example Outdated Show resolved Hide resolved
tests/suite/upgrade_test.go Outdated Show resolved Hide resolved
tests/suite/upgrade_test.go Outdated Show resolved Hide resolved
tests/suite/upgrade_test.go Outdated Show resolved Hide resolved
@sjberman
Copy link
Contributor Author

sjberman commented Jan 4, 2024

@kate-osborn Unfortunately we don't have the tests set up yet to pull changes from local, so I have to test every change by pushing :)

Copy link
Contributor

@kate-osborn kate-osborn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

tests/scripts/vars.env-example Outdated Show resolved Hide resolved
@sjberman sjberman removed the documentation Improvements or additions to documentation label Jan 5, 2024
Copy link
Contributor

@pleshakov pleshakov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is great. I run it a few times, both with cases when everything went well and with downtime (for that, I reduced the sleep values in the values file).

I left a few suggestion. And I think there is a bug with waiting for leader election to happen

tests/suite/upgrade_test.go Outdated Show resolved Hide resolved
tests/framework/load.go Outdated Show resolved Hide resolved
tests/framework/resourcemanager.go Show resolved Hide resolved
Problem: Running system tests manually is a time consuming and arduous process, and can lead to inconsistent results.

Solution: Add automation for the zero-downtime upgrades test. This test must be run on GKE. The test deploys the previous released version of NGF, then begins sending http and https traffic while performing an upgrade to NGF. Results are rendered and written.

CSV files are getting too large to commit to the repository, so not including those going forward.
@github-actions github-actions bot added the documentation Improvements or additions to documentation label Jan 8, 2024
tests/framework/load.go Dismissed Show dismissed Hide dismissed
Copy link
Contributor

@pleshakov pleshakov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@sjberman sjberman enabled auto-merge (squash) January 8, 2024 22:14
@sjberman sjberman merged commit 9c1d3e9 into nginxinc:main Jan 8, 2024
27 checks passed
@sjberman sjberman deleted the tests/automate-upgrade-test branch January 8, 2024 22:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation tests Pull requests that update tests
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Automate Zero Downtime Upgrades Test
4 participants