Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: run e2e tests with testcontainers #19003

Draft
wants to merge 25 commits into
base: master
Choose a base branch
from

Conversation

blakepettersson
Copy link
Member

@blakepettersson blakepettersson commented Jul 9, 2024

This change allows for the use of testcontainers to run E2E tests. Why would we want this? Mainly for three reasons:

  1. IDE friendlier
  2. Debugger friendlier
  3. Much easier to setup

It's a "one click" way (or a simple go test e2e way) to run E2E tests. To run using testcontainers, run go test e2e or make test-e2e-local - there is no need to do anything else.

This has a bunch of changes, but the majority of them is simply extracting the command flags from the affected controller commands to separate builders, which are then used for instantiating each controller programatically. The commands make use of these builders to set the flags as they previously did, so from a user perspective nothing changes.

Redis, K3S and the E2E server are run as testcontainers when running the E2E tests. The controllers are run in-process but spun off into separate goroutines.

Checklist:

  • Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this does not need to be in the release notes.
  • The title of the PR states what changed and the related issues number (used for the release note).
  • The title of the PR conforms to the Toolchain Guide
  • I've included "Closes [ISSUE #]" or "Fixes [ISSUE #]" in the description to automatically close the associated issue.
  • I've updated both the CLI and UI to expose my feature, or I plan to submit a second PR with them.
  • Does this PR require documentation updates?
  • I've updated documentation as required by this PR.
  • I have signed off all my commits as required by DCO
  • I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
  • My build is green (troubleshooting builds).
  • My new feature complies with the feature status guidelines.
  • I have added a brief description of why this PR is necessary and/or what this PR solves.
  • Optional. My organization is added to USERS.md.
  • Optional. For bug fixes, I've indicated what older releases this fix should be cherry-picked into (this may or may not happen depending on risk/complexity).

Copy link

bunnyshell bot commented Jul 9, 2024

❗ Preview Environment deployment failed on Bunnyshell

See: Environment Details | Pipeline Logs

Available commands (reply to this comment):

  • 🚀 /bns:deploy to redeploy the environment
  • /bns:delete to remove the environment

Copy link

bunnyshell bot commented Jul 9, 2024

✅ Preview Environment created on Bunnyshell but will not be auto-deployed

See: Environment Details

Available commands (reply to this comment):

  • 🚀 /bns:deploy to deploy the environment

@blakepettersson blakepettersson force-pushed the chore/run-e2e-tests-with-testcontainers branch 3 times, most recently from cba2c14 to 6654416 Compare July 9, 2024 22:28
Copy link

codecov bot commented Jul 9, 2024

Codecov Report

Attention: Patch coverage is 23.83107% with 505 lines in your changes missing coverage. Please review.

Project coverage is 50.64%. Comparing base (a071f93) to head (6654416).
Report is 10 commits behind head on master.

Files Patch % Lines
cmd/argocd-server/commands/argocd_server.go 0.00% 180 Missing ⚠️
...ntroller/commands/argocd_application_controller.go 0.00% 139 Missing ⚠️
.../argocd-repo-server/commands/argocd_repo_server.go 0.00% 130 Missing ⚠️
...t-controller/commands/applicationset_controller.go 74.83% 28 Missing and 10 partials ⚠️
server/server.go 44.44% 5 Missing ⚠️
cmd/argocd/commands/headless/headless.go 0.00% 4 Missing ⚠️
util/tls/tls.go 0.00% 4 Missing ⚠️
cmd/argocd/commands/admin/cluster.go 0.00% 2 Missing ⚠️
reposerver/gpgwatcher.go 0.00% 2 Missing ⚠️
util/cache/cache.go 96.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #19003      +/-   ##
==========================================
- Coverage   50.65%   50.64%   -0.01%     
==========================================
  Files         315      315              
  Lines       43286    43260      -26     
==========================================
- Hits        21926    21911      -15     
+ Misses      18861    18846      -15     
- Partials     2499     2503       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@blakepettersson blakepettersson force-pushed the chore/run-e2e-tests-with-testcontainers branch 7 times, most recently from 42d5c4c to a50a826 Compare July 17, 2024 09:21
- name: Run E2E testsuite
run: |
set -x
make test-e2e-local
goreman run stop-all || echo "goreman trouble"
Copy link
Member

@rumstead rumstead Jul 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was added to make sure the argocd processes would shut down gracefully and have the golang coverage from the e2e "flushed" to the files. We should just confirm that test-containers send the proper signals to cause the flush to happen.

(Just commenting about the contributors meeting today as a reminder :))

This change allows for the use of testcontainers to run E2E tests. Why
would we want this? Mainly for three reasons:

1. IDE friendlier
2. Debugger friendlier
3. Much easier to setup

It's a "one click" way (or a simple `go test e2e` way) to run E2E tests,
set the environment variable `ARGOCD_E2E_USE_TESTCONTAINERS=true` and
you're off to the races.

This has a bunch of changes, but the majority of them is simply creating
builders for each of the affected controller commands to separate builders,
which are used to be able instantiate each controller programatically.
The commands make use of these builders to set the flags as they
previously did.

Redis, K3S and the E2E server are run as testcontainers when running the
E2E tests. The controllers are run in-process but spun off into separate
goroutines.

Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
This works just as nicely and reduces the scope of the changes.

Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
This is required since we're running the controllers outside a container.

Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
This is to prevent the "dubious ownership" errors in Git.

Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
Since there is no need to set a specific serviceaccount right now for a
given controller, revert these changes.

Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
The repo-server and api server both needs to be able to have their
ports configured.

Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
Hardcode helm index port for now, but randomize api server and repo server ports.

Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
@blakepettersson blakepettersson force-pushed the chore/run-e2e-tests-with-testcontainers branch from ceb6b4e to 83a93a5 Compare August 7, 2024 21:12
Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
Signed-off-by: Blake Pettersson <blake.pettersson@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants