Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: refactor how e2e tests are run in github actions #1175

Closed
Tracked by #877
MSalopek opened this issue Aug 1, 2023 · 0 comments · Fixed by #1191
Closed
Tracked by #877

CI: refactor how e2e tests are run in github actions #1175

MSalopek opened this issue Aug 1, 2023 · 0 comments · Fixed by #1191
Assignees
Labels
good first issue Good for newcomers scope: testing Code review, testing, making sure the code is following the specification. type: tech-debt Slows down development in the long run

Comments

@MSalopek
Copy link
Contributor

MSalopek commented Aug 1, 2023

At present e2e tests are part of the Automated tests job which runs unit-, integration- and e2e-tests sequentially.

It would be beneficial to split e2e tests from unit- and integration-tests to simplify the the result reporting.

e2e happy path should run separately from other tests and have its own workflow file. The job's name should be e2e-happy-path and it should run on every push/commit.

Sources:


Additionally, the manual-e2e and nightly-e2e test runs should be refactored to include multiple tests runs instead of running all e2e test scenarios sequentially.

e.g. the jobs can be changed to include multiple job sections:

jobs:
  test-happy-path:
    runs-on: ubuntu-latest
    timeout-minutes: 60
    steps:
      - uses: actions/setup-go@v4
        with:
          go-version: "1.20"
      - uses: actions/checkout@v3
      - name: happy pathE2E tests
        run: make test-e2e-short
    multiconsumer-path:
      runs-on: ubuntu-latest
      timeout-minutes: 60
      steps:
        - uses: actions/setup-go@v4
          with:
            go-version: "1.20"
        - uses: actions/checkout@v3
        - name: happy pathE2E tests
          run: make test-e2e-multi-consumer

Most of the logic regarding the test starts existis in main.go which the Makefile uses to start the tests. The Makefile and main.go flags may need to be extended to support running the distinct paths individually.


Note:

Check the comments in #1179 and consider splitting happy path into 2 smaller jobs - one with normal downtime and the other with soft opt-out/throttling downtime. That will require changes in the testing suite.

@MSalopek MSalopek added good first issue Good for newcomers scope: testing Code review, testing, making sure the code is following the specification. type: tech-debt Slows down development in the long run labels Aug 1, 2023
@github-project-automation github-project-automation bot moved this to 🩹 Triage in Cosmos Hub Aug 1, 2023
@MSalopek MSalopek moved this from 🩹 Triage to 📥 Todo in Cosmos Hub Aug 1, 2023
@MSalopek MSalopek assigned MSalopek and bermuell and unassigned MSalopek Aug 2, 2023
@bermuell bermuell moved this from 📥 Todo to 🏗 In progress in Cosmos Hub Aug 8, 2023
@bermuell bermuell moved this from 🏗 In progress to 👀 In review in Cosmos Hub Aug 9, 2023
@github-project-automation github-project-automation bot moved this from 👀 In review to ✅ Done in Cosmos Hub Aug 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers scope: testing Code review, testing, making sure the code is following the specification. type: tech-debt Slows down development in the long run
Projects
Status: ✅ Done
Development

Successfully merging a pull request may close this issue.

2 participants