Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

1233-Rework tests #306

Merged
merged 31 commits into from
May 4, 2023
Merged

1233-Rework tests #306

merged 31 commits into from
May 4, 2023

Conversation

jimmykarily
Copy link
Contributor

@jimmykarily jimmykarily commented Apr 3, 2023 •
edited
Loading

Fixes #1233

@codecov-commenter
Copy link

codecov-commenter commented Apr 3, 2023 •
edited
Loading

Codecov Report

Merging #306 (f7a2696) into main (d88983a) will increase coverage by 0.02%.
The diff coverage is 0.00%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff             @@
##             main     #306      +/-   ##
==========================================
+ Coverage   12.34%   12.37%   +0.02%     
==========================================
  Files          17       17              
  Lines        1069     1067       -2     
==========================================
  Hits          132      132              
+ Misses        910      908       -2     
  Partials       27       27
Impacted Files Coverage Δ
internal/cli/bridge.go 0.00% <ø> (ø)
internal/cli/recovery.go 0.00% <ø> (ø)
internal/cli/register.go 0.00% <ø> (ø)
internal/cli/start.go 0.00% <0.00%> (ø)
internal/provider/challenge.go 71.42% <ø> (ø)
internal/provider/install.go 0.00% <ø> (ø)
internal/provider/recovery.go 0.00% <ø> (ø)

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@jimmykarily jimmykarily changed the title [WIP] Migrate to a non-global VM 1233-Rework tests Apr 3, 2023
@jimmykarily jimmykarily force-pushed the 1233-rework-tests branch 5 times, most recently from 057a68d to c17223a Compare April 7, 2023 13:34
@jimmykarily
Copy link
Contributor Author

jimmykarily commented Apr 7, 2023 •
edited
Loading

Most test are green locally. They can now run locally. Tthe only env variable needed being the ISO
except for the upgrade test which also needs CONTAINER_IMAGE.

CLOUD_INIT is not needed anymore. They tests can run with qemu too (USE_QEMU=true), since Screenshot() is now implemented. I run the tests locally with:

MACHINE_SPICY=true KVM=true USE_QEMU=true ISO=/home/dimitris/workspace/kairos/provider-kairos/build/kairos-opensuse-leap-v1.6.1-19-g5a211fd-k3s.iso ginkgo -v tests/
  • MACHINE_SPICY is optional and shows the VMs running with spicy
  • KVM makes tests run faster but can't be used on CI yet because we don't have access to kvm
  • USE_QEMU makes the tests run with qemu instead of VirtualBox.
  • ISO is the iso being tested and should be built beforehand.

TODO:

  • Fix the qrcode test (can be done on another PR, it's disabled in CI anyway)
  • Make CI green (maybe it's already green, but CI is under heave load and won't start my jobs)

@jimmykarily jimmykarily force-pushed the 1233-rework-tests branch 8 times, most recently from 79cee2e to bfdeafe Compare April 24, 2023 12:48
@jimmykarily jimmykarily mentioned this pull request Apr 24, 2023
Closed
@jimmykarily jimmykarily force-pushed the 1233-rework-tests branch 2 times, most recently from 709c744 to 967e459 Compare April 25, 2023 09:06
@jimmykarily jimmykarily mentioned this pull request Apr 25, 2023
Merged
@jimmykarily
Copy link
Contributor Author

I think it is timing out on decentralized test.

@jimmykarily jimmykarily mentioned this pull request May 2, 2023
Closed
@jimmykarily
Refactor tests
fe55649 
- Migrate to a non-global VM object
- Refactor qr registration and decentralized tests
  to not need a separate CI pipeline to run in parallel
- Add valid header on config generated by `create-config`
- Run decentralized tests with 2 VMs in parallel
- Enable available-releases test
- Bump github action and golang version used in tests
- Fix qr code registration test
- Import go-nodepair from kairos-io
- Bump go-nodepair to depend on the same version of edgevpn
  and fix compilation issues
- Rename jobs to match test files and merge available_releases with test job
- Run all jobs on ubuntu-latest with qemu

Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Remove mac commands from ubuntu worker
23b7cbc 
Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Setup qemu in Github Action worker
f71f9b9 
Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Add debug output
60322b8 
Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Bump peg to show better error
e3e82f5 
Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Bump go-nodepair to fix kairos-io/kairos#1251
79dbbb2 
Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Show free disk space
d345f13 
(suspecting we don't have enough space to create a 40g disk for qemu)

Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Bump peg in order to control the size of the vm disk to be created
113406a 
because we don't have enough space for the previous hardcoded 30g for qemu

Also install missing qemu-img package

Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Install qemu the same way we do in kairos-io/kairos
3952b7a 
because the GA action doesn't put the binaries in a place where I can
find them.

Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Decrease memory per VM to be able to fit 2 of them in the same worker
2a063f8 
so that decentralized tests can run successfully

Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Decrease required resources more and remove debug text
89cc0d0 
Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Remove dependencies from tests
1f30f94 
to allow them to run in parallel and see if the rest of them (other than
the decentralized one) pass.

Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Run tests on self-hosted workers with KVM
01d16f5 
Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Use fork of action
810881d 
until this is merged: docker-practice/actions-setup-docker#31

Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Don't setup docker
53ccefb 
On our self-hosted runners, it's already there

Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Run build job both in self-hosted and ubuntu-latest runners
f8f327b 
to save time

Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Run matrix job on any runner
92af7bb 
Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Always cancel PR older CI jobs
c3e42c4 
because the head_ref may change if we rebase the PR, which will result
in the old jobs to keep running. By using the pull_request.number we
make sure we always stop older jobs for the same PR (even on rebase).

Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Change step condition to detect self-hosted runners
58fc2a3 
Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Fix step conditional
6f23ae6 
Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Revert hacks, Github is doesn't support running in any worker
9a053ab 
(unless maybe if we label our self-hosted runners as "ubuntu-latest" or
something)

"These labels operate cumulatively, so a self-hosted runner must have all four labels to be eligible to process the job."

https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/using-self-hosted-runners-in-a-workflow

It's not an "OR" it's and "AND"

Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Remove unecessary actions (not working) and run apt update
1b60648 
before we install packages

Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Fix pipeline and detect flavor using /etc/os-release
87d57ba 
Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Fix syntax errors and remove unnecessary "And"
819e700 
Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Double the waiting time until vm reboots
8b04743 
to see if this fixes the timeouts in CI

Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Add debug output
a020aa3 
Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Increase more Reboot() timeouts
82449cc 
Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Increase decentralized_test resources
0d4d7a2 
to avoid timeouts. It passes locally with this amount of resources so...

Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Increase memory and run ginkgo recover before go routine 'Done()'
f762974 
because otherwise, the test continues and prints the next "By" statement
between running "Done" and running "Recover" making the output
confusing.

Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Ditch go routines and do things serial
9531b13 
because making ginkgo stop when our goroutines fails is unexpectedly
difficult

Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily
Don't break the config file with go mod tidy output
6d86ed1 
By inspecting the generated config file for the decentralized test in
the CI runner, I found that it had lines like:

```
go: downloading github.com/kairos-io/go-nodepair v0.0.0-20230424150716-d8f0757542c3
go: downloading github.com/kairos-io/kairos/v2 v2.0.3
go: downloading github.com/mudler/edgevpn v0.23.1
go: downloading github.com/mudler/yip v1.0.0
...
```

This comes from the fact that the first time we run the `main.go` file,
the dependencies need to be downloaded. By running `go mod tidy` first,
the `go run` command won't print those.

Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
@jimmykarily jimmykarily marked this pull request as ready for review May 3, 2023 08:50
mauromorales
mauromorales approved these changes May 4, 2023
View reviewed changes
Copy link
Member

@mauromorales mauromorales left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't remember the last time I saw these tests green LOL

💪

@jimmykarily jimmykarily merged commit 99b8b7c into main May 4, 2023
@jimmykarily jimmykarily deleted the 1233-rework-tests branch May 4, 2023 16:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mauromorales mauromorales mauromorales approved these changes

Assignees

@jimmykarily jimmykarily

Labels
None yet
Projects
🧙Issue tracking board
Archived in project
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Rework provider-kairos tests
3 participants
@jimmykarily @codecov-commenter @mauromorales