Skip to content
This repository has been archived by the owner on Nov 29, 2023. It is now read-only.

feat: add testground to ci [work in progress] #363

Closed
wants to merge 1 commit into from

Conversation

matthias-wright
Copy link
Collaborator

See #311.
This is a draft PR because it's not working currently. I'm running into testground/testground#1541. Interestingly, this only happens when using Github workflows and never when running the test plans locally. The problem is that the directories for the current run are not created at /root/.config/testground/data/outputs/local_docker/fetching/<run-id>/<plan>/<index>. The directory /root/.config/testground/data/outputs exists. Manually creating these directories might be a quick fix but won't solve the underlying issue.
I created a discussion in the testground repo for this issue: testground/testground#1568

Another thing to note:
At the moment, we are pulling the ursa crates from the ursa git repository. This can be seen for example here: https://github.com/fleek-network/ursa/blob/main/test-plans/fetching/Cargo.toml. This is fine but defeats the purpose of running the test plans on every PR. For this, we have to use the code from the current commit. The most straightforward way that I came up with to achieve this is to copy the code from the ursa crates into the test plan directories and then temporarily overwriting the corresponding Cargo.toml.

I added a script scripts/run_testground.sh that takes care of this when running testground locally.

@kckeiks
Copy link
Collaborator

kckeiks commented Feb 6, 2023

Thanks for taking this on!

Will the CI only validate that the tests build or that they ran successfully? The latter will be more complex right?

@matthias-wright
Copy link
Collaborator Author

Thanks for taking a look! If the test plans would actually run without errors, then the CI will validate that they run successfully. In this line, the --wait argument will make the command wait for the test plan to complete and the exit status will be 0 only if the test plan succeeded. The github ci then picks up on the exit status, so it should all work out.

@matthias-wright
Copy link
Collaborator Author

btw I can recommend https://github.com/nektos/act for running workflows locally without always having to push something to github.

@b0xtch
Copy link
Contributor

b0xtch commented Feb 7, 2023

looks good

@jxs
Copy link

jxs commented Feb 18, 2023

Hey folks, this might interest you: libp2p/test-plans#103

@b0xtch
Copy link
Contributor

b0xtch commented Mar 7, 2023

Should we close this, given the move away from test ground?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants