Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE REQUEST] Run Tests on Vast.AI with a GitHub Action #81

Closed
movchan74 opened this issue Mar 13, 2024 · 0 comments · Fixed by #184
Closed

[FEATURE REQUEST] Run Tests on Vast.AI with a GitHub Action #81

movchan74 opened this issue Mar 13, 2024 · 0 comments · Fixed by #184

Comments

@movchan74
Copy link
Contributor

Feature Summary

Right now the tests are run on GitHub Actions with a CPU only. All the GPU tests are run locally. It would be beneficial to run the GPU tests on Vast.AI with a GitHub Action.

Justification/Rationale

  • Running the GPU tests on Vast.AI with a GitHub Action will allow us to run GPU tests from the CI/CD pipeline.
  • Potentially it will allow us to run the tests on different GPUs and configurations.
  • Save developers time by not having to run the tests locally.

Proposed Implementation

Vast.AI provides a REST API that allows us to create and manage virtual machines. We can use the API to create a virtual machine with a GPU, run the tests, and destroy the virtual machine.

The implementation can be done in the following steps:

  • Create a virtual machine with a GPU using the Vast.AI API. We already have a template for creating an instance for SDK development.
  • Log in to the virtual machine, git clone the repository, install the dependencies, and run the tests.
  • Destroy the virtual machine after the tests are done.

The implementation can be done in a GitHub Action. I would suggest to trigger the action manually (at least at the beginning) to avoid unnecessary costs.

Also, we can keep the virtual machine in the stopped state that only charges for the storage instead of destroying it after the tests are done. This will allow us to run the tests faster and probably even cheaper because we won't have to create and destroy the virtual machine every time we run the tests (which takes time and costs money).

As a stretch goal, we can run the tests on different GPUs and configurations to test the SDK on different hardware.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant