-
-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
somehow get CI automated testing working for more platforms #835
Comments
I'm not aware of any hosted CI service which supports those architectures (especially the BSD's). Beyond building your own, leveraging QEMU within Travis may be an option. Here is a pretty good example of this strategy which covers all the listed targets and may be worth exploring further: |
I have experience doing automated openbsd installs and don't mind doing that one. |
Thanks for volunteering! So we would have a way to automatically run tests on OpenBSD when we push to master branch? |
Sure, to start I can make a script to generate an openbsd vm disk image that is ready to go. Later we can decide how to hook it up to a CI system. It may be necessary to avoid crazy slow nested VM's on travis. |
#941 works for openbsd, however we get timeouts. This approach will work for linux arm also, so is good for a reference point. ccache gives large enough speed boosts I think qemu is feasible, however it is definitely too slow for travis, especially when we add more targets. |
I'm sorry to have to report back that I read wrong in the announcement, you only get unlimited CI minutes but not unlimited minutes per build, bummer, sorry for the noise/ confusion. news from the news you can specify up to 360min runtime in the yaml file |
I'm a product manager on Azure Pipelines. You can have multiple jobs in a single pipeline/yaml and each job has a limit of 6 hours. Do you need more time than that? If you need a VM/agent that's different from what we support in the hosted pool, we have private agents for Linux, macOS, and Windows where you can use your own VMs. We're updating the open source offer to give 10 free private agents and unlimited minutes. Private agents have no time limit on individual jobs. Ping me if you run into any issues. |
Thanks @monouser7dig and @jeremyepling, I think this is worth trying out. 6 hours is plenty of time to run all the release tests as well as debug tests. Would this let us have VMs with FreeBSD or OpenBSD on it? From a quick scan of the documentation, it looks like the way it works is to install Microsoft's agent code in the VM, but the agent only supports linux/windows/mac. Is that right? With 6 hours to run tests, I would try the qemu setup that @andrewchambers did a proof-of-concept for, where we can add ARM support and other CPU architectures. |
@andrewrk, The Azure Pipelines Agent is based on .NET Core and it doesn't support FreeBSD or OpenBSD, but it might be coming soon. |
@jeremyepling this is good news! |
@jeremyepling Also, is there a recommended way to cache? 90% of the time will be spent building LLVM 7 over and over again, and this could be cached. edit currently I am using s3 to cache. |
@andrewrk |
FreeBSD x86_64 should be ticked off |
@andrewrk Not sure if this is interesting/useful information or not..... I use buildkite at work, and it works pretty well. buildkite operates as a "head node" service, with the worker agent being installed and run "on your own systems". Likely best if those systems are dedicated to CI, because they may be running untrusted code in PRs. buildkite apparently support several platforms, and appear to offer free open source project accounts. Apparently bazel uses it? Might be an option if people and/or companies are willing to donate some VM hours to the project? |
I think this is basically solved now that we have QEMU integrated into our test matrix + bootstrap repo. Each architecture/os combo has its own issues for getting to tier 1 support. |
Currently we have AppVeyor for Windows i386/x86_64 and Travis CI for MacOS x86_64 and Linux x86_64.
In order to support more targets in the standard library, we need more automatic testing.
There are more, but these are some important ones.
The text was updated successfully, but these errors were encountered: