WIP: evaluate Circle CI #23171
WIP: evaluate Circle CI #23171
Conversation
This runs in circleci's python 2.7 docker image which is based on debian jessie, mostly for convenience since that image has a non-root user added and some of the Julia tests work differently when run as root. Circle CI caching is a little odd - I'm using ccache here since it's actually fast enough to handle it pretty well, about 15 minutes to build deps from scratch and about 3 minutes to build deps with a primed ccache. We save the ARCH, HOME (default location where ccache saves its data, in case we switch containers and try to run tests as a different user), and year and week number into a tag file, then determine which saved circleci cache tag to load from based on the checksum of that tag file. So this will throw out the cache and have to rebuild it once a week, but since that only adds about 10-15 minutes to a build that's not a big deal. It would be better if circle's cache worked like Travis and AppVeyor's where it loads the latest and checksums the content to decide whether to upload a new replacement cache, but that's up to circle to fix. This has to skip the socket tests, since circle doesn't currently support ipv6 in their docker workers. They have other types of workers, but they aren't anywhere near as fast to build or run tests. This currently doesn't do a doc build or deployment, that's a TODO if we decide to switch to this. It also looks like it checks out the branch tip instead of the merge commit, but that should be fixable. [av skip] [bsd skip]
circle.yml
Outdated
| steps: &steps | ||
| # apt-get update && | ||
| # curl make patch m4 xz-utils git time ccache bar | ||
| - run: | # install build dependencies |
There was a problem hiding this comment.
Rather than commenting a name for the command, you can actually do
- run:
name: install build dependencies
command: |
sudo apt-get .......That will display the given name as the collapsible block header when viewing the build on their web UI.
There was a problem hiding this comment.
sure but I personally would rather see the command that's running as the name of the step, the comments are more explanation for anyone who might need to modify the file
circle.yml
Outdated
| build-x86_64: | ||
| docker: | ||
| # - image: ubuntu:14.04 | ||
| - image: circleci/python:2.7 |
There was a problem hiding this comment.
Would it make sense to use Elliot's images for this? They have all of the dependencies needed to build Julia and you can do both x86 and x86-64 rather than doing a 32-bit build on a 64-bit image.
There was a problem hiding this comment.
elliot's images also do 32 bit builds in a 64 bit image.
ci and buildbots for binaries serve different purposes, so no. binaries need to be built with an old glibc and new compiler, but ci should test a more common configuration - make sure we don't start using features that only work in super new gcc until llvm requires us to
There was a problem hiding this comment.
the ideal configuration for ci would probably be oldest supported versions of both glibc and compiler, but the closest thing to that combination that doesn't require building our own gcc would be to use the centos devtoolset, and that has its own issues. I was trying to avoid needing to build a custom image for this. I'd have used the bare minimum ubuntu 14.04 image except for the hassle of adding a non-root user, which the circle image already did - it's also already locally fetched so doesn't need to download any large new containers to start the build. getting things from apt-get is fast enough to not need to optimize.
circle.yml
Outdated
| # - image: ubuntu:14.04 | ||
| - image: circleci/python:2.7 | ||
| environment: | ||
| JULIA_CPU_CORES: 4 |
There was a problem hiding this comment.
I don't think this is right. Each Circle container has 2 vCPUs, and setting up 4x parallelism gives you four separate, isolated containers that do the same build at the same time. That's why in my earlier PR I had the load balancer for splitting the tests between containers.
There was a problem hiding this comment.
I thought they still used 2 vCPU containers though? Idk, I guess at this point you'd know better than I would.
There was a problem hiding this comment.
I did a bunch of test builds on my fork - running nproc in the v2 container says 32 which is probably more to do with the container host server, if you run without a JULIA_CPU_CORES limit then things get killed. pretty sure I tested this out set to different values from 2-6 and 4 was fastest overall, but can try more of those. it isn't worth manually load balancing across separate containers, the concurrency of multiple builds is better spent on testing multiple pr's simultaneously for overall throughput
|
For the Circle v2.0 API, I think they want you to make the configuration file |
|
they appear to support both, and adding a new top level file seemed like less of a mess than adding a new top level directory. can test moving it to see if that's what was wrong with the fork vs julialang execution, but I'd be a bit surprised |
|
One option that might make sense would be going with the original plan from #18007, and moving just i686 testing over to circle. We currently have 10 concurrent workers on Travis, roughly speaking 3 and a third each for linux32, linux64, and mac. If we moved linux32 from Travis to Circle, we'd have 4 (much faster...) workers doing linux32, and 5 each doing linux64 (where the doc deployment happens from) and mac. |
set JULIA_CPU_CORES to 6 - in some testing on my fork this was a few minutes faster, but may run a higher risk of getting OOM killed
delete not-so-relevant comments (resource_class would be relevant if we started paying circle, other lines relevant for using ubuntu docker image
[av skip] [bsd skip]
[av skip] [bsd skip]
|
Okay, closing this in favor of #23188. Under https://circleci.com/gh/JuliaLang/julia/edit#advanced-settings I have "Build forked pull requests - Run builds for pull requests from forks. CircleCI will automatically update the commit status shown on GitHub's pull request page" enabled, but maybe that doesn't trigger if circle.yml isn't on master yet? |
TL;DR, this should be able to build and run tests in more like 45 minutes vs
travis' 2 hours. Gives 4 free workers for open source projects, adding more
is $50/month/worker.
This runs in circleci's python 2.7 docker image which is based on
debian jessie, mostly for convenience since that image has a non-root
user added and some of the Julia tests work differently when run as root.
Circle CI caching is a little odd - I'm using ccache here since it's
actually fast enough to handle it pretty well, about 15 minutes to build
deps from scratch and about 3 minutes to build deps with a primed ccache.
We save the ARCH, HOME (default location where ccache saves its data, in
case we switch containers and try to run tests as a different user), and
year and week number into a tag file, then determine which saved circleci
cache tag to load from based on the checksum of that tag file. So this
will throw out the cache and have to rebuild it once a week, but since that
only adds about 10-15 minutes to a build that's not a big deal. It would be
better if circle's cache worked like Travis and AppVeyor's where it loads
the latest and checksums the content at the end of the build to decide
whether to upload a new replacement cache, but that's up to circle to fix.
This has to skip the socket tests, since circle doesn't currently support
ipv6 in their docker workers. They have other types of workers, but they
aren't anywhere near as fast to build or run tests.
This currently doesn't do a doc build or deployment, that's a TODO if
we decide to switch to this. It also looks like it checks out the branch
tip instead of the merge commit, but that should be fixable.
edit: I tried re-enabling the old JuliaLang circle CI webhook from #18007
before opening this, but the status link seems to be coming from circle running
on my fork rather than circle running on JuliaLang. Not sure what's up there.