better CI ?

[Dieterbe]

currently we use circleCI (1.0) machine instances (which i believe is a vm not a docker container) which gets the job done ok, but i'm interested in seeing if we can level up our CI, maybe use something better. this is not urgent, just exploratory at this point.
mainly looking at circleci 2.0 and gitlab for now.

what circleci 1.0 does well

• generally pretty easy to work with. handy ssh logins to troubleshoot

problems with our current solution, that a new solution should solve

• no parrallelisation, slowing everything down. most of our steps could run independently.
• we employ a lot of hacks. symlinking the checkout to the gopath-directory, installing the desired (latest) go version, installing docker-compose via a shellscript, ...
• sometimes things don't work well and it can be hard to figure out. possibly related to the hacks
• our caching strategy is suboptimal (e.g. install fpm/packagecloud gems every time)

nice to haves / bonus features in a new system

• allow more cluster testing using docker-compose, bring up more complicated stacks, ability to tinker with the network in an automated way (using tc/netem)
• tracking of performance stats (benchmark results) over time and visualizing them
• test against multiple go versions
• ability to run all our tests using pre-release go versions, so we can provide feedback to Go team. (which they always ask for)

alternative 1: circleCI 2.0 docker executor

looks nice in that it's lightweight to run, can use custom image with all needed dependencies (fpm, packagecloud, 3rd party go tools, etc) preinstalled. but has some major downsides:

• cannot use docker-compose with volumes that use data from the checked out code dir
• cannot talk to containers directly from the "main" container.
  see https://circleci.com/docs/2.0/building-docker-images/ for more details

alternative 2: circleCI 2.0 machine executor

can use vm images that have docker and docker-compose preinstalled. but no custom images. slower start time. but lets us do anything, basically. not possibly to use latest go version
see https://circleci.com/docs/2.0/executor-types/ for comparison with the docker one.

alternative 3: gitlab

we could run it on prem, but the open source edition, but as an open source project we can get all the premium features for free (if we use the hosted version). much more than a CI system.

[Dieterbe]

problems 1,2,3 and mostly 4 are solved via #803 and #810
there's more left to do, especially all the bonus features. but it's not pressing.
this ticket mainly exists as a place to list requirements / discuss. i'm going to close it but if anyone still has anything to add, feel free.