This repo contains a terraform module and example to run a GitLab CI multi runner on AWS Spot instances.
The setup is based on the blog post: [Auto scale GitLab CI runners and save 90% on EC2 costs] (https://about.gitlab.com/2017/11/23/autoscale-ci-runners/) The created runner will have by default a shared cache in S3 and logging is streamed to CloudWatch. The cache in S3 will expire in X days, see configuration. The logging can be disabled.
Ensure you have Terraform installed, see .terraform-version
for the used version. A handy tool to mange your terraform version is tfenv.
On mac simple install tfenv using brew.
brew install tfenv
Next install a terraform version.
tfenv install <version>
To run the terraform scripts you need to have AWS keys. Example file:
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
Currently register a new runner is a manual process. See the GitLab Runner documentation for more details.
docker run -it --rm gitlab/gitlab-runner register
Provide the details in the interactive terminal. Once done the token can be found in the GitLab runners section, choose edit to get the token or see the config.toml file.
Update the variables in terraform.tfvars
to your needs and add the following variables, see previous step for how to obtain the token.
runner_name = "NAME_OF_YOUR_RUNNER"
gitlab_url = "GIT_LAB_URL"
runner_token = "RUNNER_TOKEN"
module "runner" {
source = "npalm/gitlab-runner/aws"
aws_region = "${var.aws_region}"
environment = "${var.environment}"
ssh_public_key = "${file("${var.ssh_key_file}")}"
vpc_id = "${module.vpc.vpc_id}"
subnet_id_gitlab_runner = "${element(module.vpc.private_subnets, 0)}"
subnet_id_runners = "${element(module.vpc.private_subnets, 0)}"
runners_name = "${var.runner_name}"
runners_gitlab_url = "${var.gitlab_url}"
runners_token = "${var.runner_token}"
}
All variables and defaults:
Name | Description | Type | Default | Required |
---|---|---|---|---|
amazon_optimized_amis | AMI map per region-zone for the gitlab-runner instance AMI. | map | <map> |
no |
aws_region | AWS region. | string | - | yes |
cache_expiration_days | Number of days before cache objects expires. | string | 1 |
no |
cache_user | User name of the user to create to write and read to the s3 cache. | string | cache_user |
no |
docker_machine_instance_type | Instance type used for the instances hosting docker-machine. | string | m4.large |
no |
docker_machine_spot_price_bid | Spot price bid. | string | 0.03 |
no |
docker_machine_user | User name for the user to create spot instances to host docker-machine. | string | docker-machine |
no |
enable_cloudwatch_logging | Enable or disable the CloudWatch logging. | string | 1 |
no |
environment | A name that identifies the environment, will used as prefix and for tagging. | string | - | yes |
gitlab_runner_version | Version for the gitlab runner. | string | 10.2.0 |
no |
instance_type | Instance type used for the gitlab-runner. | string | t2.micro |
no |
runners_concurrent | Concurrent value for the runners, will be used in the runner config.toml | string | 10 |
no |
runners_gitlab_url | URL of the gitlab instance to connect to. | string | - | yes |
runners_idle_count | Idle count of the runners, will be used in the runner config.toml | string | 0 |
no |
runners_idle_time | Idle time of the runners, will be used in the runner config.toml | string | 600 |
no |
runners_limit | Limit for the runners, will be used in the runner config.toml | string | 0 |
no |
runners_name | Name of the runner, will be used in the runner config.toml | string | - | yes |
runners_privilled | Runners will run in privilled mode, will be used in the runner config.toml | string | true |
no |
runners_token | Token for the runner, will be used in the runner config.toml | string | - | yes |
ssh_public_key | Public SSH key used for the gitlab-runner ec2 instance. | string | - | yes |
subnet_id_gitlab_runner | Subnet used for hosting the gitlab-runner. | string | - | yes |
subnet_id_runners | Subnet used to hosts the docker-machine runners. | string | - | yes |
vpc_id | The VPC that is used for the instances. | string | - | yes |
An example is provided, execute the following steps to run the sample. Ensure your AWS and Terraform environment is set up correctly. All commands below are supposed to be run inside the directory example
.
Run init.sh
to create SSH keys for the runner instance.
Register a new runner:
docker run -it --rm gitlab/gitlab-runner register
Once done, lookup the token in GitLab and update the terraform.tfvars
file.
Run terraform init
to initialise terraform. Next you can run terraform plan
to inspect the resources that will be created.
To create the runner run:
terraform apply
To destroy runner:
terraform destroy
- A user is create for the gitlab-runner / docker-machine to create ec2 instances, the permissions are NOT very restrictive.