Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding the provider #1

Merged
merged 1 commit into from
Feb 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
/bin
126 changes: 125 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,125 @@
# garm-provider-gcp
# Garm External Provider For GCP

The GCP external provider allows [garm](https://github.com/cloudbase/garm) to create Linux and Windows runners on top of GCP virtual machines.

## Build

Clone the repo:

```bash
git clone https://github.com/cloudbase/garm-provider-gcp
```

Build the binary:

```bash
cd garm-provider-gcp
go build .
```

Copy the binary on the same system where garm is running, and [point to it in the config](https://github.com/cloudbase/garm/blob/main/doc/providers.md#the-external-provider).

## Configure

The config file for this external provider is a simple toml used to configure the GCP credentials it needs to spin up virtual machines.

```bash
project_id = "garm-testing"
zone = "europe-west1-d"
network_id = "projects/garm-testing/global/networks/garm"
subnetwork_id = "projects/garm-testing/regions/europe-west1/subnetworks/garm"
CredentialsFile = "/home/ubuntu/service-account-key.json"
```

## Creating a pool

After you [add it to garm as an external provider](https://github.com/cloudbase/garm/blob/main/doc/providers.md#the-external-provider), you need to create a pool that uses it. Assuming you named your external provider as ```gcp``` in the garm config, the following command should create a new pool:

```bash
garm-cli pool create \
--os-type windows \
--os-arch amd64 \
--enabled=true \
--flavor e2-medium \
--image projects/windows-cloud/global/images/family/windows-2022 \
--min-idle-runners 0 \
--repo 26ae13a1-13e9-47ec-92c9-1526084684cf \
--tags gcp,windows \
--provider-name gcp
```

This will create a new Windows runner pool for the repo with ID `26ae13a1-13e9-47ec-92c9-1526084684cf` on GCP, using the image specified by its family name `projects/windows-cloud/global/images/family/windows-2022` and instance type `e2-medium`. You can, of course, tweak the values in the above command to suit your needs.

**NOTE**: If you want to use a custom image that you created, specify the image name in the following format: `projects/my_project/global/images/my-custom-image`

Here is an example for a Linux pool that uses the image specified by its image name:

**NOTE**: The provider supports only **UBUNTU** and **DEBIAN** images for Linux pools at the moment.

```bash
garm-cli pool create \
--os-type linux \
--os-arch amd64 \
--enabled=true \
--flavor e2-medium \
--image projects/debian-cloud/global/images/debian-11-bullseye-v20240110 \
--min-idle-runners 0 \
--repo eb3f78b6-d667-4717-97c4-7aa1f3852138 \
--tags gcp,linux \
--provider-name gcp
```

Always find a recent image to use. For example, to see available Windows server 2022 images, run something like `gcloud compute images list --filter windows-2022` or just search [here](https://console.cloud.google.com/compute/images).

## Tweaking the provider

Garm supports sending opaque json encoded configs to the IaaS providers it hooks into. This allows the providers to implement some very provider specific functionality that doesn't necessarily translate well to other providers. Features that may exists on GCP, may not exist on Azure or AWS and vice versa.

To this end, this provider supports the following extra specs schema:

```bash
{
"$schema": "http://cloudbase.it/garm-provider-gcp/schemas/extra_specs#",
"type": "object",
"description": "Schema defining supported extra specs for the Garm GCP Provider",
"properties": {
"disksize": {
"type": "integer",
"description": "The size of the root disk in GB. Default is 127 GB."
},
"network_id": {
"type": "string",
"description": "The name of the network attached to the instance."
},
"subnet_id": {
"type": "string",
"description": "The name of the subnetwork attached to the instance."
},
"nic_type": {
"type": "string",
"description": "The type of NIC attached to the instance. Default is VIRTIO_NET."
}
}
}
```

An example of extra specs json would look like this:

```bash
{
"disksize": 255,
"network_id": "projects/garm-testing/global/networks/garm-2",
"subnet_id": "projects/garm-testing/regions/europe-west1/subnetworks/garm",
"nic_type": "VIRTIO_NET"
}
```

To set it on an existing pool, simply run:

```bash
garm-cli pool update --extra-specs='{"disksize" : 100}' <POOL_ID>
```

You can also set a spec when creating a new pool, using the same flag.

Workers in that pool will be created taking into account the specs you set on the pool.
62 changes: 62 additions & 0 deletions config/config.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
// SPDX-License-Identifier: Apache-2.0
// Copyright 2024 Cloudbase Solutions SRL
//
// Licensed under the Apache License, Version 2.0 (the "License"); you may
// not use this file except in compliance with the License. You may obtain
// a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
// WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
// License for the specific language governing permissions and limitations
// under the License.

package config

import (
"fmt"

"github.com/BurntSushi/toml"
)

func NewConfig(cfgFile string) (*Config, error) {
var config Config
if _, err := toml.DecodeFile(cfgFile, &config); err != nil {
return nil, fmt.Errorf("error decoding config: %w", err)
}

if err := config.Validate(); err != nil {
return nil, fmt.Errorf("error validating config: %w", err)
}
return &config, nil
}

type Config struct {
ProjectId string `toml:"project_id"`
Zone string `toml:"zone"`
CredentialsFile string `toml:"credentials_file"`
NetworkID string `toml:"network_id"`
SubnetworkID string `toml:"subnetwork_id"`
}

func (c *Config) Validate() error {
if c.Zone == "" {
return fmt.Errorf("missing region")
}
if c.ProjectId == "" {
return fmt.Errorf("missing project_id")
}
if c.NetworkID == "" {
return fmt.Errorf("missing network_id")
}
if c.SubnetworkID == "" {
return fmt.Errorf("missing subnetwork_id")
}
if c.CredentialsFile == "" {
return fmt.Errorf("missing credentials_file")
}

return nil
}
47 changes: 47 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
module github.com/cloudbase/garm-provider-gcp

go 1.21.3

require (
cloud.google.com/go/compute v1.23.3
github.com/BurntSushi/toml v1.3.2
github.com/cloudbase/garm-provider-common v0.1.2-0.20240111235646-a9efac12b060
golang.org/x/oauth2 v0.16.0
google.golang.org/api v0.156.0
google.golang.org/protobuf v1.32.0
)

require (
cloud.google.com/go/compute/metadata v0.2.3 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/go-logr/logr v1.4.1 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/google/s2a-go v0.1.7 // indirect
github.com/google/uuid v1.5.0 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect
github.com/googleapis/gax-go/v2 v2.12.0 // indirect
github.com/gorilla/handlers v1.5.1 // indirect
github.com/mattn/go-isatty v0.0.19 // indirect
github.com/minio/sio v0.3.1 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/teris-io/shortid v0.0.0-20220617161101-71ec9f2aa569 // indirect
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.46.1 // indirect
go.opentelemetry.io/otel v1.22.0 // indirect
go.opentelemetry.io/otel/metric v1.22.0 // indirect
go.opentelemetry.io/otel/sdk v1.21.0 // indirect
go.opentelemetry.io/otel/trace v1.22.0 // indirect
golang.org/x/crypto v0.18.0 // indirect
golang.org/x/net v0.20.0 // indirect
golang.org/x/sys v0.16.0 // indirect
golang.org/x/text v0.14.0 // indirect
google.golang.org/appengine v1.6.8 // indirect
google.golang.org/genproto v0.0.0-20240116215550-a9fa1716bcac // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20240116215550-a9fa1716bcac // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240116215550-a9fa1716bcac // indirect
google.golang.org/grpc v1.60.1 // indirect
gopkg.in/natefinch/lumberjack.v2 v2.2.1 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
Loading