These files will allow you to use Terraform to deploy Google Cloud's Anthos on Baremetal on Equinix Metal's Bare Metal Cloud offering.
Terraform will create an Equinix Metal project complete with Linux machines for your Anthos on Baremetal cluster registered to Google Cloud. You can use an existing Equinix Metal Project, check this section for instructions.
Users are responsible for providing their Equinix Metal account, and Anthos subscription as described in this readme.
The build (with default settings) typically takes 25-30 minutes.
The automation in the repo is COMMUNITY SUPPORTED ONLY, if the installation succeeds, and you run the Anthos Platform Validation this cluster is production grade and supportable by Google for Anthos and Equinix Metal for Infrastructure. If you have any questions please consult with Equinix Metal Support via a support ticket.
We use Slack as our primary communication tool for collaboration. You can join the Equinix Metal Community Slack group by going to slack.equinixmetal.com and submitting your email address. You will receive a message with an invite link. Once you enter the Slack group, join the #google-anthos channel! Feel free to introduce yourself there, but know it's not mandatory.
See the Releases page for a changelog describing the tagged releases.
To use these Terraform files, you need to have the following Prerequisites:
- An Anthos subscription
- Google Cloud command-line tool
gcloud
and Terraform installed and configured, see this section - A Equinix Metal org-id and API key
- Optionally, Google Cloud service-account keys, check this section
The default variables make use of 6 c3.small.x86 servers. These servers are $0.50 per hour list price (resulting in a total solution price of roughly $3.00 per hour). This deployment has been test with as little as 2 c3.small.x86 (1 Control Plane node and 1 Worker node) for a total cost of roughly $1.00.
The Terraform has been successfully tested with following versions of Anthos on Baremetal:
- 1.7.0
- 1.6.0
To simplify setup, this is designed to use manual LoadBalancing with Kube-VIP load balancer. No other load balancer support is planned at this time.
Select the version of Anthos you wish to install by setting the anthos_version
variable in your terraform.tfvars file.
The gcloud
command-line tool is used to configure GCP for use by Terraform. Download and install the tool from the download page.
Once installed, run the following command to log in, configure the tool and your project:
gcloud init
This will prompt you to select a GCP project that you will use to register the Anthos cluster. This project must be linked to an Anthos subscription.
Next, run the following command to configure credentials that can be used by Terraform:
gcloud auth application-default login
Terraform is just a single binary. Visit their download page, choose your operating system, make the binary executable, and move it into your path.
Here is an example for macOS:
curl -LO https://releases.hashicorp.com/terraform/0.14.2/terraform_0.14.2_darwin_amd64.zip
unzip terraform_0.14.2_darwin_amd64.zip
chmod +x terraform
sudo mv terraform /usr/local/bin/
rm -f terraform_0.14.2_darwin_amd64.zip
Here is an example for Linux:
curl -LO https://releases.hashicorp.com/terraform/0.14.2/terraform_0.14.2_linux_amd64.zip
unzip terraform_0.14.2_linux_amd64.zip
chmod +x terraform
sudo mv terraform /usr/local/bin/
rm -f terraform_0.14.2_linux_amd64.zip
The Anthos on Baremetal install requires several service accounts and keys to be created. See the Google documentation for more details. By default, Terraform will create and manage these service accounts and keys for you (recommended). Alternatively, you can create these keys manually, or use a provided helper script to make the keys for you.
If you choose to manage the keys yourself, the Terraform files expect the keys to use the following naming convention, matching that of the Google documentation:
util
|_keys
|_cloud-ops.json
|_connect.json
|_gcr.json
|_register.json
|_bmctl.json
If doing so manually, you must create each of these keys and place it in a folder named keys
within the util
folder.
The service accounts also need to have IAM roles assigned to each of them. To do this manually, you'll need to follow the instructions from Google
Much easier (and recommended) is to use the helper script located in the util
directory called setup_gcp_project.sh
to create these keys and assign the IAM roles. The script will allow you to log into GCP with your user account and the GCP project for your Anthos cluster.
You can run this script as follows:
util/setup_gcp_project.sh
Prompts will guide you through the setup.
Note that if you choose to manage the service accounts and keys outside Terraform, you will need to provide the gcp_keys_path
variable to Terraform (see table below).
To download this project, run the following command:
git clone https://github.com/equinix/terraform-metal-anthos-on-baremetal.git
cd terraform-metal-anthos-on-baremetal
Terraform uses modules to deploy infrastructure. In order to initialize the modules simply run:
terraform init
This should download seven modules into a hidden directory .terraform
.
There are many variables which can be set to customize your install within variables.tf
. The default variables to bring up a 6 node Anthos cluster with an HA Control Plane and three worker nodes using Equinix Metal's c3.small.x86. Change each default variable at your own risk.
There are some variables you must set with a terraform.tfvars files. You need to set metal_auth_token
& metal_organization_id
to connect to Equinix Metal and the metal_project_name
which will be created in Equinix Metal. For the GCP side you need to set gcp_project_id
so that Terraform can enable APIs and initialise the project, and it's a good idea to set cluster_name
to identify your cluster in the GCP portal. Note that the GCP project must already exist, i.e. Terraform will not create the GCP project for you.
The Anthos variables include anthos_version
and anthos_user_cluster_name
.
Here is a quick command plus sample values to start file for you (make sure you adjust the variables to match your environment):
cat <<EOF >terraform.tfvars
metal_auth_token = "cefa5c94-e8ee-4577-bff8-1d1edca93ed8"
metal_organization_id = "42259e34-d300-48b3-b3e1-d5165cd14169"
metal_project_name = "anthos-metal-project-1"
gcp_project_id = "anthos-gcp-project-1"
cluster_name = "anthos-metal-1"
EOF
A complete list of variables can be found at https://registry.terraform.io/modules/equinix/anthos-on-baremetal/metal/latest?tab=inputs.
Variable Name | Type | Default Value | Description |
---|---|---|---|
metal_auth_token | string | n/a | Equinix Metal API Key |
metal_project_id | string | n/a | Equinix Metal Project ID |
metal_organization_id | string | n/a | Equinix Metal Organization ID |
hostname | string | anthos-baremetal | The hostname for nodes |
metro | string | ny | Equinix Metal Metro to deploy into |
cp_plan | string | c3.small.x86 | Equinix Metal device type to deploy control plane nodes |
worker_plan | string | c3.small.x86 | Equinix Metal device type to deploy for worker nodes |
ha_control_plane | boolean | true | Do you want a highly available control plane? |
worker_count | number | 3 | Number of baremetal worker nodes |
operating_system | string | ubuntu_20_04 | The Operating system of the node |
billing_cycle | string | hourly | How the node will be billed (Not usually changed) |
cluster_name | string | equinix-metal-gke-cluster | The name of the GKE cluster |
metal_create_project | string | true | Create a new project for this deployment? |
metal_project_name | string | baremetal-anthos | The name of the project if 'create_project' is 'true'. |
gcp_project_id | string | n/a | The GCP project ID to use . |
gcp_keys_path | string | n/a | The path to a directory with GCP service account keys |
bgp_asn | string | 65000 | BGP ASN to peer with Equinix Metal |
ccm_version | string | v3.2.2 | The version of Cloud Provider Equinix Metal |
kube_vip_version | string | 0.3.8 | The version of Kube-VIP to install |
anthos_version | string | 1.8.3 | The version of Google Anthos to install |
ccm_deploy_url | string | Too Long to put here... | The deploy url for the Equinix Metal CCM |
storage_provider | string | n/a | Enable a Storage module (examples: "portworx", "rook") |
storage_options | map | n/a | Options specific to the storage module |
Name | Api Slug |
---|---|
CentOS 8 | centos_8 |
Ubuntu 18.04 | ubuntu_18_04 |
Ubuntu 20.04 | ubuntu_20_04 |
Name | Api Slug |
---|---|
Red Hat Enterprise Linux 8 | rhel_8 |
All there is left to do now is to deploy the cluster:
terraform apply --auto-approve
This should end with output similar to this:
Apply complete! Resources: 28 added, 0 changed, 0 destroyed.
Outputs:
Control_Plane_Public_IPs = [
"136.144.50.115",
"136.144.50.117",
"136.144.50.119",
]
Control_Plane_VIP = "145.40.65.107"
Ingress_VIP = "145.40.65.106"
Kubeconfig_location = "/home/cloud-user/git/baremetal-anthos/equinix-metal-gke-cluster-vomqb-kubeconfig"
Worker_Public_IPs = [
"136.144.50.123",
"145.40.64.221",
"136.144.50.105",
]
ssh_key_location = "/home/cloud-user/.ssh/bm-cluster-20201211211054"
You can see this output again at anytime by running terraform output
If you have an existing Equinix Metal project you can use it. YOU MUST ENABLE BGP PEERING ON YOUR PROJECT WITHOUT A PASSWORD
Get your Project ID, navigate to the Project from the console.equinixmetal.com console and click on PROJECT SETTINGS, copy the PROJECT ID.
add the following variables to your terraform.tfvars
metal_create_project = false
metal_project_id = "YOUR-PROJECT-ID"
Once Anthos is deployed on Equinix Metal, all of the documentation for using Google Anthos is located on the Anthos Documentation Page.
Storage providers are made available through optional storage modules. These storage providers include CSI (Container Native Storage) StorageClasses
.
Changing or disabling a storage provider is not currently supported.
To enable a storage module, set the storage_module
variable to the name of the name of the included module.
-
portworx
: To enable the Pure Storage Portworx installation, use the following settings interraform.tfvars
:storage_module = "portworx" storage_options = { # portworx_version = "2.6" # portworx_license = "c0ffe-fefe-activation-123" }
When enabled, Portworx will manage the local disks attached to each worker node, providing a fault tolerant distributed storage solution.
Read more about the Portworx module (also available on the Terraform Registry).
-
rook
: To enable the Rook Ceph installation, use the following settings interraform.tfvars
:storage_module = "rook" storage_options = { # rook_version = "v1.6.0" }
When enabled, Rook Ceph will manage the local disks attached to each worker node, providing a fault tolerant distributed storage solution.
Read more about the Rook module (also available on the Terraform Registry).