GCP meshPlatform Module

Terraform module to integrate GCP as a meshPlatform into meshStack instance. With this module, service accounts used by meshStack are created with the required permissions. The output of this module is a set of credentials that need to be configured in meshStack as described in meshcloud public docs.

Prerequisites

To run this module, you need the following:

• Required permissions. The user or service account applying this module needs the following permissions:

  Project-level:

  "iam.serviceAccountKeys.create",
  "iam.serviceAccountKeys.get",
  "iam.serviceAccountKeys.list",
  "iam.serviceAccounts.get",
  "iam.serviceAccounts.list",
  "iam.serviceAccounts.create",
  "iam.serviceAccounts.getIamPolicy",
  "iam.serviceAccounts.setIamPolicy",
  "serviceusage.services.list",
  "serviceusage.services.use",
  "serviceusage.services.get",
  "resourcemanager.projects.get",
  "resourcemanager.projects.getIamPolicy",
  "resourcemanager.projects.setIamPolicy",
  # Only required when you need your landing zone to invoke a cloud function
  "cloudfunctions.functions.getIamPolicy",
  "cloudfunctions.functions.setIamPolicy",
  
  # Only required for the optional submodule for exporting carbon data  
  "resourcemanager.projects.update"

  Organization-level:

  "iam.roles.list",
  "iam.roles.get",
  "iam.roles.create",
  "iam.roles.undelete",
  "iam.roles.update",
  "resourcemanager.folders.get",
  "resourcemanager.folders.getIamPolicy",
  "resourcemanager.folders.setIamPolicy",
  "billing.accounts.getIamPolicy",
  "billing.accounts.setIamPolicy",
  # Only required when you need your landing zone to access the bucket containing your GDM templates
  "storage.buckets.getIamPolicy",
  "storage.buckets.setIamPolicy"

  You can create custom roles and assign them to the identity applying those modules.

  Additionally, after the module has run, replicator service account needs to be granted a role in the Admin Console (Workspace). This can only be done by Super Administrators.

• Terraform installed

• gcloud installed

• A GCP Project

  Before using this module, you need a gcp project. We recommend creating a new project for meshcloud service accounts.

  You can create a gcp project by adding this terraform block to your terraform file:

  resource "google_project" "meshstack_root" {
    name               = <PROJECT_NAME>
    project_id         = <PROJECT_ID>
    folder_id          = <FOLDER_ID>
    billing_account    = <BILLING_ACCOUNT_ID>
  }

How to Use This Module

To provide some of the required variables, you will need to

• setup GCP Cloud Billing Export, see cloud_billing_export_dataset_id
• optionally setup GCP Cloud Carbon Footprint Exports, see cloud_carbon_export_dataset_id

Using CLI

1. login with your gcloud account.

   gcloud auth application-default login

2. Create a terraform file that calls this module and produces outputs. Similar to:

   module "meshplatform" {
     source = "git::https://github.com/meshcloud/terraform-gcp-meshplatform.git"
     # FILL INPUTS
   }
   
   output "meshplatform" {
     sensitive = true
     value     = module.meshplatform
   }

   It is highly recommended to configure a terraform backend, otherwise you risk losing track of your applied resources.

3. Execute the module.

   terraform init
   terraform apply

4. Access terraform output to insert it in meshStack platform config.

   terraform output -json

5. Authorize the Replicator Service Account.

   The replicator service account needs the "Groups Admin" role from the Admin Console (Workspace) to manage permissions for managed GCP projects. To authorize the Service Account via the Google Admin Console navigate to @Account in the sidebar and then Admin Roles -> Groups Admin and click Assign Service Accounts. In the prompt that appears, enter the service account email, which looks like user@project.iam.gserviceaccount.com.

Contributing Guide

Before opening a Pull Request, please do the following:

1. Install pre-commit

   We use pre-commit to perform several terraform related tasks such as terraform validate, terraform fmt, and generating terraform docs with terraform_docs

2. Execute pre-commit install: Hooks configured in .pre-commit-config.yaml will be executed automatically on commit. For manual execution, you can use pre-commit run -a.

Requirements

Name	Version
google	5.19.0

Providers

Name	Version
google	5.19.0

Modules

Name	Source	Version
carbon_export	./modules/meshcloud-carbon-export/	n/a
kraken_sa	./modules/meshcloud-kraken-service-account/	n/a
replicator_sa	./modules/meshcloud-replicator-service-account/	n/a

Resources

Name	Type
google_iam_workload_identity_pool.meshstack	resource
google_iam_workload_identity_pool_provider.meshstack	resource

Inputs

Name	Description	Type	Default	Required
billing_account_id	The GCP billing account in your organization.	string	n/a	yes
billing_org_id	GCP organization ID that holds billing account.	string	n/a	yes
can_delete_projects_in_landing_zone_folder_ids	The service account will have projectDeleter role only on the specified landing zone IDs.	set(string)	[]	no
carbon_export_module_enabled	Determines whether or not to include the resources of the carbon footprint export module.	bool	false	no
cloud_billing_export_dataset_id	GCP BigQuery dataset containing the Cloud Billing BigQuery export. ATTENTION You need to manually configure the billing account big query export before exceuting this module. See https://docs.meshcloud.io/docs/meshstack.how-to.integrate-meshplatform-gcp-manually.html#set-up-gcp-billing-data-export for instructions.	string	n/a	yes
cloud_billing_export_project_id	GCP Project where the BiqQuery table resides that holds the Cloud Billing export to BigQuery. See https://cloud.google.com/billing/docs/how-to/export-data-bigquery	string	n/a	yes
cloud_billing_export_table_id	GCP BigQuery table containing the Cloud Billing BigQuery export. This variable is only required to form the output for meshPlatform configuration. No resources are created or attached.	string	n/a	yes
cloud_carbon_export_dataset_id	GCP BigQuery dataset containing the Carbon Footprint BigQuery export. ATTENTION You need to manually configure the carbon footprint export transfer config before exceuting this module. See https://docs.meshcloud.io/docs/meshstack.how-to.integrate-meshplatform-gcp-manually.html#optional-enable-gcp-cloud-carbon-footprint-export for instructions."	string	n/a	yes
cloud_carbon_export_project_id	GCP Project where the BiqQuery table resides that holds the Cloud Carbon Footprint export to BigQuery.	string	n/a	yes
kraken_sa_name	Name of the service account to create for Kraken.	string	"mesh-kraken-service-tf"	no
landing_zone_folder_ids	GCP Folders that make up the Landing Zone. The service account will only receive permissions on these folders.	list(string)	n/a	yes
org_id	GCP Organization ID that holds the projects that generate billing data that the service account should import.	string	n/a	yes
project_id	GCP Project ID where to create the resources. This is typically a 'meshstack-root' project.	string	n/a	yes
replicator_sa_name	Name of the service account to create for Replicator.	string	"mesh-replicator-service-tf"	no
service_account_keys	Create service account keys for authentication.	bool	true	no
workload_identity_federation	Setup workload identity federation for authentication.	object({ workload_identity_pool_identifier = string issuer = string audience = string replicator_subject = string kraken_subject = string })	null	no

Outputs

Name	Description
carbon_footprint_export_table_name	The BigQuery table name containing the GCP Carbon Footprint BigQuery export.
cloud_billing_export_table_name	The BigQuery table name containing the GCP Cloud Billing BigQuery export.
kraken_sa_credentials_json	Kraken service account key in credentials.json format, base64 encoded.
kraken_sa_email	Kraken service account email.
replicator_manual_setup	Replicator service account setup information.
replicator_sa_credentials_json	Replicator service account key in credentials.json format, base64 encoded.
replicator_sa_email	Replicator service account email.