Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documentations for org level resources. #160

Merged
merged 2 commits into from
Apr 16, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 10 additions & 8 deletions Terraform/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,9 @@ A deployment typically contains the following files:
deployments, the remote state, and input values from other dependent
deployments.

To see what resources each deployment provisions, check out the comments in each
**main.tf** file.

## Layout

```
Expand All @@ -92,7 +95,6 @@ A deployment typically contains the following files:
|- secrets: Definitions of secrets used in the org (secret values are not set in configs).
|- org: org level resources. Resources within this directory should be managed by CICD pipeline.
|- terragrunt.hcl: root Terragrunt config which defines remote state for all deployments.
|- project.{PREFIX}-devops: additional resources that will go in the devops project.
|- project.{PREFIX}-audit: the project to hold all audit logs for the org.
|- audit: deployment to setup auditing for the org.
|- iam: org level iam definitions such as org admins.
Expand All @@ -118,7 +120,7 @@ A deployment typically contains the following files:
$ ROOT=$PWD
```

1. The bootstrap config must be deployed first in order to create the devops
1. The bootstrap config must be deployed first in order to create the `devops`
project which will host your Terraform state and CICD pipelines.

```
Expand All @@ -128,17 +130,17 @@ A deployment typically contains the following files:
$ terraform apply
```

Your devops project should now be ready.
Your `devops` project should now be ready.

1. Backup the state of the devops project to the newly created state bucket by
uncommenting out the `terraform` block in `$ROOT/bootstrap/main.tf` and
1. Backup the state of the `devops` project to the newly created state bucket
by uncommenting out the `terraform` block in `$ROOT/bootstrap/main.tf` and
running:

```
$ terraform init
```

1. Deploy secrets used in the org in the devops project.
1. Deploy secrets used in the org in the `devops` project.

```
$ cd $ROOT/secrets
Expand All @@ -147,8 +149,8 @@ A deployment typically contains the following files:
$ terraform apply
```

After the secrets have been created you must go in Console and open the
secrets manager and fill in their values.
After the secrets have been created, you must go to the Google Cloud
Console, open `Security` --> `Secret Manager` and fill in their values.

1. Follow `$ROOT/cicd/README.md` to set up CICD pipelines for Terraform
configs.
Expand Down
12 changes: 12 additions & 0 deletions Terraform/bootstrap/main.tf
Original file line number Diff line number Diff line change
@@ -1,3 +1,11 @@
# This folder contains Terraform resources to setup the devops project, which includes:
# - The devops project itself,
# - API to enable in the devops project,
# - Deletion lien of the devops project,
# - A Cloud Storage bucket to store Terraform states for all deployments,
# - Project level IAM permissions for the devops project owners,
# - Org level IAM permissions for org admins.

# ====================================================================================
# TODO(user): Uncomment after initial deployment and run `terraform init`.
terraform {
Expand All @@ -8,6 +16,7 @@ terraform {
}
# ======================================================================================

# Devops project, with APIs to enable and deletion lien created.
module "project" {
source = "terraform-google-modules/project-factory/google"
version = "~> 7.0"
Expand All @@ -24,6 +33,7 @@ module "project" {
]
}

# Terraform state bucket, hosted in the devops project.
module "state_bucket" {
source = "terraform-google-modules/cloud-storage/google//modules/simple_bucket"
version = "~> 1.4"
Expand All @@ -33,12 +43,14 @@ module "state_bucket" {
location = var.storage_location
}

# Project level IAM permissions for devops project owners.
resource "google_project_iam_binding" "devops_owners" {
project = module.project.project_id
role = "roles/owner"
members = var.devops_owners
}

# Org level IAM permissions for org admins.
resource "google_organization_iam_member" "org_admin" {
org_id = var.org_id
role = "roles/resourcemanager.organizationAdmin"
Expand Down
4 changes: 4 additions & 0 deletions Terraform/cicd/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,3 +57,7 @@ Postsubmit Cloud Build job will automatically start when a Pull Ruquest is
submitted to a configured branch. To view the result of the Cloud Build run, go
to https://console.cloud.google.com/cloud-build/builds and look for your commit
to view the Cloud Build job triggered by your merged commit.

The Postsubmit Cloud Build Trigger monitors and deploys changes made to `org/`
folder only. Other changes made to `bootstrap`, `cicd` and `secrets` folders
should be deployed manually if needed.
11 changes: 9 additions & 2 deletions Terraform/cicd/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,13 @@
# See the License for the specific language governing permissions and
# limitations under the License.

# This folder contains Terraform resources to setup CI/CD, which includes:
# - Necessary APIs to enable in the devops project for CI/CD purposes,
# - Necessary IAM permissions to set to enable Cloud Build Service Account perform CI/CD jobs.
# - Cloud Build Triggers to monitor GitHub repos to start CI/CD jobs.
#
# The Cloud Build configs can be found under the configs/ sub-folder.

# ***NOTE***: First follow
# https://cloud.google.com/cloud-build/docs/automating-builds/create-github-app-triggers#installing_the_cloud_build_app
# to install the Cloud Build app and connect your GitHub repository to your Cloud project.
Expand Down Expand Up @@ -65,8 +72,8 @@ resource "google_project_service" "devops_apis" {
disable_on_destroy = false
}

# Cloud Build - IAM bindings
# IAM bindings to allow Cloud Build SA to access state.
# Cloud Build - IAM permissions
# IAM permissions to allow Cloud Build SA to access state.
resource "google_storage_bucket_iam_member" "cloudbuild_state_iam" {
bucket = var.state_bucket
role = "roles/storage.objectViewer"
Expand Down
10 changes: 10 additions & 0 deletions Terraform/org/audit/main.tf
Original file line number Diff line number Diff line change
@@ -1,7 +1,14 @@
# This folder contains Terraform resources related to audit, which includes:
# - Organization IAM Audit log configs (https://cloud.google.com/logging/docs/audit),
# - BigQuery log sink creation and configuration for short term log storage,
# - Cloud Storage log sink creation and configuration for long term log storage,
# - IAM permissions to grant log Auditors iam.securityReviewer role to view the logs.

terraform {
backend "gcs" {}
}

# IAM Audit log configs to enable collection of all possible audit logs.
resource "google_organization_iam_audit_config" "config" {
org_id = var.org_id
service = "allServices"
Expand All @@ -17,6 +24,7 @@ resource "google_organization_iam_audit_config" "config" {
}
}

# BigQuery log sink.
module "bigquery_log_export" {
source = "terraform-google-modules/log-export/google"
version = "~> 4.0"
Expand Down Expand Up @@ -57,6 +65,7 @@ resource "google_project_iam_member" "bigquery_sink_member" {
member = module.bigquery_log_export.writer_identity
}

# Cloud Storage log sink.
module "storage_log_export" {
source = "terraform-google-modules/log-export/google"
version = "~> 4.0"
Expand Down Expand Up @@ -96,6 +105,7 @@ resource "google_storage_bucket_iam_member" "storage_sink_member" {
member = module.storage_log_export.writer_identity
}

# IAM permissions to grant log Auditors iam.securityReviewer role to view the logs.
resource "google_organization_iam_member" "security_reviewer_auditors" {
org_id = var.org_id
role = "roles/iam.securityReviewer"
Expand Down
4 changes: 4 additions & 0 deletions Terraform/org/org_policies/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@
# See the License for the specific language governing permissions and
# limitations under the License.

# This folder contains Terraform resources to configure GCP Organization Policies.
# (https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints)
# See the following resources for the details of policies enforced.

terraform {
backend "gcs" {}
}
Expand Down
23 changes: 17 additions & 6 deletions Terraform/org/project.heroes-hat-dev-audit/project/main.tf
Original file line number Diff line number Diff line change
@@ -1,21 +1,32 @@
# This folder contains Terraform resources to setup the audit project, which includes:
# - The audit project itself,
# - API to enable in the audit project,
# - Deletion lien of the audit project,
# - Project level IAM permissions for the audit project owners.

terraform {
backend "gcs" {}
}

# Devops project, with APIs to enable and deletion lien created.
module "project" {
source = "terraform-google-modules/project-factory/google"
version = "~> 7.0"

name = var.name
org_id = var.org_id
folder_id = var.folder_id
billing_account = var.billing_account
lien = var.enable_lien
activate_apis = var.apis
name = var.name
org_id = var.org_id
folder_id = var.folder_id
billing_account = var.billing_account
lien = true
activate_apis = [
"bigquery.googleapis.com",
"logging.googleapis.com",
]
default_service_account = "keep"
skip_gcloud_download = true
}

# Project level IAM permissions for audit project owners.
resource "google_project_iam_binding" "owners" {
project = module.project.project_id
role = "roles/owner"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,6 @@
name = "heroes-hat-dev-audit"
org_id = "707577601068"
billing_account = "01EA90-3519E1-89CB1F"
apis = [
"bigquery.googleapis.com",
"logging.googleapis.com",
]
owners = [
"group:rocketturtle-gcp-admin@rocketturtle.net",
]
10 changes: 0 additions & 10 deletions Terraform/org/project.heroes-hat-dev-audit/project/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -15,16 +15,6 @@ variable "billing_account" {
type = string
}

variable "apis" {
type = list(string)
default = []
}

variable "enable_lien" {
type = bool
default = true
}

variable "owners" {
type = list(string)
}
2 changes: 2 additions & 0 deletions Terraform/secrets/main.tf
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# This folder contains Terraform resources to for secrets stored in Google Cloud Secret Manager.

terraform {
backend "gcs" {
bucket = "heroes-hat-dev-terraform-state-08679"
Expand Down