Merge terraform into early access

.gitignore

@@ -0,0 +1,5 @@
+**/.terraform
+*.tfstate
+*.tfstate.*
+sql_credentials.json
+key.json

Terraform/README.md

@@ -0,0 +1,146 @@
+# FDA MyStudies Terraform Infrastructure
+
+These directories define the entire GCP infrastructure app to run the FDA
+MyStudies application.
+
+## Prerequisites
+
+1.  Install the following dependencies and add them to your PATH:
+
+    -   [GCloud](https://cloud.google.com/sdk/gcloud)
+    -   [Terraform](https://www.terraform.io/)
+    -   [Terragrunt](https://terragrunt.gruntwork.io/)
+
+1.  Get familiar with [GCP](https://cloud.google.com/docs/overview),
+    [Terraform](https://www.terraform.io/intro/index.html) and
+    [Terragrunt](https://blog.gruntwork.io/terragrunt-how-to-keep-your-terraform-code-dry-and-maintainable-f61ae06959d8).
+
+    The infrastructure is deployed using Terraform, which is an industry
+    standard for defining infrastructure-as-code. Terragrunt is used as a
+    wrapper around Terraform to manage multiple Terraform deployments and reduce
+    duplication.
+
+1.  Setup your
+    [organization](https://cloud.google.com/resource-manager/docs/creating-managing-organization)
+    for GCP resources and [G Suite Domain](https://gsuite.google.com/) for
+    groups.
+
+1.  [Create administrative groups](https://support.google.com/a/answer/33343?hl=en)
+    in the G Suite Domain:
+
+    -   {PREFIX}-org-admins@{DOMAIN}.com: This group has administrative access
+        to the entire org. This group can be used in break-glass situations to
+        give humans access to the org to make changes.
+
+    -   {PREFIX}-devops-owners@{DOMAIN}.com: This group has owners access to the
+        devops project to make changes to the CICD project or make changes to
+        the Terraform state.
+
+    -   {PREFIX}-auditors@{DOMAIN}.com: This group has security reviewer
+        (metadata viewer) access to the entire org, as well as viewer access to
+        the audit logs BigQuery and Cloud Storage resources.
+
+    WARNING: It is always recommended to use CICD to deploy changes to the
+    infrastructure. The groups above should remain empty and only have humans
+    added for emergency situations or when investigation is required.
+
+## Directory Structure
+
+The infrastructure is split into multiple directories. Each directory represents
+one Terraform deployment. Each deployment will manage specific resources in you
+infrastructure.
+
+A deployment typically contains the following files:
+
+-   **main.tf**: This file defines the Terraform resources and modules to
+    manage. For more complex deployments, there may be multiple .tf files that
+    define resources.
+
+-   **variables.tf**: This file defines any input variables that the deployment
+    can take.
+
+-   **outputs.tf**: This file defines any outputs from this deployment. These
+    values can be used by other deployments.
+
+-   **terraform.tfvars**: This file defines values for the input variables.
+
+-   **terragrunt.hcl**: This file defines dependencies between other
+    deployments, the remote state, and input values from other dependent
+    deployments.
+
+To see what resources each deployment provisions, check out the comments in each
+**main.tf** file.
+
+## Layout
+
+```
+|- bootstrap: one time setup to create projects to host Terraform state and CICD pipeline.
+|- cicd: CloudBuild configs for the CICD pipeline.
+|- secrets: Definitions of secrets used in the org (secret values are not set in configs).
+|- org: org level resources. Resources within this directory should be managed by CICD pipeline.
+  |- terragrunt.hcl: root Terragrunt config which defines remote state for all deployments.
+  |- project.{PREFIX}-audit: the project to hold all audit logs for the org.
+  |- audit: deployment to setup auditing for the org.
+  |- iam: org level iam definitions such as org admins.
+  |- folder.fda-mystudies: folder to hold all projects related to FDA MyStudies.
+    |- project.{PREFIX}-apps: apps project and resources (GKE)
+    |- project.{PREFIX}-data: data project and resources (GCS buckets, CloudSQL instances)
+    |- project.{PREFIX}-networks: network project and resources (VPC)
+    |- project.{PREFIX}-firebase: firebase project (firestores)
+```
+
+## Deployment Steps
+
+1.  Authenticate as a super admin using `gcloud auth login [ACCOUNT]`.
+
+    WARNING: remember to run `gcloud auth revoke` to logout as a super admin.
+    Being logged in as a super admin beyond the initial setup is dangerous!
+
+1.  Checkout the Terraform configs and set some helper environment variables.
+
+    ```
+    $ git clone my-repo
+    $ cd my-repo
+    $ ROOT=$PWD
+    ```
+
+1.  The bootstrap config must be deployed first in order to create the `devops`
+    project which will host your Terraform state and CICD pipelines.
+
+    ```
+    $ cd $ROOT/boootstrap
+    $ terraform init
+    $ terraform plan
+    $ terraform apply
+    ```
+
+    Your `devops` project should now be ready.
+
+1.  Backup the state of the `devops` project to the newly created state bucket
+    by uncommenting out the `terraform` block in `$ROOT/bootstrap/main.tf` and
+    running:
+
+    ```
+    $ terraform init
+    ```
+
+1.  Deploy secrets used in the org in the `devops` project.
+
+    ```
+    $ cd $ROOT/secrets
+    $ terraform init
+    $ terraform plan
+    $ terraform apply
+    ```
+
+    After the secrets have been created, you must go to the Google Cloud
+    Console, open `Security` --> `Secret Manager` and fill in their values.
+
+1.  Follow `$ROOT/cicd/README.md` to set up CICD pipelines for Terraform
+    configs.
+
+1.  Follow `$ROOT/kubernetes/README.md` to deploy the Kubernetes resources in
+    the GKE cluster.
+
+1.  Revoke your super admin access by running `gcloud auth revoke` and
+    authenticate as a normal user for daily activities.

Terraform/bootstrap/README.md

@@ -0,0 +1,9 @@
+This directory defines resources that must be deployed first in order for the
+rest of the Terraform configs to function.
+
+Run `terraform init` and `terraform apply` in this directory and backup the
+Terraform state files manually.
+
+Currently this only creates the central devops project. After this project has
+been created, Terragrunt can bootstrap the state bucket inside the project and
+manage all the following resources.

Terraform/bootstrap/main.tf

@@ -0,0 +1,74 @@
+# Copyright 2020 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This folder contains Terraform resources to setup the devops project, which includes:
+# - The project itself,
+# - APIs to enable,
+# - Deletion lien,
+# - Project level IAM permissions for the project owners,
+# - A Cloud Storage bucket to store Terraform states for all deployments,
+# - Org level IAM permissions for org admins.
+
+// TODO: replace with https://github.com/terraform-google-modules/terraform-google-bootstrap
+
+# ==============================================================================
+# TODO: Uncomment after initial deployment and run `terraform init`.
+terraform {
+  backend "gcs" {
+    bucket = "heroes-hat-dev-terraform-state-08679"
+    prefix = "bootstrap"
+  }
+}
+# ==============================================================================
+
+# Create the project, enable APIs, and create the deletion lien, if specified.
+module "project" {
+  source  = "terraform-google-modules/project-factory/google"
+  version = "~> 7.0"
+
+  name                    = var.devops_project_id
+  org_id                  = var.org_id
+  billing_account         = var.billing_account
+  lien                    = true
+  default_service_account = "keep"
+  skip_gcloud_download    = true
+  activate_apis = [
+    "cloudbuild.googleapis.com",
+    "secretmanager.googleapis.com",
+  ]
+}
+
+# Terraform state bucket, hosted in the devops project.
+module "state_bucket" {
+  source  = "terraform-google-modules/cloud-storage/google//modules/simple_bucket"
+  version = "~> 1.4"
+
+  name       = var.state_bucket
+  project_id = module.project.project_id
+  location   = var.storage_location
+}
+
+# Project level IAM permissions for devops project owners.
+resource "google_project_iam_binding" "devops_owners" {
+  project = module.project.project_id
+  role    = "roles/owner"
+  members = var.devops_owners
+}
+
+# Org level IAM permissions for org admins.
+resource "google_organization_iam_member" "org_admin" {
+  org_id = var.org_id
+  role   = "roles/resourcemanager.organizationAdmin"
+  member = var.org_admin
+}

Terraform/bootstrap/terraform.tfvars

@@ -0,0 +1,23 @@
+# Copyright 2020 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+devops_project_id = "heroes-hat-dev-devops"
+org_id            = "707577601068"
+billing_account   = "01EA90-3519E1-89CB1F"
+state_bucket      = "heroes-hat-dev-terraform-state-08679"
+storage_location  = "us-central1"
+org_admin         = "group:rocketturtle-gcp-admin@rocketturtle.net"
+devops_owners = [
+  "group:rocketturtle-gcp-admin@rocketturtle.net",
+]

Terraform/bootstrap/variables.tf

@@ -0,0 +1,41 @@
+# Copyright 2020 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+variable "devops_project_id" {
+  type = string
+}
+
+variable "devops_owners" {
+  type = list(string)
+}
+
+variable "org_id" {
+  type = string
+}
+
+variable "billing_account" {
+  type = string
+}
+
+variable "state_bucket" {
+  type = string
+}
+
+variable "storage_location" {
+  type = string
+}
+
+variable "org_admin" {
+  type = string
+}

Terraform/cicd/README.md

@@ -0,0 +1,88 @@
+This directory defines resources needed to setup CICD pipelines of Terraform
+configs.
+
+The CI and CD pipelines use
+[Google Cloud Build](https://cloud.google.com/cloud-build) and
+[Cloud Build Triggers](https://cloud.google.com/cloud-build/docs/automating-builds/create-manage-triggers)
+to detect changes in the repo, trigger builds and run the workloads.
+
+## Setup
+
+1.  In the Terraform Engine config, add a `CICD` block under the `foundation`
+    recipe and specify the following attributes:
+
+    *   `PROJECT_ID`: Project ID of the `devops` project
+    *   `STATE_BUCKET`: Name of the state bucket
+    *   `REPO_OWNER`: GitHub repo owner
+    *   `REPO_NAME`: GitHub repo name
+    *   `BRANCH_REGEX`: Regex of the branches to set the Cloud Build Triggers to
+        monitor
+    *   `CONTINUOUS_DEPLOYMENT_ENABLED`: Whether or not to enable continuous
+        deployment of Terraform configs
+    *   `TRIGGER_ENABLED`: Whether or not to enable all Cloud Build Triggers
+    *   `TERRAFORM_ROOT`: Path of the directory relative to the repo root
+        containing the Terraform configs
+
+1.  Generate the CICD Terraform configs and Cloud Build configs using the
+    Terraform Engine.
+
+1.  Before deployment CICD Terraform resources, follow
+    [installing_the_cloud_build_app](https://cloud.google.com/cloud-build/docs/automating-builds/create-github-app-triggers#installing_the_cloud_build_app)
+    to install the Cloud Build app and connect your GitHub repository to your
+    Cloud project. This currently cannot be done through automation.
+
+1.  Once the GitHub repo is connected, run the following commands in this
+    directory to enable necessary APIs, grant Cloud Build Service Account
+    necessary permissions and create Cloud Build Triggers:
+
+    ```
+    $ terraform init
+    $ terraform plan
+    $ terraform apply
+    ```
+
+    Two presubmit triggers are created by default and results are posted in the
+    Pull Request. Failing these presubmits will block Pull Request submission.
+
+    1.  `tf-validate`: Perform Terraform format and syntax check.
+    1.  `tf-plan`: Generate speculative plans to show a set of possible changes
+        if the pending config changes are deployed.
+
+    If `CONTINUOUS_DEPLOYMENT_ENABLED` is set to `true` in your Terraform Engine
+    config, `continuous_deployment_enabled` will be set to `true` in
+    `terraform.tfvars` in this directory to create an additional Cloud Build
+    Trigger and grant the Cloud Build Service Account broder permissions to
+    automaticaly apply the config changes to GCP after the Pull Request is
+    approved and submitted.
+
+    After the triggers are created, to temporarily disable or re-enable them,
+    set the `trigger_enabled` in `terraform.tfvars` to `false` or `true` and
+    apply the changes by running:
+
+    ```
+    $ terraform init
+    $ terraform plan
+    $ terraform apply
+    ```
+
+## Operation
+
+### Continuous Integration (presubmit)
+
+Presubmit Cloud Build results will be posted as a Cloud Build job link in the
+Pull Request, and they will be configured to block Pull Request submission.
+
+Every new push to the Pull Request at the configured branches will automatically
+trigger presubmit runs. To manually re-trigger CI jobs, comment `/gcbrun` in the
+Pull Ruquest.
+
+### Continuous Deployment (postsubmit)
+
+Postsubmit Cloud Build job will automatically start when a Pull Ruquest is
+submitted to a configured branch. To view the result of the Cloud Build run, go
+to https://console.cloud.google.com/cloud-build/builds and look for your commit
+to view the Cloud Build job triggered by your merged commit.
+
+The Postsubmit Cloud Build Trigger monitors and deploys changes made to `org/`
+folder only. Other changes made to `bootstrap`, `cicd` and `secrets` folders
+should be deployed manually if needed.