Skip to content
This repository has been archived by the owner on Aug 10, 2023. It is now read-only.

Commit

Permalink
Enable APIs
Browse files Browse the repository at this point in the history
  • Loading branch information
jpatokal committed Aug 20, 2020
1 parent 488f91e commit 5136fd2
Show file tree
Hide file tree
Showing 3 changed files with 14 additions and 4 deletions.
12 changes: 11 additions & 1 deletion tutorials/schedule-dataflow-jobs-with-cloud-scheduler/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,14 @@ git clone https://github.com/GoogleCloudPlatform/community
cd community/tutorials/schedule-dataflow-jobs-with-cloud-scheduler/
```

Enable the services that will be used by Terraform.

```
gcloud services enable cloudscheduler.googleapis.com
gcloud services enable dataflow.googleapis.com
gcloud services enable iam.googleapis.com
```

Set up Terraform's environment variables and a service account.

```
Expand All @@ -103,7 +111,7 @@ Create a bucket on Google Cloud Stoeage, copy the Dataflow templates there and s

```
gsutil mb -p ${GOOGLE_CLOUD_PROJECT} gs://${TF_ADMIN}
gsutil cp -R scheduler-dataflow-demo/dataflow/ gs://${TF_ADMIN}/templates/dataflow-demo-template/
gsutil cp -R scheduler-dataflow-demo/dataflow gs://${TF_ADMIN}/templates/dataflow-demo-template
cat > backend.tf << EOF
terraform {
backend "gcs" {
Expand All @@ -122,6 +130,8 @@ terraform init
terraform apply
```

Note: You may be prompted to open the Cloud Console to create an App Engine app. Be sure to create it in the same region as specified in your Terraform variables.

The job will run based on the schedule you defined in the terraform script.
In addition, you can manually run the scheduler through UI and watch it trigger your Dataflow batch job.
You can check the status of jobs through the UI.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ resource "google_cloud_scheduler_job" "scheduler" {
schedule = "0 0 * * *"
# This needs to be us-central1 even if the app engine is in us-central.
# You will get a resource not found error if just using us-central.
region = "us-central1"
region = var.region

http_target {
http_method = "POST"
Expand All @@ -32,7 +32,7 @@ resource "google_cloud_scheduler_job" "scheduler" {
"environment": {
"maxWorkers": "10",
"tempLocation": "gs://${var.bucket}/temp",
"zone": "us-west1-a"
"zone": "${var.region}-a"
}
}
EOT
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ variable "project_id" {

variable "region" {
type = string
default = "us-west1"
default = "us-central1"
}

variable "bucket" {
Expand Down

0 comments on commit 5136fd2

Please sign in to comment.