deploy: Modularize gcp terraform #717

jlerche · 2019-08-01T01:48:00Z

What problem does this PR solve?

Modularizes the GCP terraform script and uses released helm charts per #575

What is changed and how does it work?

Comprehensive refactor into modules so multiple tidb clusters can be managed

Check List

Tests

Manual test

Code changes

Has Terraform changes

Side effects

Breaking backward compatibility

Related changes

Need to cherry-pick to the release branch
Need to update the documentation

Does this PR introduce a user-facing change?:

aylei

Overall LGTM

deploy/gcp/variables.tf

deploy/gcp/tidbclusters.tf

deploy/gcp/variables.tf

deploy/modules/gcp/bastion/bastion.tf

deploy/modules/gcp/tidb-operator/main.tf

aylei

Rest LGTM

deploy/modules/gcp/bastion/bastion.tf

deploy/modules/gcp/bastion/outputs.tf

deploy/gcp/data.tf

aylei · 2019-08-02T04:07:16Z

deploy/modules/gcp/tidb-operator/main.tf

+
+  maintenance_policy {
+    daily_maintenance_window {
+      start_time = "01:00"


Then it would be better if users can customize the time, because 01:00 GMT is 9:00 Asia/Beijing

I don't see how this get resolved

deploy/modules/gcp/tidb-operator/main.tf

deploy/modules/share/tidb-cluster-release/variables.tf

tennix

For the no new line at end of file, I suggest you configure your editor to auto add new line at end of file when invoking save.

deploy/modules/gcp/bastion/variables.tf

deploy/modules/gcp/project-credentials/main.tf

deploy/modules/gcp/project-credentials/outputs.tf

deploy/gcp/data.tf

deploy/modules/gcp/tidb-operator/variables.tf

deploy/modules/gcp/vpc/outputs.tf

jlerche · 2019-08-02T18:55:29Z

For the no new line at end of file, I suggest you configure your editor to auto add new line at end of file when invoking save.

Yes I will do that going forward. I've been relying on terraform fmt but it doesn't seem to be adding a newline.

tennix

LGTM

deploy/modules/gcp/bastion/bastion.tf

aylei

LGTM

gregwebs · 2019-08-07T19:33:20Z

The file templates/tidb-cluster-values.yaml.tpl seems unused now?

gregwebs · 2019-08-08T18:34:06Z

monitor storage is 100 GiB even for the small deployment
disks are getting orphaned now

CLAassistant · 2019-08-09T09:21:00Z

All committers have signed the CLA.

deploy/modules/gcp/tidb-operator/main.tf

tennix · 2019-08-11T08:39:12Z

deploy/modules/gcp/tidb-cluster/main.tf

@@ -158,4 +158,11 @@ EOS
      KUBECONFIG = var.kubeconfig_path
    }
  }
+  provisioner "local-exec" {
+    when = destroy


What does terraform say when updating the tidb-cluster? Will it be recreating which means destroying and creating again?

In this case it does not destroy and create again if tidb-cluster changes. For a null_resource as long as triggers are not specified, or the ones that are specified don't change, then it should stay the same.

OK, cool! This is what we need.

tennix

LGTM

gregwebs · 2019-08-15T04:38:42Z

apply + destroy works well for me, as does the patch of the reclaim policy (no more orhpaned disks).

sre-bot · 2019-08-15T05:01:52Z

cherry pick to release-1.0 failed

weekface · 2019-08-15T05:23:11Z

@jlerche please open a cherry-pick to release-1.0 branch

refactor of gcp terraform into modules * Add maintenance time variable with default * Changes bastion image to centos * Removes destroy trigger for patching reclaimpolicy PV * Adds bash script to change pv reclaimpolicy from Retain to Delete (cherry picked from commit 81fe851)

refactor of gcp terraform into modules (cherry picked from commit 81fe851)

jlerche added 7 commits July 29, 2019 11:56

Adds gcp project-credentials and vpc terraform modules

a079fb0

More progress

cfac212

intermediate commit

0646f9e

More refactoring of gcp terraform into modules

a39858f

Correctly passes module outputs to variables

60492b0

Correctly set taints and selectors on node pools

9d485e1

Adds base values for GCP

d15a095

jlerche requested review from gregwebs and aylei August 1, 2019 02:23

aylei reviewed Aug 1, 2019

View reviewed changes

jlerche added 2 commits August 1, 2019 09:14

Adds formatting pass

e1e8a21

Fixes to get the helm provider to work

17629ab

jlerche marked this pull request as ready for review August 2, 2019 03:12

jlerche requested a review from aylei August 2, 2019 03:12

jlerche and others added 3 commits August 1, 2019 20:19

Removes local variable that was no longer necessary

ca71692

Uncomments and fixes data.tf

cf89eae

Merge branch 'master' into modularize_gcp_terraform

b38f8d5

aylei reviewed Aug 2, 2019

View reviewed changes

tennix reviewed Aug 2, 2019

View reviewed changes

jlerche added 2 commits August 2, 2019 13:34

Fixes formatting, removes unnecessary variables, refactors

d825c03

Refactors some names and formats

bc229a8

tennix previously approved these changes Aug 5, 2019

View reviewed changes

tennix reviewed Aug 5, 2019

View reviewed changes

deploy/modules/gcp/bastion/bastion.tf Outdated Show resolved Hide resolved

jlerche added 2 commits August 5, 2019 20:19

Add maintenance time variable with default

7480cdd

Changes bastion image to centos

c801a64

jlerche dismissed tennix’s stale review via c801a64 August 6, 2019 04:18

jlerche and others added 2 commits August 5, 2019 21:34

Adds comment to bastion module explaining empty access_config block

2799672

Merge branch 'master' into modularize_gcp_terraform

df29390

aylei previously approved these changes Aug 6, 2019

View reviewed changes

tennix added cloud/gcp Google Cloud Platform needs-cherry-pick-1.0 labels Aug 7, 2019

Removes tidb-cluster-values.yaml.tpl

239918f

jlerche dismissed tennix’s stale review via 239918f August 7, 2019 19:54

helm doesn't install a release unless there's wait = false

284415f

tennix previously approved these changes Aug 8, 2019

View reviewed changes

Removes storage size override in default.yaml

a9cc9c4

jlerche dismissed tennix’s stale review via a9cc9c4 August 8, 2019 18:56

jlerche added 2 commits August 8, 2019 12:11

Refactors the tidb operator chart version variables

cd4dacf

Moves pv reclaimpolicy=delete to tidb-cluster module

d8ba365

fix create-service-account.sh

a87796c

tennix reviewed Aug 11, 2019

View reviewed changes

jlerche added 4 commits August 14, 2019 13:59

Removes destroy trigger for patching reclaimpolicy PV

886adbf

Adds bash script to change pv reclaimpolicy from Retain to Delete

93db57b

Formatting pass

9f32741

Formatting pass

adda73b

tennix approved these changes Aug 15, 2019

View reviewed changes

Merge branch 'master' into modularize_gcp_terraform

c84d1d1

gregwebs approved these changes Aug 15, 2019

View reviewed changes

gregwebs merged commit 81fe851 into pingcap:master Aug 15, 2019

jlerche mentioned this pull request Aug 15, 2019

cherry pick PR 717 into release-1.0 #773

Merged

weekface pushed a commit that referenced this pull request Aug 15, 2019

deploy: Modularize gcp terraform (#717) (#773)

6028c79

refactor of gcp terraform into modules (cherry picked from commit 81fe851)

yahonda pushed a commit that referenced this pull request Dec 27, 2021

zh, en/: update tidb-operator chart version to v1.1.5 (#717)

c614ed9

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

deploy: Modularize gcp terraform #717

deploy: Modularize gcp terraform #717

jlerche commented Aug 1, 2019 •

edited

Loading

aylei left a comment

aylei left a comment

aylei Aug 2, 2019

aylei Aug 5, 2019

tennix left a comment

jlerche commented Aug 2, 2019

tennix left a comment

aylei left a comment

gregwebs commented Aug 7, 2019

gregwebs commented Aug 8, 2019 •

edited

Loading

CLAassistant commented Aug 9, 2019 •

edited

Loading

tennix Aug 11, 2019

jlerche Aug 13, 2019

tennix Aug 13, 2019

tennix left a comment

gregwebs commented Aug 15, 2019

sre-bot commented Aug 15, 2019

weekface commented Aug 15, 2019

deploy: Modularize gcp terraform #717

deploy: Modularize gcp terraform #717

Conversation

jlerche commented Aug 1, 2019 • edited Loading

What problem does this PR solve?

What is changed and how does it work?

Check List

Does this PR introduce a user-facing change?:

aylei left a comment

Choose a reason for hiding this comment

aylei left a comment

Choose a reason for hiding this comment

aylei Aug 2, 2019

Choose a reason for hiding this comment

aylei Aug 5, 2019

Choose a reason for hiding this comment

tennix left a comment

Choose a reason for hiding this comment

jlerche commented Aug 2, 2019

tennix left a comment

Choose a reason for hiding this comment

aylei left a comment

Choose a reason for hiding this comment

gregwebs commented Aug 7, 2019

gregwebs commented Aug 8, 2019 • edited Loading

CLAassistant commented Aug 9, 2019 • edited Loading

tennix Aug 11, 2019

Choose a reason for hiding this comment

jlerche Aug 13, 2019

Choose a reason for hiding this comment

tennix Aug 13, 2019

Choose a reason for hiding this comment

tennix left a comment

Choose a reason for hiding this comment

gregwebs commented Aug 15, 2019

sre-bot commented Aug 15, 2019

weekface commented Aug 15, 2019

jlerche commented Aug 1, 2019 •

edited

Loading

gregwebs commented Aug 8, 2019 •

edited

Loading

CLAassistant commented Aug 9, 2019 •

edited

Loading