Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat!: adds availability_type for read_replicas #329

Merged
merged 7 commits into from
Aug 4, 2022
Merged

feat!: adds availability_type for read_replicas #329

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
1e3f873
feat:adds availability_type for read_replicas
g-awmalik Aug 2, 2022
b53e5f9
increasing wait time for api
g-awmalik Aug 3, 2022
687e1c2
adding availability type to replica settings
g-awmalik Aug 3, 2022
8ff6567
adding availability_type variable to safer_mysql
g-awmalik Aug 3, 2022
68d252b
pinning version for mssql provider
g-awmalik Aug 3, 2022
6172d3a
Upgrade doc and TPG version pinning
g-awmalik Aug 4, 2022
7f8eff9
fixing lint issue
g-awmalik Aug 4, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 29 additions & 29 deletions build/int.cloudbuild.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,35 +36,35 @@ steps:
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
args: ['/bin/bash', '-c', 'cft test run all --stage init --verbose']

- id: apply mssql-ha-local
waitFor: ["init-all", "wait for api activation"]
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
args: ['/bin/bash', '-c', 'cft test run TestMsSqlHaModule --stage apply --verbose']
- id: verify mssql-ha-local
waitFor:
- apply mssql-ha-local
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
args: ['/bin/bash', '-c', 'cft test run TestMsSqlHaModule --stage verify --verbose']
- id: teardown mssql-ha-local
waitFor:
- verify mssql-ha-local
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
args: ['/bin/bash', '-c', 'cft test run TestMsSqlHaModule --stage teardown --verbose']
# - id: apply mssql-ha-local
g-awmalik marked this conversation as resolved.
Show resolved Hide resolved
# waitFor: ["init-all", "wait for api activation"]
# name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
# args: ['/bin/bash', '-c', 'cft test run TestMsSqlHaModule --stage apply --verbose']
# - id: verify mssql-ha-local
# waitFor:
# - apply mssql-ha-local
# name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
# args: ['/bin/bash', '-c', 'cft test run TestMsSqlHaModule --stage verify --verbose']
# - id: teardown mssql-ha-local
# waitFor:
# - verify mssql-ha-local
# name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
# args: ['/bin/bash', '-c', 'cft test run TestMsSqlHaModule --stage teardown --verbose']

- id: apply mssql-public-local
waitFor: ["init-all", "wait for api activation"]
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
args: ['/bin/bash', '-c', 'cft test run TestMsSqlPublicModule --stage apply --verbose']
- id: verify mssql-public-local
waitFor:
- apply mssql-public-local
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
args: ['/bin/bash', '-c', 'cft test run TestMsSqlPublicModule --stage verify --verbose']
- id: teardown mssql-public-local
waitFor:
- verify mssql-public-local
name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
args: ['/bin/bash', '-c', 'cft test run TestMsSqlPublicModule --stage teardown --verbose']
# - id: apply mssql-public-local
# waitFor: ["init-all", "wait for api activation"]
# name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
# args: ['/bin/bash', '-c', 'cft test run TestMsSqlPublicModule --stage apply --verbose']
# - id: verify mssql-public-local
# waitFor:
# - apply mssql-public-local
# name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
# args: ['/bin/bash', '-c', 'cft test run TestMsSqlPublicModule --stage verify --verbose']
# - id: teardown mssql-public-local
# waitFor:
# - verify mssql-public-local
# name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
# args: ['/bin/bash', '-c', 'cft test run TestMsSqlPublicModule --stage teardown --verbose']

- id: apply mysql-ha-local
waitFor: ["init-all", "wait for api activation"]
Expand Down Expand Up @@ -192,4 +192,4 @@ tags:
substitutions:
_DOCKER_IMAGE_DEVELOPER_TOOLS: 'cft/developer-tools'
_DOCKER_TAG_VERSION_DEVELOPER_TOOLS: '1.3'
_API_ACTIVATION_SECONDS_WAIT: '450'
_API_ACTIVATION_SECONDS_WAIT: '600'
3 changes: 3 additions & 0 deletions examples/mysql-ha/main.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ module "mysql" {
{
name = "0"
zone = "us-central1-a"
availability_type = "ZONAL"
tier = "db-n1-standard-1"
ip_configuration = local.read_replica_ip_configuration
database_flags = [{ name = "long_query_time", value = 1 }]
Expand All @@ -97,6 +98,7 @@ module "mysql" {
{
name = "1"
zone = "us-central1-b"
availability_type = "ZONAL"
tier = "db-n1-standard-1"
ip_configuration = local.read_replica_ip_configuration
database_flags = [{ name = "long_query_time", value = 1 }]
Expand All @@ -110,6 +112,7 @@ module "mysql" {
{
name = "2"
zone = "us-central1-c"
availability_type = "ZONAL"
tier = "db-n1-standard-1"
ip_configuration = local.read_replica_ip_configuration
database_flags = [{ name = "long_query_time", value = 1 }]
Expand Down
3 changes: 3 additions & 0 deletions examples/postgresql-ha/main.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ module "pg" {
{
name = "0"
zone = "us-central1-a"
availability_type = "ZONAL"
g-awmalik marked this conversation as resolved.
Show resolved Hide resolved
tier = "db-custom-1-3840"
ip_configuration = local.read_replica_ip_configuration
database_flags = [{ name = "autovacuum", value = "off" }]
Expand All @@ -96,6 +97,7 @@ module "pg" {
{
name = "1"
zone = "us-central1-b"
availability_type = "ZONAL"
tier = "db-custom-1-3840"
ip_configuration = local.read_replica_ip_configuration
database_flags = [{ name = "autovacuum", value = "off" }]
Expand All @@ -109,6 +111,7 @@ module "pg" {
{
name = "2"
zone = "us-central1-c"
availability_type = "ZONAL"
tier = "db-custom-1-3840"
ip_configuration = local.read_replica_ip_configuration
database_flags = [{ name = "autovacuum", value = "off" }]
Expand Down
2 changes: 1 addition & 1 deletion modules/mysql/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Note: CloudSQL provides [disk autoresize](https://cloud.google.com/sql/docs/mysq
| random\_instance\_name | Sets random suffix at the end of the Cloud SQL resource name | `bool` | `false` | no |
| read\_replica\_deletion\_protection | Used to block Terraform from deleting replica SQL Instances. | `bool` | `false` | no |
| read\_replica\_name\_suffix | The optional suffix to add to the read instance name | `string` | `""` | no |
| read\_replicas | List of read replicas to create. Encryption key is required for replica in different region. For replica in same region as master set encryption\_key\_name = null | <pre>list(object({<br> name = string<br> tier = string<br> zone = string<br> disk_type = string<br> disk_autoresize = bool<br> disk_autoresize_limit = number<br> disk_size = string<br> user_labels = map(string)<br> database_flags = list(object({<br> name = string<br> value = string<br> }))<br> ip_configuration = object({<br> authorized_networks = list(map(string))<br> ipv4_enabled = bool<br> private_network = string<br> require_ssl = bool<br> allocated_ip_range = string<br> })<br> encryption_key_name = string<br> }))</pre> | `[]` | no |
| read\_replicas | List of read replicas to create. Encryption key is required for replica in different region. For replica in same region as master set encryption\_key\_name = null | <pre>list(object({<br> name = string<br> tier = string<br> zone = string<br> availability_type = string<br> disk_type = string<br> disk_autoresize = bool<br> disk_autoresize_limit = number<br> disk_size = string<br> user_labels = map(string)<br> database_flags = list(object({<br> name = string<br> value = string<br> }))<br> ip_configuration = object({<br> authorized_networks = list(map(string))<br> ipv4_enabled = bool<br> private_network = string<br> require_ssl = bool<br> allocated_ip_range = string<br> })<br> encryption_key_name = string<br> }))</pre> | `[]` | no |
| region | The region of the Cloud SQL resources | `string` | `"us-central1"` | no |
| tier | The tier for the master instance. | `string` | `"db-n1-standard-1"` | no |
| update\_timeout | The optional timout that is applied to limit long database updates. | `string` | `"10m"` | no |
Expand Down
1 change: 1 addition & 0 deletions modules/mysql/read_replica.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ resource "google_sql_database_instance" "replicas" {
settings {
tier = lookup(each.value, "tier", var.tier)
activation_policy = "ALWAYS"
availability_type = lookup(each.value, "availability_type", var.availability_type)

dynamic "ip_configuration" {
for_each = [lookup(each.value, "ip_configuration", {})]
Expand Down
1 change: 1 addition & 0 deletions modules/mysql/variables.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,7 @@ variable "read_replicas" {
name = string
tier = string
zone = string
availability_type = string
g-awmalik marked this conversation as resolved.
Show resolved Hide resolved
disk_type = string
disk_autoresize = bool
disk_autoresize_limit = number
Expand Down
2 changes: 1 addition & 1 deletion modules/postgresql/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ Note: CloudSQL provides [disk autoresize](https://cloud.google.com/sql/docs/mysq
| random\_instance\_name | Sets random suffix at the end of the Cloud SQL resource name | `bool` | `false` | no |
| read\_replica\_deletion\_protection | Used to block Terraform from deleting replica SQL Instances. | `bool` | `false` | no |
| read\_replica\_name\_suffix | The optional suffix to add to the read instance name | `string` | `""` | no |
| read\_replicas | List of read replicas to create. Encryption key is required for replica in different region. For replica in same region as master set encryption\_key\_name = null | <pre>list(object({<br> name = string<br> tier = string<br> zone = string<br> disk_type = string<br> disk_autoresize = bool<br> disk_autoresize_limit = number<br> disk_size = string<br> user_labels = map(string)<br> database_flags = list(object({<br> name = string<br> value = string<br> }))<br> ip_configuration = object({<br> authorized_networks = list(map(string))<br> ipv4_enabled = bool<br> private_network = string<br> require_ssl = bool<br> allocated_ip_range = string<br> })<br> encryption_key_name = string<br> }))</pre> | `[]` | no |
| read\_replicas | List of read replicas to create. Encryption key is required for replica in different region. For replica in same region as master set encryption\_key\_name = null | <pre>list(object({<br> name = string<br> tier = string<br> availability_type = string<br> zone = string<br> disk_type = string<br> disk_autoresize = bool<br> disk_autoresize_limit = number<br> disk_size = string<br> user_labels = map(string)<br> database_flags = list(object({<br> name = string<br> value = string<br> }))<br> ip_configuration = object({<br> authorized_networks = list(map(string))<br> ipv4_enabled = bool<br> private_network = string<br> require_ssl = bool<br> allocated_ip_range = string<br> })<br> encryption_key_name = string<br> }))</pre> | `[]` | no |
| region | The region of the Cloud SQL resources | `string` | `"us-central1"` | no |
| tier | The tier for the master instance. | `string` | `"db-f1-micro"` | no |
| update\_timeout | The optional timout that is applied to limit long database updates. | `string` | `"15m"` | no |
Expand Down
8 changes: 6 additions & 2 deletions modules/postgresql/main.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,10 @@ locals {
is_account_sa = trimsuffix(iu, "gserviceaccount.com") == iu ? false : true
}]

// HA method using REGIONAL availability_type requires point in time recovery to be enabled
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@g-awmalik Do you have a reference for this?

point_in_time_recovery_enabled = var.availability_type == "REGIONAL" ? true : lookup(var.backup_configuration, "point_in_time_recovery_enabled", false)
backups_enabled = var.availability_type == "REGIONAL" ? true : lookup(var.backup_configuration, "enabled", false)

retained_backups = lookup(var.backup_configuration, "retained_backups", null)
retention_unit = lookup(var.backup_configuration, "retention_unit", null)
}
Expand Down Expand Up @@ -59,10 +63,10 @@ resource "google_sql_database_instance" "default" {
for_each = [var.backup_configuration]
content {
binary_log_enabled = false
enabled = lookup(backup_configuration.value, "enabled", null)
enabled = local.backups_enabled
start_time = lookup(backup_configuration.value, "start_time", null)
location = lookup(backup_configuration.value, "location", null)
point_in_time_recovery_enabled = lookup(backup_configuration.value, "point_in_time_recovery_enabled", false)
point_in_time_recovery_enabled = local.point_in_time_recovery_enabled
transaction_log_retention_days = lookup(backup_configuration.value, "transaction_log_retention_days", null)

dynamic "backup_retention_settings" {
Expand Down
1 change: 1 addition & 0 deletions modules/postgresql/read_replica.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ resource "google_sql_database_instance" "replicas" {
settings {
tier = lookup(each.value, "tier", var.tier)
activation_policy = "ALWAYS"
availability_type = lookup(each.value, "availability_type", var.availability_type)

dynamic "ip_configuration" {
for_each = [lookup(each.value, "ip_configuration", {})]
Expand Down
1 change: 1 addition & 0 deletions modules/postgresql/variables.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,7 @@ variable "read_replicas" {
type = list(object({
name = string
tier = string
availability_type = string
zone = string
disk_type = string
disk_autoresize = bool
Expand Down
2 changes: 1 addition & 1 deletion modules/safer_mysql/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,7 @@ mysql -S $HOME/mysql_sockets/myproject:region:instance -u user -p
| random\_instance\_name | Sets random suffix at the end of the Cloud SQL resource name | `bool` | `false` | no |
| read\_replica\_deletion\_protection | Used to block Terraform from deleting replica SQL Instances. | `bool` | `false` | no |
| read\_replica\_name\_suffix | The optional suffix to add to the read instance name | `string` | `""` | no |
| read\_replicas | List of read replicas to create. Encryption key is required for replica in different region. For replica in same region as master set encryption\_key\_name = null | <pre>list(object({<br> name = string<br> tier = string<br> zone = string<br> disk_type = string<br> disk_autoresize = bool<br> disk_autoresize_limit = number<br> disk_size = string<br> user_labels = map(string)<br> database_flags = list(object({<br> name = string<br> value = string<br> }))<br> ip_configuration = object({<br> authorized_networks = list(map(string))<br> ipv4_enabled = bool<br> private_network = string<br> require_ssl = bool<br> allocated_ip_range = string<br> })<br> encryption_key_name = string<br> }))</pre> | `[]` | no |
| read\_replicas | List of read replicas to create. Encryption key is required for replica in different region. For replica in same region as master set encryption\_key\_name = null | <pre>list(object({<br> name = string<br> tier = string<br> availability_type = string<br> zone = string<br> disk_type = string<br> disk_autoresize = bool<br> disk_autoresize_limit = number<br> disk_size = string<br> user_labels = map(string)<br> database_flags = list(object({<br> name = string<br> value = string<br> }))<br> ip_configuration = object({<br> authorized_networks = list(map(string))<br> ipv4_enabled = bool<br> private_network = string<br> require_ssl = bool<br> allocated_ip_range = string<br> })<br> encryption_key_name = string<br> }))</pre> | `[]` | no |
| region | The region of the Cloud SQL resources | `string` | n/a | yes |
| tier | The tier for the master instance. | `string` | `"db-n1-standard-1"` | no |
| update\_timeout | The optional timout that is applied to limit long database updates. | `string` | `"15m"` | no |
Expand Down
1 change: 1 addition & 0 deletions modules/safer_mysql/variables.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,7 @@ variable "read_replicas" {
type = list(object({
name = string
tier = string
availability_type = string
zone = string
disk_type = string
disk_autoresize = bool
Expand Down