Terraform gives error while trying to upgrade PostgreSQL primary and read-replica in same module

[satish-chef]

Terraform Version

Terraform v0.12.20

Terraform Configuration Files

resource aws_db_parameter_group this {
  name   = "abc"
  family = var.db_parameter_group_family


  lifecycle {
    create_before_destroy = true
  }
}


resource aws_db_instance this {
  identifier                = "abc"
  final_snapshot_identifier = var.final_snapshot_identifier

  lifecycle {
    ignore_changes = [
      snapshot_identifier,
      password
    ]
  }
  username                        = local.username
  password                        = local.password
  name                            = var.db_name
  engine                          = "postgres"
  engine_version                  = var.engine_version
  allocated_storage               = var.allocated_storage
  storage_type                    = var.storage_type
  auto_minor_version_upgrade      = var.auto_minor_version_upgrade
  instance_class                  = var.instance_class
  multi_az                        = var.multi_az
  parameter_group_name            = aws_db_parameter_group.this.id
  backup_retention_period         = var.backup_retention_period
  vpc_security_group_ids          = [data.aws_security_group.this.id]
  db_subnet_group_name            = var.db_subnet_group_name
  apply_immediately               = var.apply_immediately
  snapshot_identifier             = var.snapshot_identifier
  skip_final_snapshot             = var.skip_final_snapshot
  tags                            = var.defaults.default_tags
  deletion_protection             = var.deletion_protection
  enabled_cloudwatch_logs_exports = var.enabled_cloudwatch_logs_exports
  storage_encrypted               = var.storage_encrypted
  kms_key_id                      = local.kms_key_arn
  allow_major_version_upgrade     = var.allow_major_version_upgrade

  publicly_accessible = false
}


resource aws_db_parameter_group read_replica {
  name   = "${var.rds_read_replica}-rr-pg12"
  family = postgres11

  lifecycle {
    create_before_destroy = true
  }
}

resource aws_db_instance read_replica {
  identifier                      = "${var.rds_read_replica}-rr"
  replicate_source_db             = aws_db_instance.this.id
  storage_type                    = var.storage_type
  auto_minor_version_upgrade      = var.auto_minor_version_upgrade
  parameter_group_name            = aws_db_parameter_group.read_replica.id
  vpc_security_group_ids          = [var.aws_security_group.this.id]
  apply_immediately               = "true"
  skip_final_snapshot             = "true"
  tags                            = var..rds_tags
  storage_encrypted               = var.storage_encrypted
  kms_key_id                      = local.kms_key_arn
  enabled_cloudwatch_logs_exports = var.enabled_cloudwatch_logs_exports
  depends_on                      = [aws_db_instance.this]
}

Expected Behavior

Terraform should have upgraded PostgreSQL primary and read-replica from version 9.6.20 to version 12.5. To workaround, I upgraded the primary manually which upgraded both primary and read-replia RDS instanes.

Actual Behavior

Terraform gave below error:

Error: Error modifying DB Instance qa-abc-rep-rr: InvalidParameterCombination: The Parameter Group qa-abc-rr-pg12 with DBParameterGroupFamily postgres12 cannot be used for this instance. Please use a Parameter Group with DBParameterGroupFamily postgres9.6
	status code: 400, request id: 4590cb12-88a1-411a-a128-3b31bb0337fe

As per AWS document, primary and read-replica instanes are upgraded simultaneously but I am not sure if there is a way to execute two resources in parallel in terraform.

During a major version upgrade, Amazon RDS also upgrades all of your in-Region read replicas along with the primary DB instance.

I have put lifecycle in DB parameter group as per suggestion in this comment - hashicorp/terraform-provider-aws#6448 (comment)

Steps to Reproduce

I am calling this module from a root module below:

module qa_rds {
  source = "path_to_child_module"

  short_name                                    = "qa"
  db_name                                       = "abc"
  instance_class                                = var.instance_class
  allocated_storage                             = var.allocated_storage
  engine_version                                = var.engine_version
  db_parameter_group_family                     = var.db_parameter_group_family
  storage_type                                  = var.storage_type
  multi_az                                      = var.multi_az
  backup_retention_period                       = var.backup_retention_period
  auto_minor_version_upgrade                    = var.auto_minor_version_upgrade
  apply_immediately                             = var.apply_immediately
  skip_final_snapshot                           = var.skip_final_snapshot
  final_snapshot_identifier                     = var.final_snapshot_identifier
  deletion_protection                           = var.deletion_protection
  enabled_cloudwatch_logs_exports               = var.enabled_cloudwatch_logs_exports
  storage_encrypted                             = false
  allow_major_version_upgrade                   = "true"
}

Additional Context

I think this is a typical "Chicken-and-egg" problem in which terraform cannot decide which resource to apply first since AWS in the background is upgrading both RDS instances parallelly. I tried depends_on in read-replica but it gave same error.

References

[jbardin]

This appears to be an issue or question with the AWS provider, not with Terraform itself. You can see existing issues and file a new one in their repository here: https://github.com/hashicorp/terraform-provider-aws/issues. If you have questions about Terraform or the AWS provider, it's better to use the community forum where there are more people ready to help. The GitHub issues here are monitored only by our few core maintainers.

Thanks!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.