Upgrading from 12.2.0 to 13.1.0 causes managed node groups to be rebuilt.

[blongman-snapdocs]

I have issues

With managed node groups being rebuilt when upgrading from template 12.2.0 to 13.1.0.

I'm submitting a...

• bug report
• feature request
• support request - read the FAQ first!
• kudos, thank you, warm fuzzy

What is the current behavior?

When I run a terraform plan, I get the following output (amongst other things):

  # module.eks.module.node_groups.aws_eks_node_group.workers["m5"] must be replaced
+/- resource "aws_eks_node_group" "workers" {
        ami_type        = "AL2_x86_64"
      ~ arn             = "arn:aws:eks:us-west-2:057653682278:nodegroup/uat-zOigc4tD/uat-zOigc4tD-m5-clear-possum/aab9dffb-e0a4-f49b-31b5-645b9eaf99c9" -> (known after apply)
        cluster_name    = "uat-zOigc4tD"
        disk_size       = 250
      ~ id              = "uat-zOigc4tD:uat-zOigc4tD-m5-clear-possum" -> (known after apply)
        instance_types  = [
            "m5.2xlarge",
        ]
        labels          = {
            "Environment" = "uat-zOigc4tD"
        }
      ~ node_group_name = "uat-zOigc4tD-m5-clear-possum" -> (known after apply)
        node_role_arn   = "arn:aws:iam::057653682278:role/uat-zOigc4tD20191218170104919600000001"
      ~ release_version = "1.15.11-20200723" -> (known after apply)
      ~ resources       = [
          - {
              - autoscaling_groups              = [
                  - {
                      - name = "eks-aab9dffb-e0a4-f49b-31b5-645b9eaf99c9"
                    },
                ]
              - remote_access_security_group_id = ""
            },
        ] -> (known after apply)
      ~ status          = "ACTIVE" -> (known after apply)
        subnet_ids      = [
            "subnet-021925af4f16050c4",
            "subnet-06665e388e9ea9299",
        ]
        tags            = {
            "accountname"   = "staging"
            "costcenter"    = "uat"
            "eks-cluster"   = "uat-zOigc4tD"
            "environment"   = "uat"
             "owner"         = "sre"
            "pipeline"      = "eks-terraform"
            "project"       = "uat-zOigc4tD"
        }
      ~ version         = "1.15" -> (known after apply)

      ~ scaling_config {
          ~ desired_size = 4 -> 1
            max_size     = 10
            min_size     = 1
        }
    }

  # module.eks.module.node_groups.random_pet.node_groups["m5"] must be replaced
+/- resource "random_pet" "node_groups" {
      ~ id        = "clear-possum" -> (known after apply)
      ~ keepers   = {
            "ami_type"                  = "AL2_x86_64"
            "disk_size"                 = "250"
            "iam_role_arn"              = "arn:aws:iam::057653682278:role/uat-zOigc4tD20191218170104919600000001"
            "instance_type"             = "m5.2xlarge"
            "key_name"                  = ""
          + "launch_template"           = ""
            "node_group_name"           = "uat-zOigc4tD-m5"
            "source_security_group_ids" = ""
            "subnet_ids"                = "subnet-06665e388e9ea9299|subnet-021925af4f16050c4"
        }
        length    = 2
        separator = "-"
    }

If this is a bug, how to reproduce? Please include a code sample if relevant.

Change the module from an already built cluster with a managed node group from 12.2.0 -> 13.1.0.

What's the expected behavior?

No major disruptive changes.

Are you able to fix this problem and submit a PR? Link here if you have already.

Environment details

• Affected module version: When upgrading from 12.2.0 to 13.1.0
• OS: Darwin Ben-Longmans-MacBook-Pro.local 19.6.0 Darwin Kernel Version 19.6.0: Thu Jun 18 20:49:00 PDT 2020; root:xnu-6153.141.1~1/RELEASE_X86_64 x86_64
• Terraform version: 0.12.28

Any other relevant info

I suspect that the launch_template variable needs to not be included in module.eks.module.node_groups.random_pet.node_groups unless it's defined in the map.

[barryib]

Thanks for reporting this. I just opened a PR to address this #1088. Can you please confirm that it fixed your issue.

[blongman-snapdocs]

Interestingly another reason to replace the node group pops up (tags removed and account #s masked):

  # module.eks.module.node_groups.aws_eks_node_group.workers["m5"] must be replaced
+/- resource "aws_eks_node_group" "workers" {
        ami_type        = "AL2_x86_64"
      ~ arn             = "arn:aws:eks:us-west-2:057653682278:nodegroup/uat-zOigc4tD/uat-zOigc4tD-m5-clear-possum/aab9dffb-e0a4-f49b-31b5-645b9eaf99c9" -> (known after apply)
        cluster_name    = "uat-zOigc4tD"
        disk_size       = 250
      ~ id              = "uat-zOigc4tD:uat-zOigc4tD-m5-clear-possum" -> (known after apply)
      ~ instance_types  = [ # forces replacement
          - "m5.2xlarge",
        ]
        labels          = {
            "Environment" = "uat-zOigc4tD"
        }
        node_group_name = "uat-zOigc4tD-m5-clear-possum"
        node_role_arn   = "arn:aws:iam::057653682278:role/uat-zOigc4tD20191218170104919600000001"
      ~ release_version = "1.15.11-20200723" -> (known after apply)
      ~ resources       = [
          - {
              - autoscaling_groups              = [
                  - {
                      - name = "eks-aab9dffb-e0a4-f49b-31b5-645b9eaf99c9"
                    },
                ]
              - remote_access_security_group_id = ""
            },
        ] -> (known after apply)
      ~ status          = "ACTIVE" -> (known after apply)
        subnet_ids      = [
            "subnet-021925af4f16050c4",
            "subnet-06665e388e9ea9299",
        ]
        tags            = {
            "accountname"   = "staging"
            "costcenter"    = "uat"
            "eks-cluster"   = "uat-zOigc4tD"
            "environment"   = "uat"
            "owner"         = "sre"
            "pipeline"      = "eks-terraform"
            "project"       = "uat-zOigc4tD"
        }
      ~ version         = "1.15" -> (known after apply)

      ~ scaling_config {
          ~ desired_size = 4 -> 1
            max_size     = 10
            min_size     = 1
        }
    }

My entire change was:

 module "eks" {
-  source                                     = "terraform-aws-modules/eks/aws"
-  version                                    = "13.1.0"
+  source                                     = "git@github.com:Polyconseil/terraform-aws-eks.git?ref=tba/fix-1087"
+  #version                                    = "13.1.0"

[barryib]

Fixed in 040330d I've forgot one change. It should be better now.

[blongman-snapdocs]

Looks way better with no new change. Primary changes are here:

Terraform will perform the following actions:

  # module.eks.aws_iam_policy.cluster_elb_sl_role_creation[0] will be created
  + resource "aws_iam_policy" "cluster_elb_sl_role_creation" {
      + arn         = (known after apply)
      + description = "Permissions for EKS to create AWSServiceRoleForElasticLoadBalancing service-linked role"
      + id          = (known after apply)
      + name        = (known after apply)
      + name_prefix = "uat-zOigc4tD-elb-sl-role-creation"
      + path        = "/"
      + policy      = jsonencode(
            {
              + Statement = [
                  + {
                      + Action   = [
                          + "ec2:DescribeInternetGateways",
                          + "ec2:DescribeAccountAttributes",
                        ]
                      + Effect   = "Allow"
                      + Resource = "*"
                      + Sid      = ""
                    },
                ]
              + Version   = "2012-10-17"
            }
        )
    }

  # module.eks.aws_iam_role_policy.cluster_elb_sl_role_creation[0] will be destroyed
  - resource "aws_iam_role_policy" "cluster_elb_sl_role_creation" {
      - id          = "uat-zOigc4tD20191216194532063400000001:uat-zOigc4tD-elb-sl-role-creation20201001165320161500000001" -> null
      - name        = "uat-zOigc4tD-elb-sl-role-creation20201001165320161500000001" -> null
      - name_prefix = "uat-zOigc4tD-elb-sl-role-creation" -> null
      - policy      = jsonencode(
            {
              - Statement = [
                  - {
                      - Action   = [
                          - "ec2:DescribeInternetGateways",
                          - "ec2:DescribeAccountAttributes",
                        ]
                      - Effect   = "Allow"
                      - Resource = "*"
                      - Sid      = ""
                    },
                ]
              - Version   = "2012-10-17"
            }
        ) -> null
      - role        = "uat-zOigc4tD20191216194532063400000001" -> null
    }

  # module.eks.aws_iam_role_policy_attachment.cluster_AmazonEKSVPCResourceControllerPolicy[0] will be created
  + resource "aws_iam_role_policy_attachment" "cluster_AmazonEKSVPCResourceControllerPolicy" {
      + id         = (known after apply)
      + policy_arn = "arn:aws:iam::aws:policy/AmazonEKSVPCResourceController"
      + role       = "uat-zOigc4tD20191216194532063400000001"
    }

  # module.eks.aws_iam_role_policy_attachment.cluster_elb_sl_role_creation[0] will be created
  + resource "aws_iam_role_policy_attachment" "cluster_elb_sl_role_creation" {
      + id         = (known after apply)
      + policy_arn = (known after apply)
      + role       = "uat-zOigc4tD20191216194532063400000001"
    }

auth-map changes here:

      ~ metadata {
            annotations      = {}
            generation       = 0
          ~ labels           = {
              + "app.kubernetes.io/managed-by" = "Terraform"
              + "terraform.io/module"          = "terraform-aws-modules.eks.aws"
            }
            name             = "aws-auth"
            namespace        = "kube-system"
            resource_version = "75503174"
            self_link        = "/api/v1/namespaces/kube-system/configmaps/aws-auth"
            uid              = "171fc144-21b8-11ea-8c90-061b37c5a04c"
        }

It's pretty in line with the changes I saw planning to go to 13.0.0.

[barryib]

Great. Thanks for your help. I just merged #1088.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.