helm_release and kubernetes_secret_v1 always update in-place

[alexo1088]

Hey all,

First, thank you for the work on this module, it's been really helpful and the examples are very useful too.

I wanted to ask for some help on an issue that i'm running into. I'm currently following the AWS example, which leverages gitops-bridge to provide the metadata between TF and argo. What i'm noticing is that no matter what I do, the helm_release for argo and the kubernetes secret consistently want to update every single time I run TF apply. Here's the exact message:

  # module.eks_cluster.module.gitops_bridge_bootstrap.helm_release.argocd[0] will be updated in-place
  ~ resource "helm_release" "argocd" {
        id                         = "argo-cd"
      ~ metadata                   = [
          - {
              - app_version = "v2.8.2"
              - chart       = "argo-cd"
              - name        = "argo-cd"
              - namespace   = "argocd"
              - revision    = 6
              - values      = jsonencode(
                    {
                      - configs = {
                          - secret = {
                              - argocdServerAdminPassword = "(sensitive value)"
                            }
                        }
                      - server  = {
                          - service = {
                              - type = "LoadBalancer"
                            }
                        }
                    }
                )
              - version     = "5.45.0"
            },
        ] -> (known after apply)
        name                       = "argo-cd"
        # (28 unchanged attributes hidden)

      - set_sensitive {
          # At least one attribute in this block is (or was) sensitive,
          # so its contents will not be displayed.
        }

        # (1 unchanged block hidden)
    }

  # module.eks_cluster.module.gitops_bridge_bootstrap.kubernetes_secret_v1.cluster[0] will be updated in-place
  ~ resource "kubernetes_secret_v1" "cluster" {
        id                             = "argocd/eks-blueprints-green"
        # (4 unchanged attributes hidden)

      ~ metadata {
          ~ annotations      = {
              - "addons_repo_basepath"                         = "argocd/"
              - "addons_repo_path"                             = "argocd/bootstrap/control-plane/addons"
              - "addons_repo_revision"                         = "HEAD"
              - "addons_repo_url"                              = "git@github.com:aws-samples/eks-blueprints-add-ons"
              - "argocd_password"                              = (sensitive value)
              - "argocd_route53_weight"                        = "0"
              - "aws_account_id"                               = "xxxxx"
              - "aws_cloudwatch_metrics_iam_role_arn"          = "arn:aws:iam::xxxxx:role/aws-cloudwatch-metrics-20231107191852092600000027"
              - "aws_cloudwatch_metrics_namespace"             = "amazon-cloudwatch"
              - "aws_cloudwatch_metrics_service_account"       = "aws-cloudwatch-metrics"
              - "aws_cluster_name"                             = "eks-blueprints-green"
              - "aws_for_fluentbit_iam_role_arn"               = "arn:aws:iam::xxxxxxx:role/aws-for-fluent-bit-20231107191852093100000029"
              - "aws_for_fluentbit_log_group_name"             = "/aws/eks/eks-blueprints-green/aws-fluentbit-logs-20231107191821378800000019"
              - "aws_for_fluentbit_namespace"                  = "kube-system"
              - "aws_for_fluentbit_service_account"            = "aws-for-fluent-bit-sa"
              - "aws_load_balancer_controller_iam_role_arn"    = "arn:aws:iam::xxxxxxxx:role/alb-controller-20231107191852090300000024"
              - "aws_load_balancer_controller_namespace"       = "kube-system"
              - "aws_load_balancer_controller_service_account" = "aws-lb-sa"
              - "aws_region"                                   = "us-east-1"
              - "aws_secret_manager_git_private_ssh_key_name"  = "github-blueprint-ssh-key"
              - "aws_vpc_id"                                   = "vpc-0c98bdd52eb907def"
              - "cert_manager_iam_role_arn"                    = "arn:aws:iam::xxxxxxx:role/cert-manager-20231107191852092600000026"
              - "cert_manager_namespace"                       = "cert-manager"
              - "cert_manager_service_account"                 = "cert-manager"
              - "cluster_autoscaler_iam_role_arn"              = "arn:aws:iam::xxxxxxx:role/cluster-autoscaler-20231107204205450400000001"
              - "cluster_autoscaler_namespace"                 = "kube-system"
              - "cluster_autoscaler_service_account"           = "cluster-autoscaler-sa"
              - "cluster_endpoint"                             = "https://xxxxxxxxx.gr7.us-east-1.eks.amazonaws.com"
              - "cluster_name"                                 = "eks-blueprints-green"
              - "ecsfrontend_route53_weight"                   = "0"
              - "eks_cluster_domain"                           = "eks-blueprints.xxxxxxxxx"
              - "env"                                          = "green"
              - "environment"                                  = "eks-blueprints"
              - "external_dns_iam_role_arn"                    = "arn:aws:iam::XXXXX:role/external-dns-20231107191852090600000025"
              - "external_dns_namespace"                       = "external-dns"
              - "external_dns_policy"                          = "sync"
              - "external_dns_service_account"                 = "external-dns-sa"
              - "external_secrets_iam_role_arn"                = "arn:aws:iam::XXXXXX:role/external-secrets-20231107191852089100000023"
              - "external_secrets_namespace"                   = "external-secrets"
              - "external_secrets_service_account"             = "external-secrets-sa"
              - "gitops_workloads_path"                        = "envs/dev"
              - "gitops_workloads_revision"                    = "main"
              - "gitops_workloads_url"                         = "git@github.com:aws-samples/eks-blueprints-workloads"
              - "ingress_type"                                 = "alb"
              - "route53_weight"                               = "0"
            } -> (known after apply)
            name             = "eks-blueprints-green"
            # (5 unchanged attributes hidden)
        }
    }

Plan: 0 to add, 2 to change, 0 to destroy.

Changes to Outputs:
  ~ gitops_metadata   = (sensitive value)

Is there a way to avoid this consistently changing?

[csantanapr]

Do you know which 2 changes is referring to?
Which example your using? this would help us reproduce it. Perharps the blue-green pattern here? for @allamand to take a look

[allamand]

Hi I also see this behaviour, I didn’t figure out yet why terraform thinks it needs to update something