Support for argocd-apps

[logan-bobo]

• ✋ I have searched the open/closed issues and my issue is not listed.

Please describe your question here

Currently, in the addons section of the docs for ArgoCD there are resources for deploying ArgoCD; workflows, events, and rollouts. However, there is no section argocd-apps.

In past iterations of terraform-aws-eks-blueprints-addons there was the ability to configure ArgoCD application and projects. Is there an intention to expose argocd-apps through this module?

Provide a link to the example/module related to the question

Main helm charts for argocd-apps - https://github.com/argoproj/argo-helm/tree/main/charts/argocd-apps

If there is an intention please just reach out to me and I am happy to implement it.

[Almenon]

This would be great so we could point argocd to a bootstrap repo. The idea is that EKS blueprints manages the "core" helm charts, like external-dns, which should be in all clusters and are rarely changed.

ArgoCD points to a bootstrap repo which points to a helm repo(s), and the helm repo manages "application" helm charts, like custom webservers, internal tooling, and most of our charts in general. These charts are changed often.

[jmreicha]

I would argue that it would be nicer to have the option to see and manage these core charts and components in the ArgoCD UI rather than Terraform. Even if they rarely change it is easier to see what’s going on and how they are configured if everything is in Argo. Just my 2 cents.

[bryantbiggs]

I would argue that it would be nicer to have the option to see and manage these core charts and components in the ArgoCD UI rather than Terraform.

I completely agree. If you choose to go the route of a GitOps operator, once the controller is deployed, all of the cluster resource provisioning should then be handled through that controller and not through Terraform

[Almenon]

Let's have both, that way we are backwards compatible with v4 and people have the flexibility of choosing their preferred approach.

Would you accept a PR for the above feature? It sounds like Logan is willing to contribute if so.

[bryantbiggs]

that way we are backwards compatible with v4

I don't quite follow - we did not have support for Argo Apps in EKS Blueprints v4. And, per semver, major version changes do not guarantee backwards compatibility.

[Almenon]

Maybe we're talking about two different things? The feature I'm talking about was in v4: https://github.com/aws-ia/terraform-aws-eks-blueprints/blob/8a06a6e7006e4bed5630bd49c7434d76c59e0b5e/modules/kubernetes-addons/variables.tf#L852

major version changes do not guarantee backwards compatibility.

True. Backwards compatibility is up to the maintainer's discretion.

[bryantbiggs]

I believe the v4 argocd_applications is possible with this module now as of PR #209 - see what @csantanapr has done here https://github.com/gitops-bridge-dev/gitops-bridge/blob/d6ce1607c6d0a49bcaf9dae6ed883edd1c10b8b9/argocd/iac/terraform/examples/eks/hello-world/main.tf#L97-L100

[logan-bobo]

@bryantbiggs I agree with what has been discussed, cluster resource provisioning should be handled through Argo. What I am looking for that the initial bootstrap of ArgoCD.

The main reason for raising this was a potential solution to solve the removal of argocd_applications in v5 to bootstrap the cluster with an app of apps.

If the above link can solve this problem. This issue can be closed 🤝

Maybe the best path for this is documentation to show the pattern in v5 for bootstrapping Argo as there was for v4. This is what originally led me to raise this issue as I was trying to get an EKS cluster with Argo and an app of apps in one plan and apply cycle. If this approach is not great please let me know.

[tanvp112]

argocd_applications is just collection of ArgoCD custom resources, not difficult to apply in our own terraform code. From CD pipeline perspective, what's the issue if we have a step that provision the cluster with ArgoCD, follow by a step that apply ArgoCD project/application resources? This approach would allow ArgoCD to managed workloads in the cluster too, right?

The gitops-bridge-dev project is great, but what's the rational of going that length to do something which a CD pipeline can do. Happy to hear any input.

[logan-bobo]

argocd_applications is just collection of ArgoCD custom resources, not difficult to apply in our own terraform code.

I guess you could make this argument for any library/module. You want an interface that abstracts away complexity. The interface existed in v4 and is not there in v5 but has been solved in other ways via the gitops-bridge

From CD pipeline perspective, what's the issue if we have a step that provision the cluster with ArgoCD, follow by a step that apply ArgoCD project/application resources? This approach would allow ArgoCD to managed workloads in the cluster too, right?

It would be a fine approach but now you need to maintain your own abstraction to pass data if needed from Terraform to your downstream steps in the CD pipeline. I think this is what @bryantbiggs was discussing in #114 if you now want separate stages you lose the graph capabilities of Terraform and also the ability to easily pass values generated in Terraform (IDs, ARNs, etc.) without your own abstractions.

Anyway, this looks to be solved by #209 so closing this issue. An example of how to bootstrap Argo with an app of apps has been posted. Thanks all 🙌

[jmreicha]

Maybe the best path for this is documentation to show the pattern in v5 for bootstrapping Argo as there was for v4. This is what originally led me to raise this issue as I was trying to get an EKS cluster with Argo and an app of apps in one plan and apply cycle. If this approach is not great please let me know.

This would be very helpful and would reduce confusion for folks (like myself).

[tanvp112]

In the same pipeline the step that provision the cluster with ArgoCD deployed can easily pass-on data to next step. If needed output of each step can be saved for future use. How by doing so would result to "losing graph capabilities of Terraform"? After all, this is just about bootstrap the cluster during its creation with addons and business applications. After the bootstrap, suppose further deployment to the cluster should be handle using ArgoCD/Rollout and not terraform anymore.

Note there could be a side effect that when the cluster needs to be destroy, you will need to first destroy gitops-bridge resources.

[FernandoMiguel]

regarding passing on data values from terrafom to Argo CD deployments, we used a different strategy.
when we create a cluster with terraform, use use a got provider to write a terraform-values.yaml file to the repo where Argo CD deploys the apps.
That way the apps there will have their own specific config, chart versions, regular yaml config, but anything that would be only available in terraform state, can now be made available in an yaml file too.
Since this is usually mostly static details at cluster creation, this approach is incredibly simple.
It will also require adding another value input in your Argo CD Application or ApplicationSet to look for the terraform-values.yaml

resource "github_repository_file" "karpenter_tf" {
  repository          = data.github_repository.deployments.name
  file                = "base/${var.argo_cd_cluster_name_repository}/karpenter/terraform-values.yaml"
  overwrite_on_create = true
  content             = <<EOT
## THIS FILE IS GENERATED BY TERRAFORM. DON'T MODIFY. CHANGES WILL BE OVERWRITTEN
x-anchors:
  clustername: &clustername ${var.cluster_name}
  availabilityZones: &availabilityZones ${join(",", data.aws_availability_zones.good_zone_ids.names)}
  awsRegion: &awsRegion ${var.aws_region}
  clusterEndpoint: &clusterEndpoint ${try(local.addon_context.aws_eks_cluster_endpoint, null)}
  instanceProfile: &instanceProfile ${try(module.karpenter[0].aws_iam_instance_profile_karpenter_name, null)}
  interruptionQueueName: &interruptionQueueName ${try(module.karpenter[0].eks_blueprints_kubernetes_addons_karpenter_sqs_interruptionQueueName, null)}
  securityGroupSelector: &securityGroupSelector ${join(",", data.aws_security_groups.eks_node_security_groups.ids)}
  subnetsInternal: &subnetsInternal ${join(",", local.private_subnets_with_room)}

karpenter:
  controller:
    env:
      - name: AWS_REGION
        value: *awsRegion

  provisioner:
    enable: true
    availabilityZones: *availabilityZones
    clustername: *clustername
    instanceProfile: *instanceProfile
    securityGroupSelector: *securityGroupSelector
    subnetSelector: *subnetsInternal

  settings:
    aws:
      clusterEndpoint: *clusterEndpoint
      clusterName: *clustername
      defaultInstanceProfile:
      interruptionQueueName: *interruptionQueueName
EOT
}

In the ApplicationSet:

[...]
    spec:
      project: default
      source:
        repoURL: "https://github.com/XXX/deployments.git"
        path: "{{ .path.path }}"
        targetRevision: HEAD
        helm:
          ignoreMissingValueFiles: true
          passCredentials: true
          valueFiles:
            - "/common/values.{{ .path.basename }}.yaml"
            - "./terraform-values.yaml"
            - "./values.yaml"
[...]