Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

helmfile support #2143

Closed
estahn opened this issue Aug 13, 2019 · 44 comments
Closed

helmfile support #2143

estahn opened this issue Aug 13, 2019 · 44 comments
Labels
enhancement New feature or request

Comments

@estahn
Copy link
Contributor

estahn commented Aug 13, 2019

Is your feature request related to a problem? Please describe.
Similar to helm, helmfile support would be great.

Describe the solution you'd like
Support for helmfile.
@estahn estahn added the enhancement New feature or request label Aug 13, 2019
@adamjohnson01
Copy link
Contributor

adamjohnson01 commented Aug 15, 2019 •
edited
Loading

@estahn, we created a helmfile config managment plugin in order to use helmfile

- name: helmfile
  init:
    command: ["sh", "-c"]
    args: ["helm init --client-only && helmfile repos"]
  generate:
    command: ["sh", "-c"]
    args: ["helmfile template --skip-deps"]  # need --skip-deps to avoid "No requirements found in ..." from helm on stdout

You will need to add a volume to the repo-server

volumes:
  - name: custom-tools
    emptyDir: {}

Then add an init container in order to download helmfile.

initContainers
  - name: download-tools
    image: alpine:3.8
    command: [sh, -c]
    args:
    - wget -qO /custom-tools/helmfile https://github.com/roboll/helmfile/releases/download/v0.61.1/helmfile_linux_amd64 &&
      chmod +x /custom-tools/*
    volumeMounts:
    - mountPath: /custom-tools
      name: custom-tools

And finally add a volumeMount to mount the helmfile binary

volumeMounts:
  - name: custom-tools
    mountPath: /usr/local/bin/helmfile
    subPath: helmfile

@estahn
Copy link
Contributor Author

estahn commented Sep 9, 2019

@adamjohnson01 It sounds like you created something helmfile specific, but I assume you're just referencing the custom tools documented here. Correct?

@estahn estahn closed this as completed Sep 10, 2019
@adamjohnson01
Copy link
Contributor

@estahn, yes. That is correct.

@travisghansen
Copy link

I'm interested in this as well. How has your experience been so far @adamjohnson01 and @estahn?

  • Do you use ENV vars (https://argoproj.github.io/argo-cd/user-guide/config-management-plugins/#environment ... specifically defined in the spec) in your helmfile?
  • Have you run into any issues where separate helmfiles use the same repo name but point to differing repos?
  • How do you handle --kube-version (helm v2) or --api-versions (helm v3)?
  • Do custom config management plugins have access in some shape or form to the application data such as cluster endpoint etc (I'd like to be able to run kubectl api-versions for example to alter the command...see above question)?

I would be interested in making this a 'first class' plugin. Anyone else interested in collaborating on that effort?

@estahn
Copy link
Contributor Author

estahn commented Jan 2, 2020 via email

@travisghansen
Copy link

Ok thanks for the feedback.

@travisghansen
Copy link

Out of curiosity what is an Argo app?

@estahn
Copy link
Contributor Author

estahn commented Jan 3, 2020 via email

@travisghansen
Copy link

Ah ok sure. I misunderstood your comment as using some argo-created templating engine. You still need ksonnet/jsonnet/helm/etc as templating/packaging engine. Support for helmfile in argo simply let's you more easily declare multiple helm releases (as it were) to be a single argo app.

In any case I appreciate the feedback. The most recent helm work in argo still doesn't feel quite right to me (I'm pretty new so I could be missing something). Proper helmfile support seems like the ideal solution (for me):

  1. separation of config files from chart(s) (for 2 reasons, not storing config in the chart repo AND makes it easier to consume 3rd-party charts since you don't have to duplicate chart source code)
  2. ability to manage apps via gitops (the new helm support isn't really gitops unfortunately)
  3. ability to deploy more than 1 chart as part of an app as you mentioned above

After running through several edge-cases and toying around with argo generally, I think it's a great approach and tool. I really like what I'm seeing so far, just the helm support isn't sitting quite right yet for my various use-cases. Legit support for helmfile would knock those out real quick while also going well beyond in terms of capabilities.

@estahn
Copy link
Contributor Author

estahn commented Jan 3, 2020 via email

@travisghansen
Copy link

For any future folks stumbling upon this I've created a small project over here starting to integrate helmfile a little more than skin deep: https://github.com/travisghansen/argo-cd-helmfile

It has full support for environments/selectors (including setting them dynamically via ENV vars) and completely specifying a helmfile.yaml as an ENV var.

Anyone wishing to collaborate is welcome :) After it settles a bit I may look into building it out as a first class plugin to the project.

@abdennour
Copy link

awesome community !
I want to give you more... and i want to know if really ArgoCD can replace helmfile or they can work together ?

This is what i achieved/

I let developers use the same helmfile to run their apps locally.

I let .env file is the single source of truth for both:

  • docker-compose : which natively read from .env file
  • helm : which get the .env with --set-file , and all charts are designed to convert the .env format into Secret resource.
  • helmfile : is able to use --set-file with remote files, also i can override some .env and getting the new value from .Envrionment.Values which already fetched from Vault in Ci/CD pipeline.

This is a real example of my approach : https://github.com/abdennour/go-to-do-app

I am with reducing tools @estahn and i agree with you.
However, how can we have this great feature with ArgoCD.
I mean the feature of forcing developers to maintain use the same charts locally with DRY.

@Xtigyro
Copy link

Xtigyro commented Feb 3, 2021

+1

@dudicoco
Copy link

After hours of frustration from messing around with an init container to install helmfile + helm plugin, I've found the following docker image: https://hub.docker.com/r/chatwork/argocd-helmfile

Includes (most likely) everything you need to run helmfile within argocd outside the box!

@travisghansen travisghansen mentioned this issue May 11, 2021
Open
@GolubevV
Copy link

GolubevV commented May 11, 2021 •
edited
Loading

I would also vote for a first-class support of helmfile in ArgoCD.
I can not say that the ArgoCD can completely replace helmfile with App of apps patterns and/or helm umbrella chart.

Helm unbrella chart is very limiting - it does not support templating in values, so if you have 2 charts sharing some piece of configuration like for example DB user login and both server and client charts as dependencies, you will have to specify the very same value 2 times. Another really killing feature of helmfile is is natural support (through vals library) of many secret backends out of the box, so you can just reference the location of the secret and provide required env params (like VAULT_ROLE_ID and VAULT_SECRET_ID) to the ArgoCD app and voila - you have working secret injection:
roboll/helmfile#906

As for using App of App patterns from ArgoCD, problem with this approach is that it requires to deploy and use ArgoCD in every scenario including developer working on his personal laptop.
Helmfile in its turn can be used as kind of DSL, helm chats composing tool (like docker compose) so that developer can first test some combination of charts and their parameters locally against minikube and than just share that repo with Platform team who will plug that repo into ArgoCD.

If to give an analogy the way I see it, you can consider:

  • Helmfile as an Ansible and ArgoCD as Ansible Tower
  • Helmfile as Terraform and ArgoCD as Terraform Enterprise/Cloud.

In such vision, Helmfile first level support becomes a logical next step :)

BTW, Helmfile integration is already added to the tools like Jenkinx-X and Codefresh:

@KarstenSiemer
Copy link
Contributor

KarstenSiemer commented May 11, 2021 •
edited
Loading

I agree with @GolubevV.
We are creating our kubernetes clusters via terraform and automatically create argocd via terraform helm provider with it.
On that created cluster now exists argocd from which our devs deploy their stuff.
Sometimes the devs need access to a s3 bucket or need some kind of secret and the only way to distribute those information is by either having to manually telling them (like the role name their serviceaccount has to assume) or create a sealed secret for them ( annoying ).
Having first class helmfile support we could simply distribute such information via aws ssm, which is very easy and comfortable and by standardizing the ssm paths we do not even have to tell them in which path the devs have to be looking for.

@calohmn calohmn mentioned this issue Aug 16, 2021
Merged
@icy
Copy link

icy commented Aug 10, 2022

@adamjohnson01 It sounds like you created something helmfile specific, but I assume you're just referencing the custom tools documented here. Correct?

May we reopen the ticket? I am not sure "custom_tools" can help to address the problem. The tool https://github.com/travisghansen/argo-cd-helmfile as @travisghansen mentioned is active but I expect something from the mainstream support (aka the first-class support as @GolubevV shared).

@Xtigyro
Copy link

Xtigyro commented Aug 11, 2022

+1

@crenshaw-dev crenshaw-dev reopened this Aug 11, 2022
@crenshaw-dev
Copy link
Member

Another really killing feature of helmfile is is natural support (through vals library) of many secret backends out of the box

I don't love Argo CD directly handling plaintext secrets, because those end up on the repo-server disk, in redis, and (potentially) in Argo CD API responses (e.g. in diffs sent to the UI).

But I'm not super familiar with the other features provided by helmfile, and I don't think plugins are full-featured enough yet to offer support on-par with a native experience. So re-opening for further discussion. :-)

@travisghansen
Copy link

What leads you to believe helmfile usage would result in plain text secrets?

I’m interested in knowing if those who have used my plugin find something lacking in the integration and if so what those things are? If it’s generally considered to be feature complete then we could look at replicating the functionality into core.

@crenshaw-dev
Copy link
Member

What leads you to believe helmfile usage would result in plain text secrets?

It might not. @GolubevV just pointed out that helmfile makes it really easy to integrate with external secrets providers like Vault. That integration would rely on loading some Vault access secret into the repo-server (either by env var or by file) and then using that Vault access to inject secrets into manifests which are stored on the repo-server and in redis and potentially returned by API requests.

So helmfile support doesn't necessitate repo-server handling plaintext secrets. But if that use case is one of the main reasons to use helmfile in Argo CD, adding helmfile support might result in a lot of people following that pattern.

I’m interested in knowing if those who have used my plugin find something lacking in the integration and if so what those things are? If it’s generally considered to be feature complete then we could look at replicating the functionality into core.

I'd be curious about that too! If there are features which are not currently possible in the plugin, I'd be interested in improving Argo CD's plugin functionality set so that the plugin can provide those features.

@corinz
Copy link

corinz commented Oct 4, 2022

+1 Would like to see this

@frizzr
Copy link

frizzr commented Oct 11, 2022

+1 I also would love to see first-class support for helmfile. As @GolubevV said, having parity between using helmfile both with and without ArgoCD goes a long way toward allowing for 'turnkey' installations of large and more complex deployments with the style of management you want....either with or without GitOps.

@mayconritzmann
Copy link

+1 Would like to see this

@lucj
Copy link

lucj commented Nov 2, 2022

As it seems helmfile is not planned to be a first-class thing in argocd, the way to go is to define a plugin using the sidecar approach.
I've started a demo project here: https://github.com/lucj/argocd-helmfile-plugin do not hesitate to follow the readme and provide feedback :)

@Fredouye
Copy link

Fredouye commented Nov 3, 2022 •
edited
Loading

Hi Luc

thanks, working as expected :)

We're really looking into Helmfile support, especially for public Helm charts (without the need to mirror them internally).

I've tested your plugin with a basic Helmfile file :

repositories:
- name: bitnami
  url: https://charts.bitnami.com/bitnami

releases:
- name: apache
  namespace: apache
  chart: bitnami/apache
  version: 9.2.6
  values:
  - service:
      type: "ClusterIP"

I guess official support in Argo CD could bring a way to differentiate Kustomize / Helm / Helmfile in the GUI...

@crenshaw-dev
Copy link
Member

We plan to have somewhat better UI support for plugins in 2.6. :-) #9216

@lucj
Copy link

lucj commented Nov 3, 2022 via email

@lucj
Copy link

lucj commented Nov 3, 2022

Thanks a lot @Fredouye for your feedback.

@icy
Copy link

icy commented Nov 23, 2022

I'm looking for the support too. My case is to execute some pre-hook command before my deployment, which is kinda impossible with argocd (but possible with helmfile).

@lucj
Copy link

lucj commented Nov 23, 2022

@icy could you give this one a try ? https://github.com/lucj/argocd-helmfile-plugin

@icy
Copy link

icy commented Nov 23, 2022

@icy could you give this one a try ? https://github.com/lucj/argocd-helmfile-plugin

@lucj , thanks a lot. I will take a look at your plugin.

@armenr
Copy link

armenr commented Dec 10, 2022 •
edited
Loading

+10 for first-class helmfile support, although after 3 years, I don't have much hope.

I recall seeing somewhere - at some point - a blog that described how to set up helmfile + argocd so that you could actually see the Parameters/Values of your installed charts in the ArgoCD UI.

Does anyone know/remember what that implementation/pattern/work-around looks like?

For example, if you install the metrics-server helm chart, you get this in your UI:
Screenshot 2022-12-10 at 2 05 00 PM

@crenshaw-dev - when you mention better support for parameterized plugins, does the ability to change these values/params also fall into that scope (assuming I run helmfile via a plugin?)

@crenshaw-dev
Copy link
Member

+10 for first-class helmfile support

My hope is that we can make plugin support so good that every plugin is effectively first-class.

does the ability to change these values/params also fall into that scope

Currently, parameters for plugins are read-only in the UI. That's purely because I'm so bad at React that I couldn't get write support to work in the initial PR. But I do plan to add write support. But once that's added, helmfile will have full "Parameters" tab support, just like Helm.

@armenr
Copy link

armenr commented Dec 10, 2022

Sweet. Thank you for clearing that up @crenshaw-dev ! My "internal customers" are devs & QA that are unfamiliar with a lot of the new tooling I'm putting in place.

Even just being able to SEE what values/params are applied is a major value add for them. Thank you again for the hard work on this project.

@crenshaw-dev
Copy link
Member

I put a PR against Luc's plugin to support generating "parameter announcements," which are how plugins will present parameters to the UI starting in 2.6. The parameters for the Argo CD Helm chart are being dynamically generated based on the chart's values.yaml file.

It's still quite ugly, but that's just presentation. It can be improved. The bones are there.

image

@armenr
Copy link

armenr commented Dec 10, 2022

@crenshaw-dev standing ovation

200w

Also, my React & CSS are a bit dated (circa 2017)...I can maybe try to have a look if it's helpful?

@crenshaw-dev
Copy link
Member

Ahhh, vintage React. 👍

If you're up to try! This is the key code:

argo-cd/ui/src/app/applications/components/application-parameters/application-parameters.tsx

Lines 301 to 303 in b4c1281

view: Array.from(entries)
.map(([key, value]) => `${key}='${value}'`)
.join(' '),

A space-delimited key=value list made sense for other map-style params, because there were generally only a few key/value pairs. But for a chart with 100 parameters, it looks pretty bad. I'm not exactly sure what the better display would look like, but presumably some kind of table.

Displaying more nicely is step 1. Step 2 would be making writes work:

argo-cd/ui/src/app/applications/components/application-parameters/application-parameters.tsx

Lines 304 to 307 in b4c1281

edit: () =>
Array.from(entries)
.map(([key, value]) => `${key}='${value}'`)
.join(' ')

That's the part that really stumped me.

@st3fus
Copy link

st3fus commented Jun 27, 2023

["helmfile template --skip-deps"]

@dudicoco @travisghansen

Hey there, could you felllas guide me through setting this up properly? I have k8s cluster on DigitalOcean and I have ArgoCD installed from their marketplace (the one click setup) so I don't have any yaml files related to ArgoCD locally

@travisghansen
Copy link

Install instructions for my plugin are here: https://github.com/travisghansen/argo-cd-helmfile

@st3fus
Copy link

st3fus commented Jun 27, 2023

Install instructions for my plugin are here: https://github.com/travisghansen/argo-cd-helmfile

Yeah, I've checked it out, is there a maybe a more beginner friendly instructions? I'm still kinda fresh to this, trying to figure out what should I edit exactly

@dudicoco
Copy link

["helmfile template --skip-deps"]

@dudicoco @travisghansen

Hey there, could you felllas guide me through setting this up properly? I have k8s cluster on DigitalOcean and I have ArgoCD installed from their marketplace (the one click setup) so I don't have any yaml files related to ArgoCD locally

@st3fus as I wrote before the easiest solution which I've used is the https://hub.docker.com/r/chatwork/argocd-helmfile docker image, not sure if you can do that via the digitalocean marketplace as i'm not familiar with that, you would need to modify the image tag in the manifest.

@christianhuth
Copy link

I have written a blog article explaining how to use Helmfile with ArgoCD's Config Management Plugin feature and the official Helmfile image (https://gcr.io/helmfile/helmfile). Check it out here: https://blog.knell.it/deploying-helm-charts-using-argocd-and-helmfile/

@blakepettersson
Copy link
Member

Closing since there's a path forward with CMPs (see above)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests