Question/Feature Request: Using Helmfile with Helm Plugins

[osterman]

what

• We'd like to use helm plugins for installing/upgrading charts
• Is there a way we can achieve this today with helmfile?

use-case

• Instead of using a traditional helm chart repository, we'd like to use helmfiles stored on github. This is achievable using the plugin by Gladly (helm-github). It supports the identical interface to helm upgrade, but is invoked by calling helm github upgrade ..., where github refers to this plugin.

suggested implementation

- name: "portal"
  namespace: "monitoring"
  chart: "cloudposse-incubator/portal"
  version: "0.1.0"
  actions:
    # call plugin "github" and use "upgrade" action
    sync: "github upgrade"
    # No need to call plugin for delete
    delete: "delete" 
  values:
  ...

references

• https://github.com/sagansystems/helm-github

[mumoshu]

@osterman Thanks for the great suggestion!

I was also thinking to introduce a general plugin system to hook into helmfile, to unblock users from waiting until helmfile implements every feature that touches only part of helmfile. The part would be, in this case, the command used for installing or upgrading a single helm release.

I try to outline it fitting to your example.

---

Regarding your example, sync is composed of multiple helm commands. I want something other than actions.sync to denote the helm upgrade command used in sync.

Calling each helm command task, I imagine something like the below to support your use-case:

- name: "portal"
  namespace: "monitoring"
  chart: "cloudposse-incubator/portal"
  version: "0.1.0"
  tasks:
    installOrUpgradeRelease:
       # call plugin "github" and use "upgrade" action
       command: ["github", "upgrade"]
       # In case you'd like to customize args...
       # args: ["--additional-flag-1", "--additional-flag-2", {{ join .Tasks.UpgradeRelease.DefaultArgs ", " }}]
  values:
  ...

One of my dream feature on top of this is to add auto-selection of the plugin per release, according to its labels.

Back to your example, I wonder if you end up using an external templating tool to avoid repeating the customization for every release in your helmfile.yaml. If we had a way to reuse the customization as a plugin, you just need label it to incorporate the customization:

plugins:
- helm-github-support

releases:
- name: "portal"
  namespace: "monitoring"
  chart: "cloudposse-incubator/portal"
  version: "0.1.0"
  labels:
    plugin.helmfile.github.io/helm-github-support/enabled: "true"

WDYT on this plan? I'd appreciate any feedback! Also, thanks as always for your support..

[osterman]

plugins:
- helm-github-support

Nice. Makes sense we should be able to install them some way like this. Probably needs to be a list of maps.

I wonder if you end up using an external templating tool to avoid repeating the customization for every release in your helmfile.yaml.

Good point; hadn't thought that far ahead. Would want to avoid relying on yet another layer, but if we must, using something like gomplate we could achieve that what you suggest.

sync is composed of multiple helm commands. I want something other than actions.sync to denote the helm upgrade command used in sync.

Aha, makes sense. So the helm "primitives" used by helmfile maybe need to be overridable. In your example, one of those primitives would be installOrUpgradeRelease.

One of my dream feature on top of this is to add auto-selection of the plugin per release, according to its labels

This feels like it would be a lot more complicated to implement and utilize in the helmfile.

What about something like this:

plugins:
- name: helm-github-support
  source: https://github.com/sagansystems/helm-github
  tasks:
    installOrUpgradeRelease:
      # call plugin "github" and use "upgrade" action
      command: ["github", "upgrade"]
      # In case you'd like to customize args...
      # args: ["--additional-flag-1", "--additional-flag-2", {{ join .Tasks.UpgradeRelease.DefaultArgs ", " }}]

releases:
- name: "portal"
  namespace: "monitoring"
  chart: "cloudposse-incubator/portal"
  plugin: "helm-github-support"
  version: "0.1.0"

[mumoshu]

Probably needs to be a list of maps.

Definitely, for finer-grained configuration!

So the helm "primitives" used by helmfile maybe need to be overridable. In your example, one of those primitives would be installOrUpgradeRelease.

Exactly 👍

This feels like it would be a lot more complicated to implement and utilize in the helmfile.

plugin under releases[], as you've suggested, seems to be nicer in that regard!

One thing I'd like to clarify is that, I differentiate between helm plugins and helmfile plugins.

In the above example, helm-github-support is a helmfile plugin whose primary purpose is to override the primitive task of helmfile installOrUpgradeRelease on demand. The demand is denoted by plugin: "helm-github-support" in this case.

plugins: in the helmfile.yaml should be for helmfile plugins, and therefore source sounds like the source(path or url) of the helmfile plugin, to me. But you assume it an url to the helm plugin helm-github. I want a better notation here.

Considering all the inputs given so far, I propose something like the below.
You'll probably start with 2. Inline helmfile plugin, which seems to suffice your current use-case.

1. External helmfile plugin

This is for reusing helmfile plugins.
External helmfile plugins should be distributed via http, similarly to helm plugins.

plugins:
- name: helm-github-support
  # source can be a path or an url to a helmfile plugin
  source: https://github.com/cloudposse/helmfile-helm-github-support.git

The imaginary helmfile plugin repository cloudposse/helmfile-helm-github-support would have a plugin.yaml for helmfile that looks like:

name: helm-github-support
taskOverrides:
  installOrUpgradeRelease:
    command: ["helm", "github", "upgrade"]
helmPlugins:
# helmfile will check if the helm plugin is already installed. If not, `helm plugin install --version master https://github.com/sagansystems/helm-github.git` will be run automatically by `helmfile`
- name: helm-github
  source: https://github.com/sagansystems/helm-github.git
  version: master

2. Inline helmfile plugin

Inline helmfile plugins are managed inside, and along with, helmfile.yaml.

plugins:
# Equivalent to the content of the plugin.yaml above
- name: helm-github-support
  taskOverrides:
    installOrUpgradeRelease:
      command: ["helm", "github", "upgrade"]
  # ...

[sstarcher]

@osterman for the helm git plugin any reason it can't function similar to the helm-s3 plugin? Helm s3 ties directly in and works transparently. - https://github.com/hypnoglow/helm-s3

[sstarcher]

A plugin systems is going to great increase the complexity of helmfile. I would be worry about trying to be the kitchen sink.

[mumoshu]

@sstarcher Thanks for the comment! I'd like to hear your concerns on complexity more.

My original purpose of the plugin system was preventing the helmfile core from growing limitlessly to support varying use-cases. But I do want to achieve that with minimum change(s) to helmfile.

[mumoshu]

One more alternative to the above two, that seems minimal is to just introduce a feature to override specific command:

releases:
- name: "portal"
  namespace: "monitoring"
  chart: "cloudposse-incubator/portal"
  version: "0.1.0"
  task:
    overrides:
      helmUpgrade:
        command: ["helm", "github", "upgrade"]

This allows us to defer introducing a complex plugin system, but still allow us to address this feature request, and gather more use-cases that can be supported by overriding or hooking into specific phase of helmfile.

For example, it can be extended to:

task:
  overrides:
    helmFetch:
       # fetch the helm chart from your own store, without writing an helm plugin just for it
      command: ["./yourownhelmfetch", "$CHART_NAME", "$CHART_VERSION"]
    helmUpgrade:
       command: ["helm", "github", "ugprade"]
  hooks:
    # Run chartify to generate a helm chart on demand
    preUpgrade:
      command: ["./chartify", "$RELEASE_NAME"]
    # Notify successful deployment to Slack
    postUpgrade:
      command: ["./slacknotify.sh", "$RELEASE_NAME"]

[elemental-lf]

I agree with @sstarcher on the complexity issue. A plugin systems needs to be designed carefully and a lot of things are still changing inside of helmfile. I'd try to figure out the right set and semantics of the core commands first (for example apply was just added). As a comprise a limited pre- and post-hook system will probably be useful. It will probably solve the original use case (with some extra work) and I also have a similar use case where I'm currently using a Makefile to create and package up some charts from local git repositories and putting them into a repository to be consumed by helm via helmfile after that.

[mumoshu]

Thanks for the feedback! Makes sense a lot.

Let's keep discussing about the plugin system in another issues. In short term, solve the original issue plus your issue with something simpler!

Do you think that the hooks feature I've suggested abovd work as the base of further discussion?

[osterman]

@osterman for the helm git plugin any reason it can't function similar to the helm-s3 plugin

@sstarcher good point. It should work that way.

[elemental-lf]

Do you think that the hooks feature I've suggested abovd work as the base of further discussion?

Yes. I just looked at the helm-s3 plugin and helm-github should adapt its integration pattern as Erik and Shane have stated. But for all other cases of "I have my charts in hq, svn or my piggy bank" where no special plugin exists or other special actions are required to get or prepare the chart a preChartLoad hook would still be usefull, I think.

@osterman Have you seen https://github.com/diwakar-s-maurya/helm-git?

[osterman]

@elemental-lf was just looking at that last night as a result of this discussion. Looks interesting.

Commit the files generated by the helm commands and push to the repo. Now the repository is setup.

Only thing I didn't like is we have to remember to commit/update the index.yaml manifest.

[mumoshu]

@osterman Your point about maintaining index.yaml sounds valid to me.

How about implementing a fetch command onto helm-github that is used like helm github fetch --repo git@github.com:kubernetes/charts.git --path stable/external-dns --ref abcdefg, so that it fetches a chart hosted on github and save it as a local chart?

#295 will allow you run it automatically before helmfile run helm upgrade:

- name: external-dns
  chart: ./charts/external-dns
  hooks:
    github:
      phases: ["preChartLoad"]
      command: ["helm", "github", "fetch", "--repo=git@github.com:kubernetes/charts.git", "--output-dir=charts/external-dns"]

[osterman]

Right now, for our use-case my inclination is to just use a compatible plugin, rather than add this complexity to Helmfile per @sstarcher’s recommendation. TBH did not know plugins could work that way in helm.

[nikolajbrinch]

+1 for supporting helm-tiller (Tillerless Helm)

[nikolajbrinch]

It seems that the design og helm-tiller, actually makes this possible, by passing

bash -c 'helmfile ...' 

to

helm tiller run

[johnlinvc]

Hi. Is this thread still alive?
Not the exact same use case, but we want to specify plugins & versions in helmfile , then install them with some commands like helefile plugins.
We'd be happy to contribute this feature if anyone wants this.

Helmfile format (almost same as above, but with versioning):

plugins:
- name: helm-github-support
  # source can be a path or an url to a helmfile plugin
  source: https://github.com/cloudposse/helmfile-helm-github-support.git
  version: ~1.0.2

[mumoshu]

@johnlinvc From a user's perspective, I'd rather prefer helmfile apply to automatically install missing or update outdated plugins. But yeah, I generally agree that this is something worth exploring.

Also, your suggested helmfile format looks good at glance!

[romanlab]

Were there any updates to this?
I'm working on a helm plugin for chart validation and was wondering how i can use it with Helmfile. Another alternative will be to hook into the resulting manifest, is there a hook system I can utilize(something like before_apply)?