Question: How to handle global variables across helmfiles

[AndresPineros]

Is it possible to share global variables between helmfiles in the following manner?

Let's say I have one global variables file called globals.yaml, that has this content:

spring:
  regex: "some value"

dotnet:
  regex: "some other value"

And multiple helmfiles that depending on what we want can grab the spring or the dotnet value (this is what I want to know if is possible):

....
releases:
  - name: myrelease1 
    namespace: default
    chart: mycharts/mychart
    version: 0.1.0
    values:
      - globals.yaml
      - regex: {{ spring.regex }}

and

....
releases:
  - name: myrelease2
    namespace: default
    chart: mycharts/mychart
    version: 0.1.0
    values:
      - globals.yaml
      - regex: {{ dotnet.regex }}

So basically for each release, I can pass the global variable I want to a variable that is already defined in the chart.
Is there a way to do this or simulate this behavior?

[mumoshu]

@AndresPineros Hey! Thanks for trying helmfile.

I've asked about global variables several times so far, and I do understand those and your use-cases are important.

Back to the question, one possible way would be to use an environment variable that points to a kind of "base directory" containing globals.yaml. That is you run this against the top-level helmfile:

$ SOME_BASE_DIR=$(PWD)/values/globals.yaml helmfile apply

Your second sub-helmfile would look like:

....
releases:
  - name: myrelease2
    namespace: default
    chart: mycharts/mychart
    version: 0.1.0
    values:
      - regex: {{ list (requiredEnv "SOME_BASE_DIR") "globals.yaml" | join "" | readFile | fromYaml | getOrNil "dotnet.regex" }}

You'll be interested in #245 or more extremely using an external tool like kapitan 140, for reducing the amount of boilerplate code like this.. 😃

[mumoshu]

More thoughts:

• Using envvars like I've described above makes your helmfile less portable. You need to figure out what envvar is required by running or reading the helmfile
• Adding a convention that globals.yaml is searched upward and recursively won't work when you want multiple globalvars files

[mumoshu]

How about adding a template function findUpward that is used like {{ findUpward "globals.yaml" | readFile }}?

With that you can place globals.yaml at your project root and your helmfile and subhelmfiles can just locate it via findUpward, without having to provide an envvar like SOME_BASE_DIR.

[mumoshu]

@sstarcher @osterman Do you any experience that made you wanting global variables? How do you deal with it?

[sstarcher]

Could this not be solved by environments?

So one issue I have with my current use of globals is that it does not play nice with something like Atlantis. If you change a global it effects all releases not just a local change.

[mumoshu]

Could this not be solved by environments?

Good point! environments helps it. The issue may be how you can easily reference the globals.yaml then. Sub-helmfiles that resides within a deeply nested directory would need:

environments:
  default:
    values:
    - ../../../globals.yaml

My idea is that findUpward simplifies it:

environments:
  default:
    values:
    - {{ findUpward "globals.yaml" }}

[mumoshu]

So one issue I have with my current use of globals is that it does not play nice with something like Atlantis. If you change a global it effects all releases not just a local change.

Similarly to what happens when you use terraform modules and updates to the modules doesn't trigger changes to dependent tf projects?

I was in the impression that atlantis suggests us to use when_modified:

version: 2
projects:
- dir: project1
  autoplan:
    when_modified: ["../modules/**/*.tf", "*.tf*"]

https://www.runatlantis.io/guide/atlantis-yaml-use-cases.html#configuring-autoplanning

[sstarcher]

I don't have that issue due to separating out our terraform modules. I do the following.

• Each module is in it's own github repo and has tests on commit
• A single repo initiates all of those different modules using source to point to the github repo with a specific tag

We don't allow floating things and everything is versioned so that atlantis workaround is not required and people know EXACTLY what is being updated instead of effecting multiple services at once. Which is the exact issue a global introduces where you make a change and it breaks something you know nothing about.

[mumoshu]

Makes sense!

So an equivalent way to manage globals in helmfile would be to version your gloabls.yaml with unique IDs.

A poor man's implementation would look like:

- helmfiles/
  - helmfile.yaml
- globals/
  - globals.v1.yaml
  - globals.v2.yaml

And you reference a single version of globals.yaml(v1 or v2 or whatever) explicitly in your helmfile.yaml, so that people know EXACTLY what's being updated and why.

[sstarcher]

I would lean toward recommending using templating and environments.

• Globals go in the environment
• Use a gotmpl to define it and insert it into your resource

[mumoshu]

@AndresPineros Would you be interested in #388 (comment)?

[mumoshu]

Since #587, you can use bases to import layers of state file that may contain environment values to be reused.

Creating and importing a shared layer from many state files will give you the similar end result as global variables.

Also, environment variables can be used as global variables, as before.

I think this can be closed as resolved. WDY?

[mumoshu]

Please feel free to reopen if necessary 😃

[sgandon]

To me the environment section is there to specify environment specific values and not global values to be reused by any templating part.
I also always felt that global values where missing even if there is always a way with environment variable. Being able to set those global values in a helmfile make some sense to me and with the possibility to override them from the command line.
One way of representing this could be around environment like a "common" environment that would provide global environment values in order to reuse the environment builtin object. But usually I don't like solutions that are tuned to ease the dev and would more be in favor of a global built in object though.

[mumoshu]

@sgandon Thanks! Would you mind clarifying a bit more on this:

To me the environment section is there to specify environment specific values and not global values to be reused by any templating part.
I also always felt that global values where missing even if there is always a way with environment variable. Being able to set those global values in a helmfile make some sense to me and with the possibility to override them from the command line.

Mind giving me examples on what would you use globals and environment values for?

If you don't use environnment specific values for any templating(I read your comment so), what would you use for?

[mumoshu]

@sgandon Probably this isn't what you're trying to say, but anyway - I wondered if we can add something like .Args that is passed via helmfile --set key1=val2 --set key2=val2 that works almost like Environment.Values, but for templating helmfile state files.

The use of .Environment.Values to template state files is currently possible thanks to "double rendering" (#308), but it is getting harder and harder for me to maintain and design how it should work in cases like #523 and #587. So my idea is to use .Args solely for templating and communication between state files(including helmfiles:, and possiblybases:).

.Args should be usable for tempalting state files, so that you can use .Args to conditionally load environment values. The env values are not used for templating state files, but instead rendering releases and values...

Would the .Args in my above idea matches the globals you imagine?

[mumoshu]

cc/ @davidovich

[mumoshu]

@davidovich I'm considering to deprecate double rendering in favor of the new layering (#587) and globals. And I wanted to discuss with you as the original author of the double rendering feature 😃 WDYT?

[davidovich]

I can reply tomorrow (it's near bedtime in Montreal) and I will think about it.

…

On Tue, May 28, 2019 at 9:32 PM KUOKA Yusuke ***@***.***> wrote: @davidovich <https://github.com/davidovich> I'm considering to deprecate double rendering in favor of the new layering (#587 <#587>) and globals. And I wanted to discuss with you as the original author of the double rendering feature 😃 WDYT? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#398?email_source=notifications&email_token=AAK7U4T6KXFBKOLN66EZPITPXXMK5A5CNFSM4GDKOK2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWN4Y3Y#issuecomment-496749679>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAK7U4SGZEV2I3VRPQISA7DPXXMK5ANCNFSM4GDKOK2A> .

[sgandon]

I used to have a use case for global values that I worked around with environment variables cause we have build a tool around helmfile that can create those environment variables.
If you implement something like the .Args you are mentionning I would really invite you to consider ways to put those args in the helmfile themselfs and use the command line to overide them, just as the helm tool does. Why not propose a .Values or equivalent with a special section in the helmfile for those ?
Also some times ago I needed to have some 'default' environment values that may be overriden in other environment but I also work around it. I am not talking about the existing default environment but more a set of environment values applied to all environments and that may be overriden. This was in fact my initial use case for some kind of "global" values.

[mumoshu]

@sgandon Thanks for the response!

I would use either (1) environment "values" and explicit inheritance (#523) OR (2) .Args proposed above even for the use-case you've illustrated.

Regarding .Args, I'm not going to reinventing environment "variables". If you need global variables, I feel like it is better done with envvars, as there's a bunch of existing tools supports it.

.Args can be renamed to .Values. Introducing dedicated sections for configuring .Valueswithin ahelmfile.yaml` sounds good. But it won't propagate automatically from the parent to the nested helmfiles.

Even so, would you use the proposed .Values(a.k.a .Args)?

[mumoshu]

To clarify - "explicit inheritance" will be enabled via valuesInherited: true I've proposed in #523

[sgandon]

@mumoshu
The more I think about it the more I beleive environment value can be enough for a lot a use cases.
What is needed to my opinion is a way to specify common environment values that may be overriden by specific environments.
And the commandline set of values would just set/override those common environment values.

something like

environments:
  values:    <- common env values
    - foo: bar
  default:
    values:
      - foo: barbar
  otherenv:
    values:
      - foo: kiki

and the command line would allow of course to specify the env to use but also use a --set to specify specific env key values.
This way we do not introduce other builtin object like .Args.
What do you think ?

[mumoshu]

@sgandon Ah, that's definitely an interesting idea!

So we need:

1. Default (Environment) Values (environments.values in the above example. May be equivalent .Values. Not sure how exactly this should be implemented/exposed in the config yet)
2. Environnt-Specific Overrides (environments.NAME.values where NAME is default or otherenv in the above example
3. Runtime Overrides (feat: Allow environment values to be overrode when referring sub-helmfile #523 for via config, or via command-line)

Your use-case needs 1 and 2. And overall we need 1, 2 and 3 to possibly serve everyone's use-case, right?

[sgandon]

hummm, not sure to understand you proposal.

1. I don't use the term "Default" environment values cause there is already an environment called
   "Default", this is why I used "common". So yes we would need a common section somehow.
   I know that what I proposed may be hard to parse so maybe something like the fixed "default" env we could have "common" as fixed value, like :

environments:
common:
   values:
   - foo: bar

I really would appreciate my initial proposal if possible and not create another fixed env definition.
2. there I am not following you, to me this just describes a way of specifing the environment values, the way the are used is exactly the same way as today using .Environment.Values.foo.
The mechanisme describe above is just for specifing those values in a flexible way but they would be used as they are today.
3. yes overrides via command line or parent helmfile.

[mumoshu]

@sgandon I think I'm following you.

I'm in the impression that just changing helmfile to treat envionments.default.values to be the "common" values would solve all our issues?

Let's say environment.default.values in helmfile is something similar to values.yaml contained in a helm chart. When we run helm install --set foo.bar=baz, helm overries foo.bar in values.yaml with baz. My idea is to change helmfile to treat environment.default.values.yaml like helm chart's values.yaml.

[mumoshu]

So what's I'm proposing is making {{ .Environment.Values }} produced by environment.default.values + environment.NAME.values + overrides(via command-line or parent state file), where NAME is the environment name specified via helmfile -e NAME, which of course default to default.

[sgandon]

I absolutly agree with this proposal, this is indeed what I was expecting when I first go to work with helmfile. So this feels very natural although it break the current behaviour.
This would be fantastic a proposal.

[mumoshu]

@sgandon Thanks as always for your patience and support! Great we could agree on the thing ☺️

So this feels very natural although it break the current behaviour.

Yeah this was a concern for me as well. I have two options right now.

(1) Go head with changing the behavior, believing it won't break anything in real use-cases.

You usually don't have extra keys in default env i.e. you have fewer keys for default while other envs like production has more keys that overrides/adds default.

(2) Think big and redesign environment values. This has good side-effects on other issues like #361 while not breaking existing behavior at all. #361 (comment)

[sgandon]

looking that you #361 comment, this is great ! solution 2 is much more natural and what everyone is waiting for (I believe :) ).

[ldemailly]

To avoid duplication I'm looking for #define equivalent of some constant eg something like

globals:
   thanos-version: v0.20.1

then in 2 different release sections

releases:
  - name: prometheus
    values:
          sidecarContainers:
            - name: thanos-sidecar
              image: quay.io/thanos/thanos:{{.globals.thanos-version}}
 - name: thanos
   version: {{.globals.thanos-version}}

And this irrespective of environment (ie not duplicated unless it needs to be different; a real default value)

ps: this might be more an faq/doc question than a request but this issue is where "helmfile shared variables" googling lands

[solarmosaic-kflorence]

@ldemailly for the above I was able to use a Go template variable:

{{$chartsBasePath := env "CHARTS_BASE_PATH" | default "../../../libraries/helm"}}
releases:
  - name: "my-chart"
    chart: "{{$chartsBasePath}}/my-chart"
  - name: "my-other-chart"
    chart: "{{$chartsBasePath}}/my-other-chart"