SSM (or generic secrets plugin) support with Environment Variables #745
Comments
|
@acaire Hey! Thanks a lot for the detailed proposal. I'm considering to enhance helmfile for this use-case but still figuring out how it should be done. Would you mind reading my comment here #387 (comment) as the context - what if we added a mix of (1)a pluggable values provider interface and (2)a new configuration syntax that fits nicely with (1)? The imaginary config would look like: plugins:
# This will normally load the executable binary `helmfile-ssm` in `$PATH` that serves the helmfile plugin. Implementation-wise we'd use hashicorp/go-plugin.
# The plugin developer would implement the helmfile-ssm to advertise that it provides the `ssm` values provider that can be accessed with a `ssm` key under `releases[].valuesFrom`
#
# Perhaps we provide two flavors of helmfile binaries, one includes all the "standard" plugins like helmfile-ssm for easier onboarding experience, and a vanilla helmfile without any embedded plugins for people who doesn't use plugins and want to cut binary size
- name: ssm
releases:
- name: {{ . }}`
chart: my-chart
valuesFrom:
- ssm:
# All the values for the keys under `/helm/envname/appname/` will be fetched and imported as a flat hash of kvs
path: /helm/{{MY_ENV}}/{{$appName}}/And the new syntax Thoughts? |
|
@mumoshu thanks for the quick response! I'm curious about the comment: will be fetched and imported as a flat hash of kvs - Will this be imported directly into I'd like to delve further into our current use of helmfile - Here's a templated values file for a postgresql chart (this is a custom fork of the chart, but the basic concept remains): and the related helmfile.yaml component: Now if I wanted to set Or would I have to store my values in SSM like this: I'd welcome any implementation to take away some of the sticky tape we've put around secrets management, but something that simply injects env vars to a helm run, seems far simpler than putting too much work into a plugin system, at least if we were going to maintain a fork of helmfile 😄 If we ever change our secrets provider, or logic - we don't have to change any of our values templates, just how the values are fetched. |
|
@acaire Hey!
As the proposed However looking into your Perhaps your use-case should be addressed in terms of helmfile values? What if we enhanced the syntax to: Where your And in SSM you have the following keys set to corresponding values:
WDYT?
This seems to help your goal, in an another way. |
|
Ahh that clears it up perfectly, thanks! I'm guessing from your example we'd be able to do this per release, so this would be our final syntax: With this So My only concern is terminology - |
No, it isn't how things work!
|
|
Sorry maybe I'm not explaining correctly. Basically, you have a list of releases under I was just suggesting that re-using the term |
|
|
|
And my hidden plan is deprecating is an equivalent of: |
|
ok, cool so the example you gave earlier: Populates As a more simplistic example, would this invalidate itself? |
Still trying to understand - What do you imagine to happen when it isn't invalidated? |
|
With a given SSM path specified via i.e. But you're saying that this is also passed verbatim into the helm install? So if I didn't have a And if I did have a You'd pass helm |
For the top level For |
|
ok thanks for clearing that up - It will mean we'll have to be careful about the naming choice of our SSM keys (so as not to cause clashes as describe above with strings/nested dicts), but otherwise that implementation will work for us 👍 |
|
@acaire Thanks for confirming! Yes. Naming SSM keys will be important. I wanted to note one thing though - If we could enhance the syntax to specify where the KVs should be imported, you'll less likely encounter clashes. Suppose we added All the KVs under |
|
Great thanks! Yes that would work, assuming that Helm continues to ignore unused values data instead of erroring on it, which i'd hope is a pretty safe bet. Perhaps |
|
Yes, I bet so too 😃 Also, I like |
|
Thanks! Let me know if there's anything I can do to assist :) |
|
@mumoshu I like the proposals above. Here's another twist on it: plugins:
- type: ssm
region: us-east-1
path: /{{requiredEnv "env"}}/us-east-1/svc-1/
releases:
- name:
chart: my-chart
values:
- db:
user: {{requiredEnv "DB_USER"}}
pass: {{requiredEnv "DB_PASS"}}When these exist in ssm:
(My ❤️goes out to you @mumoshu. I'm guessing this is going to be a pain to implement since you have to render Edit: Just to be more explicit, |
|
This is very interesting
|
|
@aegershman Hey! Thanks for chiming-in.
Sure. Here it is: plugins:
# This will normally load the executable binary `helmfile-ssm` in `$PATH` that serves the helmfile plugin. Implementation-wise we'd use hashicorp/go-plugin.
# The plugin developer would implement the helmfile-ssm to advertise that it provides the `ssm` values provider that can be accessed with a `ssm` key under `releases[].valuesFrom`
#
# Perhaps we provide two flavors of helmfile binaries, one includes all the "standard" plugins like helmfile-ssm for easier onboarding experience, and a vanilla helmfile without any embedded plugins for people who doesn't use plugins and want to cut binary size
- name: ssm
valuesFrom:
- ssm:
# All the values for the keys under `/helm/envname/appname/` will be fetched and imported as a flat hash of kvs
path: /helm/{{MY_ENV}}/{{$appName}}/
releases:
- name: myapp`
chart: my-chart
valuesFrom:
- inline:
foo: FOO
- file:
path: values/values.yaml
- templateFile:
path: values/values-file.yaml.gotmpl
valuesFrom:
# Instead of implicitly passing helmfile's Values to the values-file.yaml.gotmpl
# Overrides it with `releaseName: myapp`
- inlineTemplate:
releaseName: `{{.Release.Name}}` #<- you'll need {{` `}} to defer rendering until the evaluation time of inlineTemplate. otherwise this fails due to that .Release.Name isn't available at loading-time of your helmfile.yaml :)
- sops:
# Path to the YAML file encrypted with sops or helm-secrets
path: secrets/secrets.yaml
That sounds like a good idea :) |
|
@rms1000watt Thanks!
Understood. My idea was to make - name: ssmis equivalent to: - type: ssm
name: ssmI like the distinction between type and name because it allows you to have different "plugin instances" with the same type but diversified configurations: plugins:
- name: ssmUsEast1
type: ssm
region: us-east-1
path: /{{requiredEnv "env"}}/us-east-1/
- name: ssmUsWest2
type: ssm
region: us-west-2
path: /{{requiredEnv "env"}}/us-west-2/
releases:
- name: us-east-1-svc-1
chart: my-chart
valuesFrom:
- ssmUsEast1:
path: svc-1
- name: us-west-2-svc-1
chart: my-chart
valuesFrom:
- ssmUsWest2:
path: svc-1I'd prefer plugins:
- name: ssmUsEast1
type: ssm
region: us-east-1
pathPrefix: /{{requiredEnv "env"}}/us-east-1/
- name: ssmUsWest2
type: ssm
region: us-west-2
pathPrefix: /{{requiredEnv "env"}}/us-west-2/ |
|
@acaire Can we consider an another syntax that plays nicely with the What we have so far is: # Import kv pairs
valuesFrom:
- ssm:
targetKey: foo.bar # Any other name better than `under`??
path: /helm/prod/postgres2/But I want to suggest this as the alternative: # Import kv pairs
setValues:
- key: foo.bar
valuesFrom:
- ssm:
path: /helm/prod/postgres2/This also help smoothly extending the syntax for setting single kv: # Import a kv pair
setValues:
- key: foo.bar
valueFrom:
- ssm:
path: /helm/prod/postgres2/barI'm not satisfied about that How about using # Import kv pairs
setValues:
- keyPrefix: foo.bar
keysAndValuesFrom:
- ssm:
path: /helm/prod/postgres2/
# Import a kv pair
setValues:
- key: foo.bar
valueFrom:
- ssm:
path: /helm/prod/postgres2/bar |
|
@mumoshu Happy with either implementation, but I do like Also, Would there be a mechanism for recursion? (And while I think of it, caching previously fetched credentials across a helmfile execution?) I'd imagine some people would have a path like: With: would they be available as |
Interesting idea. Perhaps you felt "value" in
I think so, too! What if we extend the existing # Import kv pairs
set:
- key: foo.bar
valuesFrom:
- ssm:
path: /helm/prod/postgres2/
# Import a kv pair
set:
- key: foo.bar
valueFrom:
- ssm:
path: /helm/prod/postgres2/barvs |
I had not thought about that yet.
Good point! Then maybe we'd use GetParametersByPath API and we'd better not recurse but import the path as-is. For: Helmfile will expose It will be accessed like:
Probably it will be addressed in #444 |
|
@mumoshu the other thing to consider is where to draw the line. You could consider the only function of these plugins is to fill the environment. Then all usage of the plugins within the helmfile is just And blame it on 12-factor apps.. "Environment variables or nothing!" lololol jk 🤣 |
|
@rms1000watt The fundamental problem I see there is that helmfile shouldn't mutate envvars. Otherwise multiple concurrent releases when Helmfile can possibly be enhanced to internally translate To me the point of 12-factor apps is that you should be able to use envvars to set/override any settings at runtime. For that I believe you can already wire-up things to honor that today, writing a template expression like But from your previous comments I didn't think it, honoring 12-facor apps rules, is your actual goal. Wouldn't your goal be no need to rewrite go templates and values.yaml files each time you switch the backend(SSM, values files, templates)? If so, that's one of goals of the |
|
Related: #760 (comment) |
|
JFYI, this is the backend of this feature I've developed externally: https://github.com/mumoshu/values It has support for:
|
|
@mumoshu Is it something you need help integrating with |
|
@rms1000watt Yup, we need to integrate it into Helmfile 😃 |
|
@mumoshu Hi! been following these threads for a while. |
|
@rms1000watt @acaire @vladfr I'm super excited about that SSM and also AWS Secrets Manager are finally supported via #906! Would you mind testing it out? We've ended up with different syntax than what we've discussed in this thread. But it turned out to provide more flexible configuration especially when combined with go templates. Also, in near future, that syntax will allow us to keep references to secrets as-is, and defer the secrets retrieval to the time of |
I'd love to have the functionality of #662 but I wanted to throw my use-case in here while it's topical as what i'm suggesting is probably closer to #705.
We're deploying a bunch of apps, but one particular app has a helm vars file which expects over 150 config values (environment variables):
As we have a bunch of different apps across a range of languages and platforms, for us at least, it is best to leave the secret fetching and templating to helm/helmfile.
With the PR as it is, individual SSM fetches would likely be slow and prevent us from overriding values with ENV vars (not to mention our paths different between environments), and we'd likely go back to longer deploys and AWS rate limiting. It would be more useful for us to be able to inject entire SSM paths (i.e.
/helm/$app/*) as environment variables for a helm run instead of what we're doing now which is using chamber. Chamber is great but means we have to iterate ourhelmfile.yamlanyway, read in the list of apps and whether or not they are to be installed, and then execute helmfile for each and every one.It'd be great to have something like the following:
Subsequently run with:
In this example, the following keys:
Would be exported as:
Using env vars might simplify additional plugin development instead of using a custom resolver for each, as it makes it hard to template out the resolver.
The text was updated successfully, but these errors were encountered: