-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Vault integration #597
Comments
Thanks for opening an issue. This is something we're thinking about. "Vault integration" is a large and complex topic. Rather than considering how Vault will integrate with Nomad as a whole, it's easier to think how we can use Vault to implement specific features in Nomad. For example, we want to provide secure communication between agents, provide authentication and authorization for agents and users, store secrets, etc. Each of these use cases needs a different design and implementation, and Vault may help us implement these features. If you can share your use cases that will help us design and prioritize those particular features. |
I want to be able to deliver the following to my apps:
|
@pires It would be nice to understand the UX around each of these and how they relate to Nomad? For eample, what is stopping an application running on Nomad to not directly use Vault to get credentials of an external service? Or if the Nomad client gets it, how does it hands it over to the process running as a Nomad Task? |
@diptanu, there is less friction in adoption if you just assume people are not going to be using directly Vault or Consul, for instance. So, providing mechanisms to leverage those technologies outside of the applications themselves is a good way of speeding up adoption. A good example is consul-template. However, for people that like to make configurations part of the application container, packaging it as part of a release, consul-template is not very helpful unless they are willing to load their configurations from external volumes. A much better scenario in terms of portability and reliability, in my opinion, is being able to pass Vault values from Nomad into tasks, through environment variables. The implementation could be something similar to what Terraform does with the Interpolation syntax, supporting a |
I'm thinking in something like this: task "registry" {
driver = "docker"
env {
LOG_LEVEL="debug"
REGISTRY_STORAGE="s3"
REGISTRY_STORAGE_S3_ACCESSKEY="${vault("/aws/keys/access")}"
REGISTRY_STORAGE_S3_SECRETKEY="${vault("/aws/keys/secret")}"
REGISTRY_STORAGE_S3_REGION="us-east-1"
REGISTRY_STORAGE_S3_BUCKET="mybucket"
REGISTRY_STORAGE_S3_ROOTDIRECTORY="/foo/bar/docker-registry"
}
config {
image = "registry:2"
network_mode = "host"
}
resources {
cpu = 500 # Mhz
memory = 128 # MB
network {
mbits = 10
port "http" {}
}
} |
Exactly what @c4milo is proposing! |
yes, +1 to that! |
+1 too, we're gonna need it for production. |
It's great!!! |
+1 If this secret was granted on a lease, would nomad restart the container when the credentials expired? |
+1 for @c4milo's use case! |
While @c4milo's proposed syntax is easy and convenient, it is also incredibly insecure. A simple |
True, the example is intended to show how the built-in function could be used. Passing secrets through environment variables is not safe if a person different to you has access to your host. |
It wasn't clear to me that's obvious, judging from all the |
I'm a novice in all of this, so it's very possible that I'm off base. Please correct me if I am wrong. However, is it too much to ask for the job to be responsible for asking Vault for the secrets itself? For instance, if Vault is being hosted inside a cluster alongside Nomad, I imagine it should be possible for the application to have logic to obtain the secrets that Vault contains. |
If someone can interact with the docker API / socket, your entire box and all it's data are owned. Nothing can be done about it. The biggest use case of this is for container orchestration. Moving a container from Dev to Qa to Prod you want different credentials everywhere, and baking them into the application is not prudent at all. If you offload the problem to a hypothetical Nomad+vault integration, the orchestration becomes much more powerful. Even better, the same job specification can be used in dev/qa/prod rather than evolving by re-writing the credentials per environment. The point about pushing integration down into the app itself is a far higher burden, as that effort needs to be duplicated in any and all apps which are deployed. |
I've been thinking about this for a while, and my best case use would be something like:
With this setup nothing is ever saved on disk or in the container since the passed ENV var is one time use and the app is still able to configure itself via vault. You still need to have your application call vault on boot, but it's really no different than any other method. You could also use something like a Config Interface, where you could create different versions depending on where you were running (VaultConfg, EnvVarConfig, FileConfig, etc) Just a note, I'm not a current user of Nomad, but this workflow is what I'll need in a scheduler to use it for production and I think it's pretty doable. |
I like your thinking @jgeiger. One challenge is supporting Vault's rich concept for leases and dynamic secrets. Expanding on your suggestion:
The one-time-use token exposed in the env (e.g. |
This would be really cool with the token wrapping functionality that was introduced in Vault 0.6.0: https://www.hashicorp.com/blog/vault-0.6.html With token wrapping, nomad won't be able to see the real token and will not need to manage the cubbyhole itself:
|
I like @F21 's notion, only concern is what happens if/when you need to restart an existing container, would that require modifying the job def and update when you just need to bounce a running container? |
Hey this is coming in 0.5 and a lot of it is in master already |
I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues. |
Are there any plans to integrate Vault with Nomad? It seems it would be very useful to be able to query secrets and pass them as ENV variables to the different tasks.
The text was updated successfully, but these errors were encountered: