Is a manifest based application to upload and download secrets from Hashicorp-Vault. Therefore, the manifest file is promoted as "source-of-authority" over secrets that a given application may consume, where engineers can use the manifest to upload secrets, and later on use it as configuration input for downloading secrets on applications' behalf.
Once a set of secrets is uploaded to Vault, you can copy
them over to Kubernetes, and keep secrets
in sync by running copy
command at later times.
You can employ vault-handler
as a Kubernetes init-container in order to
download secrets, and have it as a command-line application to upload.
To install the command-line interface via go get
use:
go get -u github.com/otaviof/vault-handler/cmd/vault-handler
Docker images can be bound at Docker-Hub. And for example, to show help message:
docker run --interactive --tty otaviof/vault-handler:latest --help
Downloading data from Vault would be, for instance:
docker run otaviof/vault-handler:latest download
Consider exporting configuration as environment variables to change the behavior of vault-handler
.
This application is using Vault's API client to interact with the service, therefore, you can export environment variables to tweak its behavior.
You can choose either AppRole type of authentication, which will request a runtime token, or directly use a existing token. Both ways are possible, but AppRole would be the recommended method.
All the options in command-line can be set via environment variables. The convention of environment
variable names to add a prefix, VAULT_HANDLER
, and the option name in capitals, replacing dashes
(-
) by underscore (_
). For instance, --vault-addr
would become VAULT_HANDLER_VAULT_ADDR
in environment.
Additionally, command-line arguments overwrite what's informed via environment variables.
The command-line interface is organized as follows:
vault-handler [command] [arguments] [manifest-files]
Where as command
you can use upload
or download
. Please check --help
output in command-line
to see all possible arguments.
A upload
command example:
vault-handler upload --input-dir /var/tmp --dry-run /path/to/manifest.yaml
And then to download
, use for instance:
vault-handler download --output-dir /tmp --dry-run /path/to/manifest.yaml
Afterwards you can copy
secrets to Kubernetes:
vault-handler copy --namespace default --dry-run /path/to/manifest.yaml
The following snippet is a manifest example, the actual secrets can be found here.
---
secrets:
ingress:
path: secret/data/kube/tls
type: kubernetes.io/tls
data:
- name: tls.crt
extension: secret
- name: tls.key
extension: secret
Description of the options used in manifest:
secrets
: root of the manifest;name
: arbitrary group "name". This group-name is also employed to name final files;name.path
: path in Vault. When using V2 key-value store, you may need to inform/secret/data
, while in V1 API it would be directly/secret
;name.type
: Kubernetes secret type, used bycopy
sub-command;name.data.name
: file name;name.data.extension
: file extension;name.data.zip
: file contents is GZIP, needs to be compressed/decompressed;name.data.nameAsSubPath
: employ name as final part of the Vault pathname.path
;name.data.key
: employ a alternative key name on Vault;name.data.fromEnv
: instead of reading file payload from file-system, you can use a environment variable instead. The value informed for this option is the environment variable to be read;
On downloading files from Vault, the following name convention applies:
${GROUP_NAME}.${FILE_NAME}.${EXTENSION}
Therefore, if you consider the example manifest, it would produce a file named name.foo.txt
in
the output directory, defined as command line parameter.
In order to build and test vault-hander
you will need the following:
- Docker-Compose: to run Vault in the background;
- GNU/Make: to run building and testing commands;
- Dep: to manage
vendor
folder (usemake dep
to install); - Vault-CLI: to apply initial configuration to Vault;
Before running tests, you will need to spin up Vault in the background, and apply initial configuration to enable AppRole authentication, and secrets K/V store.
Additionally you need a Kubernetes cluster available, please consider minikube project. During CI, KinD project is used.
docker-compose -d # run vault in development mode
.ci/bootstrap-vault.sh # bootstrap instance
source .env # import environment variables to running shell
And then you can:
make bootstrap # populate vendor
make # build application
make build-docker # build docker image
make test # run unit-tests
make integration # run integration-tests
Vault related bootstrap is needed only once, since data is kept over .data
directory, so when you
turn on development Vault instance again, it's not mandatory anymore to boostrap.
To clean it up, you can:
docker-compose down --volumes
rm -rf ./data