A tool which observes the secrets present in your Hashicorp Vault store and trigger events defined by you whenever those secrets are updated.
-
You are saving secrets in Hashicorp Vault.
-
You or your application servers or some other piece of code of yours makes use of those secrets to work.
-
Updating the secrets is a hassle for you.
- Everytime you update the secret, you have to tell your application that the secret changed by restarting it.
- Or you manually end up triggering some job/script to propagate the fact that your secret changed.
- Or at times, you just wanna tell/convey your team or some other team that someone/something changed secrets in your vault store over Slack.
-
Hashicorp vault doesn't produce any events or have any webhook support with which you can plugin some business logic handling the events of secrets changes.
-
So what to do? You use vaultie-talkie
Vaultie-talk is an event recorder and webhook simulator for changes happening in your Hashicorp Vault store.
You can configure vaultie-talkie to do the following:
- Trigger a webhook you own whenever a secret is changed in your vault store. Vaultie-talkie will send the details of both the old and new state of secrets (before and after the change respectively) as a part of the request sent to the webhook.
- Update a file, say a json file or .env file, tracking the vault secrets in, say, your application servers.
- Or just execute a command whenever the vault secret changes. Just before executing the command, vaultie-talkie would save the details of old and new secret in a temporary file so that the command which runs could assume to reach out to that temporary file, expect and get the old and new secret details, and accordingly take any action it intends to. Now, this command could be literally anything. It can be a command which mails you "Hello, your secret changed."
This will install the latest release under your $GOBIN
.
go install github.com/yashvardhan-kukreja/vaultie-talkie@latest
Make sure that your GOBIN is a part of you $PATH as well so as to be able to use this CLI, else run this:
export PATH="$GOBIN:$PATH"
vaultie-talkie expects a mandatory set of arguments no matter how you use it.
And then, depending on the "target" type you choose, it would expect some other extra arguments.
When I say "target", I mean the action you expect vaultie-talkie to take whenever it observes a secret change i.e. one of the following:
- Trigger a webhook on secret change?
- Update a file with the new secret contents?
- Execute a command on secret change?
So, here we go:
argument | value type | default | explanation | |
---|---|---|---|---|
-vault-host |
string | "" | Host (without the protocol) at which your vault store is running. For example, "23.45.67.89" | |
-vault-port |
int | 8200 | Port at which your vault store is running. For example, 8200 | |
-vault-path |
string | "" | Vault path of the secret you want vaultie-talkie to watch and respond to. For example, "foo/bar" | |
-vault-access-token |
string | "" | Vault token which has at least read privileges to the above vault path | |
-target-type |
string (allowed values: "webhook" / "file" / "command") | "" | Type of action which vaultie-talkie would take when the secret contents at -vault-path change. | |
-failure-limit |
int | 3 | Amount of failures/errors the poller should be allowed bear in a row. Once the this number is reached, vaultie-talkie would exit. Until then, it's going to just log the errors and retry. For setting no/infinite failure limits, feed the value -1. | |
-polling-interval |
time | 5s | Rate at which the vault store gets polled for watching its contents | |
-debug |
bool | false | Run vaultie-talkie in debug mode. Would log extra logs in the console where vaultie-talkie would be running. |
argument | value type | default | explanation | |
---|---|---|---|---|
-webhook-url |
string | "" | Webhook URL against which a POST request would be triggered whenever the secret contents under -vault-path changes. | |
-webhook-access-token |
string | "" | Authorization Bearer token to be used by vaultie-talkie while making requests against the Webhook URL. |
argument | value type | default | explanation | |
---|---|---|---|---|
-target-file-path |
string | "" | Path to the file where the secret contents would be saved by vaultie-talkie. | |
-target-file-format |
string (allowed values: "json" / "env") | "json" | Format in which the contents of the new secret would be written to the target file. Currently, supported format are 'json' and 'env' |
argument | value type | default | explanation | |
---|---|---|---|---|
-target-command |
string | "" | Command to execute by vaultie-talkie whenever the secret contents under -vault-path are changed. | |
-intermediate-file-for-changed-keystore |
string | "/tmp/vaultie-talkie/keystore.json" | At this file, the old and new secret contents will be written in the JSON format: {'old_key_store': , 'new_key_store': }. Whenever your target command executes, it can assume that the contents of the old and new secret would be present in this intermediate path and accordingly, use it. |
In case you got bored reading the above instructions, here are some direct examples to aid you in understanding the usage of vaultie-talkie intuitively.
I have a vault store running at http://myvault.yash.com
and under it, I have a secret at the path applications/ecommerce
with the following contents.
{
"secret1": "value1"
}
(The token "mysecrettoken" allows read access to this secret)
I want to receive a mail whenever the contents under the above secret are changed.
To accomplish this, I have written a webhook endpoint at http://mywebhook.yash.com/webhook
which I would expect to get triggered whenever the above secret is changed with the details/info associated with the changed secret.
So, I'll start vaultie-talkie with the following command
vaultie-talkie \
-vault-host=myvault.yash.com -vault-path=applications/ecommerce -vault-access-token=mysecrettoken -polling-interval=3s \
\
-target-type=webhook \
\
-webhook-url=http://mywebhook.yash.com/webhook
And I start this. Now, what's gonna happen is the following:
- vaultie-talkie will look for Vault at
http://myvault.yash.com
- It will query the secret present under
applications/ecommerce
every 3 seconds. - Whenever it would encounter a change in the above secret, it would make the following request:
POST http://mywebhook.yash.com/webhook
Request Body:
{
"old_key_store": {
"secret1": "value1"
},
"new_key_store": {
"secret1": "newvalue1",
"secret2": "value2"
}
}
And as I own the webhook, I can program it underneath to parse such incoming request and take any action like sending me emails whenever it is triggered.
I have a vault store running at http://myvault.yash.com
and under it, I have a secret at the path applications/ecommerce
with the following contents.
{
"secret1": "value1"
}
(The token "mysecrettoken" allows read access to this secret)
Now, my use-case is interesting. I am running an application somewhere which relies on a config.json
to run. The config.json is meant to resemble the above secret in vault identically.
Which means whenever the secret contents in the above vault path are changed, I want the config.json
to be updated resembling the new secret contents and my application is restarted.
To accomplish the above task, I have written a python script on the same host as my application. This python script, when executed, would update the config.json assuming the latest secret contents present at /tmp/latest-vault-contents
and then, restart the server. And the command python /home/path/config-updater.py
would run the above python script.
So, this is how I'll use the vaultie-talkie to achieve my use-case.
vaultie-talkie \
-vault-host=myvault.yash.com -vault-path=applications/ecommerce -vault-access-token=mysecrettoken \
\
-target-type=command \
\
-target-command="python /home/path/config-updater.py" -intermediate-file-for-changed-keystore "/tmp/latest-vault-contents"
And I start this. Now, what's gonna happen is the following:
-
vaultie-talkie will look for Vault at
http://myvault.yash.com
-
It will query the secret present under
applications/ecommerce
every 5 seconds (default). -
Whenever it would encounter a change in the above secret, it would do two things:
- Update the intermediate file
/tmp/latest-vault-contents
with the following stuff:
{ "old_key_store": { "secret1": "value1" }, "new_key_store": { "secret1": "newvalue1", "secret2": "value2" } }
- Execute the command
python /home/path/config-updater.py
- Update the intermediate file
And I'll be writing the python script /home/path/config-updater.py
like this
import json
import os
new_config_json_content = {}
## read the new contents of the secret from the intermediate file
with open("/tmp/latest-vault-contents") as f:
vault_change = json.load(f)
new_config_json_content = vault_change["new_key_store"]
## update config.json with the new secret
with open('config.json', 'w') as f:
json.dump(new_config_json_content, f)
## restart my application using config.json
os.system("restart-my-application")
This project has a great scope of contributing to it in terms adding new kinds of targets to it. The one which come to my head at the moment are mail targets or slack targets ensuring that whenever a secret is changed a mail or a slack messaged is broadcasted respectively.
For making any contributions, please raise an issue associated with it and feel free to work on its PR once that issue is acknowledged by the maintainers (currently, me :P)
While working on the PR, please ensure to install pre-commit
and set it up with respect to the project locally by running
pre-commit install
Moreover, you can ensure the resilience of your changes beforehand by running the existing tests via
make test
- Define cooler targets like slack, mail, etc.
- Improve the docs (obviously) :3
- Make this project compatible with watching other storage backends like Hashicorp Consul KV (long shot goal).