-
Notifications
You must be signed in to change notification settings - Fork 4.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Vault CLI doesn't work to store binary data #1423
Comments
@jaxley It's up to the caller to encode binary data as base64 before sending it in via the CLI, just like it's up to API users to base64 their data encoded in their POST calls. |
That's not at all clear from the tool or the contract or documentation. Since you can specify a file for storage in the CLI tool, you would expect as a user that the client using the API would properly encode and decode the data so it does not corrupt it. I can see that using the API directly this would be the case, but for someone using the CLI, I would think that responsibility belongs in the CLI tool using the API? In fact, the documentation for the CLI tool (vault read -h) seems incorrect in saying: -field=field If included, the raw value of the specified field This wording implies to me that this is a way to read unadulterated ("raw") data out of Vault. |
Any given path in Vault may be accessed by any client. If the CLI client transparently encoded/decoded, it would mean different behavior between the API and the CLI. The CLI is purely an API client so we don't do that.
It means that the data will not be formatted into a table, which is the normal output method. Regardless, it is reading the raw data of the field, but it's still the caller's responsibility to ensure that it's in the right format before passing it in to the API. |
I can see the concern about the vault CLI introducing some "special sauce" that direct API clients and SDKs would not expect nor know how to handle. However, if the CLI is storing bits in Vault that it cannot properly extract with fidelity, then I see that as a bug in the CLI that it is not properly handling data or encoding so that the input will match the output. I'm not suggesting that the CLI should do anything different than any other client of the REST API - it should properly encode the data so it can be faithfully reproduced when retrieved! That would most likely just be Base64 encoding that would be expected by an API caller to be able to decode the data. Or perhaps you could extend the API to include metadata (optional) to communicate encoding information to the caller so they don't have to guess or hard-code that knowledge. BTW, the wording in the documentation could be improved along the lines of your follow-up comment. "raw" does not imply what you said, which is much clearer, "data will not be formatted into a table" |
I have a question about vault in hashicorp.. In vault, Write file and read it, then compare byte code, Do you know why file is changed? $ vault write secret/catalina/files/key-dat2 value=@key.dat Is there workaround? |
Thanks, I solved my issue :) |
Because storing binary is not supported hashicorp/vault#1423
For people trying to inject a binary file from a vault secret into a kubernetes pod:
As a rule of thumb, and in order to avoid any unexpected problem, you should encode every file you put into a vault secret. I've been struggling with this simple thing for far too long. I had to compile knowledge from stackoverflow, github, consul documentation and vault documentation. |
Is there any recommendation for how to handle a binary secret file stored in vault when the external secrets operator is used to sync that vault entry to a kubernetes secret? |
@shuston try with the isBinary: true option in external secret definition |
@jalonsoa sorry, I don't see that option in https://external-secrets.io/v0.6.0-rc1/spec/#external-secrets.io/v1beta1.ExternalSecretDataFromRemoteRef Sorry if I'm missing something. |
Running vault in dev mode
Then, write a binary file into vault:
$ vault write secret/burp/cert certificate=@/Users/jaxley/Downloads/burp-certificate.crt
And immediately read it back out:
$ vault read -field=certificate secret/burp/cert > ~/Downloads/burp-from-vault.crt
However, the file is munged and does not equal the original:
Not sure if the issue is how the file is represented on the way in or out, but here is a sample of what the JSON looks like when reading via the API:
And a diff of the bytes:
I saw some older issues about binary data that seemed to indicate this was fixed, such as:
#205
#583
But it doesn't seem like the CLI properly handles this. Does the API? I presume the CLI is supposedly encoding the binary data inside the JSON request that callers directly using the API would have to mimic.
The text was updated successfully, but these errors were encountered: