Using aws-vault with WSL

[Majsvaffla]

Me and my colleagues recently started using aws-vault and love it 😃 However, as my development environment is in WSL, I wasn't able to use the wincred backend for aws-vault at first. I'm opening this issue to find out whether there are others like me, wanting to use aws-vault from WSL, and to ask how are you using aws-vault with WSL? Would you also like to see some official support for using the wincred backend "natively" with aws-vault in WSL?

Here's what I did to get it to work:
I tried running the aws-vault.exe binary from WSL and that works for all of the sub commands that I've tried but one. It doesn't work for aws-vault exec since the command I wish to exec is not available from the executables point of view (in Windows).

I want to share with you a Python script I wrote together with a colleague. To use it you would put in on your $PATH in WSL along with the aws-vault.exe Windows binary (instead of the real aws-vault Linux binary) and call it as it it were aws-vault. If called with the "exec" sub command it will use aws-vault.exe (with support for wincred backend) to get the environment with AWS credentials and then execute the original command in WSL using that environment. For all other sub commands it will simply forward the sub command and all other arguments to aws-vault.exe.

There are some limitations in how it accepts command line arguments. It works if all options (--backend, --prompt etc.) are put before "exec". It might and probably will crash otherwise.

#! /usr/bin/python3.7

import os
import subprocess
import sys

EXEC = "exec"
DOUBLE_DASH = "--"
AWS_VAULT_EXE = "aws-vault.exe"  # Has to be on $PATH in WSL
CMD_EXE = "cmd.exe"

# Ensure AWS Vault environment variables are shared between WSL and Windows
AWS_VAULT_ENVIRONMENT_VARIABLES = (k for k in os.environ.keys() if k.startswith("AWS_VAULT_"))
WSLENV = os.environ.get("WSLENV")
os.environ["WSLENV"] = ":".join(
    s for s in [WSLENV, *AWS_VAULT_ENVIRONMENT_VARIABLES] if s
)

if EXEC in sys.argv:
    exec_index = sys.argv.index(EXEC)
    double_dash_index = (
        sys.argv.index(DOUBLE_DASH) if DOUBLE_DASH in sys.argv else exec_index + 2
    )

    # Get AWS_* environment variables using the AWS Vault Windows binary
    win_args = sys.argv[exec_index:double_dash_index]
    win_process = subprocess.run(
        [AWS_VAULT_EXE, *win_args, CMD_EXE, "/C", "set", "AWS_"],
        stdout=subprocess.PIPE,
        cwd="/mnt/c",
        encoding="utf8",
    )
    if win_process.returncode != 0:
        exit(win_process.returncode)
    win_env = win_process.stdout.strip()

    # Make a dict out of the environment string
    wsl_env = {}
    for line in win_env.splitlines():
        k, *v = line.split("=")
        wsl_env[k] = "".join(v)

    # Exec command with arguments as is with AWS_* environment variables set
    wsl_args = sys.argv[double_dash_index + 1 :]
    os.execlpe(wsl_args[0], *wsl_args, {**os.environ, **wsl_env})
else:
    # Exec AWS Vault Windows binary with all arguments as is
    os.execlp(AWS_VAULT_EXE, *sys.argv)

[prelegalwonder]

@Majsvaffla in the above scenario, are you using WSL1 or WSL2? I can't get aws-vault.exe to execute within WSL without doing cmd.exe /C <path-to-aws-vault_exe-on-windows> without getting a bad exit code.

I'm limited to WSL1 due to company restricted windows version level.

[Majsvaffla]

@Majsvaffla in the above scenario, are you using WSL1 or WSL2? I can't get aws-vault.exe to execute within WSL without doing cmd.exe /C <path-to-aws-vault_exe-on-windows> without getting a bad exit code.

I'm limited to WSL1 due to company restricted windows version level.

I’ve only used the wrapper in WSL2 but I think it should work in both versions.

You should be able to execute aws-vault.exe just like cmd.exe. You need to put it in /usr/local/bin or some other place that’s on your $PATH in WSL.

[qnm]

Working for me with WSL2, thanks for sharing!

[mtibben]

Thanks @Majsvaffla, feel free to submit this contrib script in a PR

[unfor19]

I'm glad to see that I'm not the only one who had issues with it 😄
This is how I'm using aws-vault in WSL2 and Ubuntu 20.04

Short version

# All the commands are executed in a WSL2 terminal

# Download
AWS_VAULT_VERSION="v6.3.1" && \
wget -O aws-vault "https://github.com/99designs/aws-vault/releases/download/${AWS_VAULT_VERSION}/aws-vault-linux-amd64"

# Install
sudo mv aws-vault /usr/local/bin/ && \
sudo chmod +x /usr/local/bin/aws-vault

# Verify
aws-vault --version

# Output:
# v6.3.1

# Install the pass backend and update gnupg, which encrypts passwords
sudo apt-get update && sudo apt-get install -y pass gnupg

# Make sure your terminal windows is large enough
# Generate a key with gpg (gnupg)
gpg --gen-key
# Follow the prompts ...

# Create a storage key in pass from the previously generated public (pub) key
MY_PUBLIC_KEY="844E426A53A64C2A916CBD1F522014D5FDBF6E3D"
pass init "$MY_PUBLIC_KEY"

# All set, let's test

# Create an aws-vault profile
MY_PROFILE_NAME="staging-admin"
aws-vault add "$MY_PROFILE_NAME"

# Invoke some command with the AWS CLI using the previously created profile
aws-vault exec staging-admin -- aws s3 ls
# outputs a list of buckets if any

Long Version

Expand/Collapse

All the commands are executed in WSL2.

Download and "install" aws-vault

# Download
AWS_VAULT_VERSION="v6.3.1" && \
wget -O aws-vault "https://github.com/99designs/aws-vault/releases/download/${AWS_VAULT_VERSION}/aws-vault-linux-amd64"

# Install
sudo mv aws-vault /usr/local/bin/ && \
sudo chmod +x /usr/local/bin/aws-vault

# Verify
aws-vault --version

# Output:
# v6.3.1

Install the pass backend for aws-vault. This is where we'll store the encrypted AWS credentials. We also need gnupg (gpg), which is the encryption tool that pass uses to encrypt passwords. gpg is shipped with Ubuntu, but it's best to keep it updated, so I added it to the installation process.

sudo apt-get update && sudo apt-get install -y pass gnupg

Create a storage key with gpg for the pass backend; that key is used for encrypting passwords.

IMPORTANT: Make sure your terminal window is large enough; otherwise, you won't be prompted to set a passphrase, and the whole process will fail.

gpg --gen-key
# Follow the prompts ...

Valid output

public and secret key created and signed.

pub   rsa3072 2021-04-22 [SC] [expires: 2023-04-22]
      844E426A53A64C2A916CBD1F522014D5FDBF6E3D
uid                      Meir Gabay <willy@wonka.com>
sub   rsa3072 2021-04-22 [E] [expires: 2023-04-22]

Initialize a "key-store" for aws-vault with pass, and instruct pass to use the previously created public key to encrypt aws-vault credentials.

NOTE: A public key is used for encryption, "anyone" can have it; for decryption, you need a private/secret key—this why it's so important to keep the private key safe.

pass init "844E426A53A64C2A916CBD1F522014D5FDBF6E3D"
# You should be prompted to insert the passphrase that was set during the `gpg --gen-key` process

Valid output

Password store initialized for 844E426A53A64C2A916CBD1F522014D5FDBF6E3D
gpg: checking the trustdb
gpg: marginals needed: 3  completes needed: 1  trust model: pgp
gpg: depth: 0  valid:   3  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 3u
gpg: next trustdb check due at 2023-04-22
staging-admin: reencrypting to 24552E67E0372C6C

Luckily, the default "vaulting backend" for Linux is pass, so we can simply add a profile.

aws-vault add staging-admin
Enter Access Key ID: AKIAABCDEFGH12345678
Enter Secret Access Key: 
Added credentials to profile "staging-admin" in vault

Verify

aws-vault exec staging-admin -- aws s3 ls
# buckets list ...

[warwickgrigg]

Thank you @unfor19 . I needed a just a couple more things to get your solution working for me:

export AWS_VAULT_BACKEND=pass
export GPG_TTY="$( tty )"

which I've also added to my ~/.bashrc

[unfor19]

Thank you @unfor19 . I needed a just a couple more things to get your solution working for me:

export AWS_VAULT_BACKEND=pass
export GPG_TTY="$( tty )"

which I've also added to my ~/.bashrc

Superb!
It's weird that you had to add the AWS_VAULT_BACKEND=pass var, the default backend for vault in Linux is pass. Though if it works, it works 🎉

$ aws-vault --help
usage: aws-vault [<flags>] <command> [<args> ...]

A vault for securely storing and accessing AWS credentials in development environments.

Flags:
  --help                Show context-sensitive help (also try --help-long and --help-man).
  --version             Show application version.
  --debug               Show debugging output
  --backend=pass        Secret backend to use [pass file] ($AWS_VAULT_BACKEND) # <--- pass is default isn't it?

[ijovanovski]

My solution to the problem works nice.
install aws-vault using windows chocolatey in command prompt or power sheel.
Open WSL
create or open .bash_profile if you already have one open with some editors like nano ~/.bash_profile
add file path to aws vault like so:
alias aws-vault="/mnt/c/ProgramData/chocolatey/bin/aws-vault.exe"
save the file before you exit
refresh the file . ~/.bash_profile
thats all let the microsoft credential manager takes care of the rest
you can also check version if it is working by typing in wsl: aws-vault --version

[ispaulle]

I'm glad to see that I'm not the only one who had issues with it 😄 This is how I'm using aws-vault in WSL2 and Ubuntu 20.04

Short version

# All the commands are executed in a WSL2 terminal

# Download
AWS_VAULT_VERSION="v6.3.1" && \
wget -O aws-vault "https://github.com/99designs/aws-vault/releases/download/${AWS_VAULT_VERSION}/aws-vault-linux-amd64"

# Install
sudo mv aws-vault /usr/local/bin/ && \
sudo chmod +x /usr/local/bin/aws-vault

# Verify
aws-vault --version

# Output:
# v6.3.1

# Install the pass backend and update gnupg, which encrypts passwords
sudo apt-get update && sudo apt-get install -y pass gnupg

# Make sure your terminal windows is large enough
# Generate a key with gpg (gnupg)
gpg --gen-key
# Follow the prompts ...

# Create a storage key in pass from the previously generated public (pub) key
MY_PUBLIC_KEY="844E426A53A64C2A916CBD1F522014D5FDBF6E3D"
pass init "$MY_PUBLIC_KEY"

# All set, let's test

# Create an aws-vault profile
MY_PROFILE_NAME="staging-admin"
aws-vault add "$MY_PROFILE_NAME"

# Invoke some command with the AWS CLI using the previously created profile
aws-vault exec staging-admin -- aws s3 ls
# outputs a list of buckets if any

Long Version

Expand/Collapse

Thanks for this. Running WSL2 Ubuntu 20.04 LTS and this works.

[musaabalokaidi]

Has anyone tried this with a YubiKey?

When using the aws-vault exec on WSL with the AWS_VAULT_PROMPT set to ykman I get the following error:

ykman: ykman resolves to executable in current directory (./ykman)

It does look like an issue with ykman itself, but I don't know what's going on in the background so it could be an issue with either aws-vault or ykman

Here's the debug output. What's interesting is that the ykman oath that it seems to fail on does work when I run it directly.

Also, when I try to run aws-vault.exe exec --prompt=ykman profile-name wsl from a PowerShell prompt, I get the following error:
aws-vault: error: exec: exec: "wsl": executable file not found in %PATH%

Any ideas why? I definitely has wsl installed and I can launch it without any issues and it's in the correct path.

[tmzt]

This is not using exec, but it accomplishes the same thing for me.

From within your running bash or similar shell, run:

eval `aws-vault.exe export $USER| awk '{print "export " $1 ""}'`

You can check this using aws sts get-caller-identity.

[gw-bcharboneau]

Just sharing for others that may have this issue, will post a separate bug for it shortly.

Issue 1237: Error when executing any AWS Vault Command on 7.2.0, in WSL Ubuntu on Windows 11

Not working with AWS Vault 7.2.0

I get the following error when trying to use any aws-vault command (example using the add command)

~$ aws-vault add "test"
/usr/local/bin/aws-vault: line 1: --2024-02-01: command not found
/usr/local/bin/aws-vault: line 2: syntax error near unexpected token (' /usr/local/bin/aws-vault: line 2: Resolving github.com (github.com)... 140.82.112.3'