pinentry-ncurses in async process mangles inputs for the rest of the session

[CosmicToast]

General information

• Pure version: git (2017-11-13)
• ZSH version: 5.4.2
• Terminal program & version:
  • putty
  • conemu -> wsl
  • alacritty (etc)
• Operating system: NixOS
• ZSH framework: Toasty Zsh

I have:

• Tested with another terminal program and can reproduce the issue.
• Followed the Integration instructions for my framework (none exist but I can write them if required).

Problem description

If pure ends up invoking pinentry-ncurses (e.g through ssh-agent) in an async task (e.g checking git@... repository's status), input becomes mangled on both sides.

Reproduction steps

(note: not tested with a "normal" ssh-agent, nor outside nixos)

1. SSH into your system (for it to be autodetected).
2. Start gpg-agent with ssh-agent emulation, and default pinentry (ssh sessions are autodetected, and use pinentry-ncurses). (without your key currently being cached/open)
3. Enter an ssh+git repository (git@...).

Expected behavior:
pinentry-ncurses pops up, you enter the password and continue on your merry way

Actual behavior:
pinentry-ncurses pops up (in the async process), and mangles inputs (I haven't looked into it very deeply, but it seems like half your inputs go to asynced-pinentry and half go into the shell).

My .zshrc:

autoload -U promptinit; promptinit
prompt pure
export GPG_TTY=$(tty)
export SSH_AUTH_SOCK=$(gpgconf --list-dirs agent-ssh-socket)
gpg-connect-agent updatestartuptty /bye > /dev/null

---

IssueHunt Summary

sheevizi has been rewarded.

Backers (Total: $210.00)

• issuehunt ($60.00)
• chipsenkbeil ($100.00)
• antoineco ($50.00)

Submitted pull Requests

• #645 Make sure GPG-agent does not mess up the tty

---

Tips

• Checkout the Issuehunt explorer to discover more funded issues.
• Need some help from other developers? Add your repositories on IssueHunt to raise funds.

[DanielJonesEB]

@5paceToast Did you find a solution to this issue? I'm having similar issues with Git Radar doing a fetch in the background, and my terminal then getting mangled. Finding the pinentry process and killing it helps, but it'd be nice to know if there's a way this can be avoided in the first place.

[mafredri]

It’s known that ncurses apps mess with the async worker and there’s not much to do about that (except prevent them from running).

Currently Pure makes no effort to figure out if any kind of pinentry method is used but we try to disable it via GIT_TERMINAL_PROMPT=0.

How does pinentry behave? Is it a one time prompt or is it prompted every time? If it’s the former we might be able to request it before handing work over to the async worker (this would block your prompt every time you entered a new git repo without pinentry active).

[CosmicToast]

Pinentry is a dumb "input password here" program - it doesn't make sense to have pure invoke it itself. It's invoked by gpg-agent instead, and the frequency of its invocation depends on its configuration (various *ttl* options).

When using gpg-agent as your ssh-agent, you always use pinentry.
Gpg-agent has a single configuration option (pinentry-program) that lets one set which pinentry program is used. In my case, it uses pinentry-gtk2...
However it seems that gpg-agent detects an ssh connection existing and sets itself to use pinentry-curses instead (based on the behavior on my machine).

It seems like the feature that ends up requiring a key to be present is the check on whether or not there's stuff to push/pull (with everything else working fine).
Maybe that can be put behind a double-guard? (e.g if GPG_TTY and any of the SSH_* environment variables are set (SSH_TTY, SSH_CLIENT, etc), disable that one feature?)

[SevereOverfl0w]

It could be possible to have pinentry bail by setting PINENTRY_USER_DATA to something like "pure_async_fetching" for async fetches, and allowing a user to specify a custom pinentry which will bail dependent on the value of that var. See https://unix.stackexchange.com/a/236747

I have a similar problem in that I keep being prompted to decrypt my hardware token, which is in my bag, just for look at a directory.

I can't see any way to ask gnupg/pinentry to bail if it has to prompt for anything. It would be nice to retain the fetching behaviour if my hardware token is plugged in and decrypted.

[davidtwco]

I've had this issue as well when using gpg-agent as my ssh agent. I've switched back to ssh-agent for now.

[mafredri]

Does applying the patch from #373 (comment) help with this issue at all?

[kazuoteramoto]

#373 (comment) did not help. I still get a pinentry prompt if the key is not cached on gpg-agent.

(And just to make sure, if I comment the git fetch line I don't have the input mangling problem)

[mafredri]

Thanks for reporting @kazuoteramoto, could you try the next patch that always forces batch mode: #373 (comment)?

[davidtwco]

@mafredri sadly, this isn't working either. Trying with #392, when I browse to a repository that has a ssh remote, pinentry is opened when pure does the background check and that starts mangling input.

[mafredri]

😞

@davidtwco could you try setting:

export PINENTRY_USER_DATA="USE_CURSES=0"

In prompt_pure_async_git_fetch?

[davidtwco]

@mafredri This doesn't seem to be doing it either.

[mafredri]

Alright, well, I'd need a reproduction setup to be able to debug this further; grasping at straws here :/.

[davidtwco]

@mafredri I don't know how helpful this is but davidtwco/dotfiles@6c77ea0 is the commit in my dotfiles where I reverted back to ssh-agent, davidtwco/dotfiles@5927527 is where I silenced a startup message and davidtwco/dotfiles@52d31da is when I initially switched - between those, you'd be able to piece together a working configuration to use gpg-agent as a ssh agent. Not sure how practical that is.

[mafredri]

Thanks, I'll try to have a look. We could try one last straw though:

export GPG_TTY=

Again, inside prompt_pure_async_git_fetch, any change?

[davidtwco]

@mafredri that doesn't seem to do it either.

[mafredri]

Alright, and it's pinentry prompting for PW, not SSH?

[davidtwco]

@mafredri Here's the current diff I've got on top of #392.

diff --git a/pure.zsh b/pure.zsh
index e2f600c..b435090 100644
--- a/pure.zsh
+++ b/pure.zsh
@@ -270,6 +270,8 @@ prompt_pure_async_git_fetch() {
        export GIT_TERMINAL_PROMPT=0
        # set ssh BachMode to disable all interactive ssh password prompting
        export GIT_SSH_COMMAND="${GIT_SSH_COMMAND:-"ssh"} -o BatchMode=yes"
+       export PINENTRY_USER_DATA="USE_CURSES=0"
+       export GPG_TTY=

        command git -c gc.auto=0 fetch &>/dev/null || return 99

Here's a asciinema recording that shows me navigating into a git repository with the gpg agent being used, then after running a command the pinentry shows up and immediately disappears (normally it doesn't, that must be an artefact of the recording, but after I type a few characters it disappears when not recording) - after that, every key I type shows up as a random character or a *.

[mafredri]

Thanks @davidtwco, I can confirm this behavior. I managed to get pinentry to pop up a few times, but it no longer does and I can't get it to show up again. I don't really understand gpg well enough to figure out how to reproduce this (anymore), and I'm not interested in learning the tools.

If I am to debug this issue further I'll need someone to provide me info on how to set up a simple config / environment that reproduces the issue 100% of the time.

[davidtwco]

@mafredri I can't think of anything much at the moment, could it be you need to change the gpg agent configuration to not cache the credentials and prompt every time?

[mafredri]

No idea what it might be; I'm not sure even if/how anything should be configured for pinentry prompts. I've tried setting default-cache-ttl 0 to no avail. Pretty much giving up unless I get a sure-fire way to do this 😄.

[davidtwco]

@mafredri I believe there's a separate ssh cache setting. The one you mentioned is for gpg keys in general.

[mafredri]

I've tried setting the separate SSH cache settings as well, I'm pretty sure I've tried every applicable setting, but I can't get it working. I've also wasted more time on just trying to get pinentry to pop up than I'm comfortable with, so:

The only way I'll dig deeper into this is if someone can provide me with a 100% repro, for example a .tar.gz that I can extract into a users home directory and get going. Or a list of commands that will just work™️.

For now, this issue is on ice unless someone, other than me, wants to investigate further.

[CosmicToast]

I'll try and make a .tar.gz like that (or a container image or something) once I have some free time. Goal will be 100% reproducibility on `cd` into git.

…

On Mon, May 7, 2018 at 1:36 PM Mathias Fredriksson ***@***.***> wrote: I've tried setting the separate SSH cache settings as well, I'm pretty sure I've tried every applicable setting, but I can't get it working. I've also wasted more time on just trying to get pinentry to pop up than I'm comfortable with, so: The only way I'll dig deeper into this is if someone can provide me with a 100% repro, for example a .tar.gz that I can extract into a users home directory and get going. Or a list of commands that will *just work*™️. For now, this issue is on ice unless someone, other than me, wants to investigate further. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#366 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFB113MEq0Y7V7XE75Ap3LnUs2vcR5Yqks5twIYCgaJpZM4QhTlC> .

[mafredri]

That's awesome @5paceToast, thanks! 😄

[CosmicToast]

Try with this: https://send.firefox.com/download/b420bc8429/#6vMQerOUXNnJEu9cDcpwwg
(it's a .txz archive)

Usage instructions:

1. unpack (as root - it's a rootfs)
2. cd alpine
3. systemd-nspawn /bin/login -f root
4. See README inside (tl;dr start sshd, use tmux to avoid dropping/destroying session)

(note: I don't really have a preferred filesharing service, and github doesn't do tarballs; I set the "delete after X downloads" to 20, so ideally I won't have to figure out a different place to put it).

[mafredri]

Thanks btw @5paceToast for the repo, it worked great.

I've been scratching my head with this for a while, and haven't really found any solution to the issue.

Here's some of my findings (might have forgot something):

• We really can't utilize PINENTRY_USER_DATA, this is read by gpg-agent (e.g. on launch) and as such, is static, for all intents and purposes
• Short of killing the zpty, I have found no way for pinentry to return to the main TTY
  • A zpty is basically a clone of the running shell, with a few updated FDs and the like, for some reason this causes confusion for programs writing to to a certain TTY.
  • Might be worth posting on zsh-users ML about this as it could perhaps be improved in the zpty implementation
• It doesn't seem to help to initialize gpg after Pure has created the zpty (occasionally it works when using gpg-connect-agent updatestartuptty /bye, but then it stops working)
• We could theoretically swap out the pinentry program with a custom script and every time Pure does a fetch, we temporarily update gpg-agent with a new value for PINENTRY_USER_DATA which turns the pinentry into a no-op
  • This is not a good solution, probably racy, might interfere with the users regular workflow

I've recently started using GPG and YubiKeys for SSH, so I'm ever more motivated to fix this now. Ideally I would love for Pure to be able to show a 🔐-icon when a user is prompted (e.g. graphical pinentry, or waiting for touch on YubiKey) but I really don't see how this is possible.

If anyone has experience tweaking and hacking around with gpg-agent/pinentry, I would love to hear some ideas on how we:

1. Can redirect/force the interaction back to the original TTY (e.g. not the zpty)
2. Detect that gpg-agent is waiting on pin / card reader while we're performing the git pull.

[CosmicToast]

Here are a couple of alternative ideas:

1. Check for ssh origin before doing anything else. If ssh is detected, try to silently (but not asynchronously) log into it. That way, if there is to be a prompt, it'll show up in the foreground.
2. Since this is an issue with gpg specifically (at least as far as I can tell), we can iterate over gpg keys and see if all of them are in a "cached" state. If they are not in a cached state, cache them in the foreground.

Both of these have issues though.
The problem with 1 is it's not scalable - if this issue arises with anything other than git, the solution has to be made from scratch.
The problem with 2 is it introduces a race condition (imagine a key that IS cached, but whose cache expires in 1 millisecond).

It might make sense to look into 1. to see if it's actually feasible with the current codebase?

[mafredri]

Thanks for the ideas, one big problem with 1. is that, according to my testing, the zpty instance will mess with pinentry as long as it's running no matter if we are authenticating asynchronously or not. It could even be a zpty instance from another tab (depending on where gpg decides to show pinentry) that catches the auth and then the pinentry term kind of shows in the terminal, but input goes partially to zpty and partially not, and it all becomes a garbled mess. The only way I found to avoid this is to make sure the zpty is closed (no async worker).

But regarding that, we could also check for some indications from ssh (ssh -vvv) that the user will be prompted (either pinentry or card reader) when ssh pauses at debug3: sign_and_send_pubkey.

Example output:

debug1: Offering public key: RSA SHA256:/... cardno:XXXXXXXX
debug3: send_pubkey_test
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 60
debug1: Server accepts key: pkalg ssh-rsa blen 535
debug2: input_userauth_pk_ok: fp SHA256:/...
debug3: sign_and_send_pubkey: RSA SHA256:/... (pauses here)

Regarding 2., is there a way for us to check when the cached key will expire?

[CosmicToast]

Regarding 1, "zpty is closed (no async worker)." is what I meant by foreground; e.g run the check in a hook, and if the check succeeds, then launch the worker (and thus zpty).

Regarding 2, my understanding is that gpg-connect-agent only recentlyish gained the capability to even report whether a key is cached or not (2.1.18?). You nead the CACHEID (full keygrip) and you can send KEYINFO --no-ask $KEYGRIP Err Pmt Des to gpg-connect-agent.

My situation is quite the opposite of yours (I dropped using smartcards for gpg-ssh-auth, at least for now, because I couldn't find one that fulfills all of my requirements and supports ed25519), so I can't verify the above (very easily, at least).

I could fetch the archive of that rootfs I made, and start examining things there, but I'm quite busy right now (and will remain such for the forseeable future), though, so it may be faster to experiment on your end, where it's already all set up.

[IssueHuntBot]

@IssueHunt has funded $60.00 to this issue.

---

• Submit pull request via IssueHunt to receive this reward.
• Want to contribute? Chip in to this issue via IssueHunt.
• Checkout the IssueHunt Issue Explorer to see more funded issues.
• Need help from developers? Add your repository on IssueHunt to raise funds.

[davidtwco]

@mafredri have you seen this blog post? - I've recently revisited using gpg-agent for SSH (w/ a Yubikey) and have been running into this again.

[davidtwco]

I've experimented a little with the solution from the previously linked post. In my experience, I've found that when a Yubikey is connected for the first time then it needs to be unlocked, and the command from the blog post will tell you that. Once unlocked (by running a GPG command that requires the key), the command from the blog post will tell you that it is unlocked. However, I've found that despite being unlocked, the key will need unlocked again when used as a SSH key for the first time, and that the command will not inform you of that.

[chipsenkbeil]

I'm hitting this as well, which really sucks as I have to switch to bash to handle pinentry-curses or pinentry-tty. It begins and some of the characters I type in the prompt show up as * while others are the underlying characters. No matter how much I delete and try to get a clean password in, it fails the prompt. If I suspend it, then some or all of the characters I type in zsh become * and it becomes unusable (can't even Ctrl+D out).

My setup is an iPad using blink shell to connect to a remote server using mosh (SSH has same problem). The remote server is running the following:

1. Operating System: ArchLinux running kernel 5.7.9-arch1-1
2. zsh: 5.8
3. zplug: 2.4.2
4. pure: 3a5355b - July 8th, 2020
5. gpg-agent: 2.2.21
6. pinentry-curses: 1.1.0 (also tried pinentry-tty 1.1.0)
7. tmux: 3.1b
8. git: 2.27.0
9. TERM environment variable: screen-256color

Let me add to the fund.

[issuehunt-oss]

@chipsenkbeil has funded $100.00 to this issue.

---

• Submit pull request via IssueHunt to receive this reward.
• Want to contribute? Chip in to this issue via IssueHunt.
• Checkout the IssueHunt Issue Explorer to see more funded issues.
• Need help from developers? Add your repository on IssueHunt to raise funds.

[antoineco]

I recently switched to gpg-agent as my SSH agent to be able to store my private key on a Yubikey, and I'm facing this issue as well (Pure 1.20.1, GnuPG 2.3.7, pinentry-curses 1.20).

My workaround is to establish a connection to the SSH server — in the foreground — prior to changing directory to the Git repository in my shell. Example, for a repository hosted on GitHub:

ssh -T git@github.com

This allows me to keep the key unlocked for the duration the GPG agent's cache is active, and prevent the pinentry dialog from being triggered by Pure.

For this to be viable, you probably have to set the GPG agent's cache TTL high.

[issuehunt-oss]

@antoineco has funded $50.00 to this issue.

---

• Submit pull request via IssueHunt to receive this reward.
• Want to contribute? Chip in to this issue via IssueHunt.
• Checkout the IssueHunt Issue Explorer to see more funded issues.
• Need help from developers? Add your repository on IssueHunt to raise funds.

[sheeviZi]

Please test #645. Hopefully it gets merged.

[issuehunt-oss]

@sindresorhus has rewarded $189.00 to @sheevizi. See it on IssueHunt

• 💰 Total deposit: $210.00
• 🎉 Repository reward(0%): $0.00
• 🔧 Service fee(10%): $21.00