Support for loading shell completion scripts

[jcpetruzza]

It would be nice to be able to load new {bash/zsh/etc}-completions from an .envrc script. A typical use case would be when use nix provisions some commands that you don't have globally installed (or are for different versions so their command-line args differ).

This could work for example like this:

• The stdlib provides a new command, say, add_completions to register completion scripts inside .envrc. E.g.

add_completions bash /path/to/foo/share/bash-completions/completions/
add_completions zsh /path/to/foo/share/zsh/site-functions/

• For each supported shell, their direnv export action emits, along with the export/unset commands, code to add (and remove) the registered completions.

[1oglop1]

I was exactly looking for this feature. For example I use awscli which requires complete -C aws_completer aws but I do not want to have enabled for the system because my awscli is installed in specific virtualenv

[jcpetruzza]

It seems to me that the most general way to get this working (as well as other features requiring shell-specific commands) would by implementing something like what was suggested in #73 (comment).

Essentially, this would allow people to write bash functions to be ran in .envrc that can add the required shell-specific code to the direnv_postload/direnv_preunload variables based on the setting of SHELL. That way, direnv remains simple and the heavy lifting is moved to "libraries".

@zimbatm Would you take a patch implementing something along that lines?

[zimbatm]

As long as the implementation includes proper tests. direnv is missing too many tests at the moment that I feel confident shipping new features. The DIRENV_DIFF format needs to be changed to accommodate for the diffing of more things than environment variables as well.

[jcpetruzza]

If we can get something like shell_specific in #464 merged in, then one can implement this as follows.

• add_completions bash dir emits bash-specific code that:

  1. Runs complete and captures the output. This gives us all existing completions before loading the env
  2. Runs source on all files in dir
  3. Runs complete again and diffs the output with that of step 1 (there will only be additions and modifications).
  4. Registers an ON_UNLOAD action that for each modified entry xxx, runs complete -r xxx if it was a new entry (deleting the entry), or the definition from step 1 if was a redefinition.

• add_completions zsh emits zsh-specific code that:

  1. Gets all currents completions with something like

     for k in ${(k)_comps:#-*(-|-,*)}; do
       printf "%q;%q" $k ${_comps[$k]}
     done

  2. Adds dir to the rpath array
  3. Runs compinit so that all new completion defs get indexed
  4. Gets the current completions again and diffs the output with that of step 1
  5. Registers an ON_UNLOAD action that:
     1. Runs compdef -d xxx on every command xxx that appeared in the diff of step 4.
     2. Runs compinit -D to force a reindex of completions (otherwise, definitions that were added when loading the env may persist).

We actually want add_completions to accept more than one directory, for performance reasons, but the idea is the same.

[jcpetruzza]

And one can then make use_nix look at the diff of the PATH before and after instantiating the nix derivation to find the path to the bin directory of the loaded packages, and use that to find the directories holding the bash/zsh completions for the packages and, if found, pass them to the add_completions command.

[maxrothman]

Any movement on this? I'd love to have this feature, it's the one thing keeping me from using direnv currently. At work we have python-based CLIs in our repos, and being able to auto-activate their virtualenvs and add their shell completions when cd'ing into a repo without permanently polluting the environment would make using the CLIs much, much easier.

So all that to say "bump" I guess 😄

[Profpatsch]

Yeah, this would be fantastic to have.

[PierreR]

Yeah, it is quite annoying to have not completion at all with direnv zsh shell ... Somehow you need to install everything with nix-env to get completion in a straightforward manner which defeats the whole direnv philosophy.

[zimbatm]

One open question is; even if direnv supported this, do shells support unmapping completion functions?

[jraygauthier]

@zimbatm: How about an opt in post hook to allow dynamic completion to be reloaded from XDG_DATA_DIRS ?

In nix, I usually have the following shell hook:

  shellHook = ''
    # Bring xdg data dirs of dependencies and current program into the
    # environement. This will allow us to get shell completion if any
    # and there might be other benefits as well.
    xdg_inputs=( "''${buildInputs[@]}" )
    for p in "''${xdg_inputs[@]}"; do
      if [[ -d "$p/share" ]]; then
        XDG_DATA_DIRS="''${XDG_DATA_DIRS}''${XDG_DATA_DIRS+:}$p/share"
      fi
    done
    export XDG_DATA_DIRS

    # Make sure we support the pure case as well as non nixos cases
    # where dynamic bash completions were not sourced.
    if ! type _completion_loader > /dev/null; then
      . ${bash-completion}/etc/profile.d/bash_completion.sh
    fi
  '';

Which unfortunately does not work well through direnv + use nix.

With the opt-in, once the new env is ready, the following post hook would be systematically run, effectively reloading the completion from XDG_DATA_DIRS:

. ${bash-completion}/etc/profile.d/bash_completion.sh

Would it unregister existing completion, I'm unsure.

EDIT: Please disregard, the above nix shell hook already work fine with direnv + use nix without any post hook.

[zimbatm]

Basically, to do this the direnv way(TM) with proper backup&restore support, bash would have to send all its local variables and functions to direnv so it can diff and re-export properly. But that's not all because bash can also spawn sub-shells where that information would be lost (unlike environment variables which are inherited). So to make this work in a coherent manner direnv would also need to be able to detect and handle that as well.

[flokli]

This feels like a lot of work. I'd personally be fine having some escape hatch in the meantime, that I could use to register completions for example.

I don't really want to use use nix and its shellHook thing, because it exports a lot of environment variables I don't want to set while entering such an environment.

[zimbatm]

So far, the best option is to assume that the user has the bash-completion package installed. And then use direnv to set $XDG_DATA_DIRS.

That's what I added to numtide/devshell#48 recently and it works quite well.

The only downside is that completions don't get unloaded when exiting an environment.

[jonringer]

This has been solved for bash on unstable with NixOS/nixpkgs#103501. Packages just need to be added to nativeBuildInputs for their /share directories to be added to $XDG_DATA_DIRS.

NixOS/nixpkgs#104225 is another alternative, I may re-open it if there's interest, as it would potentially work with zsh and fish as well.

Another alternative would be to make zsh and fish completions also load XDG_DATA_DIRS lazily.

[zimbatm]

One thing I noticed is that the bash-completion package loads a shim whenever a completion is not found. Once the shim is installed, it won't try to look for another completion.

https://github.com/scop/bash-completion/blob/89ff88fc34c881dc05ffb536b9ff1226d3a086c0/bash_completion#L2312-L2315

So if a user goes to the project, direnv doesn't load for some reason (eg: the .envrc needs to be allowed), and then type the command<tab> then the completion is gone for command for the session.

[adrian-gierakowski]

NixOS/nixpkgs#104225 is another alternative, I may re-open it if there's interest, as it would potentially work with zsh and fish as well.

Another alternative would be to make zsh and fish completions also load XDG_DATA_DIRS lazily.

I’m interested in fish completions. Anything I could do to help?

[zimbatm]

yes, talk to upstream so they automatically load completions in $XDG_DATA_DIRS. You can point them to this issue.

[jtojnar]

See fish-shell/fish-shell#8261

[adrian-gierakowski]

@jtojnar do I read it correctly: they claim loading/unloading is already possible by calling some fish functions so they are unwilling to implement a solution based on XDG_DATA_DIRS?

[p-hash]

I am looking for a way to add bash completions without using nix.

I use asdf to install several CLI tools and asdf allows me to manage them in the same directory: .tool_versions file has list of tools to be installed for this particular project. So, it would be nice to have their completions added to current shell. Since I am already using direnv, it is an obvious choice, but unfortunately, direnv does not support it.

Is there any workaround or alternative?

[pfgray]

Another alternative would be to make zsh and fish completions also load XDG_DATA_DIRS lazily.

FWIW, I made https://github.com/pfgray/fish-completion-sync, a fish plugin which has the same effect as bash-completion for fish.

It works by listening on changes to $XDG_DATA_DIRS (which direnv already automatically changes), and syncs it with $fish_complete_path (which fish does use dynamically).

It's not perfect, but it's been working well for me so far

[carschandler]

My issue with this is that when loading an activation script like the one that micromamba shell hook -s bash generates, the complete command is required, so it fails.

[bryango]

IMHO this is not an issue of direnv but a problem of some shells that still could not respect $XDG_DATA_DIRS. In any case, I came up with a dirty hack for zsh. For this to work one has to first export $FPATH in e.g. ~/.zshrc as direnv runs in a bash subshell.

#!/bin/bash
# ~/.config/direnv/lib/fpath_prepend.sh
# add missing zsh functions for executables
# for this to work, one has to export $FPATH in e.g. ~/.zshrc

fpath_prepend() {
  if [[ -z $FPATH ]]; then
    >&2 echo "\$FPATH empty, not adding: $*"
    exit 0
  fi

  local executable exec_path extra_fpath;
  extra_fpath=""

  for executable in "$@"; do
    if exec_path=$(which "$executable" 2>/dev/null); then
      extra_fpath="$extra_fpath$(dirname "$exec_path")/../share/zsh/site-functions:"
    fi
  done

  export FPATH="$extra_fpath$FPATH"
}

With this under ~/.config/direnv/lib one can then e.g. simply fpath_prepend cargo rustup in .envrc for cargo & rustup completions to work nicely. Note that I am using nix flake and nix-direnv.

[joshbode]

I've been adding the following to the end of my .envrc which seems to work pretty well for zsh:

if [[ "${SHELL##*/}" == zsh ]]; then
  direnv_load ${SHELL} -c 'export FPATH; direnv dump'
  PREFIX=${PWD}/dist  # or wherever your local prefix is
  path_add FPATH "${PREFIX}/share/zsh/site-functions"
  path_add MANPATH "${PREFIX}/share/man"
  path_add XDG_DATA_DIRS "${PREFIX}/share"
fi

I have noticed some strangeness in restoring FPATH when leaving the directory, which is perhaps due to the late load of FPATH, but I just start a new shell.
I should probably just export FPATH in my .zshenv which would avoid needing the direnv_load...

[carschandler]

@bryango well it's also an issue for bash. I think it's more do to with the fact that direnv does its activation using a non-interactive shell which doesn't support loading completions.

[bryango]

well it's also an issue for bash

I think bash should work if you supply it with the correct $XDG_DATA_DIRS, and at least for me, it works out of the box (I use nix, by the way). It seems that bash does load completions from $XDG_DATA_DIRS and nix properly exports $XDG_DATA_DIRS as per #443 (comment).

I was probably too harsh on zsh though, which is unfair for the people volunteering to maintain it... Sorry, amended my answer.

direnv does its activation using a non-interactive shell

I am not sure if "interactive-ness" is the matter here. Even if I enter a new interactive zsh with the correct $XDG_DATA_DIRS, as long as $FPATH (or $fpath) is not updated, zsh fails to load the completion. Also, as evidenced by previous comments, zsh (and fish?) indeed did not respect $XDG_DATA_DIRS.

[carschandler]

Interesting. The specific issue for me is that I’m trying to load a shell hook from micromamba that has a line using the complete command, which isn't available to direnv's shell.

[bryango]

The specific issue for me is that I’m trying to load a shell hook from micromamba that has a line using the complete command, which isn't available to direnv's shell.

Oh, I see... this is an interesting edge case. I cannot see a way out of this for direnv, as it runs in a subshell and I don't think the complete results can somehow be exported to the other shell. There is a hack out of it though: extract the complete related commands into a file and place it somewhere that bash can auto load, e.g. some directory under $XDG_DATA_DIRS 😆

[adrusi]

With zsh on macos with nix-darwin, I wasn't finding that the shell was loading completions even though XDG_DATA_DIRS and FPATH were getting set, leaving me to manually run compinit to actually load them. In case anyone else is having a similar issue, my workaround is to modify the direnv hook in my zsh startup to run compinit after direnv:

_direnv_hook() {
  trap -- '' SIGINT
  eval "$("/opt/homebrew/bin/direnv" export zsh)"
  compinit
  trap - SIGINT
}
typeset -ag precmd_functions
if (( ! ${precmd_functions[(I)_direnv_hook]} )); then
  precmd_functions=(_direnv_hook $precmd_functions)
fi
typeset -ag chpwd_functions
if (( ! ${chpwd_functions[(I)_direnv_hook]} )); then
  chpwd_functions=(_direnv_hook $chpwd_functions)
fi

[BronzeDeer]

With zsh on macos with nix-darwin, I wasn't finding that the shell was loading completions even though XDG_DATA_DIRS and FPATH were getting set, leaving me to manually run compinit to actually load them. In case anyone else is having a similar issue, my workaround is to modify the direnv hook in my zsh startup to run compinit after direnv:

_direnv_hook() {
  trap -- '' SIGINT
  eval "$("/opt/homebrew/bin/direnv" export zsh)"
  compinit
  trap - SIGINT
}
typeset -ag precmd_functions
if (( ! ${precmd_functions[(I)_direnv_hook]} )); then
  precmd_functions=(_direnv_hook $precmd_functions)
fi
typeset -ag chpwd_functions
if (( ! ${chpwd_functions[(I)_direnv_hook]} )); then
  chpwd_functions=(_direnv_hook $chpwd_functions)
fi

Hey, just wanted to say, thanks for that code snippet, it helped me start create my own solution for this, and I extended it into a whole plugin that automatically tracks changes to FPATH, XDG_DATA_DIRS, and optionally PATH to discover new/removed completion paths and reloads compinit as needed

https://github.com/BronzeDeer/zsh-completion-sync