GNOME module: adding bash-completion script generation

[amcn]

I was looking at clang's code recently and stumbled upon their neat bash auto-completion support. Its functionality is well summarised by this blog post. The key functionality is that clang itself gained a special --autocomplete argument which internally causes clang to walk over its programmatically constructed argument table and print the possible valid completions to stdout. The bash auto-complete script then invokes clang with this special argument and parses stdout making it compatible with bash's completion. This has two main advantages:

1. clang understands the types of the arguments and can propose sensible completions.
2. No need to keep the bash-completion script up-to-date when arguments are added/removed/changed.

I proposed to the GNOME community that similar functionality be added to glib's GOptionContext and the idea was well received. Its implementation is under way.

Part of my proposal to GNOME was that meson would gain functionality to help users automatically generate and install a basic shell completion script for a given target. So before the implementation in glib gets too far ahead, I wanted to run the meson part of this by the meson community and get feedback here. I am imagining something like this:

my_app = executable('my_app', 'my_app.c', dependencies: [ dependency('glib-2.0') ])

gnome = import('gnome')

completion_scripts = gnome.shell_completions(
  my_app,
  shells: ['bash', 'zsh', 'fish', 'powershell', 'insert-some-other-shell-here'],
  install: true,
)

So, meson community, what do you think? Is this idea sensible? Can you see any pitfalls or hidden complexity?

[eli-schwartz]

On the general topic of such completion generators, it's quite common for projects to have a command that queries the internal data structures of the current binary and then generates a freestanding completion script which doesn't call into the original binary but uses the standard shell completion facilities. It's essentially just a "better way to maintain complex completion scripts".

There is a significant advantage to pregenerating the completion script. You don't have to fork+exec into the program every time you hit <TAB>, which for some programs can take a very long time.

On the other hand, if you know the application in question, you know the startup time is cheap, and you know there are nearly a thousand options and you know writing out a bash table of a thousand options results in huge chunky data files that maybe bloat your interactive shell memory since they can't be unloaded... then querying the program may indeed make a whole lot more sense as all that data is transient.

In terms of practical concerns, many programs that distribute any sort of generated completion script at all are unable to do so when building the software via cross-compilation, as you may not actually be able to run the built program without setting up qemu. But that can be solved by checking for / demanding meson's existing exe_wrapper support -- we anyways sometimes need this for stuff like code generators, gobject-introspection, cc.run(), and whatnot.

[amcn]

You raise a good point about the cost of fork+exec. I haven't tested it personally but I can't imagine clang's solution maintains interactivity when it's using a clang binary built with full debug info which I've seen reach up to 1.5GB in size (probably bigger these days, it's been a while).

I think the reason that many LLVM based tools don't autogenerate a script once at package-time is because the set of valid arguments can be dynamic. Take opt for example: custom optimisation passes which can themselves be loaded from the command line can add new command line options to the available set:

$ opt -load my-pass.so --my-pass-arg=foo some-ir.ll 

I'm not sure if clang has similar dynamism in its arguments, but its option parsing is based on LLVM so I wouldn't be surprised.

GOptionContext also supports this kind of dynamic argument set with its notion of GOptionGroup. Typically glib-adjacent libraries such as Gtk, Gstreamer and Gir have an API to get the libraries' GOptionGroup but depending on the version of the dynamically loaded library the actual set of options could change. They typically don't change very often though it must be said.

[eli-schwartz]

A common tactic is to generate a static completion script that knows the command option half of an option-argument, where the argument half is fully dynamic. For example foobarprog --infile <TAB> could use the built-in filedir generator to deduce valid values, but it could also query the foobarprog. You only pay the cost of forking out when there is no other choice.

Being able to load a plugin that then extends the set of valid options is an interesting wrinkle. I suppose it could be handled by also scanning the existing command line to see if -load appears and in that scenario pay the full cost of dynamically querying.

If it's always a library that gets permanently installed then the extender library could ship its own completion script that augments the primary one. This is similar to how the git program allows you to install new git-mysubcommand programs to register them as git mysubcommand. (The obvious question is obvious, I know.) The augmented completion script would then be somehow detected by e.g. having the main script at first load, iterate over the set of valid _completion_loader search directories and scan for suffic-patterned completion scripts.