beet zsh completion bug

[snowyoneill]

Problem

when i try to complete beets command using zsh command i keep seeing _beet:zregexparse:4: invalid regex : )

im running into the same issue documented here: #3050 and here https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=914569

Setup

ubuntu 19.10 (Eoan Ermine)
oh-my-zsh
	zsh 5.7.1 (x86_64-ubuntu-linux-gnu)

beets
	beets version 1.4.9
	Python version 3.7.5
	plugins: no plugins

if i run which _beet after trying the completion i get this.

_beet () {
	local _ra_p1 _ra_p2 _ra_left _ra_right _ra_com expl tmp nm="$compstate[nmatches]" 
	local _ra_actions _ra_line="${(pj:\0:)${(@)words[1,CURRENT - 1]:Q}}"$'\0'"$PREFIX" 
	_ra_actions=() 
	zregexparse -c _ra_p1 _ra_p2 "$_ra_line" $'/[^\0]##\0/' $'(' $'(' $'/-c\0/' $'/[^\0]##\0/' $':_ra_comp $\'file:file:_files\'' $'|' $'/-v\0/' $'|' $'/-l\0/' $'/[^\0]##\0/' $':_ra_comp $\'file:file:_files\'' $'|' $'/-h\0/' $'|' $'/-d\0/' $'/[^\0]##\0/' $':_ra_comp $\'dir:directory:_dirs\'' $'|' $'/[]/' $':_ra_comp $\'options:global options:(( -c\\\\:path\\\\ to\\\\ configuration\\\\ file -v\\\\:print\\\\ debugging\\\\ information -l\\\\:library\\\\ database\\\\ file\\\\ to\\\\ use -h\\\\:show\\\\ this\\\\ help\\\\ message\\\\ and\\\\ exit -d\\\\:destination\\\\ music\\\\ directory ))\'' $')' $'#' $')' $')'
	case "$?" in
		(0 | 2) _message "no more arguments" ;;
		(1) if [[ "$_ra_line[_ra_p1 + 1, -1]" = *$'\0'* ]]
			then
				_message "parse failed before current word"
			else
				_ra_left="$_ra_line[_ra_p1 + 1, _ra_p2]" 
				_ra_right="$_ra_line[_ra_p2 + 1, -1]" 
				compset -p $(( $#PREFIX - $#_ra_line + $_ra_p1 ))
				(( $#_ra_actions )) && _alternative "$_ra_actions[@]"
			fi ;;
		(3) _message "invalid regex" ;;
	esac
	[[ nm -ne "$compstate[nmatches]" ]]
}

---

The weird thing is that if i boot into the Ubuntu 19.10 live usb and try from there it works.

setup commands:

	sudo apt-get install python3-setuptools python3-pip zsh
	pip3 install beets
	sudo cp /media/cdrom/_beet /usr/local/share/zsh/site-functions

ubuntu@ubuntu ~ % which _beet
_beet () {
	local _ra_p1 _ra_p2 _ra_left _ra_right _ra_com expl tmp nm="$compstate[nmatches]" 
	local _ra_actions _ra_line="${(pj:\0:)${(@)words[1,CURRENT - 1]:Q}}"$'\0'"$PREFIX" 
	_ra_actions=() 
	zregexparse -c _ra_p1 _ra_p2 "$_ra_line" $'/[^\0]##\0/' $'(' $'(' $'/-c\0/' $'/[^\0]##\0/' $':_ra_comp $\'file:file:_files\'' $'|' $'/-v\0/' $'|' $'/-l\0/' $'/[^\0]##\0/' $':_ra_comp $\'file:file:_files\'' $'|' $'/-h\0/' $'|' $'/-d\0/' $'/[^\0]##\0/' $':_ra_comp $\'dir:directory:_dirs\'' $'|' $'/[]/' $':_ra_comp $\'options:global options:(( -c\\\\:path\\\\ to\\\\ configuration\\\\ file -v\\\\:print\\\\ debugging\\\\ information -l\\\\:library\\\\ database\\\\ file\\\\ to\\\\ use -h\\\\:show\\\\ this\\\\ help\\\\ message\\\\ and\\\\ exit -d\\\\:destination\\\\ music\\\\ directory ))\'' $')' $'#' $')' $'(' $'/config\0/' $':_ra_comp $\'subcmds:subcommands:((config:show\\\\ or\\\\ edit\\\\ the\\\\ user\\\\ configuration))\'' $'(' $'/[^\0]##\0/' $':_ra_comp $\'option:option:{_beet_subcmd config}\'' $')' $'#' $'|' $'/fields\0/' $':_ra_comp $\'subcmds:subcommands:((fields:show\\\\ fields\\\\ available\\\\ for\\\\ queries\\\\ and\\\\ format\\\\ strings))\'' $'(' $'/[^\0]##\0/' $':_ra_comp $\'option:option:{_beet_subcmd fields}\'' $')' $'#' $'|' $'/help\0/' $':_ra_comp $\'subcmds:subcommands:((help:give\\\\ detailed\\\\ help\\\\ on\\\\ a\\\\ specific\\\\ sub-command))\'' $'(' $'/[^\0]##\0/' $':_ra_comp $\'option:option:{_beet_subcmd help}\'' $')' $'#' $'|' $'/import\0/' $':_ra_comp $\'subcmds:subcommands:((import:import\\\\ new\\\\ music))\'' $'(' $'/[^\0]##\0/' $':_ra_comp $\'option:option:{_beet_subcmd import}\'' $')' $'#' $'|' $'/list\0/' $':_ra_comp $\'subcmds:subcommands:((list:query\\\\ the\\\\ library))\'' $'(' $'/[^\0]##\0/' $':_ra_comp $\'option:option:{_beet_subcmd list}\'' $')' $'#' $'|' $'/modify\0/' $':_ra_comp $\'subcmds:subcommands:((modify:change\\\\ metadata\\\\ fields))\'' $'(' $'/[^\0]##\0/' $':_ra_comp $\'option:option:{_beet_subcmd modify}\'' $')' $'#' $'|' $'/move\0/' $':_ra_comp $\'subcmds:subcommands:((move:move\\\\ or\\\\ copy\\\\ items))\'' $'(' $'/[^\0]##\0/' $':_ra_comp $\'option:option:{_beet_subcmd move}\'' $')' $'#' $'|' $'/remove\0/' $':_ra_comp $\'subcmds:subcommands:((remove:remove\\\\ matching\\\\ items\\\\ from\\\\ the\\\\ library))\'' $'(' $'/[^\0]##\0/' $':_ra_comp $\'option:option:{_beet_subcmd remove}\'' $')' $'#' $'|' $'/stats\0/' $':_ra_comp $\'subcmds:subcommands:((stats:show\\\\ statistics\\\\ about\\\\ the\\\\ library\\\\ or\\\\ a\\\\ query))\'' $'(' $'/[^\0]##\0/' $':_ra_comp $\'option:option:{_beet_subcmd stats}\'' $')' $'#' $'|' $'/update\0/' $':_ra_comp $\'subcmds:subcommands:((update:update\\\\ the\\\\ library))\'' $'(' $'/[^\0]##\0/' $':_ra_comp $\'option:option:{_beet_subcmd update}\'' $')' $'#' $'|' $'/version\0/' $':_ra_comp $\'subcmds:subcommands:((version:output\\\\ version\\\\ information))\'' $'(' $'/[^\0]##\0/' $':_ra_comp $\'option:option:{_beet_subcmd version}\'' $')' $'#' $'|' $'/write\0/' $':_ra_comp $\'subcmds:subcommands:((write:write\\\\ tag\\\\ information\\\\ to\\\\ files))\'' $'(' $'/[^\0]##\0/' $':_ra_comp $\'option:option:{_beet_subcmd write}\'' $')' $'#' $')'
	case "$?" in
		(0 | 2) _message "no more arguments" ;;
		(1) if [[ "$_ra_line[_ra_p1 + 1, -1]" = *$'\0'* ]]
			then
				_message "parse failed before current word"
			else
				_ra_left="$_ra_line[_ra_p1 + 1, _ra_p2]" 
				_ra_right="$_ra_line[_ra_p2 + 1, -1]" 
				compset -p $(( $#PREFIX - $#_ra_line + $_ra_p1 ))
				(( $#_ra_actions )) && _alternative "$_ra_actions[@]"
			fi ;;
		(3) _message "invalid regex" ;;
	esac
	[[ nm -ne "$compstate[nmatches]" ]]
}

Notice that the which _beet function returns different commands in each case. On my primary system it doesn't include the subcmds:subcommands section

If i use the extra _beet version from https://github.com/beetbox/beets/blob/0301bb517dd2fd5c1ec82be9f00c865615c1d5d9/extra/_beet then it works on my machine. But without the caching that was introduced in newer versions.

Steps I've tried to resolve this issue:
1. Reset .zshrc and removed oh my zsh
2. Remapped my /home/ folder to a blank directory to ensure it wasn't a config issue.
3. Created a new user account to double check if i wasn't going nuts on step 2.
4. Completely reinstalled Ubuntu 19.10 to triple check i wasn't losing my bananas.

None of these things worked yet for some reason beets zsh completion works fine in the live USB, but the moment i completely install the OS and try again i encounter issues.

Does anyone have any ideas what might be causing this?

UPDATE:

I've managed to replicate the bug on the Live USB by creating a bare bones config.yaml, assigning only the library key and pointing it to a directory that doesn't exist e.g. ~/Desktop/Music/musiclibrary.db -> ~/Desktop/Music2/musiclibrary.db

This also causes the which _beet output miss the subcmds:subcommands like on my main installation.

But i don't understand why that happening on my main system as the directories are setup correctly.

[sampsyo]

Interesting! Pinging @vapniks, who knows the zsh bindings best, to see if they have any insight. Perhaps the insight about the missing directory helps?

[DomenicF]

Hi, I am also having this issue.

When using beet import and ZSH (autocomplete) I get the following errors every time:

_beet:zregexparse:4: invalid regex : )
_beet:zregexparse:4: invalid regex : )
_beet:zregexparse:4: invalid regex : )

Setup

• OS: Pop!_OS 20.04 (based on Ubuntu, using Ubuntu package)
• Python version: 3.8.2
• beets version: 1.4.9

My configuration (output of beet config) is:

library: ~/beets.db
directory: /mnt/ORGANIZED_MUSIC

plugins: acousticbrainz fetchart lastgenre scrub

paths:
    default: $albumartist/$albumartist - $album%aunique{} ($year) [$format]/$track. $title
    singleton: Non-Album/$artist ($year) [$format]/$track. $title
    comp: Compilations/$album%aunique{} ($year) [$encoder]/$track. $title
lastgenre:
    whitelist: yes
    min_weight: 10
    count: 1
    fallback:
    canonical: no
    source: album
    force: yes
    auto: yes
    separator: ', '
    prefer_specific: no
fetchart:
    auto: yes
    minwidth: 0
    maxwidth: 0
    enforce_ratio: no
    cautious: no
    cover_names:
    - cover
    - front
    - art
    - album
    - folder
    sources:
    - filesystem
    - coverart
    - itunes
    - amazon
    - albumart
    google_key: REDACTED
    google_engine: 001442825323518660753:hrh5ch1gjzm
    fanarttv_key: REDACTED
    store_source: no
acousticbrainz:
    auto: yes
    force: no
    tags: []
scrub:
    auto: yes

Any help would be appreciated. Thank you!

[stale]

Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Ruin0x11]

This is still an issue for me as of beets 1.6.0.

[krp312]

Same for me. Beets 1.6.0.

[sampsyo]

I'll reopen this and mark it as a bug, but I admit I have no insight into how the zsh completions work—so I'll need help from an expert to understand what's going wrong and fix it.

[Sylk1-9]

Hi. Same issue on ubuntu 22.04, zsh 5.8.1 (x86_64-ubuntu-linux-gnu), beets version 1.6.0, Python version 3.10.6.

A temporary solution I found is to end the path provided to the beet import command by a start *.

For example,

beet import Freaky\ Chakra\ \(1996\)\ Low*

then press the tab key (auto-complete), which do not trigger the above error, and should actually complete the input path to

beet import -wc Freaky\ Chakra\ \(1996\)\ Lowdown\ Motivator

[OrigamiEngineer]

I have the same experience as @Sylk1-9; same issue on Arch, zsh 5.9, beets 1.6.0 python 3.10.10.

I wanted to say thank you to you Sylk1-9, it's really a bother not to have autocomplete! (and something is better than nothing) Your workaround really helps my workflow :D

[mthies-unibi]

The error invalid regex : ) is triggered, if the installed awk command is not mawk (by Thomas E. Dickey and Michael D. Brennan). The function _join_lines in the _beet completion script uses a non-standard regular expression operator {,<number>} that only mawk seems to support. This is equivalent to the more widely supported longer form {0,<number>}.

After adding the missing 0 before the comma, the awk code inside _beet contains this regexp: "^[[:space:]]{0,"ARG2"}[^[:space:]]" and completion of subcommands started to work without that invalid regex error on my system.

To get completion of field values from the database to work (e.g. completing beet list genre: with a genre already in use), two further changes were required:

1. In function _beet_field_values remove the sed filtering of the sqlite3 output and simply invoke sqlite3 -list -noheader. The sed invocation was missing some command terminators ; which only some implementations of sed support (e.g. GNU). Also, the filtering depends on some specific ASCII line-drawing in the sqlite3 output to recognize any field values. These visual separator lines were not present in the default output format of my sqlite3.

2. The function _beet_field_values can only find the library database, if the zsh completion cache is activated. The _beet completion script already sets the cache-policy (near the top of the script), but should probably also ensure that the cache is activated, at least for its own context and the sub-contexts for the query arguments:

   zstyle ":completion:${curcontext%:*}:*" use-cache true

[arrrgi]

@mthies-unibi would you be interesting in making a PR for this or at a minimum, sharing a Gist with the changes you implemented?