Interactive global search

[the-mikedavis]

This refactors global search (<space>/) to be interactive. The idle timeout triggers an async callback which refills the picker with new search results.

https://asciinema.org/a/536431

Closes #196

[the-mikedavis]

I have been using this for a while and it's nice to be able to change the query quickly but there are some usability regressions:

• You can only search using the regex query, so you can't filter on filenames as you can with master's global_search. For really large searches it's very useful to be able to filter on filenames.
• The picker filters options using the search text, so the picker and the dynamic picker fight over showing results when using regular expression syntax like escapes (for example, search for \{self: there are matches but the picker filters them out).

It would also be useful to search in other pickers with multiple criteria (for example you could filter the symbol picker by symbol kind), so maybe we can come up with an improvement to the picker functionality that will benefit all pickers. I also wonder how telescope handles this?

[pascalkuthe]

Another example where I also ran into a similar problem would.be a git commit picker. You could filter:

• Author
• Date
• commit message
• checksum

Irrespective of whether we want something like a commit picker in core I think this example also shows potential for filtering multiple queries . I think the easiest route to implement this would be to create some kind of cycle_picker_filter keybinding that would change what you are filtering (while keeping the previous filters intact). Maybe we could reuse the old binding here

[matoous]

Just to add one more usecase: in symbol picker it would be nice to be able to filter by the symbol type. What I have also seen elsewhere (which is imho inferior to what @pascalkuthe suggested) is having custom strings in the filter to filter for various things, e.g. in the example of symbol picker one could do function:XX to filter only for functions, in the case of commits this would be author:XX, and in the case of global search one could do file:XX. But.. this is way more verbose and fragily in my opinion. I really like the idea of being able to cycle through the filters. Also also, very far from the scope of this PR: maybe another option (or complementary to the cycle filter) would be to have shortcuts for picking different filters, e.g. (for commits):

prompt: XXX
---
1: message 2: author 3: date 4: checksum
---
commit 1
commit 2
...

and a key binding (e.g. alt+X) to select one of the filters.

[pascalkuthe]

Just to add one more usecase: in symbol picker it would be nice to be able to filter by the symbol type. What I have also seen elsewhere (which is imho inferior to what @pascalkuthe suggested) is having custom strings in the filter to filter for various things, e.g. in the example of symbol picker one could do function:XX to filter only for functions, in the case of commits this would be author:XX, and in the case of global search one could do file:XX. But.. this is way more verbose and fragily in my opinion. I really like the idea of being able to cycle through the filters. Also also, very far from the scope of this PR: maybe another option (or complementary to the cycle filter) would be to have shortcuts for picking different filters, e.g. (for commits):

prompt: XXX
---
1: message 2: author 3: date 4: checksum
---
commit 1
commit 2
...

and a key binding (e.g. alt+X) to select one of the filters.

I think alt-X might be preferable to a cycle keybding. We could show something similar to a buffer line on top to make this intutive:

|1: message | 2: author | 3: date | 4: sha |
————————————————————————————————————————————

Another nice thing to imorove the UI would be to display the full item while only filtering parts of it. For.example again for commits (but I can think of many other examoles) it would.be nice to print the full commit (sha author date message) in the picker even when filtering by author so you don't loose all context of what is being filtered. That will require some adjustment to to way we handle fuzzy indecies (currently if the display_text and filter_text missmatch in widthnit doesn't display correctly and we can get rid of sort_text I believe).

That said all that won't be necessary for global_search. I think the basic mechanism of having multiple filters could be implemented in this PR and usednin global_search and more advanced UI left to future PRs.

[otavioschwanck]

i have no idea if this code already but would be awesome to chain multiple searches based on previous searchs like telescope do now (with S-RET). Also, doom-emacs find-files has the best implementation of that i ever seen. Example of that:

Press SPC / and search for - , returns:

• banana_good
• kiwi_good
• others

change the completion to - #good, returns:

• banana_good
• kiwi_good

change the completion to - #good#ba, returns:

• banana_good

Every # picks the previous result and filters again. This is so powerful and awesome, the thing i most miss on emacs.

[the-mikedavis]

I think what you're describing can be done by adding spaces between searches. That's specific to filtering pickers though which is not really related to this PR.

I think the basic mechanism of having multiple filters could be implemented in this PR and used in global_search and more advanced UI left to future PRs

It might be best to start with the symbol picker for hacking on this idea - that picker only needs to be sorted by symbol type and name. Then the workspace symbol picker would be a good next step since that also uses a DynamicPicker. Plus the diff for this PR is already pretty ugly with how the callback was refactored.

I think the extra filtering feature might also need #3053

[rcorre]

This is great, I've really missed the behavior of :Rg from fzf.vim.

With regards to cycling filters, :Rg seems to keep searches where either the filename or the text match each search term, and then orders them by some criteria. I'm not quite sure how it works, but it feels like magic. I can just type some combination of text/filename critiera, and it does a great job of finding the file I want, without ever having to be explicit about what field I'm searching:

[pascalkuthe]

This is great, I've really missed the behavior of :Rg from fzf.vim.

With regards to cycling filters, :Rg seems to keep searches where either the filename or the text match each search term, and then orders them by some criteria. I'm not quite sure how it works, but it feels like magic. I can just type some combination of text/filename critiera, and it does a great job of finding the file I want, without ever having to be explicit about what field I'm searching:

:Rg works completly differently. It runs ripgrep with a search regex once and then performs fuzzy matching on the concatenation of the filepath and the matched line. This is very similar to how helix currently works with the only difference that we only allow fuzzy matching on the path and not on the line.

By comparison this PR reruns the ripgrep based search automatically (we use ripgrep internals since it's written in rust) so you can change the regex and automatically get updated results.

[rcorre]

:Rg works completly differently. It runs ripgrep with a search regex once and then performs fuzzy matching on the concatenation of the filepath and the matched line

That is how :Rg <pattern> works.

By comparison this PR reruns the ripgrep based search automatically (we use ripgrep internals since it's written in rust) so you can change the regex and automatically get updated results.

This is how :Rg (without arguments) works. In my screenshot, I've run :Rg without arguments. "cursor" matched lines with "cursor", while "tutor" matched the file "tutor". I can see how explicitly limiting terms to specific fields would be useful too, but sometimes it's nice to not have to think about whether you're filtering the filename/line/etc.

[pascalkuthe]

:Rg works completly differently. It runs ripgrep with a search regex once and then performs fuzzy matching on the concatenation of the filepath and the matched line

That is how :Rg <pattern> works.

By comparison this PR reruns the ripgrep based search automatically (we use ripgrep internals since it's written in rust) so you can change the regex and automatically get updated results.

This is how :Rg (without arguments) works. In my screenshot, I've run :Rg without arguments. "cursor" matched lines with "cursor", while "tutor" matched the file "tutor". I can see how explicitly limiting terms to specific fields would be useful too, but sometimes it's nice to not have to think about whether you're filtering the filename/line/etc.

:Rg without arguments does the same thing. It just returns all lines in that case without any regex filtering on which you then fuzzy match.

You can configure it to behave the same as this PR but in that case you are not matching on filenames anymore either. From the README of fzf.vim:

In the default implementation of Rg, ripgrep process starts only once with the initial query (e.g. :Rg foo) and fzf filters the output of the process.

This is okay in most cases because fzf is quite performant even with millions of lines, but we can make fzf completely delegate its search responsibliity to ripgrep process by making it restart ripgrep whenever the query string is updated. In this scenario, fzf becomes a simple selector interface rather than a "fuzzy finder".

We will name the new command all-uppercase RG so we can still access the default version. --bind 'change:reload:rg ... {q}' will make fzf restart ripgrep process whenever the query string, denoted by {q}, is changed. With --disabled option, fzf will no longer perform search. The query string you type on fzf prompt is only used for restarting ripgrep process. Also note that we enabled previewer with fzf#vim#with_preview. The last argument to the function, ctrl-/, is the key to toggle the preview window.

[the-mikedavis]

This is now part of #7265: it fixes the usability problems I mentioned in #4687 (comment)