Improve dynamic candidate handling

CHANGELOG.md

@@ -0,0 +1,57 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+The format is based on [Keep a Changelog].
+
+## Unreleased
+### Changed
+* `selectrum-read` now also accepts a function as CANDIDATES.  This
+   can be used to generate them dynamically and to adjust user input
+   used for matching/highlighting ([#27]).
+* The API of `selectrum-refine-candidates-function` changed to always
+  return a list of strings and is no longer able to adjust user input.
+
+
+## 1.0 (released 2020-03-23)
+### Added
+* Package `selectrum`
+* Minor mode `selectrum-mode`
+* Faces:
+    * `selectrum-current-candidate`
+    * `selectrum-primary-highlight`
+    * `selectrum-secondary-highlight`
+* Interface user options:
+    * `selectrum-num-candidates-displayed`
+    * `selectrum-minibuffer-bindings`
+    * `selectrum-count-style`
+* Commands bound in minibuffer:
+    * `selectrum-previous-candidate`
+    * `selectrum-next-candidate`
+    * `selectrum-previous-page`
+    * `selectrum-next-page`
+    * `selectrum-goto-beginning`
+    * `selectrum-goto-end`
+    * `selectrum-kill-ring-save`
+    * `selectrum-select-current-candidate`
+    * `selectrum-submit-exact-input`
+    * `selectrum-insert-current-candidate`
+* Entry points:
+    * `selectrum-read`
+    * `selectrum-completing-read`
+    * `selectrum-read-buffer`
+    * `selectrum-read-file-name`
+    * `selectrum-read-directory-name`
+    * `selectrum-read-library-name`
+* Hooks:
+    * `selectrum-candidate-selected-hook`
+    * `selectrum-candidate-inserted-hook`
+* API user options, variables, and functions:
+    * `selectrum-refine-candidates-function`
+    * `selectrum-default-candidate-refine-function`
+    * `selectrum-preprocess-candidates-function`
+    * `selectrum-default-candidate-preprocess-function`
+    * `selectrum-highlight-candidates-function`
+    * `selectrum-default-candidate-highlight-function`
+    * `selectrum-should-sort-p`
+
+[keep a changelog]: https://keepachangelog.com/en/1.0.0/

Makefile

@@ -35,7 +35,6 @@ checkdoc: ## Check docstring style
 	    echo "[checkdoc] $$file" >&2 ;\
 	    emacs -Q --batch \
 	        --eval "(or (fboundp 'checkdoc-file) (kill-emacs))" \
-	        --eval "(setq sentence-end-double-space nil)" \
 	        --eval "(checkdoc-file \"$$file\")" 2>&1 \
 	        | grep . && exit 1 || true ;\
 	done

README.md

@@ -113,8 +113,10 @@ how to fix it.
   (`<up>`, `<down>`, `C-v`, `M-v`, `M-<`, `M->`). If you prefer, you
   can use `C-p` and `C-n` instead of the arrow keys.
 * *To accept the currently selected candidate:* type `RET`.
-* *To submit what you've typed, even if it's not a candidate:* type
-  `C-j`.
+* *To submit what you've typed, even if it's not a candidate:* you can
+  use `<up>` or `C-p` to select the user input just like a regular
+  candidate, and type `RET` as usual. (Alternatively, you can type
+  `C-j` to submit your exact input without selecting it first.)
 * *To abort:* as per usual, type `C-g`.
 * *To navigate into the currently selected directory while finding a
   file\:* type `TAB`. (What this actually does is insert the currently
@@ -280,7 +282,8 @@ its public API some of the functions that wrap `selectrum-read`:
 * `selectrum-read-library-name` (to override `read-library-name`)
 
 You can use these functions in defining variants of Selectrum-based
-commands.
+commands. Internally those functions all use `selectrum-read` which
+you can use to build other custom completion commands.
 
 ### Sorting, filtering, and highlighting
 
@@ -290,12 +293,13 @@ user option:
 
 * `selectrum-preprocess-candidates-function` takes the original list
   of candidates and sorts it (actually, it can do any sort of
-  preprocessing it wants). This preprocessing only happens once.
+  preprocessing it wants). Usually preprocessing only happens once
+  unless the candidates are generated dynamically from user input. In
+  that case the candidate set might have changed, thus they are
+  processed after each input change.
 * `selectrum-refine-candidates-function` takes the preprocessed list
-  and filters it using the user's input (actually, it can produce the
-  final list of candidates however it wants, including generating it
-  on the fly). This refinement happens every time the user input is
-  updated.
+  and filters it using the user's input. This refinement happens every
+  time the user input is updated.
 * `selectrum-highlight-candidates-function` takes a list of the
   refined candidates that are going to be displayed in the minibuffer,
   and propertizes them with highlighting.
@@ -336,19 +340,10 @@ which may be applied to candidates using `propertize`:
 Note that sorting, filtering, and highlighting is done on the standard
 values of candidates, before any of these text properties are handled.
 
-There is one final detail: the `selectrum-refine-candidates-function`
-may return, in addition to the refined list of candidates, a
-transformed user input which will be used for highlighting (for
-`find-file`, this is the basename of the file in the user input).
-
 To really understand how these pieces work together, it is best to
 inspect the source code of `selectrum-read-buffer` and
 `selectrum-read-file-name` (an effort has been made to make the code
-readable). Note that both of these functions operate by temporarily
-rebinding `selectrum-candidate-preprocess-function` and
-`selectrum-candidate-refine-function` in order to generate candidates
-on the fly and then sort and filter them using the original values of
-these functions.
+readable).
 
 ### Hooks
 
@@ -415,18 +410,19 @@ for the problem I want solved.
 [Ivy](https://github.com/abo-abo/swiper#ivy) is the most promising
 alternative to Selectrum, and it's what I used before developing
 Selectrum. It is marketed as a minimal alternative to Helm which
-provides a simpler interface. The problem with Ivy is its
-architecture, API, and implementation, all of which are very poorly
-designed. Ivy was originally designed to be used as a backend to
+provides a simpler interface. The problem with Ivy is that its
+architecture and API are very messy, and as a result the
+implementation is complex and buggy. Ivy was originally designed to be
+used as a backend to
 [Swiper](https://github.com/abo-abo/swiper#swiper), a buffer search
 package that originally used Helm. Unfortunately, when Ivy became a
-more general-purpose interactive selectrum package, its abstractions
+more general-purpose interactive selection package, its abstractions
 were not reworked to make sense in this new context. Over time, more
 and more special cases were added to try to make various commands work
-properly, and not nearly enough effort has been put in to making the
-core functionality consistent and correct. As a result, the `ivy-read`
-API has around 20 arguments and a heap of special cases for particular
-values (which are completely undocumented). Numerous functions in Ivy,
+properly, and as a result the consistency and correctness of the core
+functionality have suffered. As a result, the `ivy-read` API has
+around 20 arguments and a heap of special cases for particular values
+(which are completely undocumented). Numerous functions in Ivy,
 [Counsel](https://github.com/abo-abo/swiper#counsel), and Swiper have
 special cases hardcoded into them to detect when they're being called
 from specific other functions *in the other two packages*. As a result
@@ -436,7 +432,7 @@ bugs and quirks in UX that led me to develop Selectrum.
 
 Fundamentally, selecting an item from a list *is not a complicated
 problem*, and it *does not require a complicated solution*. That's why
-Selectrum is around 900 lines of code even though Ivy+Counsel (which
+Selectrum is around 1,000 lines of code even though Ivy+Counsel (which
 do basically the same thing) are around 11,000 lines together.
 Selectrum achieves its conciseness by:
 
@@ -497,7 +493,4 @@ best way to present buffer search. Instead, I decided on an improved
 variant of the Isearch interface that takes inspiration from the
 standard text search interface found in almost every other modern
 piece of software, such as web browsers. The result is
-[CTRLF](https://github.com/raxod502/ctrlf), although this package is
-currently alpha-quality and has not yet been documented. See [this
-issue](https://github.com/raxod502/ctrlf/issues/11) for discussion
-about the difference between CTRLF and Swiper.
+[CTRLF](https://github.com/raxod502/ctrlf).