Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Eval'ing (keyword "\"|") gives "Search failed" error #1971

Closed
brianfay opened this issue Mar 28, 2017 · 17 comments
Closed

Eval'ing (keyword "\"|") gives "Search failed" error #1971

brianfay opened this issue Mar 28, 2017 · 17 comments

Comments

@brianfay
Copy link

brianfay commented Mar 28, 2017

Expected behavior

When evaluating (keyword "\"|"), the keyword :"| should be displayed in the minibuffer.

Actual behavior

When evaluating (keyword "\"|"), the minibuffer displays a regex error:

error in process filter: Search failed: "\\(\\s-\\|^\\|#\\|(\\|\\[\\|{\\)\\(\"\\)"

Steps to reproduce the problem

Evaluate

(keyword "\"|")

using cider-eval-last-sexp or a similar command.

Alternatively, evaluate

(def k (keyword "\"|"))

and try evaluating k.

Environment & Version information

Using the clojure configuration layer in spacemacs

CIDER version information

;; CIDER 0.15.0snapshot (package: 20170322.155), nREPL 0.2.12
;; Clojure 1.8.0, Java 1.8.0_101

Lein/Boot version

Lein 2.6.1

Emacs version

Gnu Emacs 24.5.1

Operating system

OS X Sierra

Other notes

xiongtx on Slack brought up a good point that " and | are technically not considered valid characters for clojure keywords and symbols, even though they generally work in practice.

@dpsutton
Copy link
Contributor

This appears to be a bug in clojure-font-lock-keywords from clojure-mode.

Here's a stacktrace:

clojure-string-start(t)
  clojure--font-locked-as-string-p(regexp)
  clojure-font-lock-regexp-groups(4)
  font-lock-fontify-keywords-region(1 4 nil)
  font-lock-default-fontify-region(1 4 nil)
  apply(font-lock-default-fontify-region 1 4 nil)
  (closure ((func . font-lock-default-fontify-region) cider--debug-mode cider--debug-mode-response t) (beg end &rest rest) (let* ((modified (buffer-modified-p)) (buffer-undo-list t) (inhibit-read-only t) (inhibit-modification-hooks t)) (unwind-protect (progn (remove-text-properties beg end (quote (cider-locals nil cider-block-dynamic-font-lock nil))) (add-text-properties beg end (quote (help-echo cider--help-echo))) (if cider-font-lock-dynamically (progn (cider--update-locals-for-region beg end)))) (if modified nil (restore-buffer-modified-p nil)))) (apply func beg end rest))(1 4 nil)
  apply((closure ((func . font-lock-default-fontify-region) cider--debug-mode cider--debug-mode-response t) (beg end &rest rest) (let* ((modified (buffer-modified-p)) (buffer-undo-list t) (inhibit-read-only t) (inhibit-modification-hooks t)) (unwind-protect (progn (remove-text-properties beg end (quote (cider-locals nil cider-block-dynamic-font-lock nil))) (add-text-properties beg end (quote (help-echo cider--help-echo))) (if cider-font-lock-dynamically (progn (cider--update-locals-for-region beg end)))) (if modified nil (restore-buffer-modified-p nil)))) (apply func beg end rest)) 1 4 nil)
  (closure ((func closure ((func . font-lock-default-fontify-region) cider--debug-mode cider--debug-mode-response t) (beg end &rest rest) (let* ((modified (buffer-modified-p)) (buffer-undo-list t) (inhibit-read-only t) (inhibit-modification-hooks t)) (unwind-protect (progn (remove-text-properties beg end (quote ...)) (add-text-properties beg end (quote ...)) (if cider-font-lock-dynamically (progn ...))) (if modified nil (restore-buffer-modified-p nil)))) (apply func beg end rest)) cider--debug-mode cider--debug-mode-response t) (beg end &rest rest) (let* ((modified (buffer-modified-p)) (buffer-undo-list t) (inhibit-read-only t) (inhibit-modification-hooks t)) (unwind-protect (progn (remove-text-properties beg end (quote (cider-locals nil cider-block-dynamic-font-lock nil))) (add-text-properties beg end (quote (help-echo cider--help-echo))) (if cider-font-lock-dynamically (progn (cider--update-locals-for-region beg end)))) (if modified nil (restore-buffer-modified-p nil)))) (apply func beg end rest))(1 4 nil)
  apply((closure ((func closure ((func . font-lock-default-fontify-region) cider--debug-mode cider--debug-mode-response t) (beg end &rest rest) (let* ((modified (buffer-modified-p)) (buffer-undo-list t) (inhibit-read-only t) (inhibit-modification-hooks t)) (unwind-protect (progn (remove-text-properties beg end (quote ...)) (add-text-properties beg end (quote ...)) (if cider-font-lock-dynamically (progn ...))) (if modified nil (restore-buffer-modified-p nil)))) (apply func beg end rest)) cider--debug-mode cider--debug-mode-response t) (beg end &rest rest) (let* ((modified (buffer-modified-p)) (buffer-undo-list t) (inhibit-read-only t) (inhibit-modification-hooks t)) (unwind-protect (progn (remove-text-properties beg end (quote (cider-locals nil cider-block-dynamic-font-lock nil))) (add-text-properties beg end (quote (help-echo cider--help-echo))) (if cider-font-lock-dynamically (progn (cider--update-locals-for-region beg end)))) (if modified nil (restore-buffer-modified-p nil)))) (apply func beg end rest)) 1 4 nil)
  (closure ((func closure ((func closure ((func . font-lock-default-fontify-region) cider--debug-mode cider--debug-mode-response t) (beg end &rest rest) (let* ((modified ...) (buffer-undo-list t) (inhibit-read-only t) (inhibit-modification-hooks t)) (unwind-protect (progn ... ... ...) (if modified nil ...))) (apply func beg end rest)) cider--debug-mode cider--debug-mode-response t) (beg end &rest rest) (let* ((modified (buffer-modified-p)) (buffer-undo-list t) (inhibit-read-only t) (inhibit-modification-hooks t)) (unwind-protect (progn (remove-text-properties beg end (quote ...)) (add-text-properties beg end (quote ...)) (if cider-font-lock-dynamically (progn ...))) (if modified nil (restore-buffer-modified-p nil)))) (apply func beg end rest)) cider--debug-mode cider--debug-mode-response t) (beg end &rest rest) (let* ((modified (buffer-modified-p)) (buffer-undo-list t) (inhibit-read-only t) (inhibit-modification-hooks t)) (unwind-protect (progn (remove-text-properties beg end (quote (cider-locals nil cider-block-dynamic-font-lock nil))) (add-text-properties beg end (quote (help-echo cider--help-echo))) (if cider-font-lock-dynamically (progn (cider--update-locals-for-region beg end)))) (if modified nil (restore-buffer-modified-p nil)))) (apply func beg end rest))(1 4 nil)
  apply((closure ((func closure ((func closure ((func . font-lock-default-fontify-region) cider--debug-mode cider--debug-mode-response t) (beg end &rest rest) (let* ((modified ...) (buffer-undo-list t) (inhibit-read-only t) (inhibit-modification-hooks t)) (unwind-protect (progn ... ... ...) (if modified nil ...))) (apply func beg end rest)) cider--debug-mode cider--debug-mode-response t) (beg end &rest rest) (let* ((modified (buffer-modified-p)) (buffer-undo-list t) (inhibit-read-only t) (inhibit-modification-hooks t)) (unwind-protect (progn (remove-text-properties beg end (quote ...)) (add-text-properties beg end (quote ...)) (if cider-font-lock-dynamically (progn ...))) (if modified nil (restore-buffer-modified-p nil)))) (apply func beg end rest)) cider--debug-mode cider--debug-mode-response t) (beg end &rest rest) (let* ((modified (buffer-modified-p)) (buffer-undo-list t) (inhibit-read-only t) (inhibit-modification-hooks t)) (unwind-protect (progn (remove-text-properties beg end (quote (cider-locals nil cider-block-dynamic-font-lock nil))) (add-text-properties beg end (quote (help-echo cider--help-echo))) (if cider-font-lock-dynamically (progn (cider--update-locals-for-region beg end)))) (if modified nil (restore-buffer-modified-p nil)))) (apply func beg end rest)) 1 4 nil)
  (closure ((func closure ((func closure ((func closure (... cider--debug-mode cider--debug-mode-response t) (beg end &rest rest) (let* ... ...) (apply func beg end rest)) cider--debug-mode cider--debug-mode-response t) (beg end &rest rest) (let* ((modified ...) (buffer-undo-list t) (inhibit-read-only t) (inhibit-modification-hooks t)) (unwind-protect (progn ... ... ...) (if modified nil ...))) (apply func beg end rest)) cider--debug-mode cider--debug-mode-response t) (beg end &rest rest) (let* ((modified (buffer-modified-p)) (buffer-undo-list t) (inhibit-read-only t) (inhibit-modification-hooks t)) (unwind-protect (progn (remove-text-properties beg end (quote ...)) (add-text-properties beg end (quote ...)) (if cider-font-lock-dynamically (progn ...))) (if modified nil (restore-buffer-modified-p nil)))) (apply func beg end rest)) cider--debug-mode cider--debug-mode-response t) (beg end &rest rest) (let* ((modified (buffer-modified-p)) (buffer-undo-list t) (inhibit-read-only t) (inhibit-modification-hooks t)) (unwind-protect (progn (remove-text-properties beg end (quote (cider-locals nil cider-block-dynamic-font-lock nil))) (add-text-properties beg end (quote (help-echo cider--help-echo))) (if cider-font-lock-dynamically (progn (cider--update-locals-for-region beg end)))) (if modified nil (restore-buffer-modified-p nil)))) (apply func beg end rest))(1 4 nil)
  apply((closure ((func closure ((func closure ((func closure (... cider--debug-mode cider--debug-mode-response t) (beg end &rest rest) (let* ... ...) (apply func beg end rest)) cider--debug-mode cider--debug-mode-response t) (beg end &rest rest) (let* ((modified ...) (buffer-undo-list t) (inhibit-read-only t) (inhibit-modification-hooks t)) (unwind-protect (progn ... ... ...) (if modified nil ...))) (apply func beg end rest)) cider--debug-mode cider--debug-mode-response t) (beg end &rest rest) (let* ((modified (buffer-modified-p)) (buffer-undo-list t) (inhibit-read-only t) (inhibit-modification-hooks t)) (unwind-protect (progn (remove-text-properties beg end (quote ...)) (add-text-properties beg end (quote ...)) (if cider-font-lock-dynamically (progn ...))) (if modified nil (restore-buffer-modified-p nil)))) (apply func beg end rest)) cider--debug-mode cider--debug-mode-response t) (beg end &rest rest) (let* ((modified (buffer-modified-p)) (buffer-undo-list t) (inhibit-read-only t) (inhibit-modification-hooks t)) (unwind-protect (progn (remove-text-properties beg end (quote (cider-locals nil cider-block-dynamic-font-lock nil))) (add-text-properties beg end (quote (help-echo cider--help-echo))) (if cider-font-lock-dynamically (progn (cider--update-locals-for-region beg end)))) (if modified nil (restore-buffer-modified-p nil)))) (apply func beg end rest)) 1 4 nil)
  (closure ((func closure ((func closure ((func closure (... cider--debug-mode cider--debug-mode-response t) (beg end &rest rest) (let* ... ...) (apply func beg end rest)) cider--debug-mode cider--debug-mode-response t) (beg end &rest rest) (let* ((modified ...) (buffer-undo-list t) (inhibit-read-only t) (inhibit-modification-hooks t)) (unwind-protect (progn ... ... ...) (if modified nil ...))) (apply func beg end rest)) cider--debug-mode cider--debug-mode-response t) (beg end &rest rest) (let* ((modified (buffer-modified-p)) (buffer-undo-list t) (inhibit-read-only t) (inhibit-modification-hooks t)) (unwind-protect (progn (remove-text-properties beg end (quote ...)) (add-text-properties beg end (quote ...)) (if cider-font-lock-dynamically (progn ...))) (if modified nil (restore-buffer-modified-p nil)))) (apply func beg end rest)) cider--debug-mode cider--debug-mode-response t) (beg end &rest rest) (let* ((modified (buffer-modified-p)) (buffer-undo-list t) (inhibit-read-only t) (inhibit-modification-hooks t)) (unwind-protect (progn (remove-text-properties beg end (quote (cider-locals nil cider-block-dynamic-font-lock nil))) (add-text-properties beg end (quote (help-echo cider--help-echo))) (if cider-font-lock-dynamically (progn (cider--update-locals-for-region beg end)))) (if modified nil (restore-buffer-modified-p nil)))) (apply func beg end rest))(1 4 nil)
  font-lock-fontify-region(1 4)
  cider-font-lock-as(clojure-mode ":\"|")
  cider-font-lock-as-clojure(":\"|")
  cider-repl-emit-result(#<buffer *cider-repl fizzbuzz*> ":\"|" t)

@expez
Copy link
Member

expez commented Mar 30, 2017

Garbage in, garbage out. I think we should close this, not complicate the code with this strange edge-case, and direct work toward useful features.

@bbatsov
Copy link
Member

bbatsov commented Mar 30, 2017

@expez I agree to some extent, but I think we should at least handle this error, so it doesn't propagate to the users. I think in the case of garbage in we should apply no font-locking. That'd be simple and better UX imo.

@bbatsov
Copy link
Member

bbatsov commented Mar 30, 2017

Probably adding handling for those chars would not be complex either, but I don't have time to look into this.

@bbatsov bbatsov added the bug label Apr 12, 2017
@xiongtx
Copy link
Member

xiongtx commented Mar 14, 2018

While the reader allows it, the spec does not include " as a legal constituent of a symbol or keyword:

Symbols begin with a non-numeric character and can contain alphanumeric characters and *, +, !, -, _, ', and ? (other characters may be allowed eventually)

Keywords are like symbols

But then again, a number of libraries grossly violate the ban against . in keywords:

They cannot contain '.' or name classes.

@WhittlesJr
Copy link

WhittlesJr commented Oct 8, 2018

Maybe this should be a separate issue, but I get the same error when evaluating the following:

(apply str (map char (range 33 41)))

Ran into this because I needed to generate ASCII characters sequentially. I reduced it down to this particular range in this particular order, which is:

!\"#$%&'(

If you start higher than 33 or end lower than 41(40), it will print just fine.

Runs fine on repl.it, and also works fine if you just reverse the sequence.

It will also choke if you just try to evaluate the string itself:

"!\"#$%&'("

And I found that you can strip out all characters except the following to reproduce it:

!\"(

@stale
Copy link

stale bot commented May 8, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contribution and understanding!

@stale stale bot added the stale label May 8, 2019
@stale
Copy link

stale bot commented Jun 7, 2019

This issues been automatically closed due to lack of activity. Feel free to re-open it if you ever come back to it.

@stale stale bot closed this as completed Jun 7, 2019
@pdenno
Copy link

pdenno commented Feb 23, 2020

I get this often when printing a large map or vector to the REPL.
What *Messages* says is:
clojure-string-start: Search failed: "\\(\\s-\\|^\\|#\\|(\\|\\[\\|{\\)\\(\"\\)"
error in process filter: Search failed: "\\(\\s-\\|^\\|#\\|(\\|\\[\\|{\\)\\(\"\\)"

I updated today to clojure-mode-20191112.1948 but it doesn't help. clojure-string-start hasn't changed since at least 20180202.

You should be able to reproduce it with the attached file:
(read-string (slurp "bad.txt))
bad.txt

@harold
Copy link
Contributor

harold commented May 15, 2020

Here is another interesting case of this.

This prints fine for me:

[{:byte-length 38597,
  :directory? false,
  :url "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}
 {:byte-length 38597,
  :directory? false,
  :url "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}
 {:byte-length 38597,
  :directory? false,
  :url "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}
 {:byte-length 38640,
  :directory? false,
  :url "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}
 {:byte-length 38598,
  :directory? false,
  :url "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}
 {:byte-length 38541,
  :directory? false,
  :url "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}
 {:byte-length 38528,
  :directory? false,
  :url "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}
 {:byte-length 38548,
  :directory? false,
  :url "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}]

While this fails:

'({:byte-length 38597,
   :directory? false,
   :url "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}
  {:byte-length 38597,
   :directory? false,
   :url "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}
  {:byte-length 38597,
   :directory? false,
   :url "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}
  {:byte-length 38640,
   :directory? false,
   :url "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}
  {:byte-length 38598,
   :directory? false,
   :url "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}
  {:byte-length 38541,
   :directory? false,
   :url "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}
  {:byte-length 38528,
   :directory? false,
   :url "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}
  {:byte-length 38548,
   :directory? false,
   :url "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"})

The only difference is that the first one (working) is a vector and the second one (failing) is a list.

Seen in *Messages*:

error in process filter: clojure-string-start: Search failed: "\\(\\s-\\|^\\|#\\|(\\|\\[\\|{\\)\\(\"\\)"
error in process filter: Search failed: "\\(\\s-\\|^\\|#\\|(\\|\\[\\|{\\)\\(\"\\)"

Hope that helps.

@pdenno
Copy link

pdenno commented May 16, 2020

Interesting! I still get these "process filter" errors often, but I can't grok the regular expression reported; you can find it in clojure-mode.el. I just don't know what it is trying to do, much less why it should throw an error rather than just give up. Your test cases suggest that it needs to gets engaged in the search to fail. It gets engaged with it with the list owing to \( in the regex. I wonder whether changing \( to \[ and \) to \] would cause the first test case to fail and the second to succeed (reversing the situation).

@bbatsov
Copy link
Member

bbatsov commented May 17, 2020

Yeah, we'll have to see what exactly's wrong with the fontification regex, although this bit of code fontifies fine in clojure-mode.

Btw, I don't recall if I mentioned this, but that's another problem related to cider-repl-use-clojure-font-lock, and it goes away when you set it to nil. The crux of it is that the Clojure font-locking happens by putting the code in a temp clojure-mode buffer and then copying back the font-locked result to the REPL.

@yuhan0 Do you have some time to look into the the regexp in clojure-mode?

@harold
Copy link
Contributor

harold commented May 18, 2020

Hi Both! I appreciate your time and consideration.

Here is the regex in question:

It looks like the elisp function in play here is re-search-backward:

Which has a noerror option that could be used to suppress the error at least.


@bbatsov - is the stuff we're handing over to be fontified sliced on the arbitrary 1kb buffers we were discussing on the other thread (#2628)? If so, there may be assumptions in the clojure-mode code that what is to be fontified is somehow more well-formed than the input we're giving it ends up being in some cases.


Another idea would be to catch the error, maybe here:

  • cider/cider-util.el

    Lines 330 to 332 in 4eebab8

    (defun cider-font-lock-as-clojure (string)
    "Font-lock STRING as Clojure code."
    (cider-font-lock-as 'clojure-mode string))

And then (if possible) do something better than just silently dropping the rest of the output and leaving the REPL in a funny state.


Just thoughts, hope that helps. Thanks, all.

@bbatsov
Copy link
Member

bbatsov commented May 18, 2020

@bbatsov - is the stuff we're handing over to be fontified sliced on the arbitrary 1kb buffers we were discussing on the other thread (#2628)? If so, there may be assumptions in the clojure-mode code that what is to be fontified is somehow more well-formed than the input we're giving it ends up being in some cases.

Yeah, exactly. Funny enough both unrelated issues are related to the same fontification code. :-)

Both options you mentioned are valid approaches to tackle this, although ideally we should see if we can refine the regexp. I'll think about this a bit more.

@harold
Copy link
Contributor

harold commented May 18, 2020

Perfect - let me know if there's more I can do to help.

Fontifying arbitrarily cut snippets seems tough in general - no surprise there are complications in this area.

Keep up the awesome work!

@bbatsov
Copy link
Member

bbatsov commented May 18, 2020

Btw, I made a mistake above - obviously the two issues are quite related. :-) In general both would be addressed by something trying to apply the font-locking only to a single chunk result. I'm also thinking that if we simply check if the form we send for fontification is balanced that would also solve both problems. You can really enter something unbalanced in the REPL anyways. Fix incoming. :-)

@bbatsov
Copy link
Member

bbatsov commented May 18, 2020

I came up with a simple fix that seems to work - if an expression is not balanced we simply don't try font-lock it. As chunks are not balanced - magic... :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants