Filtering out ESC sequences on mouse events

[jroimartin]

I'm giving mouse support to gocui and I've found the following problem: when I have an editable buffer and I click it to move the cursor, the event loop receives an EventKey for each byte of the ESC sequence that corresponds to the mouse event. Because it is an editable buffer, these sequences end up being written in the buffer. You can see it in the following screenshot:

Is there a way to filter out this ESC sequences? Or, is it possible to distinguish them from the real key-press events?

Thanks! And keep up the good work! :)

[nsf]

Hm, it's weird that your terminal sends these events without app enabling mouse support. You can try running termbox-go with mouse enabled like in this demo: https://github.com/nsf/termbox-go/blob/master/_demos/paint.go#L84

I mean termbox-go has full mouse support now.

So, I'm confused, do you use termbox.InputMouse mode and still get unrecognized events? What I see in your terminal looks like a very valid extended xterm mouse event. And I've added support for them recently.

Anyways, I need more info. Do you use the latest version of the termbox-go? What input mode are you using? What terminal?

[jroimartin]

Here you have some more info:

• termbox-go version: commit 362329b
• Input mode: termbox.InputAlt | termbox.InputMouse
• Terminal: iTerm2 - emulation xterm-256color + xterm mouse reporting (enabled by default)
• Demo paint.go: when I click anywhere, the demo exits (looks like it receives the ESC key event and exits).
• gocui's mouse support: https://github.com/jroimartin/gocui/blob/master/gui.go#L500-L509

If you need more info, please tell me!

[jroimartin]

BTW It's important to note that the click is recognized by termbox-go. Actually, in gocui I can move the mouse around doing click, but I also receive the ESC sequence as a number of key-press events.

[nsf]

That is an interesting behaviour indeed. iTerm2 - that's mac, right? Can you try different terminals if there are any? Those escape sequences rely pretty much on the fact that terminal will send them all at once. If one byte arrives first (e.g. ESC) and then the next ones, the logic will fail. And it appears to be the case.

There is also a demo in termbox-go: https://github.com/nsf/termbox-go/blob/master/_demos/raw_input.go

Can you try it and see in general what it prints on various events (keys, mouse buttons)?

[nsf]

https://github.com/jroimartin/gocui/blob/master/gui.go#L242-L244

Also I can see that you enable mouse only sometimes, and I tried your _examples/demo.go as-is, it doesn't have mouse enabled right?

[jroimartin]

Yes, indeed, iTerm2 is mac.

Playing with raw_input.go, I get different behaviours:

• Some times it works as expected:

• Other times, I receive this (the previous key events of the sequence are also shown briefly):

On the other hand, Terminal.app (the default terminal app in OSX) also shows the same problem. Actually, my initial screenshot was taken while using Terminal.app.

[jroimartin]

Yes, sorry, you can enable mouse support in _examples/demo.go with:

$ git diff demo.go
diff --git a/_examples/demo.go b/_examples/demo.go
index 01067c7..25703e3 100644
--- a/_examples/demo.go
+++ b/_examples/demo.go
@@ -202,6 +202,7 @@ func main() {
        g.SelBgColor = gocui.ColorGreen
        g.SelFgColor = gocui.ColorBlack
        g.Cursor = true
+       g.Mouse = true

        err = g.MainLoop()
        if err != nil && err != gocui.ErrQuit {

[jroimartin]

I've just tested _example/demo.go (with mouse support) in xterm on Linux and it works as expected, so it is quite clear that iTerm2/Terminal.app must be emulating xterm mouse events in a weird way...

[nsf]

Well, it's not weird, they just don't write them atomically. And termbox-go is pretty simple when it comes to event handling. It feels internal buffer, then tries to read an event out of it. If all ESC sequence recognition fails, it will interpret ESC symbol as ESC key (that's why they are called escape sequences, because they use an actual ESC byte, which is the same as pressing the key). In termbox.InputAlt mode ESC key is disabled and ESC byte means somebody pressed an alt key.

Technically I can add another mode where I will disable both ESC and Alt and I can then wait for more bytes to try to recognize esc sequence before giving up on it. But I don't know. I honestly don't know if there is a better way to handle that problem.

Maybe on macosx though, there is some magic switch which can make terminal input more synchronous. Maybe worth googling something along those lines.

[nsf]

And yes, I tried your demo, it works as expected on linux+xterm.

[mengstr]

So is there any solution or workaround for this issue? Or is mouse-support on OSX borked until further notice?

[nsf]

I don't have OSX, so, can't solve that problem. Well, what I would do if I had it: maybe take a look at other open source terminal apps, perhaps there is some trick to force OSX terminals to send data in chunks instead of as available. I don't know.

[rationull]

This should be resolved after PR #176 which I submitted after commenting on issue #132. I would be interested in knowing if anyone hits further problems, as the delay can be increased if necessary (currently set to 50ms).

[ravern]

Hi, I have this same issue. In the latest version, the delay is set to 100 * time.Millisecond, however, EventKeys are still being sent.

I have tried this in the default Terminal app and in iTerm 2. I also tried increasing the delay in 100 millisecond increments all the way to 1000 but it does not seem to have an effect.

I am running macOS High Sierra 10.13.4 on a 2016 15-inch Macbook Pro.

[rationull]

@ravernkoh when you set the delay up to 1000 ms, do you see the EventKeys after a perceptible 1000 ms delay or do they happen immediately?

I haven't run this in a while but I'm running 10.13.4 now, and I can test it out in my use case later today after work.

[ravern]

They still happen immediately. There isn't a noticeable difference between using 100 * time.Millisecond and 1000 * time.Millisecond. Both still have EventKeys that are sent that look like parts of an EventMouse.

This is tested using _demos/raw_input.go.

[rationull]

@ravernkoh forgive me for asking, but are you sure your change to 1000 ms was included in your build correctly? When I was implementing this fix originally I regularly turned the delay up for testing, and when set to something long (1 or 2 seconds) the delay behaved like a straightforward pause in input handling. IIRC it was a minor pain ensuring the change was picked up by the build system though.

It's possible something has changed that prevents this fix from working at all, although I would be less surprised if 100 ms was simply just not enough of a delay in some situations (since the number is made up and not tested very broadly in the first place).

[ravern]

Ah, I think I found the problem. _demos/raw_input.go uses PollRawEvent to poll for events, not PollEvent. Your code is working as intended @rationull . Tested using

package main

import (
	"fmt"

	termbox "github.com/nsf/termbox-go"
)

func main() {
	if err := termbox.Init(); err != nil {
		panic(err)
	}
	defer termbox.Close()

	termbox.SetInputMode(termbox.InputCurrent | termbox.InputMouse)

	for {
		switch ev := termbox.PollEvent(); ev.Type {
		case termbox.EventKey:
			fmt.Println("key")
			if ev.Key == termbox.KeySpace {
				return
			}
		case termbox.EventMouse:
			fmt.Println("mouse")
		}
	}
}

Apologies for any confusion caused.

On another note, is it possible for this same fix to be applied on PollRawEvent or is it just not supposed to work that way (I might be completely misunderstanding the raw-ness)?

[nsf]

PollRawEvent is for "get me whatever bytes there are and I'll parse them myself". But then I added ParseEvent also. Which is supposed to parse the event the way termbox would. However those timer hacks make it impossible, because things rely on a timer instead of a stream of bytes. That's just how things are in terminals. They won't become any better probably and I usually suggest people to look for something fancier. HTML for example. Especially at the point if you start to care about mouse in your terminal app, just use a real GUI platform. And to be honest I would favor HTML over anything else. It's portable, everyone has a browser, it will run on the phone even and you can make it work remotely almost for free. Web development is quite a breeze these days due to things like typescript and react.

Sorry for my "anti-terminals" ad there. I tend to insert it everywhere I talk about termbox.

[rationull]

+1 the timer hack is distasteful, and it's a shame that the multi-character mouse events don't seem to be delivered atomically on macOS.

It may be possible to implement the same kind of thing in PollRawEvent but it's dicey enough even in PollEvent to differentiate between keyboard and mouse events; I'm not sure how we would meaningfully differentiate the delayed events in ParseEvent unless the calling code somehow guaranteed that, in the case of mouse events, the entire event string is provided. To make this guarantee, PollRawEvent would have to be polluted with logic to partially parse the event. Further, delays seem inappropriate in a "raw" parsing function.

I doubt @nsf would accept a PR even if I did adapt this fix to PollRawEvent, I know I wouldn't in the same position, given the previous comments on terminal UI :)

@ravernkoh hopefully you can resolve what you need with PollEvent instead!

[nsf]

Well, if you want to you can make it work. Here's the thing. PollRawEvent is allowed to return any stream of bytes. So, in theory you can encode messages (original terminal bytes) and the time you got them (or whatever other info you need there to figure out the actual events) into that stream of raw input bytes and then decode that in ParseEvent. The question is whether somebody needs it or not.

[rationull]

Good point. But the raw events would still have to be polled by the caller with some knowledge of whether a delay is needed which amounts to partial parsing (or just always delay which seems worse). Seems like a bridge too far to me. I would be curious whether @ravernkoh's use case is solvable another way.

To be clear I'm not specifically interested in implementing this. It's possible I could be convinced as a recreational project if it's super helpful (or could provide input to @ravernkoh on a solution) but in general I find your argument about terminal UIs convincing.