Consider dropping DeviceEvent in favor of an external library

[Ralith]

In #164, among other changes I introduced DeviceEvent to represent raw input events not associated with any particular window. This allowed applications to access much-needed high resolution unfiltered sensor data e.g. from mice, allowing for interactions such as high-fidelity first-person camera control. However, as the events are not window-related, it is somewhat at odds with the need to limit winit's scope to the task of handling windows.

Further research has since made it apparent to me that raw input can be handled entirely by an external crate without winit's cooperation on Linux and Windows, by creating an independent X or Wayland connection and (so long as winit does not indiscriminately consume events for windows not its own) HWND_MESSAGE window respectively. Such a crate would also be better positioned to handle firmly out-of-scope, but widely required, functionality such as game controller and joystick input.

If/when such a crate is written, winit's scope could therefore be narrowed and more clearly defined by the wholesale removal of non-GUI-related events, replaced with a recommendation to use the external crate. This strikes me as an ideal factoring of the problem.

I'm not sure whether it's appropriate to remove DeviceEvent in the mean time--I don't think it's had time to see much use in the wild yet, so future breakage could be limited by acting quickly, but the information it exposes is important and there is not yet any good alternative crate with which to access it.

[Ralith]

I have put together a prototype of such a crate supporting raw mouse, keyboard, joystick, and gamepad input on Linux+X11, with allowances for the extension of support to other platforms.

One remaining question is how these libraries might be composed on a single thread in a user-friendly and efficient manner.

[elinorbgr]

Note that on wayland, a client obtaining raw device events is voluntarily forbidden for security reasons (these mechanism are typically used by keyloggers and other similar malwares), and the clients must specifically ask the compositor for such access, in a way linked to its window (for example ask a pointer grab for a specific surface).

For the record, there is a protocol extension (link) in the works for providing access to high-precision relative pointer events, but it cannot be decoupled from surfaces as they define the focus. A client cannot receive input events if it is not focused.

So decoupling the input from the windowing would not work under wayland, unless the user is willing to configure udev to allow the winit apps direct access to the devices.

[Ralith]

That is frustrating. I don't suppose it's possible to get wayland to give you that high-precision data in association with focus on a surface managed by another client, with that client's cooperation?

[elinorbgr]

That would be up to the wayland server to allow such a thing, but I don't know of any work being done to allow that, as there are mechanisms being developed to provide a client directly with this high-precision input.

A work-around would be to have your lib ask to be given a wayland display as input during initialization in case of a wayland context (with is what mesa does to setup EGL contextes for example). Once you have this display you can basically do whatever you want.

[Ralith]

Does sharing a wayland display with winit entail sharing the same event queue? If so, how might they avoid inadvertently consuming eachothers' events? My prototype is able to support X11 input well because it is, as far as the X11 server is concerned, a totally independent client.

[elinorbgr]

Nope, wayland includes a mechanism to have more than one event queue, and to precisely decide where each event must be dispatched (a same event can also be sent to more than one event queue if necessary). It also gracefully handles managing the various event queues from different threads.

[Ralith]

Okay, maybe this is still practicable then, if winit's willing to provide access to the underlying handles. I'd also need to be able to track window creation and destruction so as to maintain the request for data, but presumably wayland has something similar to X11's map/unmap events to support that.

The complexity involved in this case still somewhat weakens the case against having support for raw mouse input (at least) built into winit, although it concerns only one backend, and there's still going to be demand for an external library for non-windowing-system-related input such as game controllers.

[elinorbgr]

Given the content of the protocol extension, tracking window creation/deletion may not be needed. It's just that the events will only be received if the focus is on one of the winit windows, but there is no need to explicitly map the cursor events to any window.

Also, winit does already expose access to the wayland display already (as mesa needs it for example).

[Ralith]

Oh, I see! That's not so bad after all, then. There's still the question of composition--polling is easy, but winit currently provides no mechanism to wait on events other than its own. Real futures-rs support could be a solution, or users could simply be expected to run winit on a dedicated thread xor rely on polling for non-winit events.

[tomaka]

Real futures-rs support could be a solution

I'm opposed to this.

[MaikKlein]

Shouldn't winit then drop DeviceEvent and add the specific events to WindowEvent? For example a mouse motion event could be added WindowEvent::MouseMotion(i32, i32).

[LPGhatguy]

I understand the desire to factor out things like gamepad input into a separate library.

I don't like the idea of having to use another crate to get precise mouse deltas. I feel like all of the current WindowEvent and DeviceEvent types should just be rolled into a single Event that exposes as much data as possible. It would have events like CursorMoved and MouseMoved to differentiate between GUI-related and device-related events.

The line between 'device' events and 'window' events is further blurred by devices like keyboards, which can convey input in a number of different ways (keys, characters, composition events) that don't necessarily need to map to the 'GUI' vs 'non-GUI' paradigm.

[Ralith]

@MaikKlein The proposal was to remove the raw input events entirely, retaining only device add/removal, which cannot be added to WindowEvent because there is no WindowId associated with them.

@LPGhatguy

I don't like the idea of having to use another crate to get precise mouse deltas.

Given the incredibly common need for raw mouse motion and the low implementation cost, I can see the case for retaining support for that specific event.

I feel like all of the current WindowEvent and DeviceEvent types should just be rolled into a single Event that exposes as much data as possible. It would have events like CursorMoved and MouseMoved to differentiate between GUI-related and device-related events.

Whether the event enums should be flattened is out of scope of this discussion.

The line between 'device' events and 'window' events is further blurred by devices like keyboards, which can convey input in a number of different ways (keys, characters, composition events) that don't necessarily need to map to the 'GUI' vs 'non-GUI' paradigm.

The window-related-ness of keyboard input in the current implementation is well defined: keyboard events delivered by the windowing system to a particular window, generally due to being focused by a pointer associated with a particular virtual keyboard device, are window-related. Keyboard events gathered directly from the physical devices irrespective of focus, association with pointers, or any other factors are not.

There's a separate question of whether you process those events in terms of scancodes or semantic operations plus complex text input, but that's more or less orthogonal. Only window-related keyboard events need to be handled by an application attempting to play well with the host system and the user's expectations.

[tomaka]

Per discussion on IRC, since raw input requires knowledge of the window on at least wayland, then it's not practical to use an external crate and winit should support raw input like a monolithic crate.

[MaikKlein]

@tomaka Will the API change for https://docs.rs/winit/0.7.1/winit/enum.Event.html? For example will DeviceEvent move into WindowEvent? Or will it stay the way it currently is?