Basic multi touch support (issue #279) (#306)

* translate touch events from glium to egui

Unfortunately, winit does not seem to create _Touch_ events for the touch pad
on my mac.  Only _TouchpadPressure_ events are sent.

Found some issues (like
[this](rust-windowing/winit#54)), but I am not sure
what they exactly mean:  Sometimes, touch events are mixed with
touch-to-pointer translation in the discussions.

* translate touch events from web_sys to egui

The are a few open topics:
- egui_web currently translates touch events into pointer events.
  I guess this should change, such that egui itself performs this kind of
  conversion.
- `pub fn egui_web::pos_from_touch_event` is a public function, but I
  would like to change the return type to an `Option`.  Shouldn't this
  function be private, anyway?

* introduce `TouchState` and `Gesture`

InputState.touch was introduced with type `TouchState`, just as
InputState.pointer is of type `Pointer`.

The TouchState internally relies on a collection of `Gesture`s.  This commit
provides the first rudimentary implementation of a Gesture, but has no
functionality, yet.

* add method InputState::zoom()

So far, the method always returns `None`, but it should work as soon as the
`Zoom` gesture is implemented.

* manage one `TouchState` per individual device

Although quite unlikely, it is still possible to connect more than one touch
device. (I have three touch pads connected to my MacBook in total, but
unfortunately `winit` sends touch events for none of them.)

We do not want to mix-up the touches from different devices.

* implement control loop for gesture detection

The basic idea is that each gesture can focus on detection logic and does not
have to care (too much) about managing touch state in general.

* streamline `Gesture` trait, simplifying impl's

* implement first version of Zoom gesture

* fix failing doctest

a simple `TODO` should be enough

* get rid of `Gesture`s

* Provide a Zoom/Rotate window in the demo app

For now, it works for two fingers only.  The third finger interrupts the
gesture.

Bugs:
- Pinching in the demo window also moves the window -> Pointer events must be
  ignored when touch is active
- Pinching also works when doing it outside the demo window -> it would be nice
  to return the touch info in the `Response` of the painter allocation

* fix comments and non-idiomatic code

* update touch state *each frame*

* change egui_demo to use *relative* touch data

* support more than two fingers

This commit includes an improved Demo Window for egui_demo, and a complete
re-write of the gesture detection.  The PR should be ready for review, soon.

* cleanup code and comments for review

* minor code simplifications

* oops – forgot the changelog

* resolve comment https://github.com/emilk/egui/pull/306/files/fee8ed83dbe715b5b70433faacfe74b59c99e4a4#r623226656

* accept suggestion #306 (comment)

Co-authored-by: Emil Ernerfeldt <emil.ernerfeldt@gmail.com>

* fix syntax error (dough!)

* remove `dbg!` (why didnt clippy see this?)

* apply suggested diffs from review

* fix conversion of physical location to Pos2

* remove redundanct type `TouchAverages`

* remove trailing space

* avoid initial translation jump in plot demo

* extend the demo so it shows off translation

Co-authored-by: Emil Ernerfeldt <emil.ernerfeldt@gmail.com>

CHANGELOG.md

@@ -15,6 +15,9 @@ NOTE: [`eframe`](eframe/CHANGELOG.md), [`egui_web`](egui_web/CHANGELOG.md) and [
 * [Pan and zoom plots](https://github.com/emilk/egui/pull/317).
 * [Users can now store custom state in `egui::Memory`.](https://github.com/emilk/egui/pull/257).
 * Zoom input: ctrl-scroll and (on `egui_web`) trackpad-pinch gesture.
+* Support for raw [multi touch](https://github.com/emilk/egui/pull/306) events,
+  enabling zoom, rotate, and more.  Works with `egui_web` on mobile devices,
+  and should work with `egui_glium` for certain touch devices/screens.
 
 ### Changed 🔧
 * Make `Memory::has_focus` public (again).

egui/src/data/input.rs

@@ -96,7 +96,7 @@ impl RawInput {
 /// An input event generated by the integration.
 ///
 /// This only covers events that egui cares about.
-#[derive(Clone, Debug, Eq, PartialEq)]
+#[derive(Clone, Debug, PartialEq)]
 pub enum Event {
     /// The integration detected a "copy" event (e.g. Cmd+C).
     Copy,
@@ -133,6 +133,22 @@ pub enum Event {
     CompositionUpdate(String),
     /// IME composition ended with this final result.
     CompositionEnd(String),
+
+    Touch {
+        /// Hashed device identifier (if available; may be zero).
+        /// Can be used to separate touches from different devices.
+        device_id: TouchDeviceId,
+        /// Unique identifier of a finger/pen.  Value is stable from touch down
+        /// to lift-up
+        id: TouchId,
+        phase: TouchPhase,
+        /// Position of the touch (or where the touch was last detected)
+        pos: Pos2,
+        /// Describes how hard the touch device was pressed. May always be `0` if the platform does
+        /// not support pressure sensitivity.
+        /// The value is in the range from 0.0 (no pressure) to 1.0 (maximum pressure).
+        force: f32,
+    },
 }
 
 /// Mouse button (or similar for touch input)
@@ -296,3 +312,47 @@ impl RawInput {
             .on_hover_text("key presses etc");
     }
 }
+
+/// this is a `u64` as values of this kind can always be obtained by hashing
+#[derive(Clone, Copy, Debug, Eq, PartialEq, PartialOrd, Ord)]
+pub struct TouchDeviceId(pub u64);
+
+/// Unique identifiction of a touch occurence (finger or pen or ...).
+/// A Touch ID is valid until the finger is lifted.
+/// A new ID is used for the next touch.
+#[derive(Clone, Copy, Debug, Eq, PartialEq, PartialOrd, Ord)]
+pub struct TouchId(pub u64);
+
+#[derive(Clone, Copy, Debug, Eq, PartialEq)]
+pub enum TouchPhase {
+    /// User just placed a touch point on the touch surface
+    Start,
+    /// User moves a touch point along the surface.  This event is also sent when
+    /// any attributes (position, force, ...) of the touch point change.
+    Move,
+    /// User lifted the finger or pen from the surface, or slid off the edge of
+    /// the surface
+    End,
+    /// Touch operation has been disrupted by something (various reasons are possible,
+    /// maybe a pop-up alert or any other kind of interruption which may not have
+    /// been intended by the user)
+    Cancel,
+}
+
+impl From<u64> for TouchId {
+    fn from(id: u64) -> Self {
+        Self(id)
+    }
+}
+
+impl From<i32> for TouchId {
+    fn from(id: i32) -> Self {
+        Self(id as u64)
+    }
+}
+
+impl From<u32> for TouchId {
+    fn from(id: u32) -> Self {
+        Self(id as u64)
+    }
+}

egui/src/input_state.rs

@@ -1,8 +1,12 @@
+mod touch_state;
+
 use crate::data::input::*;
 use crate::{emath::*, util::History};
-use std::collections::HashSet;
+use std::collections::{BTreeMap, HashSet};
 
 pub use crate::data::input::Key;
+pub use touch_state::MultiTouchInfo;
+use touch_state::TouchState;
 
 /// If the pointer moves more than this, it is no longer a click (but maybe a drag)
 const MAX_CLICK_DIST: f32 = 6.0; // TODO: move to settings
@@ -15,9 +19,13 @@ pub struct InputState {
     /// The raw input we got this frame from the backend.
     pub raw: RawInput,
 
-    /// State of the mouse or touch.
+    /// State of the mouse or simple touch gestures which can be mapped to mouse operations.
     pub pointer: PointerState,
 
+    /// State of touches, except those covered by PointerState (like clicks and drags).
+    /// (We keep a separate `TouchState` for each encountered touch device.)
+    touch_states: BTreeMap<TouchDeviceId, TouchState>,
+
     /// How many pixels the user scrolled.
     pub scroll_delta: Vec2,
 
@@ -55,6 +63,7 @@ impl Default for InputState {
         Self {
             raw: Default::default(),
             pointer: Default::default(),
+            touch_states: Default::default(),
             scroll_delta: Default::default(),
             screen_rect: Rect::from_min_size(Default::default(), vec2(10_000.0, 10_000.0)),
             pixels_per_point: 1.0,
@@ -70,7 +79,7 @@ impl Default for InputState {
 
 impl InputState {
     #[must_use]
-    pub fn begin_frame(self, new: RawInput) -> InputState {
+    pub fn begin_frame(mut self, new: RawInput) -> InputState {
         #![allow(deprecated)] // for screen_size
 
         let time = new
@@ -84,6 +93,10 @@ impl InputState {
                 self.screen_rect
             }
         });
+        self.create_touch_states_for_new_devices(&new.events);
+        for touch_state in self.touch_states.values_mut() {
+            touch_state.begin_frame(time, &new, self.pointer.interact_pos);
+        }
         let pointer = self.pointer.begin_frame(time, &new);
         let mut keys_down = self.keys_down;
         for event in &new.events {
@@ -97,6 +110,7 @@ impl InputState {
         }
         InputState {
             pointer,
+            touch_states: self.touch_states,
             scroll_delta: new.scroll_delta,
             screen_rect,
             pixels_per_point: new.pixels_per_point.unwrap_or(self.pixels_per_point),
@@ -121,7 +135,13 @@ impl InputState {
     /// * `zoom > 1`: pinch spread
     #[inline(always)]
     pub fn zoom_delta(&self) -> f32 {
-        self.raw.zoom_delta
+        // If a multi touch gesture is detected, it measures the exact and linear proportions of
+        // the distances of the finger tips.  It is therefore potentially more accurate than
+        // `raw.zoom_delta` which is based on the `ctrl-scroll` event which, in turn, may be
+        // synthesized from an original touch gesture.
+        self.multi_touch()
+            .map(|touch| touch.zoom_delta)
+            .unwrap_or(self.raw.zoom_delta)
     }
 
     pub fn wants_repaint(&self) -> bool {
@@ -188,6 +208,52 @@ impl InputState {
         // TODO: multiply by ~3 for touch inputs because fingers are fat
         self.physical_pixel_size()
     }
+
+    /// Returns details about the currently ongoing multi-touch gesture, if any.  Note that this
+    /// method returns `None` for single-touch gestures (click, drag, …).
+    ///
+    /// ```
+    /// # use egui::emath::Rot2;
+    /// # let ui = &mut egui::Ui::__test();
+    /// let mut zoom = 1.0; // no zoom
+    /// let mut rotation = 0.0; // no rotation
+    /// if let Some(multi_touch) = ui.input().multi_touch() {
+    ///     zoom *= multi_touch.zoom_delta;
+    ///     rotation += multi_touch.rotation_delta;
+    /// }
+    /// let transform = zoom * Rot2::from_angle(rotation);
+    /// ```
+    ///
+    /// By far not all touch devices are supported, and the details depend on the `egui`
+    /// integration backend you are using.  `egui_web` supports multi touch for most mobile
+    /// devices, but not for a `Trackpad` on `MacOS`, for example.  The backend has to be able to
+    /// capture native touch events, but many browsers seem to pass such events only for touch
+    /// _screens_, but not touch _pads._
+    ///
+    /// Refer to [`MultiTouchInfo`] for details about the touch information available.
+    ///
+    /// Consider using `zoom_delta()` instead of `MultiTouchInfo::zoom_delta` as the former
+    /// delivers a synthetic zoom factor based on ctrl-scroll events, as a fallback.
+    pub fn multi_touch(&self) -> Option<MultiTouchInfo> {
+        // In case of multiple touch devices simply pick the touch_state of the first active device
+        if let Some(touch_state) = self.touch_states.values().find(|t| t.is_active()) {
+            touch_state.info()
+        } else {
+            None
+        }
+    }
+
+    /// Scans `events` for device IDs of touch devices we have not seen before,
+    /// and creates a new `TouchState` for each such device.
+    fn create_touch_states_for_new_devices(&mut self, events: &[Event]) {
+        for event in events {
+            if let Event::Touch { device_id, .. } = event {
+                self.touch_states
+                    .entry(*device_id)
+                    .or_insert_with(|| TouchState::new(*device_id));
+            }
+        }
+    }
 }
 
 // ----------------------------------------------------------------------------
@@ -517,6 +583,7 @@ impl InputState {
         let Self {
             raw,
             pointer,
+            touch_states,
             scroll_delta,
             screen_rect,
             pixels_per_point,
@@ -537,6 +604,12 @@ impl InputState {
                 pointer.ui(ui);
             });
 
+        for (device_id, touch_state) in touch_states {
+            ui.collapsing(format!("Touch State [device {}]", device_id.0), |ui| {
+                touch_state.ui(ui)
+            });
+        }
+
         ui.label(format!("scroll_delta: {:?} points", scroll_delta));
         ui.label(format!("screen_rect: {:?} points", screen_rect));
         ui.label(format!(