Improve interop touches by using UIScrollView-like strategy #1440

elijah-semyonov · 2024-07-11T11:52:20Z

Description

In this approach CMPGestureRecognizer delays touchesBegan until explicitly failed, and is required to fail by UIGestureRecognizer of children views (aka interop views). Failure happens after the first touch started and no motion above scroll slop happens within 150ms. If this happens, intercepted touches are delivered to children views (and their gesture recognisers), Compose itself gets all tracked touches as cancelled and ignores them until the touch sequence ends (imposed by UIKit).
This behavior is inspired by UIScrollView implementation.

Screen.Recording.2024-07-11.at.13.46.48.mov

Fixes

Improves behavior of touches in certain scenarios

Testing

This should be tested by QA

Release Notes

iOS - Features

Improvements in touches processing to detect if touches were meant to be delivered to interop views, or should be processed by Compose.

ASalavei · 2024-07-11T14:16:49Z

compose/ui/ui-uikit/src/uikitMain/objc/CMPUIKitUtils/CMPUIKitUtils/CMPGestureRecognizer.m

+- (void)handleStateChange {
+    switch (self.state) {
+        case UIGestureRecognizerStateBegan:
+            NSLog(@"state = Began");


it seems like logs should be removed

ASalavei · 2024-07-11T14:17:42Z

compose/ui/ui-uikit/src/uikitMain/objc/CMPUIKitUtils/CMPUIKitUtils/CMPGestureRecognizer.h

 @protocol CMPGestureRecognizerHandler <NSObject>
-
+ 


Sry, it looks like here some formatting issues with spaces.

ASalavei · 2024-07-11T14:21:18Z

compose/ui/ui-uikit/src/uikitMain/objc/CMPUIKitUtils/CMPUIKitUtils/CMPGestureRecognizer.m

+    _scheduledFailureBlock = dispatchBlock;
+
+    // Calculate the delay time in dispatch_time_t
+    dispatch_time_t delay = dispatch_time(DISPATCH_TIME_NOW, (int64_t)(0.15 * NSEC_PER_SEC));


Please, move 0.15 to constants. Or add some meaningful comment because it looks like it require some explanation.

Moved to a local variable with an explanation

ASalavei · 2024-07-11T14:41:18Z

compose/ui/ui/src/uikitMain/kotlin/androidx/compose/ui/window/InteractionUIView.uikit.kt

+     * [CMPGestureRecognizer.scheduleFailure], which will pass intercepted touches to the interop
+     * views if the gesture recognizer is not recognized within a certain time frame
+     * (UIScrollView reverse-engineered 150ms is used).
+     * The similar approach is used by [UIScrollView](https://developer.apple.com/documentation/uikit/uiscrollview?language=objc)


Link suggestion: https://developer.apple.com/documentation/uikit/uiscrollview

Aha, changed

ASalavei · 2024-07-11T15:54:35Z

compose/ui/ui/src/uikitMain/kotlin/androidx/compose/ui/window/InteractionUIView.uikit.kt

+     * scroll and compose horizontal scroll with different fingers)
+     *
+     * 4. Those are not the first touches in the sequence. A gesture is not recognized.
+     * See if centroid of the tracked touches has moved enough to recognize the gesture.


I'm worry regarding this scenario. The second touch will cause significant offset of centroid, and with high probability it will trigger pan gesture, instead of multi-touch gesture, like pinch zoom. Just wonder if it will be a problem on interop maps.

If second touch lands after the gesture was failed, it's not even delivered to it, so the gesture will properly be recognised by maps, for example.

I'm afraid this might break the behavior of detectTransformGestures. On 1.7.0-alpha02, it seemed detectTransformGestures always call onGesture with zoom = 1.0f on iOS.
I have create an issue on youtrack: https://youtrack.jetbrains.com/issue/CMP-5879/detectTransformGestures-doesnt-work-properly-on-iOS-with-Compose-Multiplatform-1.7.0-alpha02

Hi, thanks, already looking into it.

ASalavei · 2024-07-11T16:00:31Z

compose/ui/ui/src/uikitMain/kotlin/androidx/compose/ui/window/InteractionUIView.uikit.kt

+     * location to null.
+     */
+    private fun stopTrackingTouches(touches: Set<*>) {
+        onTouchesCountChanged(-touches.size)


I would move this call on after we actually remove touches from trackedTouches

They are functionally equivalent now, because effects of this call are dispatched later on main thread by OS itself (display link changes). But let's do it for sake of consistency with onFailure.

ASalavei · 2024-07-11T16:00:42Z

compose/ui/ui/src/uikitMain/kotlin/androidx/compose/ui/window/InteractionUIView.uikit.kt

+     * @return `true` if the touches are initial, `false` otherwise.
+     */
+    private fun startTrackingTouches(touches: Set<*>): Boolean {
+        onTouchesCountChanged(touches.size)


I would move this call on after we actually remove touches from trackedTouches

ASalavei · 2024-07-11T16:02:00Z

compose/ui/ui/src/uikitMain/kotlin/androidx/compose/ui/window/InteractionUIView.uikit.kt

+     */
+    var gestureRecognizer: CMPGestureRecognizer? = null
+
+    private var state: UIGestureRecognizerState


Name suggestion: gestureRecognizerState

MatkovIvan · 2024-07-11T16:59:17Z

compose/ui/ui/src/uikitMain/kotlin/androidx/compose/ui/window/InteractionUIView.uikit.kt

+            val initialLocation = initialLocation ?: return false
+            val centroidLocation = trackedTouchesCentroidLocation ?: return false
+
+            val slop = 10.0


As far as I remember we already had such constant. Can it be reused?

Extracted to CUPERTINO_PAN_GESTURE_SLOP_VALUE

MatkovIvan · 2024-07-11T16:59:45Z

compose/ui/ui/src/uikitMain/kotlin/androidx/compose/ui/window/InteractionUIView.uikit.kt

+            val dx = centroidLocation.useContents { x - initialLocation.useContents { x } }
+            val dy = centroidLocation.useContents { y - initialLocation.useContents { y } }
+
+            return dx * dx + dy * dy > slop * slop


A good candidate for utility/helper function

Comparing squared lengths?
I doubt it happens again, let it be here for now.

MatkovIvan · 2024-07-11T17:01:41Z

compose/ui/ui/src/uikitMain/kotlin/androidx/compose/ui/window/InteractionUIView.uikit.kt

+     * that it can be used later to determine if the gesture recognizer should be recognized
+     * or failed.
+     */
+    private fun rememberHitTestResult(hitTestBlock: () -> UIView?): UIView? {


rememberX has very specific semantic in Compose, please rename to avoid confusions

MatkovIvan

The main change/delay logic LGTM, I just have a couple of nitpicks

MatkovIvan · 2024-07-12T08:53:38Z

compose/ui/ui/src/uikitMain/kotlin/androidx/compose/ui/platform/Constants.uikit.kt

+/**
+ * iOS default value in scale-independent points for the pan gesture slop.
+ */
+internal const val CUPERTINO_PAN_GESTURE_SLOP_VALUE = 10


Let's make it typed and align naming (it's used not only in "pan" gestures)

Suggested change

internal const val CUPERTINO_PAN_GESTURE_SLOP_VALUE = 10

internal const val DefaultCupertinoTouchSlop = 10.dp

No DP here, new logic uses CGFloat(aka Double)

Just get raw value from dp there - we need to keep such things typed

It doesn't work well here.
Dp is not primitive - turns into just val instead of const val.
Let's leave it as it is. Renamed to CUPERTINO_TOUCH_SLOP. It's not Default, it's what it is.

MatkovIvan · 2024-07-12T08:59:58Z

compose/ui/ui-uikit/src/uikitMain/objc/CMPUIKitUtils/CMPUIKitUtils/CMPGestureRecognizer.m

+    // 150ms is a timer delay for notifying a handler that the gesture was failed to recognize.
+    // `handler` implementtion is responsible for cancelling this via calling `cancelFailure` and transitioning
+    // this gesture recognizer to a proper state.
+    double failureInterval = 0.15;


Nitpick: mixing ms and s

Nitpick indeed

compose/ui/ui-uikit/src/uikitMain/objc/CMPUIKitUtils/CMPUIKitUtils/CMPGestureRecognizer.m

compose/ui/ui/src/uikitMain/kotlin/androidx/compose/ui/window/InteractionUIView.uikit.kt

…tils/CMPGestureRecognizer.m Co-authored-by: Ivan Matkov <ivan.matkov@jetbrains.com>

…InteractionUIView.uikit.kt Co-authored-by: Ivan Matkov <ivan.matkov@jetbrains.com>

elijah-semyonov added 13 commits July 9, 2024 14:32

Move state logic to Kotlin

2e7f94e

Setup golden path

0df0c23

Setup golden path

8b0a412

Manually stop tracking when failing

200df35

Minor refactor

f4f0ebd

Minor refactor

3554031

Setup failure scheduling

0f6c275

Add doc

5740501

Minor change

f286942

Minor change

c0e22b8

Implement everything

f5bfacc

Remove prints

b5b672d

Remove prints

f97fc46

elijah-semyonov requested review from MatkovIvan and ASalavei July 11, 2024 11:52

elijah-semyonov self-assigned this Jul 11, 2024

elijah-semyonov added 2 commits July 11, 2024 13:59

Remove prints

1678443

Remove redundant import

3a08659

elijah-semyonov marked this pull request as ready for review July 11, 2024 12:00

Remove comment

733fb01

ASalavei reviewed Jul 11, 2024

View reviewed changes

MatkovIvan reviewed Jul 11, 2024

View reviewed changes

elijah-semyonov added 2 commits July 12, 2024 10:13

Remove debug prints

40d8d03

Remove spaces

288b3f9

elijah-semyonov added 6 commits July 12, 2024 10:19

Add comment

82dde9f

Change the link

cc88ed0

Move expressions for consistency

d5caf40

Rename for clarity

9c9fa41

Extract CUPERTINO_PAN_GESTURE_SLOP_VALUE

f2ebb8f

Change name

325473f

MatkovIvan reviewed Jul 12, 2024

View reviewed changes

elijah-semyonov and others added 7 commits July 12, 2024 11:10

Update compose/ui/ui-uikit/src/uikitMain/objc/CMPUIKitUtils/CMPUIKitU…

65338d0

…tils/CMPGestureRecognizer.m Co-authored-by: Ivan Matkov <ivan.matkov@jetbrains.com>

Rename

65b8397

Modify

ced8856

Update compose/ui/ui/src/uikitMain/kotlin/androidx/compose/ui/window/…

7d59be2

…InteractionUIView.uikit.kt Co-authored-by: Ivan Matkov <ivan.matkov@jetbrains.com>

Update compose/ui/ui/src/uikitMain/kotlin/androidx/compose/ui/window/…

db4cc40

…InteractionUIView.uikit.kt Co-authored-by: Ivan Matkov <ivan.matkov@jetbrains.com>

Rename

ac5a031

Fix typo

92ea0a3

MatkovIvan approved these changes Jul 12, 2024

View reviewed changes

elijah-semyonov merged commit 174766b into jb-main Jul 12, 2024
6 checks passed

elijah-semyonov deleted the es/interop-gestures-improvements branch July 12, 2024 10:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve interop touches by using UIScrollView-like strategy #1440

Improve interop touches by using UIScrollView-like strategy #1440

elijah-semyonov commented Jul 11, 2024 •

edited

Loading

ASalavei Jul 11, 2024

elijah-semyonov Jul 12, 2024

ASalavei Jul 11, 2024

elijah-semyonov Jul 12, 2024

ASalavei Jul 11, 2024

elijah-semyonov Jul 12, 2024

ASalavei Jul 11, 2024

elijah-semyonov Jul 12, 2024

ASalavei Jul 11, 2024

elijah-semyonov Jul 12, 2024

Ponyets Aug 2, 2024

elijah-semyonov Aug 2, 2024

ASalavei Jul 11, 2024

elijah-semyonov Jul 12, 2024

ASalavei Jul 11, 2024

elijah-semyonov Jul 12, 2024

ASalavei Jul 11, 2024

elijah-semyonov Jul 12, 2024

MatkovIvan Jul 11, 2024

elijah-semyonov Jul 12, 2024

MatkovIvan Jul 11, 2024

elijah-semyonov Jul 12, 2024

MatkovIvan Jul 11, 2024

elijah-semyonov Jul 12, 2024

MatkovIvan left a comment

MatkovIvan Jul 12, 2024

elijah-semyonov Jul 12, 2024

MatkovIvan Jul 12, 2024

elijah-semyonov Jul 12, 2024 •

edited

Loading

MatkovIvan Jul 12, 2024

elijah-semyonov Jul 12, 2024

	internal const val CUPERTINO_PAN_GESTURE_SLOP_VALUE = 10
	internal const val DefaultCupertinoTouchSlop = 10.dp

Improve interop touches by using UIScrollView-like strategy #1440

Improve interop touches by using UIScrollView-like strategy #1440

Conversation

elijah-semyonov commented Jul 11, 2024 • edited Loading

Description

Fixes

Testing

Release Notes

iOS - Features

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MatkovIvan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elijah-semyonov Jul 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elijah-semyonov commented Jul 11, 2024 •

edited

Loading

elijah-semyonov Jul 12, 2024 •

edited

Loading