Touches forwarding to underlying interop views on iOS #1426

elijah-semyonov · 2024-07-02T08:05:46Z

First approach to allow touches to be processed by both Compose and nested interop views.
Other UX improvements will follow.

actual typealias InteropView = UIView
Change ComposeScene API to allow hit testing specific InteropView.
Make InteractionUIView a container view of InteropContainer to enforce it being in the same responder chain as interop views.
Remove InteractionUIView.Delegate and replace it with callbacks to avoid by lazy reentry in ComposeSceneMediator, make InteractionUIView construction eager.
Use event.allTouches instead of event.touchesForView to avoid receiving an empty list in case InteractionUIView is part of responder chain but not the hit test result itself.
Remove KeyboardEventHandler.uikit.kt to avoid by lazy reentry, stick everything in a single lambda.

Fixes some of the cases from the domain of JetBrains/compose-multiplatform#4818

Testing

Demo/IosBugs/UIKitRenderSync now properly registers touches to allow LazyColumn scrolling and native view reaction.

Release Notes

iOS - Fixes

Touches inside interop views are not exclusive to them and are processed on Compose side as well.

Multiple Platforms - Breaking changes

ComposeScene fun hitTestInteropView(position: Offset): Boolean changed to fun hitTestInteropView(position: Offset): InteropView?

...ui/src/skikoMain/kotlin/androidx/compose/ui/input/pointer/InteropViewCatchPointerModifier.kt

compose/ui/ui/src/skikoMain/kotlin/androidx/compose/ui/node/RootNodeOwner.skiko.kt

compose/ui/ui/src/skikoMain/kotlin/androidx/compose/ui/scene/ComposeScene.skiko.kt

compose/ui/ui/src/uikitMain/kotlin/androidx/compose/ui/viewinterop/InteropView.uikit.kt

compose/ui/ui/src/skikoMain/kotlin/androidx/compose/ui/viewinterop/InteropView.skiko.kt

MatkovIvan

LGTM 👍 Just a few docs nitpicks

compose/ui/ui/src/skikoMain/kotlin/androidx/compose/ui/scene/ComposeScene.skiko.kt

compose/ui/ui/src/skikoMain/kotlin/androidx/compose/ui/viewinterop/InteropView.skiko.kt

MatkovIvan · 2024-07-02T12:20:41Z

compose/ui/ui/src/skikoMain/kotlin/androidx/compose/ui/viewinterop/InteropView.skiko.kt

+ */
+@InternalComposeUiApi
+fun Modifier.interopView(view: InteropView): Modifier =


The name is too generic. I assume it will show up in suggestions for unrelated cases

Any suggestions? I guess it's exactly what it is, an InteropView association for a given element.

modifier is no a VIEW. So at least something like interopViewAnchor.

Wait. We don't need this function in public at all - let's just keep a node. (InteropViewAnchorModifierNode)

compose/ui/ui/src/uikitMain/kotlin/androidx/compose/ui/interop/UIKitView.uikit.kt

ASalavei · 2024-07-02T16:12:16Z

compose/ui/ui/src/uikitMain/kotlin/androidx/compose/ui/window/InteractionUIView.uikit.kt

-    private var updateTouchesCount: (count: Int) -> Unit,
+    private var hitTestInteropView: (point: CValue<CGPoint>, event: UIEvent?) -> InteropView?,
+    private var onTouchesEvent: (view: UIView, event: UIEvent, phase: CupertinoTouchesPhase) -> Unit,
+    private var onTouchesCountChange: (count: Int) -> Unit,
    private var inBounds: (CValue<CGPoint>) -> Boolean,


I would consider some simplification of the API here:

Move all calls to delegate-like interface.

Omit inBounds, and move hitTest logic out to the class to its owner.

I removed delegate specifically to avoid instantiation of objects that capture this.
Due to heavy coupling of dependencies in ComposeSceneMediator it's already a problem to figure out what's the order of lazy instantiations and I've tried this approach before I efficiently just made this. Delegates implementation required wrapping coupled objects access into lambda to avoid lazy-reentry, which just makes things way too complicated. But I agree with the second point, we can make those things easier. Let's do it in a separate PR?

ASalavei · 2024-07-02T16:15:48Z

...e/ui/ui/src/uikitMain/kotlin/androidx/compose/ui/window/IntermediateTextInputUIView.uikit.kt

@@ -87,7 +87,7 @@ internal class IntermediateTextInputUIView(
                hideEditMenu()
            }
        }
-    var keyboardEventHandler: KeyboardEventHandler? = null
+    var onPresses: (Set<*>) -> Unit = NoOpOnPresses


Are presses can be done from another source, different from the attached keyboard?
I would keep keyboard naming part in API to have more context when reading the code.

Yes, hardware keyboard too, afaik.
Anyway, a good point, will add it.

elijah-semyonov · 2024-07-03T10:03:11Z

I implied, that simple touches forwarding is needed, but some views like MKMapView and UITextField use internal gesture recognisers, so we have to use one too.

elijah-semyonov · 2024-07-03T10:05:55Z

Screen.Recording.2024-07-03.at.12.04.39.mov

Not ideal, but it works for now

MatkovIvan

LGTM, let's just fine tune the naming

MatkovIvan · 2024-07-09T08:48:42Z

compose/ui/ui/src/skikoMain/kotlin/androidx/compose/ui/viewinterop/InteropView.skiko.kt

+ */
+@InternalComposeUiApi
+fun Modifier.interopView(view: InteropView): Modifier =


modifier is no a VIEW. So at least something like interopViewAnchor.

Wait. We don't need this function in public at all - let's just keep a node. (InteropViewAnchorModifierNode)

MatkovIvan · 2024-07-09T08:49:08Z

...ui/ui/src/skikoMain/kotlin/androidx/compose/ui/viewinterop/InteropViewFactoryHolder.skiko.kt

@@ -18,6 +18,7 @@ package androidx.compose.ui.viewinterop

 import androidx.compose.runtime.ComposeNodeLifecycleCallback

+// TODO: implement this for iOS


Please add a link to this task

compose/ui/ui/src/uikitMain/kotlin/androidx/compose/ui/interop/UIKitView.uikit.kt

compose/ui/ui/src/uikitMain/kotlin/androidx/compose/ui/window/InteractionUIView.uikit.kt

…/UIKitView.uikit.kt Co-authored-by: Ivan Matkov <ivan.matkov@jetbrains.com>

…that could be unchanged)

This reverts commit f2777cb

elijah-semyonov added the changes in API label Jul 2, 2024

elijah-semyonov self-assigned this Jul 2, 2024

elijah-semyonov requested review from MatkovIvan and ASalavei July 2, 2024 08:06

elijah-semyonov changed the title ~~Touches forwarding to underlying interop views~~ Touches forwarding to underlying interop views on iOS Jul 2, 2024

MatkovIvan reviewed Jul 2, 2024

View reviewed changes

compose/ui/ui/src/skikoMain/kotlin/androidx/compose/ui/viewinterop/InteropView.skiko.kt Show resolved Hide resolved

elijah-semyonov marked this pull request as ready for review July 2, 2024 10:30

MatkovIvan reviewed Jul 2, 2024

View reviewed changes

ASalavei reviewed Jul 2, 2024

View reviewed changes

elijah-semyonov marked this pull request as draft July 3, 2024 09:48

elijah-semyonov marked this pull request as ready for review July 3, 2024 09:49

elijah-semyonov marked this pull request as draft July 3, 2024 10:02

elijah-semyonov marked this pull request as ready for review July 3, 2024 10:05

elijah-semyonov requested review from MatkovIvan and ASalavei July 3, 2024 10:06

MatkovIvan reviewed Jul 9, 2024

View reviewed changes

elijah-semyonov force-pushed the es/gesture-recognizer-touch-forwarding branch from 3dab05d to 66f99e1 Compare July 9, 2024 09:00

MatkovIvan approved these changes Jul 9, 2024

View reviewed changes

elijah-semyonov added 8 commits July 9, 2024 11:55

Add CMPGestureRecognizer stubs

d81095c

Process touches via gesture recognizers

7d0e2f7

Nullify proxy

3d48f10

Return interop pointer back

e921072

Change hit-testing logic

23adafd

Fix bug

0e6ce54

Modify sample

62fb802

Update internal state on gesture recognizer

b1d7426

elijah-semyonov and others added 24 commits July 9, 2024 11:56

Remove interop view

a5dc04b

Minor rename

cb756e4

Make InteractionView an interopContainerView

0954971

Rewire callbacks to avoid lazy reentry

9035039

Add stub for shouldRecognizeSimultaneously

46c3b56

Add @ExperimentalComposeUiApi

2fa9cc9

Use allTouches instead of touchesForView

79ba88a

Remove useless println

d66bed4

Temporarily deintegrate gesture recognizer proxying

f61cbc3

Update doc

a6c33f3

Refactor interop API

74c9319

Update compose/ui/ui/src/uikitMain/kotlin/androidx/compose/ui/interop…

f8dfa14

…/UIKitView.uikit.kt Co-authored-by: Ivan Matkov <ivan.matkov@jetbrains.com>

Update document

f1dd20d

Update documentation

164fc78

Update doc

6728786

Update API dump

7e39a69

Minor rename

66fbe36

Send touches explicitly instead of dispatching all touches of event (…

d4abaf0

…that could be unchanged)

Revert "Temporarily deintegrate gesture recognizer proxying"

dc172f1

This reverts commit f2777cb

Add map example

a42879e

Minor renames

a6d6fd0

Make API private

c4acb08

Dump API

7f1db29

Fix double import

c324419

elijah-semyonov force-pushed the es/gesture-recognizer-touch-forwarding branch from 28dca12 to c324419 Compare July 9, 2024 09:58

elijah-semyonov merged commit a78dd41 into jb-main Jul 9, 2024
6 checks passed

elijah-semyonov deleted the es/gesture-recognizer-touch-forwarding branch July 9, 2024 11:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Touches forwarding to underlying interop views on iOS #1426

Touches forwarding to underlying interop views on iOS #1426

elijah-semyonov commented Jul 2, 2024 •

edited

Loading

MatkovIvan left a comment

MatkovIvan Jul 2, 2024

elijah-semyonov Jul 3, 2024 •

edited

Loading

MatkovIvan Jul 9, 2024

ASalavei Jul 2, 2024

elijah-semyonov Jul 3, 2024 •

edited

Loading

ASalavei Jul 2, 2024

elijah-semyonov Jul 3, 2024

elijah-semyonov commented Jul 3, 2024

elijah-semyonov commented Jul 3, 2024

MatkovIvan left a comment

MatkovIvan Jul 9, 2024

MatkovIvan Jul 9, 2024

		@@ -18,6 +18,7 @@ package androidx.compose.ui.viewinterop

		import androidx.compose.runtime.ComposeNodeLifecycleCallback

		// TODO: implement this for iOS

Touches forwarding to underlying interop views on iOS #1426

Touches forwarding to underlying interop views on iOS #1426

Conversation

elijah-semyonov commented Jul 2, 2024 • edited Loading

Testing

Release Notes

iOS - Fixes

Multiple Platforms - Breaking changes

MatkovIvan left a comment

Choose a reason for hiding this comment

MatkovIvan Jul 2, 2024

Choose a reason for hiding this comment

elijah-semyonov Jul 3, 2024 • edited Loading

Choose a reason for hiding this comment

MatkovIvan Jul 9, 2024

Choose a reason for hiding this comment

ASalavei Jul 2, 2024

Choose a reason for hiding this comment

elijah-semyonov Jul 3, 2024 • edited Loading

Choose a reason for hiding this comment

ASalavei Jul 2, 2024

Choose a reason for hiding this comment

elijah-semyonov Jul 3, 2024

Choose a reason for hiding this comment

elijah-semyonov commented Jul 3, 2024

elijah-semyonov commented Jul 3, 2024

MatkovIvan left a comment

Choose a reason for hiding this comment

MatkovIvan Jul 9, 2024

Choose a reason for hiding this comment

MatkovIvan Jul 9, 2024

Choose a reason for hiding this comment

elijah-semyonov commented Jul 2, 2024 •

edited

Loading

elijah-semyonov Jul 3, 2024 •

edited

Loading

elijah-semyonov Jul 3, 2024 •

edited

Loading