Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

iOS - Scroll does not work at all when touch starts on a UIKitView inside eg a LazyColumn #4818

Closed
AttilaBarany opened this issue May 16, 2024 · 29 comments
Assignees
Labels
bug Something isn't working ios touches interop ios

Comments

@AttilaBarany
Copy link

Affected platforms

  • iOS

Versions

  • 1.6.0, 1.6.2, 1.6.10-rc01...

To Reproduce
Just extend the "uikit-in-compose" example app like this:
Screenshot 2024-05-16 at 16 15 51

(adding a big box after the elements inside the column, and make the column scrollable)

and try to scroll by grabbing eg the textfield

You can reproduce this with any UIKitView, eg a simple UIView(), wkwebview etc... Adding eg Androidview webview does allow us to scroll by grabbing it, but UKitView does not bubble up anything, even tried with custom nestedscrollconnection just print out pre- or post scroll, but got not a single trigger. This failure just makes useless the UIKitView, or at least very limited. You can just integrate only in static pages, without any scrolling, otherwise the user will stuck on the page with invisible contents which are below in the column.

@AttilaBarany AttilaBarany added bug Something isn't working submitted labels May 16, 2024
@brewin
Copy link

brewin commented May 16, 2024

Funny, I was just about to submit this same issue. It does severely limit the use of UIKitView. Setting interactive = false makes it scrollable, but of course then it isn't interactive.

Here is my minimal reproducer:

@OptIn(ExperimentalForeignApi::class)
fun MainViewController() = ComposeUIViewController {
    Column(modifier = Modifier.verticalScroll(state = rememberScrollState())) {
        UIKitView(
            factory = {
                UIView().apply {
                    backgroundColor = UIColor.redColor
                }
            },
            modifier = Modifier.fillMaxWidth().height(400.dp)
        )
        repeat(100) {
            Text("Item #$it")
        }
    }
}

@elijah-semyonov
Copy link
Contributor

Hi, it's a known problem, and not exactly a trivial one to fix due to differences in touch processing in Compose and UIKit. We are investigating our options currently.

@AttilaBarany
Copy link
Author

Great news, thank you @elijah-semyonov . I am gonna suspend my project until than, and stay with separated UI (I am using a webview in column, which can not be done without UIKitView for iOS).

@elijah-semyonov
Copy link
Contributor

As a workaround, you could present native UIViewController with web view modally on top of compose UI.
Use LocalUIViewController.current.

@AttilaBarany
Copy link
Author

As a workaround, you could present native UIViewController with web view modally on top of compose UI. Use LocalUIViewController.current.

So how that would could be part of the scrollable content? I dont get it.

@elijah-semyonov
Copy link
Contributor

It won't be. But you don't have to reimplement entire screen natively.

@AttilaBarany
Copy link
Author

It won't be. But you don't have to reimplement entire screen natively.

Just the column content, get it. Which is the whole screen (except top and bottom bar).:) Never mind. I started and coded the UI already separately/natively, but wanted to refactor to only maintain one UI instead of two. Took almost 2 weeks, and this was the last piece which is missing, everything else looks working, thats why its so annoying :)) But I might keep going this direction and use this screen only natively as suggested.

Thanks for your help anyway, and I am looking forward to have this fixed. :)
(p.s I beleive flutter had the very same issue before)

@brewin
Copy link

brewin commented May 17, 2024

Flutter's UIKitView has a gestureRecognizers property that allows customizing the gestures that are passed to the UIKit view.

https://api.flutter.dev/flutter/widgets/UiKitView/gestureRecognizers.html

For example, with the following setup vertical drags will not be dispatched to the UIKitview as the vertical drag gesture is claimed by the parent [GestureDetector].

GestureDetector(
  onVerticalDragStart: (DragStartDetails details) {},
  child: const UiKitView(
    viewType: 'webview',
  ),
)

@elijah-semyonov
Copy link
Contributor

elijah-semyonov commented May 21, 2024

Well, we can do that! The problem is, there is no way that is currently apparent to us, to conditionally allow touches to slip to the interop view gesture recognisers (such as compose scroll in a native scroll, for example).

The behavior in your example is possible by the usage of Modifier.draggable and using interactable = false in UIKitView arguments.

@dilip640
Copy link

facing same issue

@vickyleu
Copy link

same

@kavindudiyes
Copy link

@elijah-semyonov could you please expand on your suggestion of "The behavior in your example is possible by the usage of Modifier.draggable and using interactable = false in UIKitView arguments.". I tried a few combinations of setting interactive = false and implementing draggable on the Modifier, without success. With interactive = false, scrolling is fine, but it doesn't register any click events as @brewin has pointed out above.

My use-case is to embed a Native Ad using Google AdMob, where the UIView for the Ad is coming from Swift for iOS. In the ideal case, I want to be able to scroll on top of the Ad, but still be able to click on the Ad.

@vickyleu
Copy link

My solution is to turn off the touch event of UIView and unify the click penetration through the click event hitTest of compose to the rect of UIView.

For example, my UIView put a webview, and I need to click on the picture inside. I will first register a query of the coordinates of all pictures through js, get a set of rectangular arrays, and confirm that the picture is clicked by matching the array.

elijah-semyonov added a commit to JetBrains/compose-multiplatform-core that referenced this issue Jul 9, 2024
First approach to allow touches to be processed by both Compose and
nested interop views.
Other UX improvements will follow.

1. `actual typealias InteropView = UIView`
2. Change `ComposeScene` API to allow hit testing specific
`InteropView`.
3. Make `InteractionUIView` a `container` view of `InteropContainer` to
enforce it being in the same responder chain as interop views.
4. Remove `InteractionUIView.Delegate` and replace it with callbacks to
avoid `by lazy` reentry in `ComposeSceneMediator`, make
`InteractionUIView` construction eager.
5. Use `event.allTouches` instead of `event.touchesForView` to avoid
receiving an empty list in case `InteractionUIView` is part of responder
chain but not the hit test result itself.
6. Remove `KeyboardEventHandler.uikit.kt` to avoid `by lazy` reentry,
stick everything in a single lambda.

Fixes some of the cases from the domain of
JetBrains/compose-multiplatform#4818

## Testing
`Demo/IosBugs/UIKitRenderSync` now properly registers touches to allow
LazyColumn scrolling and native view reaction.

## Release Notes

### iOS - Fixes
- Touches inside interop views are not exclusive to them and are
processed on Compose side as well.

### Multiple Platforms - Breaking changes
- ComposeScene `fun hitTestInteropView(position: Offset): Boolean`
changed to `fun hitTestInteropView(position: Offset): InteropView?`

---------

Co-authored-by: Ivan Matkov <ivan.matkov@jetbrains.com>
elijah-semyonov added a commit to JetBrains/compose-multiplatform-core that referenced this issue Jul 12, 2024
## Description
In this approach `CMPGestureRecognizer` delays `touchesBegan` until
explicitly failed, and is required to fail by `UIGestureRecognizer` of
children views (aka interop views). Failure happens after the first
touch started and no motion above scroll slop happens within 150ms. If
this happens, intercepted touches are delivered to children views (and
their gesture recognisers), Compose itself gets all tracked touches as
cancelled and ignores them until the touch sequence ends (imposed by
UIKit).
This behavior is inspired by `UIScrollView` implementation.


https://github.com/JetBrains/compose-multiplatform-core/assets/4167681/1245b986-1f11-4f02-a939-c069a7797e3a

### Fixes
Improves behavior of touches in [certain
scenarios](JetBrains/compose-multiplatform#4818)

## Testing
This should be tested by QA

## Release Notes

### iOS - Features
- Improvements in touches processing to detect if touches were meant to
be delivered to interop views, or should be processed by Compose.

---------

Co-authored-by: Ivan Matkov <ivan.matkov@jetbrains.com>
@elijah-semyonov
Copy link
Contributor

elijah-semyonov commented Jul 12, 2024

Awaiting your feedback (and perhaps future issues) after these changes are released!
Track release notes for iOS 🌚

@okushnikov
Copy link

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.

@AttilaBarany
Copy link
Author

@elijah-semyonov I have imported 1.7.0-dev1727, But I do not experience any difference. The same lazycolumn which holds a wkwebview or a single uiview, does not scroll if the drag starts on the UIKitview (Modify.draggable also not triggered). But it does scroll with Androidview.

@paskowski
Copy link

I also see no difference when running 1.7.0-dev1727. In my case it's lazy column with UIKitView that has UITextField inside of it.

@elijah-semyonov
Copy link
Contributor

elijah-semyonov commented Jul 15, 2024

@AttilaBarany @paskowski
You need to move a finger over the screen fast to recognise a scroll

@AttilaBarany
Copy link
Author

@AttilaBarany @paskowski You need to move a finger over the screen fast to recognise a scroll

Nothing happens. Slow, fast, long, short, delay...

Screenshot 2024-07-15 at 14 18 04

@brewin
Copy link

brewin commented Jul 15, 2024

I don’t think 1.7.0-dev1727 includes the changes. The next dev release will.

@AttilaBarany
Copy link
Author

I don’t think 1.7.0-dev1727 includes the changes. The next dev release will.

I got it from his commit linked in this issue above. JetBrains/compose-multiplatform-core@174766b

Screenshot 2024-07-15 at 14 45 49

@elijah-semyonov
Copy link
Contributor

@AttilaBarany
Please provide a full repro project, I can't reproduce your behavior from a snippet:

val factory = remember {
    {
        UIView().apply {
            backgroundColor = UIColor.blueColor
            userInteractionEnabled = true
        }
    }
}

LazyColumn(modifier = Modifier.fillMaxSize()) {
    item {
        UIKitView(
            modifier = Modifier.fillMaxWidth().height(300.dp),
            factory = factory
        )
    }

    item {
        Box(Modifier.fillMaxWidth().height(1500.dp).background(Color.Red))
    }
}
Screen.Recording.2024-07-15.at.15.57.28.mov

@elijah-semyonov
Copy link
Contributor

elijah-semyonov commented Jul 15, 2024

The logic is identical to UIScrollView now - if there is fast initial movement, gesture is recognised as scroll and dispatched accordingly. If there is no movement and hitTest is an interop view, it intercepts the touches and no scrolling will ever happen until touches sequence ends (all fingers are up)

@paskowski
Copy link

I agree with #4818 (comment) that https://github.com/JetBrains/compose-multiplatform/releases/tag/v1.7.0-dev1727 does not have the changes that were supposed to fix this issue.

@AttilaBarany
Copy link
Author

@elijah-semyonov The video looks very promising, thank you for testing and showing me the error maybe is in my setup!!! I had a mistake providing the library version. I have changed the lib to 1.7.0-dev1727, but the gradle was set to use the compose.plugin which had an earlier version set. After fixed (set the plugin version to 1.7.0-dev1727), or I thought I fixed, gradle does not want to sync:
Could not find org.jetbrains.compose.runtime:runtime:1.7.0-dev1727.
Required by:
project :shared

@AttilaBarany
Copy link
Author

@elijah-semyonov I got the versions sorted, I set 1.7.0-alpha01 whis is the latest, and the scroll still does not work. So @paskowski and @brewin are correct. The fix is still not included.

Screenshot 2024-07-15 at 20 37 04

@elijah-semyonov
Copy link
Contributor

elijah-semyonov commented Jul 16, 2024

That's super awkward, CI shows that it was included in 1727. Anyway, let's w8 for other dev build and see if it flies.

@MatkovIvan
Copy link
Member

I got the versions sorted, I set 1.7.0-alpha01 whis is the latest,

1.7.0-dev1727 is newer than 1.7.0-alpha01. 1.7.0-alpha01 does not contain @elijah-semyonov 's fix, but dev1727 does. devXXXX is regular builds from master that are not uploaded to maven central (available only from dev one), so sorting is not really correct here. We might improve naming in the future, but now it is what it is.

@paskowski
Copy link

I can confirm that both 1.7.0-dev1727 and 1.7.0-dev1731 work correctly. I think I set an incorrect dependency's version yesterday, apologies for that. Thank you for fixing this, it's a real game changer @elijah-semyonov 🎉 .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working ios touches interop ios
Projects
None yet
Development

No branches or pull requests

9 participants