test_selection_change fails, goes to full screen on macOS

[HalfWhitt]

Describe the bug

Okay, I've got a strange one. I'm getting a failure for tests/widgets/test_selection::test_selection_change. Running it in slow, and inserting pauses of my own, I see what is happening, I just don't know why: the probe clicks on the selection, which opens it up. It then attempts to click a little higher, on the next item up. Somehow, the selection always collapses the instant of the mouse click, so the click hits the green fullscreen button instead.

I recorded my screen while it happened. Stepping through the frames, I can see that the fullscreen button is clicked before the selection options have even finished visually disappearing:

I'm also getting a failure for tests/widgets/test_detailedlist.py::test_actions, for a similar reason: it looks like the action menu collapses just before the action can be selected.

I don't know what would have changed to make this start happening; it appears to persist even if I checkout earlier versions of Toga.

Steps to reproduce

1. Run briefcase dev --test -- --slow tests/widgets/test_selection.py::test_selection_change (or tests/widgets/test_detailedlist.py::test_actions). (Due to Testbed works in run, but hangs in test #2382, I'm currently resorting to run or dev -r.)
2. See test failure. In selection_change, you can also see the test window go fullscreen.

Expected behavior

The second click should select the other option in the selection, rather than the window's fullscreen button.

Environment

• Operating System: macOS 14.2.1
• Python version: 3.10.13
• Software versions:
  • Briefcase: 0.3.17
  • Toga: from source, up-to-date with main

[freakboy3742]

This is interesting - we have seen some intermittent test failures (maybe 1 in 30-50 test runs?) in CI on these exact tests. Run the tests again, and they almost always pass. Is this 100% reproducible for you, or intermittent?

Either way - if the bug is being cause because the menu is collapsing and we're hitting the "maximise" button, my immediate thought is that we should adding some padding to the widget so that there's no risk of the popup menu colliding with the maximise button.

There may also be a need for a timeout on the test - or, at least a restructure so that we don't wait forever for an event that won't happen.

[HalfWhitt]

Yes, so far it's 100% reproducible. Every time, across multiple Python versions. I hadn't touched Toga in probably a couple of weeks prior — I can't remember if maybe I updated the OS or something?

I wonder... padding could avoid it hitting the fullscreen button. But unless the specific position of that button is what's causing the selection to collapse and the click to go "behind" it, it still wouldn't select the correct item. I can try it and see.

[HalfWhitt]

Unfortunately I was correct; padding can clear the fullscreen button, but the click still doesn't register on the selection option.

[HalfWhitt]

One more data point: I created a Mac virtual machine in UTM, set up Toga there, and ran the testbed; I got the same result. So it's evidently not just some weird quirk of my current system setup. (Both host and guest are running 14.3.1.)

[freakboy3742]

Good news - I can now reproduce this. It looks like it's 100% reproducible for me under Sonoma.

This also means that this problem is part of what is blocking #2404.

[freakboy3742]

It looks like the problem may stem from a change to the implementation of NSMenu. The issue isn't that the menu is disappearing before the events take place - the problem is that the mouse click generated by the testbed is no longer triggering the menu item. I think I've got a fix; see the patch in #2404 for details.

[HalfWhitt]

And we would've gotten away with it too, if it weren't for those meddling macOS developers!