Move/resize gestures ignored by Microsoft Excel (and other non-standard window apps)

[adamm]

Love the app, and works exactly as expected everywhere, except on Microsoft Excel on Office 365

Mac OS: 10.14.4
Hummingbird: 2.0.0(3) bc3358d
Microsoft Excel: v16.24

What's interesting is that it works fine on Microsoft Word, Microsoft Teams, Powerpoint, etc. Just Excel doesn't honour the move/resize gestures.

[finestructure]

Sorry for the very late reply - for some reason my notifications were turned off :/

I'm afraid there's probably not much I can do about this. In rare cases windows seem to be elusive. I don't have Excel to test this further but if you come across any other windows for apps that I could try this with, please let me know.

Also, have you tried changing the modifier keys? I'm wondering if perhaps Excel is capturing some of them in a weird way preventing Hummingbird from activating.

[adamm]

No worries :)

I can confirm other Move modifier keys have the same problem. However, I see all the Resize modifiers do work with Excel. Odd.

If I find another program with the same behavior, i'll update this issue log.

[finestructure]

Mmm, that's bizarre. And I'm sure you've swapped the modifiers between move/resize, i.e. the set that's working for Resize isn't working for Move.

In that case the only thing I can think of is that there's something particular about the window that makes it unresizeable. Although move and resize are using the same APIs - I would expect them both to behave the same...

I have come across some other app in the past that wouldn't respond to moves or resizes but I can't recall right now which one it was. Certainly not one I use on a regular basis.

I'll see if I can get my hands on Excel to inspect what the window handling code does.

[finestructure]

Ok, I've had a look now and it's quite bizarre. It does work, sometimes. The key seems to be to initiate the first Hummingbird move/resize near the toolbar menu. After that, the window can be move/resized as normal:

https://www.icloud.com/iclouddrive/0qHrtLWeH5SRyHnGt4FxDhqmw#Screen_Recording_2019-10-25_at_13.57

[adamm]

Interesting. I tried to replicate your video (I even set my Move hotkeys to Ctrl+Fn) and I can confirm the toolbar works as expected, but when i go back to the main spreadsheet it still doesn't move. One can assume that Microsoft is probably doing something funky with the rendering layer that's causing interference.

I found another program that has the same issue. System Preferences
On all preferences windows, the hotkey on the toolbar moves the window just fine, but on some of them, using the hotkey on the body of the window does not.

General: Fails
Desktop & Screensaver: Fails
Dock: Works
Mission Control: Works
Language & Region: Works
Security & Privacy: Fails
Spotlight: Works
Notifications: Fails
...
Trackpad: Works
Printers & Scanners: Fails

Clearly there is some sort of UI object used on these windows that is conflicting with Hummingbird.

Hope this helps!

[finestructure]

Yes, I've actually noticed that myself before and I suspect it's because of security context.

Hummingbird is using the Accessibility APIs to read and modify window geometry. There are other things the Accessibility APIs allow you to do and I believe any particularly critical controls completely opt out of the mechanism even if you grant the accessibility permissions.

Regarding Excel: my video test didn't work 100% of the time. There were definitely cases where the menu bar "trick" didn't allow subsequent move/resize of the main window. Really weird.

I'm hoping to take a closer look this week!

[finestructure]

I don't see any way to fix this, I'm afraid. Excel windows seem to be playing by their own rules and I can't figure out how to get a handle on them to trigger move/resize.

[vaclavhanzl]

Aquamacs is another app which does not move/resize. When I try to move/resize Aquamacs, some other window is move/resized instead, so I guess that it is somehow undiscovered in the list of windows HB considers. Setting different modifiers does not matter. Also, trying with pointer at titlebar also does not work.
I'd be happy to help with this somehow, otherwise HB is a lifesaver for me. I had to switch from Linux to Mac and I found window management really painful. HB mostly saved me but Aquamacs is important in my work and not being able to move it spoils the joy.
I am quite experienced programmer on Linux, new to Mac but I already did first hacks to fix things driving me crazy on Mac (End key going to end of document instead of end of line, no normal way to make truly global shortcut to run a terminal etc.) and I considered writing something like HB but fortunately found it exists already (thanks!!!). So I am willing to invest some effort into fixing HB on Aquamacs.

[cotfas]

(End key going to end of document instead of end of line, no normal way to make truly global shortcut to run a terminal etc.)

For keyboard remaping you can try Karabiner (you can also assing keys to execute scripts)

[vaclavhanzl]

Thanks @cotfas , I already solved all my key-related itches with 100 lines of C code, this way I have full control :) Fixing HB for Aquamacs is my only pain left uncured...

[finestructure]

Hey @vaclavhanzl , thanks for reporting the issue with Aquamacs! To be honest, I'm not that surprised there are issues. There seem to be a few apps that don't use standard windows and therefore bypass the detection method.

I'm currently at a loss as to how to fix it. I've just pushed a branch were I added more detailed error logging when detection failed but as I recall (it's been a while since I looked at it) there was no rhyme or reason for why it's failing.

If you'd like to take a stab at it and manage to solve that puzzle that'd be fabulous! :)

[vaclavhanzl]

Hi @finestructure, I found out what is the problem with Aquamacs. In the GUI hierarchy, well behaved apps have 'window' elements under 'application', see e.g. (run in terminal, maybe allowing some security exception if asked) :

osascript -e 'tell application "System Events" to get the entire contents of every UI element whose name is "Terminal"'|tr ',' '\012'|head

which prints

window ... of application process Terminal

and it is similar for other movable apps like "Google Chrome", however for "Aquamacs", I get:

text field ... of application process Aquamacs

so the 'window' type element is missing in the hierarchy. I can move Chrome window with:

osascript -e 'tell application "System Events" to set position of first window of application process "Google Chrome" to {100, 100}'

however I need this to move Aquamacs:

osascript -e 'tell application "System Events" to set position of first text field of application process "Aquamacs" to {100, 100}'

So the problem is that AXUIElement.window(at: event.location) defined here only finds 'window' elements but not 'text field' elements.
I am unable to find out how to make it to work for 'application' child nodes of any type but you may know :)

[vaclavhanzl]

I can move Aquamacs, however is does not mean that it can be discovered under pointer. So to answer this, press Cmd+Shift+4 Space and start roaming around the screen. Aquamacs really is properly discovered. (Press Esc to get out of it.) However I have no idea what API is used by the Apple's print screen tool.

[finestructure]

Hi @vaclavhanzl , that's a very interesting find! window(at:) is looking specifically for NSAccessibility.Attribute.window but there might be a similar attribute for text fields.

The method could fall back to that if no window is found or special case it for Aquamacs, Excel (if the same applies), ...

Very interesting also that you can move Aquamacs by telling the text field to move!

I'm not surprised the window screenshot feature can find the window - I'm sure there's another API at play here than the Accessibility access route that HB is limited to.

Thanks for reporting back, I'll definitely take a look. I'm very busy with another project at the moment so I'm afraid it'll be a while though :(

[vaclavhanzl]

Thanks for your attention at busy time @finestructure. Looking at docs, I guess the fall back could be to test for Attribute.role being application and if yes, use the element itself as a window (sorry, just guessing, never seen swift before and what I see using applescript - which I also never used before - only loosely corresponds). I guess so because I found movable targets using

osascript -e 'tell application "System Events" to get the name of every UI element whose role is "AXApplication"'

and it would be nice if other people could use it to find names for any other non-movable apps (like Excel) and then try to analyze and move them using commands I wrote before.
Aquamacs itself could certainly be fixed (it likely violates some standard and is opensource so I could file a bug and/or fix it) however I'd not hope for any fix in Excel :) so workaround in HB may be the only way.

[finestructure]

Super useful tips there for debugging! That should come in handy. Thanks again for sharing :)

[tigger04]

Could it be that all these applications that don't work with hummingbird are XQuartz apps? (X11 mac implementation)

You can launch XQuartz alone and it has some interesting options athat I could see potentially interfering with Hummingbird ... i have fiddled around with these now so that i can't even remember what the defaults were I'm afraid, but still haven not persuaded it to do what Hummingbird tells it (or would tell it, if it could find it?)

[finestructure]

Hi @tigger04 , thanks for posting. I don't think that's the issue here - Aquamacs isn't an XQuartz app, I believe, and Excel certainly isn't. However, I'm almost certain XQuartz apps won't work with HB, either.

As to the particular settings: these look more like they might interfere with particular shortcuts you've set up in XQuartz. The issue at play here, however, is that the windows themselves aren't recognised by HB, which would be a different issue (with the same end result, though 😅 ).

[tigger04]

A frustrating one! perhaps as Apple continues to lock down the platform further with each release, apps will be forced to conform like they do on iOS. It might be a good day for uniformity, but that also brings with it the fear over what that means for developers and open source software in general.

Anyway thanks for putting together a nice little tool. I normally use BetterTouchTool but it's been quite unstable recently. This one seem to do one thing and does it well.

[tigger04]

Aquamacs is another app which does not move/resize. When I try to move/resize Aquamacs, some other window is move/resized instead, so I guess that it is somehow undiscovered in the list of windows HB considers. Setting different modifiers does not matter. Also, trying with pointer at titlebar also does not work.
I'd be happy to help with this somehow, otherwise HB is a lifesaver for me. I had to switch from Linux to Mac and I found window management really painful. HB mostly saved me but Aquamacs is important in my work and not being able to move it spoils the joy.
I am quite experienced programmer on Linux, new to Mac but I already did first hacks to fix things driving me crazy on Mac (End key going to end of document instead of end of line, no normal way to make truly global shortcut to run a terminal etc.) and I considered writing something like HB but fortunately found it exists already (thanks!!!). So I am willing to invest some effort into fixing HB on Aquamacs.

@vaclavhanzl for emacs take a look at https://github.com/railwaycat/homebrew-emacsmacport also the repo at https://bitbucket.org/mituharu/emacs-mac/src/master/README-mac - in sort

• emacs main package used to have a carbon UI option but carbon is the old mac UX API/library or 'appkit' that has gone out of fashion/support, so it takes some refactoring to build for cocoa (the new appkit/api based on obj-c/swift)
• anyway, this is the same emacs just ported to cocoa, so works just like any other mac app and works well with HB
• i'm not so familiar with aquamacs but i use doom emacs which is another variant, and this works perfectly for it and has solved all my problems
  if not working with aquamacs i can recommend doom is worth taking a look, it's emacs with some sensible defaults, EVIL (vim based) keybindings and a bunch of lazy loaded packages that is pretty snappy to use

re universal keybindings on them ac as already mentioned in this thread karabiner is the gold standard- if you don't mind writing your own json it is extremely powerful, basically remap your keyboard and any combo of keys/apps/variables to do what you want.
If you want something quick and dirty try bettertouchtool where you can map what you want through the UI - it's freeware not FOSS, there is a paid option i believe

[reagle]

Drats. I just discovered and purchased Hummingbird and note that it doesn't work with KeePassXC 2.6.6 . I just found this bug and confirm it doesn't work for Excel...

[finestructure]

Sorry to hear it's not working for you, @reagle. If you send me an email with email address you used to purchase I can issue a refund.

[reagle]

@finestructure: Count it as an investment for a possible fix -- though I know you said it's not likely. Perhaps update the README to say it doesn't work with some apps?

@tigger04: I also have BTT, but haven't any idea how to implement this same functionality there. Do you have a pointer?

[finestructure]

Perhaps update the README to say it doesn't work with some apps?

That's a good idea, I'll do that. Thanks!

[finestructure]

Added https://github.com/finestructure/Hummingbird/blob/develop/readme.md#known-limitations

[reagle]

As you can see, KeePassXC closed the issue as an QT/upstream issue. It'd be good to file a bug with QT, but I'm a simple user and know nothing about Hummingbird or QT development.

[tigger04]

@tigger04: I also have BTT, but haven't any idea how to implement this same functionality there. Do you have a pointer?

I used to have BTT and I think it was - open BTT config window, in the menu there's an item to get to general preferences (cmd+ comma ',' will probably do it). Behind a labyrinth of tabs was buried away a fairly well configurable drag window / resize functionality quite similar to this one.

Note BTT is a paid app. I stopped using it due to its ghastly user interface and the fact that it was becoming more and more unstable as time went on and updates would frequently break the whole thing - I just relied on it too much. When Karabiner bowed out permanently from functioning on Big Sur I migrated the whole thing over to Hammersoon - and stumbled upon a script that does the same thing for moving and sizing

[reagle]

@tigger04, with your help, I found it, thanks! BTT also works with Excel and QBittorrent (but not KeepassXC). I still prefer Hummingbird's "resize from corner" feature.

[gdelafond]

FYI: for KeePassXC 2.7.4 on macOS Ventura, it works but only from the title window bar.

[cyaconi]

Same with Kitty terminal, it works only from the title window bar