Add support for high-resolution custom mouse cursors

[lostminds]

Describe the project you are working on

A multi-platform app that will be run on macOS, Windows and linux

Describe the problem or limitation you are having in your project

In Godot you can set the cursor shown for controls or situations by setting a cursor_shape for these from a limited set of defined cursor types. If you want to supply a different cursor or modify the appearance of the cursors you need to pick one of these present cursor types and assign a new image for this cursor type:

DisplayServer.cursor_set_custom_image(cursor_texture,DisplayServer.CURSOR_MOVE,Vector2(16,16))

However, just like in many control textures the issue is that the size of the cursor is set by the size of this texture. This means that if you want to supply a high-resolution cursor for use on high resolution screens it will be twice as big instead (and still rendered at low resolution). On high resolution screens the cursors are scaled up to ensure the same size on screen, but this means the custom cursors are blurry with scaling artifacts on high dpi monitors. Since the mouse cursor will usually be the center of attention for the player it's unfortunate this specifically cannot be made sharp.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Add some system to separate cursor size from the texture pixels resolution, to allow defining the cursor size and have a higher resolution texture to render it at appropriate resolution on highDPI monitors while retaining the same on screen visual size.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

One option is to add a separate cursor_size parameter to DisplayServer.cursor_set_custom_image that is used for the cursor on-screen size in resolution-independent units and use this instead of the texture size to determine the size of the cursor. This to allow supplying a larger texture (typically 2x) that can be used to provide a high resolution cursor on highDPI screens and still rendered at the correct size on normal DPI screens. This would be backward compatible since this size could be optional, and if left unset (or 0,0) it could fall back to the current behavior of using the texture size to set the cursor size.

Another more complex option could be to use something like the proposed godotengine/godot#86022 to have svg-based cursor textures that are automatically rendered at the correct resolution for the custom screen. However, I'm not sure how far away this is from being viable and implemented.

If this enhancement will not be used often, can it be worked around with a few lines of script?

No, as far as I can tell there's no way to access the cursors directly to fix this with scaling.

Is there a reason why this should be core and not an add-on in the asset library?

I think cursors are a core component, used most games/applications. And with highDPI monitors becoming more and more common highDPI support will become more and more relevant in this case as well.

[Mickeon]

While "far away" the SVG solution would be waaay more encompassing and tick many checkboxes at once if figured out. RichTextLabel and other icons across the Godot Editor suffer from blurriness due to lack of this.

[Calinou]

This is definitely an issue I noticed when using Godot on hiDPI displays, but I'm unsure about the specifics outlined here.

On high resolution screens the cursors are scaled up to ensure the same size on screen, but this means the custom cursors are blurry with scaling artifacts on high dpi monitors.

Do we have any control over this behavior on Windows, macOS and Linux? If so, what do the APIs for specifying unscaled cursors look like? If the OS is always unconditionally scaling custom cursor images, then there's no way we can ever use hiDPI-friendly cursor images. I don't recall this being the behavior on Windows and Linux at least, so I'm surprised to hear about this. Instead, cursor images are always displayed at 1:1 size regardless of the desktop scale factor, which can result in them being too small in terms of screen real estate.

Also, you may want to double-check that cursor size overrides may be taken into account for this regardless of the desktop scale factor. For instance, I use a large cursor (32×32 instead of the default 24×24) on my Fedora 39 KDE setup but I use 100% scaling. I do the same on Windows where I use one of the "accessible" cursor options with size +2. macOS also has a cursor size option in its Universal Access preferences.

A workaround for now would be to do this on your script's side based on a scale factor preference (or #2661 once it's implemented). Remember that mouse cursor images above 128×128 are not supported on the web platform though (maximum on desktop is 256×256).

One option is to add a separate cursor_size parameter to DisplayServer.cursor_set_custom_image that is used for the cursor on-screen size in resolution-independent units and use this instead of the texture size to determine the size of the cursor.

I doubt OSes have an API that let you customize the scale the cursor image is displayed at. Hardware cursor drawing is designed to be very simple so it can be fast. The only thing we could do is have Godot automatically resize the image based on the desktop scale factor before passing it to the OS API, which can be a decent solution but it'll mean that changing the cursor often may be slow (e.g. for people emulating animated cursors).

[lostminds]

I doubt OSes have an API that let you customize the scale the cursor image is displayed at.

On macOS you create cursors using an NSImage and the basic NSImage objects have a size property separate from the bitmap pixel size, so you can set the size of the cursor independently of the pixel size of the image (if it even has one). So you can set a 32x32pixel image to have a 16x16 size and it'll render at the correct on screen size with higher resolution (based on monitor resolution scaling, system cursor size scaling settings like you mention etc).

On Windows I haven't done it myself, but it appears to offer a similar system where you can use LoadImage with type Cursor where you can also set a cursor size cx,cy independent of the image pixel size, which could provide a similar way to have high resolution cursors by defining a cx,cy size smaller than the pixel size of the image. But I could be misunderstanding this, and I'd welcome hearing more about that from someone more familiar with windows cursors and windows screen scaling.

Hardware cursor drawing is designed to be very simple so it can be fast.

Yes, this is likely correct, but this just means that the system has a version cached somewhere for fast rendering at the correct current scale since this will likely remain quite static. The original image you use to create the cursor could well be a different size or format. However since this scaling is handled by the system it also has the benefit that we don't need to know the screen scaling or system cursor scaling settings. If I'm not mistaken we just need to provide a correctly set up cursor image resource for the system to use and render/scale automatically at the correct resolution.

[bruvzg]

On macOS you create cursors using an NSImage and the basic NSImage objects have a size property separate from the bitmap pixel size, so you can set the size of the cursor independently of the pixel size of the image (if it even has one). So you can set a 32x32pixel image to have a 16x16 size and it'll render at the correct on screen size with higher resolution (based on monitor resolution scaling, system cursor size scaling settings like you mention etc).

This is already done for menu icons, any image size can be passed, but NSImage size is set to 16x16, so should be really easy to do.

On Windows, LoadImage is for loading resources and won't do anything. And there's no way to supply multiple resolutions for the cursor. But I think it should do scaling automatically, if image size is appropriate.

[lostminds]

On Windows, LoadImage is for loading resources and won't do anything

Yes, this is a little confusing to me as well. According to the Using Cursors docs there are several ways to create/load a cursor resource before setting it via SetCursor. However, in the docs for LoadCursor it says that this has now been superseded by LoadImage which I guess means you can/should now use that? This is also mentioned in this related question, which seems to indicate windows will also scale cursors when it thinks it's appropriate for the current DPI settings if they have their size set.

There's also the confusingly named CreateIconFromResourceEx that seems to be able to create cursors (as well) from data pointers instead of needing to load the data from a file. But it seems that the loaded resource data might need to be formatted as a .cur file structure instead of just being any image data.

[bruvzg]

Godot is using CreateIconIndirect to make cursor for the custom image data, the Load* functions are loading cursor resources embedded into executable.

[lostminds]

It appears that Windows solution to multi-resolution cursors (and similarly icons) is using resources in RT_GROUP_CURSOR format, which basically just seems like a .cur resource file with a set of different resolution versions of the cursor including size information for each one. When loading these resources to be used as a cursor LookupIconIdFromDirectoryEx seems to be used by the system to pick the best match for the current display from the set, optionally using LoadIconWithScaleDown to resize the image before it's used.

However, I guess this means that unless Godot uses these Load* methods to load the resources from file we won't trigger the system automatically picking and/or scaling the cursor from the set. That is, unless it's possible to create a RT_GROUP_CURSOR data structure and trigger the same automatic picking/scaling by feeding it to CreateIconFromResourceEx.

[Riteo]

@Calinou

Do we have any control over this behavior on Windows, macOS and Linux?

FTR, on Wayland, cursors use normal wl_surfaces and so they receive all the usual HiDPI scaling events. In fact, we already load higher-resolution versions of cursors when hovering entering an HiDPI screen.

[lostminds]

After some more testing on Windows it looks like you @Calinou were right in your assumption above, there the custom cursors are just used at a 1:1 pixel ratio regardless of display resolution scaling. Instead this display resolution scaling only seems to be used when selecting what size cursor to load when loading a cursor resource with multiple resolutions. This means that the custom cursor implementation currently works differently on macOS and Windows, as the macOS version will scale the custom cursor image based on current display resolution scale.

While not a solution to the base problem above, perhaps a short term workaround could be to just change the macOS implementation to work more like the windows one by setting the size of the cursor NSImage to be divided with the current display scale when setting the cursor. In other words reversing the automatic scaling to get back to the pixel size of the image used. This way the custom cursors should at least behave the same way on macOS and Windows (and Linux?) and this would then in turn allow a second work-around where in your script you can try to figure out the display scale and switch between different size cursor images to use, something people might already do on Windows but won't currently work on macOS. Getting #2661 implemented would then of course also be very helpful in picking what cursor size to use.