Add frame throttling source #2535
Conversation
This commit adds `Window::request_frame_throttling_hint` request and the corresponding `WindowEvent::CanRedrawFrame` event. The request is intended to be used as an alternative to timer based rendering and vsync extensions. For now only X11 support is implemented.
|
I'm quite surprised that this is touching the X Present protocol. The last I saw of that protocol it was a draft that Keithp was working on while still in the Intel graphics team which he left before finishing this work. At that time there was relatively little interest left in the community to work on this kind of things so I'm quite surprised that it got developed further in the end. The present protocol is something that's more intended to be used at the same level as the DRI2/3 protocols - i.e. it addresses shortcomings we found in DRI3. I would expect it's more applicable for compositors and for clients I'd expect it to be used in the graphics driver (e.g. Mesa), I would need to check into this more but I'd expect that subscribing to these present complete notifications is only going to work if you happen to be running on a driver that's using the present protocol internally - but you can't just assume that surely? My initial instinct here would have been to look at these protocols from Owen Taylor here: https://fishsoup.net/misc/wm-spec-synchronization.html. Those protocols were defined during a kind of heyday for Linux X11 graphics stack development in collaboration between Red Hat and Intel and I'd expect them to be most widely supported in compositors (e.g in Gnome Shell / KDE). I think those protocols are probably a bit more appropriate to use within Winit, outside of the driver. |
It's there for a long time actually, since 2015 and used. Though there are pretty few users in the wild probably due to lack of advertising.
I'm not sure how it works internally, I'd assume compositor support here? I know that it works with XWayland for example out of the box and and is delivered similar to frame callbacks resulting in identical frame rates between frame callbacks and xpresent driven rendering. The calls could return error and that's intentional, since such information may be missing or not supported by the system.
I went with |
|
There's hopefully some actively maintained reference code for using these protocols somewhere (e.g. maybe in gtk/gdk) but here's some old C code I wrote in the past that uses these protocols, that you might be able to decipher / steal from: https://github.com/rib/cglib/blob/rig/cglib/winsys/cg-winsys-egl-x11.c That cglib, or the slightly earlier cogl may even have other things that are potentially of interest in this context of Winit and X11 in particular, considering that it was actively developed at the time that these protocols were defined for the Gnome 3 desktop. Since energy and interest moved on from that, e.g. as folks started working on Wayland then relatively little will have changed with these things since then, and I expect there are some things that are only really documented in the code of these old projects. Incidentally those projects also supported similar frame synchronization for Wayland which may potentially be helpful to poke through. |
yeah, seems so - just quite surprising for me. It's not as much of a surprise though that there aren't many users - on the one hand I don't think there are many people left that are actively working with x11 at this level but also I think this protocol is only really intended to be used for things like EGL and GLX (e.g. from within Mesa). For a while keithp was working with Valve, looking to knock x11 into shape for better VR support and I wonder if maybe finishing this off tied into that work at the time.
I had a poke around e.g. in mutter (gnome shell) and the xorg xserver to double check this. It's a protocol that's supported in the xorg xserver, not the compositor - which is partly also why I wouldn't think it's a great fit here, since it doesn't really let you sync with your compositor (except if running on something like xwayland). The protocol is essentially a DRI3 extension which is what your EGL or GLX drivers use. Yeah xwayland integrates with present in the same xorg xserver repo. It generates fake msc events and integrates with the wayland compositor's frame callbacks as I understand it from skimming through the code. As far as I know DRI3 was never supported by Nvidia and I also guess the situation is the same with this Present extension. At least James Jones did give feedback on the spec but I doubt very much that they implemented it.
In the specific case of an x11 application that's known to run on xwayland then Xpresent can kind of do what you're looking for I think, since it would indirectly let the client integrate with the real Wayland compositor's frame callbacks - as a replacement for being driven by the vblank. That's not consistent though with how it would work for an x11 client running on an x11 compositor - in that case they don't get indirect feedback from the x11 compositor. If you dig through some of the discussions for the spec then there were ideas to extend it with the concept of "Present redirection" (e.g. see https://lists.x.org/archives/xorg-devel/2013-October/038262.html) which would have better connected x11 clients with an x11 compositor but it looks like those ideas got dropped. Even in for the assumed xwayland case though it does seems like a notable layering violation for an x11 application to be touching the present protocol outside of the driver - it's more like it should be the basis of an EGL/GLX extension that you could use, besides just using it when calling egl/glxSwapBuffers. E.g. it looks like the GLX driver may conflict with this; hooking into The fact that EGL/GLX uses e.g. DRI2/3/Present isn't really something that applications are supposed to know / assume, since it was generally only true for open source drivers, and Nvidia had their own equivalent protocols. Even if assuming running on open source drivers the availability of extensions like these depends on the version of the xserver, graphics drivers and client libraries which shouldn't really be assumed by something like Winit. The protocols like |
|
Thanks for your input here, it's really helpful to me with how to approach X11 interface for that given that docs for it are either lacking or you should clearly know where to look at.
I'll try to look into it and see how it goes, but from what I see it seems to be similar to what I do with xpresent. |
|
No worries - I hope it's at least constructive input. At least for some of this kind of stuff I was partially involved in its design/development (some of these protocols were effectively developed in support of a (failed) mobile OS called moblin/meego/tizen (depending on the year) largely done between Intel (where I was) and Red Hat (since it was also the foundation for Gnome 3). For things that weren't just discussed internally or just on IRC then it's sometimes possible to dredge history up from freedesktop.org mailing lists, or old gnome / freedesktop.org bugzilla archives for this kind of thing. (and digging through the git log for mesa / xorg xserver etc) For these protocols in particular I would have been working closely with Owen at the time and might be able to dig up more details/context if we get stuck with something here. |
|
@rib from what I've read and managed to find wrt I don't understand why using X present results in like 300 fps with picom and not even approaching monitor refresh rate without compositor. I have a feeling that the issue is that it has something to do with interfering with GLX, like you said? It kind of feels like X present is sort of special and should be avoided, and Xpresent(or used via the _sync_contol extension, that's strange though) shouldn't be used at all. Maybe some different approach with X present should be taken? |
|
I would have expected For plain, non-composited X then at that point you would be able to rely on yeah There certainly no unified solution that covers both composited and non-composited x11 clients unfortunately. |
|
The log with X present https://paste.rs/3Mv running with The other part of it that XWayland works, but that was sort of expected. From what I understand the X present is used by |
|
Grepping through the KWin repo it indeed looks like they don't support Considering that Ubuntu defaults to Gnome Shell I would probably guess that Gnome Shell does basically cover most Linux desktop users (and some of the other popular desktops use mutter or gnome shell under the hood) but kwin is surely also a lot of users too. Something that some compositors do is to unredirect windows that are fullscreen (i.e. games) which should then be able to synchronize with vblank via Quite a lot of windowed x11 apps may just use software rendering with APIs like cairo and likely just update based on UI interaction, instead of rendering continuously which would also make this less of an issue for a lot of cases. It's all quite a big mess though unfortunately :-D |
|
It seems like chromium is using xpresent, but to only get triplet(ust/msc/sbc) and schedules event itself. The way it does that by asking for those events and removing select right away. However with a closer look they say that sometimes the drivers return garbage data so they say that getting vsync params is not available.
The problem is that we want to sync to vblank in a non-blocking way. I'm not sure there's a way to do so with glx/x11, etc? On Wayland you simply drive by frame callbacks and call it a day. |
hehe, wow :) are they maybe using it in a context where they aren't also using egl/glx? - that might be quite reasonable (and then it might just be a separate issue that some drivers have inconsistent definitions of the msc counters)
There are the GLX_INTEL_swap_events that are likely to be supported by other open source drivers besides just intel, but almost certainly not Nvidia. If I recall right, that enabled non-composited apps to synchronize without blocking. but yeah it's all more of a mix and match approach with lots of special cases. |
|
As the discussion has helped remind me of the challenge it was dealing with this stuff in the past I figured I'd at least try and break down the a list of cases that may involve different solutions:
This actually probably needs to be expanded further by considering applications rendering with Open GL[ES] with GLX vs EGL and then Vulkan, and then potentially consider applications that are simply software rendering. For supporting X11 it may make sense to just try and pick some of the higher-priority, common cases and see if they can be supported, and potentially just leave other cases unsupported/unthrottled. Unfortunately even with that idea it may be tricky in some cases where the current best solution may currently depend on GLX/EGL which isn't currently in-scope for Winit. |
|
the cases where a game may get unredirected when full screen also highlight that things can change dynamically and ideally a client would recognise when it switches between being composited or not to change how it synchronizes its rendering. |
|
I've opened a separate branch for that #2883 |
This commit adds
Window::request_frame_throttling_hintrequest and the correspondingWindowEvent::CanRedrawFrameevent.The request is intended to be used as an alternative to timer based rendering and vsync extensions.
For now only X11 support is implemented.
CHANGELOG.mdif knowledge of this change could be valuable to usersThe related issue.
#2412