-
-
Notifications
You must be signed in to change notification settings - Fork 950
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add capture using WinRT Windows.Graphics.Capture API. #2149
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for this contribution!
One other thing that WGC apparently provides is support for cross-adapter capture (capturing outputs attached to GPU 1 to a texture on GPU 2 for encoding). Did you happen to try that scenario? If not, that's fine - I was just curious.
This comment was marked as resolved.
This comment was marked as resolved.
Does UAC elevation and service mode work as expected? I've read many microsoft sources that pretty much told that this capture approach is not compatible with unattended use cases Source: robmikh/Win32CaptureSample#24 and robmikh/Win32CaptureSample#54 |
Thanks for the quick reviews! I'll be addressing code and GitHub actions issue in, uh, a bit.
@cgutman: I have not tried this scenario and while I did see this somewhere, I figured it was a little out of scope considering that, as far as I can tell, Sunshine is consumer software.
@TheElixZammuto: I think the headless scenario will work, but I need to test it after I make requested changes. Privilege elevation probably won't, but the existing API supports it. I designed this feature to be opt-in-only and to fall back if unsupported, so I hope that there's sufficient insurance for this. In no way do I advocate replacing the existing display capture code with Windows.Graphics.Capture. I quite like the idea of it being a beta feature that can be enabled in sunshine.conf but not the user interface. I also hope its presence lowers the bar for more knowledgeable contributors to improve this capture backend. |
WGC actually can capture the UAC secure desktop, even from an unelevated Sunshine.exe process, so it's actually better than DXGI DDA in that regard. It can even capture the login screen after locking the PC. However, you are correct that it doesn't work with our service model, at least not without additional changes. WGC works using per-user services which are activated via COM. The specific user service that runs the capture backend is Fortunately, the fact that elevation doesn't seem to actually provide any benefit to WGC (unlike DDA) means we do have some options here to support WGC from a service context. The first and simplest is to try impersonating using the user token from If impersonation doesn't work out, we'd need to use a WGC helper process that we would spawn into the user session and share textures with. This is definitely much more complex than the impersonation solution, but it may not be too horrible since we're already using shared textures between capture and encoding today.
Cross-adapter encoding is good to have on hybrid graphics systems, which are quite typical these days (most CPUs include an iGPU and basically all laptops do). Cross-adapter encoding lets users encoding on the dGPU even when the display is physically connected to the iGPU. We have to use a bunch of hacks (ddprobe.exe manually setting our graphics preference) to work around the cross-adapter encoding limitation for DDA on hybrid graphics systems to even be able to stream at all.
I actually would like to move to WGC as the default on OSes where support is good (Win11/2022+). It has a ton of advantages over the DXGI solution, including:
Obviously it could still have massive Achilles heel that sinks it for our usecase, but it looks quite compelling on the surface. In any case, none of this aspirational stuff matters for right now. I'm mainly just writing it down somewhere to not forget any of it. You should leave it as opt-in for this PR as you have it now and don't worry about the impersonation stuff. |
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
I finally tested this, and notice a fairly substantial performance regression on your updated branch vs the first commit (b843c12) tested on its own; my guess is the Testing 60fps client and server with the server rendering content at 60fps, the client can only runs at ~40fps maximum. My system is not too ancient (Ryzen 5700X and RX 6600), and I'm only testing at 768p, but this system usually has no trouble keeping 60fps at 4K with the standard capture path. Added some quick debug:
This shows a LOT of zero count timeouts on the latest branch (and the host is definitely rendering content at 60fps, so the volume of timeouts I'm seeing should not be happening). I can raise the capture to 60fps via one of the following steps (performed separately): |
@psyke83 Thank you for reporting this. I noticed it too, but I thought it might be a personal problem (i.e. "It only doesn't work on my machine") The issue no longer appears for me after applying the commit "When the frame timeout is zero, use the most recently produced frame instead of waiting for a new one." If you get a chance, I'd be curious to hear your thoughts on the actual change and if it works for you too! |
Capture is now running at 60fps on my system - nice work! Unfortunately, whilst the incoming framerate stays at ~60, there is some noticeable periodic stuttering in games running at 60fps that isn't present in the regular capture path... but I may have discovered the solution already. My GPU/driver combo doesn't support HAGS, meaning that Sunshine is using realtime gpu scheduling on my system. This normally works well (especially to prevent stuttering when the GPU is heavily loaded), but it seems to conversly cause stuttering with your new WGC capture path. Manually setting the priority to high seems to result in capture as smooth as the DDAPI path (but I need to test more thoroughly to say that the stuttering is 100% alleviated). I would recommend you and any other testers to check realtime vs high gpu scheduling with this capture path, just in case the issue is not unique to my specific setup. Thanks, and again - great work! |
src/platform/windows/display_wgc.cpp
Outdated
release_frame(); | ||
|
||
AcquireSRWLockExclusive(&frame_lock); | ||
if (timeout.count() > 0 && SleepConditionVariableSRW(&frame_present_cv, &frame_lock, timeout.count(), 0) == FALSE) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This needs to first check if a frame is available before sleeping on the condition variable. Unlike an event, a condition variable doesn't have stored signalled/not-signalled state. It only wakes waiters that are asleep at the time WakeConditionVariable()
is called. This means you always want to have it gated by a check that happens under your lock (see Microsoft's sample code)
Likewise, on the producer side, you don't have to call WakeConditionVariable()
when produced_frame
is already non-null, because you're not making a state transition where the consumer could possibily be waiting (the predicate is already true).
This is probably what's killing your performance. If the consumer isn't waiting at the time the producer gets a frame (N), that frame is going to be completely lost because the consumer is going to go to sleep on the CV even though there's already a frame ready for it. Only once asleep will the consumer get properly woken up with the next new frame (N+1).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for this.
To make a long story short, today I learned that condition variables don't actually store signaled state. With your suggested changes, I still no longer see the performance degradation observed earlier! I wonder if the stuttering @psyke83 observed is still present.
This comment was marked as resolved.
This comment was marked as resolved.
This is a negative for those depending on Sunshine and Moonlight to support VRR, which apparently is working in Wayland and Gamescope at the moment. If we move to a fixed frame rate capture, then VRR would break for those users... is there any way we can make the fixed frame rate capture optional? |
@psyke83 any chance this is caused by C++20 instead of C++17? |
This reverts commit 8f1692a.
Also HDR does not work too. It'll clip to SDR. |
HDR seems to be working fine for me with the new capture method. I don't have any way to measure nits or anything, but on my two clients with HDR enabled, I'm not seeing any blown out colors or clipping when comparing the two capture methods. (Sorry to be the "works for me!" guy.) |
@joshuacant It's nice to not be the only one! I'll be making follow-up comments on #2320 |
I'm on Win 11, just try the SDR dimming slider under HDR setting. Or check HDR metadata reading on SteamOS visualization tool. |
I know. But since I can't compile the code right now, the reason I was asking is an answer to a previous post saying this PR is not the cause, I wanted to test previous releases to pinpoint the exact commit. And after reverting the changes in this PR, Sunshine works like normal. Well, kind of, I can't pair new connections. "Unexpected end of stream" and the pairing message from Moonlight disappears within 5 seconds and displays that error. Installed v0.22.2, it works fine and pairing works like normal. |
He meant the feature changes were not the cause... but the change to build system (also in this PR) was the cause.
This was caused by the localization changes. Will be fixed shortly. |
Anyone have a nightly build with this change? The buildbot has overwritten the binaries and I want to test if this also fixes incoming call notifications in Microsoft Teams. Edit - found it: https://github.com/LizardByte/Sunshine/actions/runs/8462134237/job/23183018276 |
Co-authored-by: ReenigneArcher <42013603+ReenigneArcher@users.noreply.github.com>
Is there a way create the service as per-user for sunshine manually? |
Hi, I found a simple "temporary" solution, changing from ddx to wgc (and vice versa) on the fly, starting and stopping the service and Sunshine exe automatically, with two autoit exe scripts launched from windows scheduler when pc is locked (and also suspended) and unlocked (and also awaked). However, on client device you must "enter" two times... Sunshine switch wgc au3 file: sleep (3000) Global $contenuto = FileRead($file) FileClose($file) Global $contenutoModificato = StringReplace($contenuto, "ddx", "wgc") $file = FileOpen("C:\Program Files\Sunshine\config\Sunshine.conf", 2) ProcessClose("sunshine.exe") Sunshine Switch ddx: ; Apre il file ; Legge il contenuto del file ; Chiude il file Global $contenutoModificato = StringReplace($contenuto, "wgc", "ddx") ; Riapre il file per sovrascrivere il contenuto modificato ProcessClose("sunshine.exe") |
I created this simple "temporary" solution written above. |
@roob0 thank you for sharing the script. But i want to stream lock screen , at least the screen when windows is shutting down without disconnection. I hope there is a way. |
@Kobain-Seo Yes, It perfectly streams the lockscreen! I made this script for this purpouse. You can unlock PC from the client with moonlight, then reconnect (always in moonlight) after the pc is unlocked. |
@Kobain-Seo I usually put my pc in hybrid sleep (and cutting off power when I want to close it, so I don't press "Shutdown"). However, if you want to shutdown your pc in the "classic" way , in switch ddx's scheduler activity you can add the Event in Activator with System, User32, ID 1074. I'm testing the scripts in several scenario (lock-on, lockon and sleep, shutdown) and always work ;) |
@cgutman I find this simple solution for swicthing from service to the exe and from ddx to wgc (and viceversa). Perhaps a reconnection could be added in Moonlight (avoiding the disconnection) but, of course, a more sophisticated solution would be better. This is only a temporary "solution", waiting for the service to be "fixed". |
Can't make it work, i've never used autoit + task scheduler so I don't know if I didi it correctly, can you help me here? maybe making a tutorial, or via DM |
Also hoping for an improvement on that part. :) Can i use the same settings and configfile with that script. |
@Vaskyy don't use AutoIt, just create 2 config files, one with wgc setting like
Run the batch file again to reverse the settings 😊, if you want to use dxgi, don't forget to |
Damn that took me alot of time, bc im absolutely not advanced in any cmd/ps stuff. i did create an dxgi.bat:
and also an wgc.bat:
But now the wgc.bat is opening a cmd prompt, that if i close it, will terminate the thing of course. Also isnt it possible to autostart the sunshine.exe (not the service) to be able to use WGC always? |
I actually managed to do this BUT, and is a big BUT, you have to give up some home privacy, if you set sunshine.exe on the start forlder on windows Start Menu sunshine will autostart at widnows LOGON, (not boot up), although you cant set it up before it would be useless on WGC since you can't unlock the pc on WGC, so, my solution? autologon, pc boots and goes staight into windows with sunshine auto starting and letting you use always WGC... another workaround is to unlock the pc with parsec and then change to sunshine, I actually do this with another decive I have at home to turn my pc ON |
Funny because since I use virtual display driver, no one see my PC will know my PC was on unlocked haha they just think display is always off, you need change the cableing OS might output to it or you need to adjust in Moonlight remotely. |
there you go, try my workaround, is not ideal as a service, but in practice it does the job. |
Description
This adds a capture backend on Windows that uses the Windows.Graphics.Capture API. This is a more recent API that's capable of capturing the Xbox Game Bar, which the DXGI Desktop Duplication API seemingly cannot do.
To accomplish this, I used a slightly different build environment known as MSYS UCRT64. I also bumped the compilation standard to C++20 so that the GNU C++ compiler can handle the coroutine definitions present in WinRT headers. The UCRT64 and MinGW64 environment appear to be binary-compatible, and Sunshine's dependencies translate smoothly across this boundary. The build documentation in this change reflects these modifications.
This change can be enabled by setting
capture = wgc
in sunshine.conf on Windows servers. When this setting is off, no functionality change should be observable.I've observed CPU and memory usage trends over a long streaming period and there don't appear to be any new leaks.
Issues Fixed or Closed
Fixes #832 (manually verified)
Type of Change
.github/...
)Checklist
Branch Updates
LizardByte requires that branches be up-to-date before merging. This means that after any PR is merged, this branch
must be updated before it can be merged. You must also
Allow edits from maintainers.