Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add capture using WinRT Windows.Graphics.Capture API. #2149

Merged
merged 17 commits into from
Mar 28, 2024
Merged

Add capture using WinRT Windows.Graphics.Capture API. #2149

merged 17 commits into from
Mar 28, 2024

Conversation

tez011
Copy link
Contributor

@tez011 tez011 commented Feb 18, 2024

Description

This adds a capture backend on Windows that uses the Windows.Graphics.Capture API. This is a more recent API that's capable of capturing the Xbox Game Bar, which the DXGI Desktop Duplication API seemingly cannot do.

To accomplish this, I used a slightly different build environment known as MSYS UCRT64. I also bumped the compilation standard to C++20 so that the GNU C++ compiler can handle the coroutine definitions present in WinRT headers. The UCRT64 and MinGW64 environment appear to be binary-compatible, and Sunshine's dependencies translate smoothly across this boundary. The build documentation in this change reflects these modifications.

This change can be enabled by setting capture = wgc in sunshine.conf on Windows servers. When this setting is off, no functionality change should be observable.
I've observed CPU and memory usage trends over a long streaming period and there don't appear to be any new leaks.

Issues Fixed or Closed

Fixes #832 (manually verified)

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Dependency update (updates to dependencies)
  • Documentation update (changes to documentation)
  • Repository update (changes to repository files, e.g. .github/...)

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated the in code docstring/documentation-blocks for new or existing methods/components

Branch Updates

LizardByte requires that branches be up-to-date before merging. This means that after any PR is merged, this branch
must be updated before it can be merged. You must also
Allow edits from maintainers.

  • I want maintainers to keep my branch updated

@CLAassistant
Copy link

CLAassistant commented Feb 18, 2024

CLA assistant check
All committers have signed the CLA.

Copy link
Collaborator

@cgutman cgutman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this contribution!

One other thing that WGC apparently provides is support for cross-adapter capture (capturing outputs attached to GPU 1 to a texture on GPU 2 for encoding). Did you happen to try that scenario? If not, that's fine - I was just curious.

docs/source/building/windows.rst Show resolved Hide resolved
src/platform/windows/display.h Outdated Show resolved Hide resolved
src/platform/windows/display.h Outdated Show resolved Hide resolved
src/platform/windows/display_uwp.cpp Outdated Show resolved Hide resolved
src/platform/windows/display_uwp.cpp Outdated Show resolved Hide resolved
src/platform/windows/display_uwp.cpp Outdated Show resolved Hide resolved
src/platform/windows/display_uwp.cpp Outdated Show resolved Hide resolved
cmake/targets/common.cmake Outdated Show resolved Hide resolved
@cgutman

This comment was marked as resolved.

docs/source/building/windows.rst Outdated Show resolved Hide resolved
cmake/compile_definitions/windows.cmake Outdated Show resolved Hide resolved
@TheElixZammuto
Copy link
Contributor

TheElixZammuto commented Feb 19, 2024

Does UAC elevation and service mode work as expected? I've read many microsoft sources that pretty much told that this capture approach is not compatible with unattended use cases

Source: robmikh/Win32CaptureSample#24 and robmikh/Win32CaptureSample#54

@tez011
Copy link
Contributor Author

tez011 commented Feb 20, 2024

Thanks for the quick reviews! I'll be addressing code and GitHub actions issue in, uh, a bit.

One other thing that WGC apparently provides is support for cross-adapter capture (capturing outputs attached to GPU 1 to a texture on GPU 2 for encoding).

@cgutman: I have not tried this scenario and while I did see this somewhere, I figured it was a little out of scope considering that, as far as I can tell, Sunshine is consumer software.

Does UAC elevation and service mode work as expected? I've read many microsoft sources that pretty much told that this capture approach is not compatible with unattended use cases

@TheElixZammuto: I think the headless scenario will work, but I need to test it after I make requested changes. Privilege elevation probably won't, but the existing API supports it. I designed this feature to be opt-in-only and to fall back if unsupported, so I hope that there's sufficient insurance for this.

In no way do I advocate replacing the existing display capture code with Windows.Graphics.Capture. I quite like the idea of it being a beta feature that can be enabled in sunshine.conf but not the user interface. I also hope its presence lowers the bar for more knowledgeable contributors to improve this capture backend.

@cgutman
Copy link
Collaborator

cgutman commented Feb 21, 2024

Does UAC elevation and service mode work as expected? I've read many microsoft sources that pretty much told that this capture approach is not compatible with unattended use cases

WGC actually can capture the UAC secure desktop, even from an unelevated Sunshine.exe process, so it's actually better than DXGI DDA in that regard. It can even capture the login screen after locking the PC.

However, you are correct that it doesn't work with our service model, at least not without additional changes. WGC works using per-user services which are activated via COM. The specific user service that runs the capture backend is CaptureService which is implemented in %SystemRoot%\system32\CaptureService.dll. The use of a per-user service is probably why the 0x80070424 error is thrown from a service context, because the service indeed doesn't exist for LocalSystem's LUID (which is how per-user services are identified).

Fortunately, the fact that elevation doesn't seem to actually provide any benefit to WGC (unlike DDA) means we do have some options here to support WGC from a service context.

The first and simplest is to try impersonating using the user token from WTSQueryUserToken() (same as we do to start apps). We might just need this for our first activation of WGC (which is the call to winrt::GraphicsCaptureSession::IsSupported()) or we could also need it across other WGC API calls.

If impersonation doesn't work out, we'd need to use a WGC helper process that we would spawn into the user session and share textures with. This is definitely much more complex than the impersonation solution, but it may not be too horrible since we're already using shared textures between capture and encoding today.

I have not tried this scenario and while I did see this somewhere, I figured it was a little out of scope considering that, as far as I can tell, Sunshine is consumer software.

Cross-adapter encoding is good to have on hybrid graphics systems, which are quite typical these days (most CPUs include an iGPU and basically all laptops do). Cross-adapter encoding lets users encoding on the dGPU even when the display is physically connected to the iGPU. We have to use a bunch of hacks (ddprobe.exe manually setting our graphics preference) to work around the cross-adapter encoding limitation for DDA on hybrid graphics systems to even be able to stream at all.

In no way do I advocate replacing the existing display capture code with Windows.Graphics.Capture. I quite like the idea of it being a beta feature that can be enabled in sunshine.conf but not the user interface. I also hope its presence lowers the bar for more knowledgeable contributors to improve this capture backend.

I actually would like to move to WGC as the default on OSes where support is good (Win11/2022+). It has a ton of advantages over the DXGI solution, including:

  • Ability to capture the Game Bar
  • Fixed frame rate capture
  • Cross-adapter capture (untested but claimed by OBS folks)
  • Avoids all the mess of having to blend the cursor into the frame
  • Non-blocking API that doesn't hold critical locks for long periods of time like IDXGIOutputDuplication::AcquireNextFrame() does
  • Could potentially work around the Nvidia driver bug that causes hangs with high VRAM usage (the deadlock in the capture thread is on the keyed mutex inside IDXGIOutputDuplication::AcquireNextFrame())
  • Potential for much better frame pacing and lower latency by avoiding sleeps in our capture loop (we can instead look at the current time in on_frame_arrived() and decide to encode a frame or not based on how long it's been since the last frame)

Obviously it could still have massive Achilles heel that sinks it for our usecase, but it looks quite compelling on the surface.

In any case, none of this aspirational stuff matters for right now. I'm mainly just writing it down somewhere to not forget any of it. You should leave it as opt-in for this PR as you have it now and don't worry about the impersonation stuff.

cmake/targets/windows.cmake Outdated Show resolved Hide resolved
cmake/targets/windows.cmake Outdated Show resolved Hide resolved
docs/source/building/windows.rst Outdated Show resolved Hide resolved
docs/source/building/windows.rst Show resolved Hide resolved
docs/source/building/windows.rst Show resolved Hide resolved
src/platform/windows/display_base.cpp Outdated Show resolved Hide resolved
@tez011

This comment was marked as resolved.

@ReenigneArcher

This comment was marked as resolved.

@tez011

This comment was marked as resolved.

@ReenigneArcher

This comment was marked as resolved.

@tez011

This comment was marked as resolved.

@ReenigneArcher

This comment was marked as resolved.

@ReenigneArcher

This comment was marked as resolved.

@ReenigneArcher ReenigneArcher added this to the adjust lint rules milestone Feb 28, 2024
@psyke83
Copy link
Collaborator

psyke83 commented Feb 29, 2024

I finally tested this, and notice a fairly substantial performance regression on your updated branch vs the first commit (b843c12) tested on its own; my guess is the SRWLOCK changes are related.

Testing 60fps client and server with the server rendering content at 60fps, the client can only runs at ~40fps maximum. My system is not too ancient (Ryzen 5700X and RX 6600), and I'm only testing at 768p, but this system usually has no trouble keeping 60fps at 4K with the standard capture path.

Added some quick debug:

diff --git a/src/platform/windows/display_wgc.cpp b/src/platform/windows/display_wgc.cpp
index c1390136..1ccd9cb0 100644
--- a/src/platform/windows/display_wgc.cpp
+++ b/src/platform/windows/display_wgc.cpp
@@ -147,9 +147,10 @@ namespace platf::dxgi {
     AcquireSRWLockExclusive(&frame_lock);
     if (SleepConditionVariableSRW(&frame_present_cv, &frame_lock, timeout.count(), 0) == FALSE) {
       ReleaseSRWLockExclusive(&frame_lock);
-      if (GetLastError() == ERROR_TIMEOUT)
+      if (GetLastError() == ERROR_TIMEOUT) {^M
+        BOOST_LOG(error) << "timeout: " << timeout.count();^M
         return capture_e::timeout;
-      else
+      } else^M
         return capture_e::error;
     }
     if (produced_frame) {

This shows a LOT of zero count timeouts on the latest branch (and the host is definitely rendering content at 60fps, so the volume of timeouts I'm seeing should not be happening).

I can raise the capture to 60fps via one of the following steps (performed separately):

  • Using only the original commit (and capture = uwp).
  • Hardcoding the default timeout from 0ms to 16ms to correspond to a 60fps frame interval.
  • Increasing the client framerate to >60fps (such as 120fps), even though my host and client client screens only supports 60fps.

@tez011
Copy link
Contributor Author

tez011 commented Mar 1, 2024

@psyke83 Thank you for reporting this. I noticed it too, but I thought it might be a personal problem (i.e. "It only doesn't work on my machine")

The issue no longer appears for me after applying the commit "When the frame timeout is zero, use the most recently produced frame instead of waiting for a new one." If you get a chance, I'd be curious to hear your thoughts on the actual change and if it works for you too!

@psyke83
Copy link
Collaborator

psyke83 commented Mar 1, 2024

@tez011

Capture is now running at 60fps on my system - nice work!

Unfortunately, whilst the incoming framerate stays at ~60, there is some noticeable periodic stuttering in games running at 60fps that isn't present in the regular capture path... but I may have discovered the solution already. My GPU/driver combo doesn't support HAGS, meaning that Sunshine is using realtime gpu scheduling on my system. This normally works well (especially to prevent stuttering when the GPU is heavily loaded), but it seems to conversly cause stuttering with your new WGC capture path. Manually setting the priority to high seems to result in capture as smooth as the DDAPI path (but I need to test more thoroughly to say that the stuttering is 100% alleviated).

I would recommend you and any other testers to check realtime vs high gpu scheduling with this capture path, just in case the issue is not unique to my specific setup. Thanks, and again - great work!

release_frame();

AcquireSRWLockExclusive(&frame_lock);
if (timeout.count() > 0 && SleepConditionVariableSRW(&frame_present_cv, &frame_lock, timeout.count(), 0) == FALSE) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to first check if a frame is available before sleeping on the condition variable. Unlike an event, a condition variable doesn't have stored signalled/not-signalled state. It only wakes waiters that are asleep at the time WakeConditionVariable() is called. This means you always want to have it gated by a check that happens under your lock (see Microsoft's sample code)

Likewise, on the producer side, you don't have to call WakeConditionVariable() when produced_frame is already non-null, because you're not making a state transition where the consumer could possibily be waiting (the predicate is already true).

This is probably what's killing your performance. If the consumer isn't waiting at the time the producer gets a frame (N), that frame is going to be completely lost because the consumer is going to go to sleep on the CV even though there's already a frame ready for it. Only once asleep will the consumer get properly woken up with the next new frame (N+1).

Copy link
Contributor Author

@tez011 tez011 Mar 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this.
To make a long story short, today I learned that condition variables don't actually store signaled state. With your suggested changes, I still no longer see the performance degradation observed earlier! I wonder if the stuttering @psyke83 observed is still present.

@tez011

This comment was marked as resolved.

@Nonary
Copy link
Collaborator

Nonary commented Mar 5, 2024

@cgutman

Fixed frame rate capture

This is a negative for those depending on Sunshine and Moonlight to support VRR, which apparently is working in Wayland and Gamescope at the moment. If we move to a fixed frame rate capture, then VRR would break for those users... is there any way we can make the fixed frame rate capture optional?

@ReenigneArcher
Copy link
Member

@psyke83 any chance this is caused by C++20 instead of C++17?

@Originalimoc
Copy link

Also HDR does not work too. It'll clip to SDR.

@joshuacant
Copy link

HDR seems to be working fine for me with the new capture method. I don't have any way to measure nits or anything, but on my two clients with HDR enabled, I'm not seeing any blown out colors or clipping when comparing the two capture methods. (Sorry to be the "works for me!" guy.)

@tez011
Copy link
Contributor Author

tez011 commented Mar 29, 2024

@joshuacant It's nice to not be the only one!

I'll be making follow-up comments on #2320

@Originalimoc
Copy link

HDR seems to be working fine for me with the new capture method. I don't have any way to measure nits or anything, but on my two clients with HDR enabled, I'm not seeing any blown out colors or clipping when comparing the two capture methods. (Sorry to be the "works for me!" guy.)

I'm on Win 11, just try the SDR dimming slider under HDR setting. Or check HDR metadata reading on SteamOS visualization tool.

@RebelliousX
Copy link

RebelliousX commented Mar 30, 2024

So is there a way to download previous nightly installers instead of the latest one?

There's almost zero reason to use nightly versus v0.22.2.

I know. But since I can't compile the code right now, the reason I was asking is an answer to a previous post saying this PR is not the cause, I wanted to test previous releases to pinpoint the exact commit.

And after reverting the changes in this PR, Sunshine works like normal. Well, kind of, I can't pair new connections. "Unexpected end of stream" and the pairing message from Moonlight disappears within 5 seconds and displays that error. Installed v0.22.2, it works fine and pairing works like normal.

@ReenigneArcher
Copy link
Member

ReenigneArcher commented Mar 30, 2024

saying this PR is not the cause

He meant the feature changes were not the cause... but the change to build system (also in this PR) was the cause.

I can't pair new connections

This was caused by the localization changes. Will be fixed shortly.

@puigru
Copy link

puigru commented Apr 18, 2024

Anyone have a nightly build with this change? The buildbot has overwritten the binaries and I want to test if this also fixes incoming call notifications in Microsoft Teams.

Edit - found it: https://github.com/LizardByte/Sunshine/actions/runs/8462134237/job/23183018276
Click on "Upload Artifacts" then find "Artifact download URL". archive link
Unfortunately adding adding capture = wgc in sunshine.conf makes it crash with an access violation error for me on Windows 10 LTSC ☹️
Looking forward to this being upstreamed properly!

KuleRucket pushed a commit to KuleRucket/Sunshine that referenced this pull request Jun 6, 2024
Co-authored-by: ReenigneArcher <42013603+ReenigneArcher@users.noreply.github.com>
@Kobain-Seo
Copy link

Does UAC elevation and service mode work as expected? I've read many microsoft sources that pretty much told that this capture approach is not compatible with unattended use cases

WGC actually can capture the UAC secure desktop, even from an unelevated Sunshine.exe process, so it's actually better than DXGI DDA in that regard. It can even capture the login screen after locking the PC.

However, you are correct that it doesn't work with our service model, at least not without additional changes. WGC works using per-user services which are activated via COM. The specific user service that runs the capture backend is CaptureService which is implemented in %SystemRoot%\system32\CaptureService.dll. The use of a per-user service is probably why the 0x80070424 error is thrown from a service context, because the service indeed doesn't exist for LocalSystem's LUID (which is how per-user services are identified).

Fortunately, the fact that elevation doesn't seem to actually provide any benefit to WGC (unlike DDA) means we do have some options here to support WGC from a service context.

The first and simplest is to try impersonating using the user token from WTSQueryUserToken() (same as we do to start apps). We might just need this for our first activation of WGC (which is the call to winrt::GraphicsCaptureSession::IsSupported()) or we could also need it across other WGC API calls.

If impersonation doesn't work out, we'd need to use a WGC helper process that we would spawn into the user session and share textures with. This is definitely much more complex than the impersonation solution, but it may not be too horrible since we're already using shared textures between capture and encoding today.

I have not tried this scenario and while I did see this somewhere, I figured it was a little out of scope considering that, as far as I can tell, Sunshine is consumer software.

Cross-adapter encoding is good to have on hybrid graphics systems, which are quite typical these days (most CPUs include an iGPU and basically all laptops do). Cross-adapter encoding lets users encoding on the dGPU even when the display is physically connected to the iGPU. We have to use a bunch of hacks (ddprobe.exe manually setting our graphics preference) to work around the cross-adapter encoding limitation for DDA on hybrid graphics systems to even be able to stream at all.

In no way do I advocate replacing the existing display capture code with Windows.Graphics.Capture. I quite like the idea of it being a beta feature that can be enabled in sunshine.conf but not the user interface. I also hope its presence lowers the bar for more knowledgeable contributors to improve this capture backend.

I actually would like to move to WGC as the default on OSes where support is good (Win11/2022+). It has a ton of advantages over the DXGI solution, including:

  • Ability to capture the Game Bar
  • Fixed frame rate capture
  • Cross-adapter capture (untested but claimed by OBS folks)
  • Avoids all the mess of having to blend the cursor into the frame
  • Non-blocking API that doesn't hold critical locks for long periods of time like IDXGIOutputDuplication::AcquireNextFrame() does
  • Could potentially work around the Nvidia driver bug that causes hangs with high VRAM usage (the deadlock in the capture thread is on the keyed mutex inside IDXGIOutputDuplication::AcquireNextFrame())
  • Potential for much better frame pacing and lower latency by avoiding sleeps in our capture loop (we can instead look at the current time in on_frame_arrived() and decide to encode a frame or not based on how long it's been since the last frame)

Obviously it could still have massive Achilles heel that sinks it for our usecase, but it looks quite compelling on the surface.

In any case, none of this aspirational stuff matters for right now. I'm mainly just writing it down somewhere to not forget any of it. You should leave it as opt-in for this PR as you have it now and don't worry about the impersonation stuff.

Is there a way create the service as per-user for sunshine manually?

@roob0
Copy link

roob0 commented Jul 20, 2024

Does UAC elevation and service mode work as expected? I've read many microsoft sources that pretty much told that this capture approach is not compatible with unattended use cases

WGC actually can capture the UAC secure desktop, even from an unelevated Sunshine.exe process, so it's actually better than DXGI DDA in that regard. It can even capture the login screen after locking the PC.

However, you are correct that it doesn't work with our service model, at least not without additional changes. WGC works using per-user services which are activated via COM. The specific user service that runs the capture backend is CaptureService which is implemented in %SystemRoot%\system32\CaptureService.dll. The use of a per-user service is probably why the 0x80070424 error is thrown from a service context, because the service indeed doesn't exist for LocalSystem's LUID (which is how per-user services are identified).

Fortunately, the fact that elevation doesn't seem to actually provide any benefit to WGC (unlike DDA) means we do have some options here to support WGC from a service context.

The first and simplest is to try impersonating using the user token from WTSQueryUserToken() (same as we do to start apps). We might just need this for our first activation of WGC (which is the call to winrt::GraphicsCaptureSession::IsSupported()) or we could also need it across other WGC API calls.

If impersonation doesn't work out, we'd need to use a WGC helper process that we would spawn into the user session and share textures with. This is definitely much more complex than the impersonation solution, but it may not be too horrible since we're already using shared textures between capture and encoding today.

I have not tried this scenario and while I did see this somewhere, I figured it was a little out of scope considering that, as far as I can tell, Sunshine is consumer software.

Cross-adapter encoding is good to have on hybrid graphics systems, which are quite typical these days (most CPUs include an iGPU and basically all laptops do). Cross-adapter encoding lets users encoding on the dGPU even when the display is physically connected to the iGPU. We have to use a bunch of hacks (ddprobe.exe manually setting our graphics preference) to work around the cross-adapter encoding limitation for DDA on hybrid graphics systems to even be able to stream at all.

In no way do I advocate replacing the existing display capture code with Windows.Graphics.Capture. I quite like the idea of it being a beta feature that can be enabled in sunshine.conf but not the user interface. I also hope its presence lowers the bar for more knowledgeable contributors to improve this capture backend.

I actually would like to move to WGC as the default on OSes where support is good (Win11/2022+). It has a ton of advantages over the DXGI solution, including:

  • Ability to capture the Game Bar
  • Fixed frame rate capture
  • Cross-adapter capture (untested but claimed by OBS folks)
  • Avoids all the mess of having to blend the cursor into the frame
  • Non-blocking API that doesn't hold critical locks for long periods of time like IDXGIOutputDuplication::AcquireNextFrame() does
  • Could potentially work around the Nvidia driver bug that causes hangs with high VRAM usage (the deadlock in the capture thread is on the keyed mutex inside IDXGIOutputDuplication::AcquireNextFrame())
  • Potential for much better frame pacing and lower latency by avoiding sleeps in our capture loop (we can instead look at the current time in on_frame_arrived() and decide to encode a frame or not based on how long it's been since the last frame)

Obviously it could still have massive Achilles heel that sinks it for our usecase, but it looks quite compelling on the surface.

In any case, none of this aspirational stuff matters for right now. I'm mainly just writing it down somewhere to not forget any of it. You should leave it as opt-in for this PR as you have it now and don't worry about the impersonation stuff.

Hi, I found a simple "temporary" solution, changing from ddx to wgc (and vice versa) on the fly, starting and stopping the service and Sunshine exe automatically, with two autoit exe scripts launched from windows scheduler when pc is locked (and also suspended) and unlocked (and also awaked). However, on client device you must "enter" two times...

Sunshine switch wgc au3 file:

sleep (3000)
Global $file = FileOpen("C:\Program Files\Sunshine\config\Sunshine.conf", 0)

Global $contenuto = FileRead($file)

FileClose($file)

Global $contenutoModificato = StringReplace($contenuto, "ddx", "wgc")

$file = FileOpen("C:\Program Files\Sunshine\config\Sunshine.conf", 2)
FileWrite($file, $contenutoModificato)
FileClose($file)

ProcessClose("sunshine.exe")
ProcessClose("sunshine.exe")
RunWait(@comspec & ' /c net stop "' & "Sunshine Service" & '"', "", @SW_HIDE)
sleep(1500)
ShellExecute("C:\Program Files\Sunshine\Sunshine.exe","","C:\Program Files\Sunshine","",@SW_HIDE)


Sunshine Switch ddx:

; Apre il file
Global $file = FileOpen("C:\Program Files\Sunshine\config\Sunshine.conf", 0)

; Legge il contenuto del file
Global $contenuto = FileRead($file)

; Chiude il file
FileClose($file)

Global $contenutoModificato = StringReplace($contenuto, "wgc", "ddx")

; Riapre il file per sovrascrivere il contenuto modificato
$file = FileOpen("C:\Program Files\Sunshine\config\Sunshine.conf", 2)
FileWrite($file, $contenutoModificato)
FileClose($file)

ProcessClose("sunshine.exe")
ProcessClose("sunshine.exe")
sleep (500)
Run(@comspec & ' /c net start "' & "Sunshine Service" & '"', "", @SW_HIDE)

@roob0
Copy link

roob0 commented Jul 20, 2024

Is there a way create the service as per-user for sunshine manually?

I created this simple "temporary" solution written above.

@Kobain-Seo
Copy link

@roob0 thank you for sharing the script. But i want to stream lock screen , at least the screen when windows is shutting down without disconnection. I hope there is a way.

@roob0
Copy link

roob0 commented Jul 20, 2024

@Kobain-Seo Yes, It perfectly streams the lockscreen! I made this script for this purpouse. You can unlock PC from the client with moonlight, then reconnect (always in moonlight) after the pc is unlocked.

@roob0
Copy link

roob0 commented Jul 20, 2024

@Kobain-Seo I usually put my pc in hybrid sleep (and cutting off power when I want to close it, so I don't press "Shutdown"). However, if you want to shutdown your pc in the "classic" way , in switch ddx's scheduler activity you can add the Event in Activator with System, User32, ID 1074. I'm testing the scripts in several scenario (lock-on, lockon and sleep, shutdown) and always work ;)

@roob0
Copy link

roob0 commented Jul 21, 2024

@cgutman I find this simple solution for swicthing from service to the exe and from ddx to wgc (and viceversa). Perhaps a reconnection could be added in Moonlight (avoiding the disconnection) but, of course, a more sophisticated solution would be better. This is only a temporary "solution", waiting for the service to be "fixed".

@Crasthiff
Copy link

Does UAC elevation and service mode work as expected? I've read many microsoft sources that pretty much told that this capture approach is not compatible with unattended use cases

WGC actually can capture the UAC secure desktop, even from an unelevated Sunshine.exe process, so it's actually better than DXGI DDA in that regard. It can even capture the login screen after locking the PC.
However, you are correct that it doesn't work with our service model, at least not without additional changes. WGC works using per-user services which are activated via COM. The specific user service that runs the capture backend is CaptureService which is implemented in %SystemRoot%\system32\CaptureService.dll. The use of a per-user service is probably why the 0x80070424 error is thrown from a service context, because the service indeed doesn't exist for LocalSystem's LUID (which is how per-user services are identified).
Fortunately, the fact that elevation doesn't seem to actually provide any benefit to WGC (unlike DDA) means we do have some options here to support WGC from a service context.
The first and simplest is to try impersonating using the user token from WTSQueryUserToken() (same as we do to start apps). We might just need this for our first activation of WGC (which is the call to winrt::GraphicsCaptureSession::IsSupported()) or we could also need it across other WGC API calls.
If impersonation doesn't work out, we'd need to use a WGC helper process that we would spawn into the user session and share textures with. This is definitely much more complex than the impersonation solution, but it may not be too horrible since we're already using shared textures between capture and encoding today.

I have not tried this scenario and while I did see this somewhere, I figured it was a little out of scope considering that, as far as I can tell, Sunshine is consumer software.

Cross-adapter encoding is good to have on hybrid graphics systems, which are quite typical these days (most CPUs include an iGPU and basically all laptops do). Cross-adapter encoding lets users encoding on the dGPU even when the display is physically connected to the iGPU. We have to use a bunch of hacks (ddprobe.exe manually setting our graphics preference) to work around the cross-adapter encoding limitation for DDA on hybrid graphics systems to even be able to stream at all.

In no way do I advocate replacing the existing display capture code with Windows.Graphics.Capture. I quite like the idea of it being a beta feature that can be enabled in sunshine.conf but not the user interface. I also hope its presence lowers the bar for more knowledgeable contributors to improve this capture backend.

I actually would like to move to WGC as the default on OSes where support is good (Win11/2022+). It has a ton of advantages over the DXGI solution, including:

  • Ability to capture the Game Bar
  • Fixed frame rate capture
  • Cross-adapter capture (untested but claimed by OBS folks)
  • Avoids all the mess of having to blend the cursor into the frame
  • Non-blocking API that doesn't hold critical locks for long periods of time like IDXGIOutputDuplication::AcquireNextFrame() does
  • Could potentially work around the Nvidia driver bug that causes hangs with high VRAM usage (the deadlock in the capture thread is on the keyed mutex inside IDXGIOutputDuplication::AcquireNextFrame())
  • Potential for much better frame pacing and lower latency by avoiding sleeps in our capture loop (we can instead look at the current time in on_frame_arrived() and decide to encode a frame or not based on how long it's been since the last frame)

Obviously it could still have massive Achilles heel that sinks it for our usecase, but it looks quite compelling on the surface.
In any case, none of this aspirational stuff matters for right now. I'm mainly just writing it down somewhere to not forget any of it. You should leave it as opt-in for this PR as you have it now and don't worry about the impersonation stuff.

Hi, I found a simple "temporary" solution, changing from ddx to wgc (and vice versa) on the fly, starting and stopping the service and Sunshine exe automatically, with two autoit exe scripts launched from windows scheduler when pc is locked (and also suspended) and unlocked (and also awaked). However, on client device you must "enter" two times...

Sunshine switch wgc au3 file:

sleep (3000) Global $file = FileOpen("C:\Program Files\Sunshine\config\Sunshine.conf", 0)

Global $contenuto = FileRead($file)

FileClose($file)

Global $contenutoModificato = StringReplace($contenuto, "ddx", "wgc")

$file = FileOpen("C:\Program Files\Sunshine\config\Sunshine.conf", 2) FileWrite($file, $contenutoModificato) FileClose($file)

ProcessClose("sunshine.exe") ProcessClose("sunshine.exe") RunWait(@comspec & ' /c net stop "' & "Sunshine Service" & '"', "", @SW_HIDE) sleep(1500) ShellExecute("C:\Program Files\Sunshine\Sunshine.exe","","C:\Program Files\Sunshine","",@SW_HIDE)

Sunshine Switch ddx:

; Apre il file Global $file = FileOpen("C:\Program Files\Sunshine\config\Sunshine.conf", 0)

; Legge il contenuto del file Global $contenuto = FileRead($file)

; Chiude il file FileClose($file)

Global $contenutoModificato = StringReplace($contenuto, "wgc", "ddx")

; Riapre il file per sovrascrivere il contenuto modificato $file = FileOpen("C:\Program Files\Sunshine\config\Sunshine.conf", 2) FileWrite($file, $contenutoModificato) FileClose($file)

ProcessClose("sunshine.exe") ProcessClose("sunshine.exe") sleep (500) Run(@comspec & ' /c net start "' & "Sunshine Service" & '"', "", @SW_HIDE)

Can't make it work, i've never used autoit + task scheduler so I don't know if I didi it correctly, can you help me here? maybe making a tutorial, or via DM

@Vaskyy
Copy link

Vaskyy commented Sep 19, 2024

Also hoping for an improvement on that part. :)

Can i use the same settings and configfile with that script.
Never used Autoit.
Is there a way to publish an better in depth tutorial to set it up ontop of an existing installation as a service?
Best without the use of different configfiles.

@RebelliousX
Copy link

RebelliousX commented Sep 19, 2024

@Vaskyy don't use AutoIt, just create 2 config files, one with wgc setting like sunshine.conf.bak and the other with dxgi sunshine.conf which is the original. Create a batch file to do the following one step at a time:
Note: run the batch as admin.

  • stop the sunshine service (it has to be done from command prompt, not services gui services.msc which won't work)
    net stop sunshineservice
  • rename the backup sunshine config sunshine.conf.bak to temp.conf
  • rename the current sunshine.conf file, to sunshine.conf.bak
  • rename temp.conf config file to sunshine.conf
  • start sunshine.exe

Run the batch file again to reverse the settings 😊, if you want to use dxgi, don't forget to net start sunshineservice
No need to complicate things using some 3rd party tools 😉

@Vaskyy
Copy link

Vaskyy commented Sep 19, 2024

Damn that took me alot of time, bc im absolutely not advanced in any cmd/ps stuff.

i did create an dxgi.bat:

@echo off
cd C:\Program Files\Sunshine\config
taskkill /im sunshine.exe /f
ren sunshine.conf sunshine-temp.conf
ren sunshine-dxgi.conf sunshine.conf
ren sunshine-temp.conf sunshine-wgc.conf
net start sunshineservice

and also an wgc.bat:

@echo off
cd C:\Program Files\Sunshine\config
net stop sunshineservice
ren sunshine.conf sunshine-temp.conf
ren sunshine-wgc.conf sunshine.conf
ren sunshine-temp.conf sunshine-dxgi.conf
cd C:\Program Files\sunshine
.\sunshine.exe > NUL

But now the wgc.bat is opening a cmd prompt, that if i close it, will terminate the thing of course.
Dont know how to get rid of it.

Also isnt it possible to autostart the sunshine.exe (not the service) to be able to use WGC always?

@Crasthiff
Copy link

Crasthiff commented Oct 1, 2024

Damn that took me alot of time, bc im absolutely not advanced in any cmd/ps stuff.

i did create an dxgi.bat:

@echo off
cd C:\Program Files\Sunshine\config
taskkill /im sunshine.exe /f
ren sunshine.conf sunshine-temp.conf
ren sunshine-dxgi.conf sunshine.conf
ren sunshine-temp.conf sunshine-wgc.conf
net start sunshineservice

and also an wgc.bat:

@echo off
cd C:\Program Files\Sunshine\config
net stop sunshineservice
ren sunshine.conf sunshine-temp.conf
ren sunshine-wgc.conf sunshine.conf
ren sunshine-temp.conf sunshine-dxgi.conf
cd C:\Program Files\sunshine
.\sunshine.exe > NUL

But now the wgc.bat is opening a cmd prompt, that if i close it, will terminate the thing of course. Dont know how to get rid of it.

Also isnt it possible to autostart the sunshine.exe (not the service) to be able to use WGC always?

I actually managed to do this BUT, and is a big BUT, you have to give up some home privacy, if you set sunshine.exe on the start forlder on windows Start Menu sunshine will autostart at widnows LOGON, (not boot up), although you cant set it up before it would be useless on WGC since you can't unlock the pc on WGC, so, my solution? autologon, pc boots and goes staight into windows with sunshine auto starting and letting you use always WGC...

another workaround is to unlock the pc with parsec and then change to sunshine, I actually do this with another decive I have at home to turn my pc ON

@OrimoB
Copy link

OrimoB commented Oct 15, 2024

Damn that took me alot of time, bc im absolutely not advanced in any cmd/ps stuff.
i did create an dxgi.bat:

@echo off
cd C:\Program Files\Sunshine\config
taskkill /im sunshine.exe /f
ren sunshine.conf sunshine-temp.conf
ren sunshine-dxgi.conf sunshine.conf
ren sunshine-temp.conf sunshine-wgc.conf
net start sunshineservice

and also an wgc.bat:

@echo off
cd C:\Program Files\Sunshine\config
net stop sunshineservice
ren sunshine.conf sunshine-temp.conf
ren sunshine-wgc.conf sunshine.conf
ren sunshine-temp.conf sunshine-dxgi.conf
cd C:\Program Files\sunshine
.\sunshine.exe > NUL

But now the wgc.bat is opening a cmd prompt, that if i close it, will terminate the thing of course. Dont know how to get rid of it.

Also isnt it possible to autostart the sunshine.exe (not the service) to be able to use WGC always?

I actually managed to do this BUT, and is a big BUT, you have to give up some home privacy, if you set sunshine.exe on the start forlder on windows Start Menu sunshine will autostart at widnows LOGON, (not boot up), although you cant set it up before it would be useless on WGC since you can't unlock the pc on WGC, so, my solution? autologon, pc boots and goes staight into windows with sunshine auto starting and letting you use always WGC...

another workaround is to unlock the pc with parsec and then change to sunshine, I actually do this with another decive I have at home to turn my pc ON

Funny because since I use virtual display driver, no one see my PC will know my PC was on unlocked haha they just think display is always off, you need change the cableing OS might output to it or you need to adjust in Moonlight remotely.

@Crasthiff
Copy link

Damn that took me alot of time, bc im absolutely not advanced in any cmd/ps stuff.
i did create an dxgi.bat:

@echo off
cd C:\Program Files\Sunshine\config
taskkill /im sunshine.exe /f
ren sunshine.conf sunshine-temp.conf
ren sunshine-dxgi.conf sunshine.conf
ren sunshine-temp.conf sunshine-wgc.conf
net start sunshineservice

and also an wgc.bat:

@echo off
cd C:\Program Files\Sunshine\config
net stop sunshineservice
ren sunshine.conf sunshine-temp.conf
ren sunshine-wgc.conf sunshine.conf
ren sunshine-temp.conf sunshine-dxgi.conf
cd C:\Program Files\sunshine
.\sunshine.exe > NUL

But now the wgc.bat is opening a cmd prompt, that if i close it, will terminate the thing of course. Dont know how to get rid of it.

Also isnt it possible to autostart the sunshine.exe (not the service) to be able to use WGC always?

I actually managed to do this BUT, and is a big BUT, you have to give up some home privacy, if you set sunshine.exe on the start forlder on windows Start Menu sunshine will autostart at widnows LOGON, (not boot up), although you cant set it up before it would be useless on WGC since you can't unlock the pc on WGC, so, my solution? autologon, pc boots and goes staight into windows with sunshine auto starting and letting you use always WGC...
another workaround is to unlock the pc with parsec and then change to sunshine, I actually do this with another decive I have at home to turn my pc ON

Funny because since I use virtual display driver, no one see my PC will know my PC was on unlocked haha they just think display is always off, you need change the cableing OS might output to it or you need to adjust in Moonlight remotely.

there you go, try my workaround, is not ideal as a service, but in practice it does the job.
Actually this setting is perfect fot a VDD setup since the the only way to access is through moonlite or, as you said, cabling to a monitor. I had a similar setup with just the tower as a server and al my peripherals connected to an intel NUC working as my desktop gaming pc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Xbox Game Bar doesn't show up in the stream