Restore the ability to have fully transparent windows on Windows

[dotdash]

Besides its original purpose, commit 6343059 "Fix Windows transparency
behavior to support fully-opaque regions (#1621)" also included some
changes considered cleanups, one of them was:

• Remove the CreateRectRgn call, since we want the entire window's region to
  have blur behind it, and DwnEnableBlurBehindWindow does that by default.

But the original code actually disabled the blur effect for the whole
window by creating an empty region for it, because that allows for the
window to be truely fully transparent. With the blur effect in place,
the areas meant to be transparent either blur the things behind it
(until Windows 8) or are darkened (since Windows 8). This also means
that on Windows 8 and newer, the resulting colors are darker than
intended in translucent areas when the blur effect is enabled.

This restores the behaviour from winit <0.24 and fixes #1814.

Arguably, one might want to expose the ability to control the blur
region, but that is outside the scope of this commit.

• Tested on all platforms changed
• Compilation warnings were addressed
• cargo fmt has been run on this branch
• cargo doc builds successfully
• Added an entry to CHANGELOG.md if knowledge of this change could be valuable to users
• Updated documentation to reflect any user-facing changes, including notes of platform-specific behavior
• Created or updated an example program if it would help users understand this functionality
• Updated feature matrix, if new features were added or implemented

[maroider]

This PR looks like it's mostly a revert of 6343059, but without the following "hack":

// HACK: When opaque (opacity 255), there is a trail whenever
// the transparent window is moved. By reducing it to 254,
// the window is rendered properly.
let opacity = 254;

// The color key can be any value except for black (0x0).
let color_key = 0x0030c100;
winuser::SetLayeredWindowAttributes(
    real_window.0,
    color_key,
    opacity,
    winuser::LWA_ALPHA,
);

The comment in this snippet claims that it solves an issue observed during the development of #675, and it seems like this is what caused alacritty/alacritty#1927, which got fixed in #1621. This PR does seem to fix the issue described in #1814, but I'm not too sure if it's the best way to do it.

Commenting out this call to DwmEnableBlurBehindWindow makes the window fully transparent to me. I'm not sure why DwmEnableBlurBehindWindow is being called here in the first place since (as I understand it) WS_EX_LAYERED and SetLayeredWindowAttributes with LWA_ALPHA should be enough to achieve transparency. I've managed to track down where the call was introduced, but no explanation is provided there. I unfortunately don't have older versions of Windows to test on, so I can't really check if my assertion is true. I also haven't tested if it works with rendered content, or if it looks OK when dragged around.

[dotdash]

This PR looks like it's mostly a revert of 6343059

Correct. As I mentioned in the commit message, the part that is being reverted here was claimed to be essentially a no-op, which it is not. I guess the author thought that the region covers the whole client area when it actually covers none of it. So there was a false premise for this change.

Commenting out this call to DwmEnableBlurBehindWindow makes the window fully transparent to me. I'm not sure why DwmEnableBlurBehindWindow is being called here in the first place since (as I understand it) WS_EX_LAYERED and SetLayeredWindowAttributes with LWA_ALPHA should be enough to achieve transparency.

I suppose that only works for the transparent example, because it never draws anything. With the blur behind effect enabled, the DWM does the effect itself, but then nothing is drawn, and with that effect disabled simply nothing at all happens. Notice that the window has decorations disabled, so the call to SetLayeredWindowAttributes never happens. And even if it did happen, SetLayeredWindowAttributes with LWA_ALPHA only turns the whole window, including its decorations, transparent to a certain degree, it does not allow the application itself to control per pixel transparency. So that alone would not work for my use case.

I've managed to track down where the call was introduced, but no explanation is provided there.

Without decorations, just the blur behind call with the empty regions works for me, but when I enable decorations, the call to SetLayeredWindowAttributes is required, otherwise the stuff I draw via OpenGL is not visible at all. I've tracked that down to the WS_EX_LAYERED style flag. That requires a call to SetLayeredWindowAttributes or UpdateLayeredWindow to make the window work, see the second paragraph of https://docs.microsoft.com/en-us/windows/win32/winmsg/window-features#layered-windows .

I'm not really sure why that style is required. It's from 28775be, which is yet another commit that claims to fix some issue with a trail being left behind (only in the changelog, not even in the commit message itself) with no further explanation being given. :-(

Removing both the code that sets the WS_EX_LAYERED attribute as well as the call to SetLayeredWindowAttributes has no visible ill-effects for me, neither with nor without window decorations. Also, WS_EX_LAYERED is not supposed to be used with windows that have CS_OWNDC (which is set for windows created by winit), see the description of WS_EX_LAYERED on https://docs.microsoft.com/en-us/windows/win32/winmsg/extended-window-styles

For per-pixel transparency, the documentation also says that you need to use UpdateLayeredWindow rather than SetLayeredWindowAttributes to achieve that. So really the existing call with full opacity seems to be just there to make the window work at all, but is otherwise useless, as is the WS_EX_LAYERED flag.

CS_OWNDC was added by @tomaka back in 2014, and a quick Google search reveals that it is considered evil except when using OpenGL, see https://stackoverflow.com/questions/48663815/getdc-releasedc-cs-owndc-with-opengl-and-gdi - Given the author of that commit, it seems quite possible that OpenGL usage was involved, even though the commit gives no rationale for the change at all.

I have a branch at https://github.com/dotdash/winit/tree/test that has a commit that removes the WS_EX_LAYERED related stuff, so people can test this. If it works, I'll write up a proper commit message for that commit and update the comment on the blur behind call, to say that this call enables per-pixel alpha for the application, while explicitly disabling the actual blur effect. Using WS_EX_LAYERED with UpdateLayeredWindow seems like a more "flexible" (or "proper"?) approach, but looking at its documentation, it seems to be at odds with the abstractions made by winit.

---

On a side note, to whom it may concern, and (mostly) JFYI: You seem to be using the "rebase and merge" option for pull requests, and as a side effect of that, commit ids between the master branch and the pull requests differ, which made it quite a bit harder for me to correlate things between the master branch and github, and looking up where changes came from.

It also makes it way harder to identify whether things happened on indepedent branches independently, and each branch was correct on its own, but the merge results is invalid not because of textual conflicts but because of semantic conflicts. The rebase approach completely dismisses that information.

It also makes the automated checks a bit less useful. If there are changes on master that semantically break the changes in the pull request, then the rebased commits will be broken and e.g. git bisect will simply point at them instead of pointing at the merge commit. So while the checks said that the commits are fine, the results on master will say that the commits are broken, which is quite confusing until you realize that the commits aren't the same at all. And since the rebase threw away the information about the original base commit for the PR, you need to awkwardly pull that out of github somehow (I didn't find a reliable way quickly) if you want to get some clues about which intermediate commit might be responsible.

For example, for #1740 you can find the information by digging through the logs of the checks, seeing that it was tested by merging into 96809ac but was then rebased onto ee3996c, so the test results are meaningless for the commit that ended up in the master branch, while with a true merge, you'd at least know that the merged commits are fine, and were compatible with master at the time of the check.

Ok, that got way longer than I anticipated. I guess I just got a bit frustrated because analyzing this was made so much harder by this. What you make of this is up to you of course.

[tomaka]

CS_OWNDC was added by @tomaka back in 2014, and a quick Google search reveals that it is considered evil except when using OpenGL, see https://stackoverflow.com/questions/48663815/getdc-releasedc-cs-owndc-with-opengl-and-gdi - Given the author of that commit, it seems quite possible that OpenGL usage was involved, even though the commit gives no rationale for the change at all.

If it's back in 2014, then winit was not extracted yet from the glutin library, whose sole purpose was to open a window with an OpenGL context.

[dotdash]

CS_OWNDC was added by @tomaka back in 2014, and a quick Google search reveals that it is considered evil except when using OpenGL, see https://stackoverflow.com/questions/48663815/getdc-releasedc-cs-owndc-with-opengl-and-gdi - Given the author of that commit, it seems quite possible that OpenGL usage was involved, even though the commit gives no rationale for the change at all.

If it's back in 2014, then winit was not extracted yet from the glutin library, whose sole purpose was to open a window with an OpenGL context.

Thanks! Just in case, do you know whether it would be possible or a good idea to remove the CS_OWNDC flag in winit and asks users that need it to set it later on?

[tomaka]

Honestly I haven't touched anything related to Windows windowing for several years, and I don't remember at all what CS_OWNDC even is

[maroider]

It might be possible to use SetClassLongW.

[maroider]

I have a branch at https://github.com/dotdash/winit/tree/test that has a commit that removes the WS_EX_LAYERED related stuff, so people can test this.

I tried your changes with Glutin's transparent example (with and without decorations), and they seem to have no adverse effects for me. I am experiencing some "trailing" around the task bar, but I suspect that is due to monitor ghosting. The ghosting appears both with current master as well as both of your commits.

For reference: I also used the following here so I could drag the window around without decorations:

winuser::WM_NCHITTEST => winuser::HTCAPTION,

It would be nice to have someone with an older version of Windows test this since I only have Windows 10.

cc @0xpr03, since you've listed yourself as a tester for Windows 7

[msiglreith]

Hi and thanks for investigating!

The initial PR for this was #675, which seems to initially be added for win10 (#260 (comment))

I'm in general in favor of removing the SetLayeredWindowAttributes with the current fixes. Even though the UpdateLayeredWindow sounds nice on paper I'm a bit wary about performance compared to interacting with the DWM. Will test it out here as well later on.

[msiglreith]

Sorry, took me a bit to finally try it out.
Tried it with an OpenGL application as well and works fine!
Same behavior when running with the additional change for removing SetLayeredWindowAttributes.

Please, add a short entry to the CHANGELOG and I think adding the other commit as well would good.

Thanks!

[maroider]

Just in case it was unclear what it was that I wanted you to test (on Windows 7), @0xpr03:

• Get the latest master of Glutin.
• Add the following here:

[patch.'crates-io'.winit]
git = "https://github.com/dotdash/winit"
branch = "fix_transparency"

• Run the transparent example.h
• Check if the transparent bits are blurred or clear.

I honestly don't expect to see any issues here, but it would be nice to have it confirmed.

[0xpr03]

Ah thanks, it looked like this already got approved based on the comment of msiglreith

[maroider]

it looked like this already got approved

Yeah, I just want to have some form of confirmation that it actually works on Windows 7, since DwnEnableBlurBehindWindow has slightly different behaviour on Windows 7, 8 (and maybe 10).

[0xpr03]

I'm sorry but it looks like my windows 7 VM can't do this..

Err value: NoAvaila
blePixelFormat', glutin_examples\examples\transparent.rs:15:74

It's already a pain to get the right toolchains for win7. rust-msvc won't work anymore, it requires a newer .NET that won't install on win7 or is straight up "unsupported", even if you manage to get license access to msvc 2013-2017

[0xpr03]

Update: There is hope, my Linux system has a working windows 7 VM with transparency. Seems my windows box has a windows 7 that is stuck with outdated stuff.

[0xpr03]

Sorry for the noise but nope.. I can have transparent windows and run games in that VM but apparently not this pixel format.

[dotdash]

Sorry for letting this lay around for so long, I've now added the other commit with a proper commit message.

[Ciantic]

I just did similar thing few weeks ago (transparency to druid) on Windows side.

I got an answer from Microsoft devs that the most performant way to do is with WS_EX_NOREDIRECTIONBITMAP. They pointed out that the tutorial I found: "Windows with C++ : High-Performance Window Layering Using the Windows Composition Engine" is still essentially the latest and most performant way to do it.

If you use WS_EX_NOREDIRECTIONBITMAP, using negative margins trick is not necessary, you can even have a title-bar if you for some reason want in the transparent window.

EDIT I'm reading the logic here, maybe the idea is that I can use WS_EX_NOREDIRECTIONBITMAP with transparency if I set window to transparent, and flag it with WindowFlags::NO_BACK_BUFFER.

[maroider]

Thanks for the tutorial link, it was an interesting read :)

If I read that correctly, then all you should need to do is to call WindowBuilderExtWindows::with_no_redirection_bitmap(true) when building the window. The blur-behind tick introduced here isn't used if the window is created with WindowBuilderExtWindows::with_no_redirection_bitmap. You probably don't even need to use WindowBuilder::with_transparent.

[Ciantic]

The black art of Windows transparency! I know one more way (that doesn't involve legacy layered windows):

// Copy pasted from C++
// Creates DWM surface 
MARGINS m = { -1 };
if (!SUCCEEDED(DwmExtendFrameIntoClientArea(hwnd, &m))) {
    throw runtime_error("Unable to extend");
}

I used it once in my C++ code. Maybe it's similar to used DwnEnableBlurBehindWindow but it doesn't do blurring by default, maybe it could be used here instead if the Blurring has issues.

It works only right after CreateWindow(Ex).

With the blur effect in place,
the areas meant to be transparent either blur the things behind it
(until Windows 8) or are darkened (since Windows 8). This also means
that on Windows 8 and newer, the resulting colors are darker than
intended in translucent areas when the blur effect is enabled.

DwmExtendFrameIntoClientArea trick doesn't require using that Blur feature at all, it just makes it transparent for some reason.