Draw layer-shell popups on top of regular windows

[emersion]

xdg-shell popups are already drawn on top of other toplevels, regardless of the toplevels z-index. We should do the same for layer-shell popups.

Original issue: swaywm/wlr-protocols#60

[ammen99]

I do not like this solution, because I do not see such behavior specified anywhere in the wayland protocols. Can you elaborate why Sway even started drawing in this way for xdg-shell windows?

[David96]

@ammen99 And what would you suggest about the problem I described in the issue linked by emersion? I think it is reasonable for popups to be positioned over regular windows. Popups very often overlap other windows and usually you want to be able to see them (just like for the popups of waybar that show additional information when hovering some item).

[ammen99]

The solution would be that the compositor lets you configure order of surfaces by their namespace. Drawing popups on top will solve your problem, yes, but doesn't fix the fundamental issue - layer-shell's layers are nowhere near enough.

[emersion]

I do not see such behavior specified anywhere in the wayland protocols

It's a compositor policy. xdg-popup doesn't specify that popups should be drawn over everything else.

Can you elaborate why Sway even started drawing in this way for xdg-shell windows?

Let's say you have two overlapping windows: a text editor on top of a web browser. Let's say the user moves the mouse over a link in the web browser and the web browser shows a tooltip with xdg-popup. You want to draw the tooltip on top of both windows.

[David96]

The solution would be that the compositor lets you configure order of surfaces by their namespace. Drawing popups on top will solve your problem, yes, but doesn't fix the fundamental issue - layer-shell's layers are nowhere near enough.

But I really don't want to start thinking about exactly at which layer what application should be drawn - that'd be additional work that has to be done for every panel or application starter I want to try out. On the other hand, I can't imagine a usecase where popups shouldn't be drawn over other windows. This would really make life easier.

[ammen99]

@David96 You don't want to but you'll have to think it as soon as you want to add a third application which belongs to the top layer :)

Anyway I see your point, maybe this isn't a bad idea after all.

[David96]

I would be willing to spend some time trying to implement this but I have zero experience with Wayland and basically none with wlroots and the sway codebase, so some pointer as to where to start would be appreciated.
There's the popup_get_layer function in layer_shell.c - would that be the right place to try to return the top layer instead of the parent layer? Or would I somehow have to make sure that popup.parent_layer points to the top layer?

[emersion]

would that be the right place to try to return the top layer instead of the parent layer?

The code choosing the z-order is in sway/desktop/render.c. Depending on the ordering of the rendering operations, a surface will appear on top or below another. We'd need to stop using wlr_layer_surface_v1_for_each_surface, just like xdg-shell surfaces, and instead render the main surface and the popups separately.