wl-paste hangs when pasting text from a closed XWayland application

[fredizzimo]

1. Open a XWayland application
2. Copy some text
3. exit the application
4. run wl-paste and observe that it hangs

This happens 100% of the time, but only when pasting from XWayland applications.

This is probably a more general problem, since wl-clipboard-rs also has the same problem. But it's basically a showstopper for me since wl-clipboard is used by Neovim and causes Neovim to hang as well.

Pasting into other applications like KWrite does not paste anything either, but it also doesn't hang. The text is still in the Klipper history, and can be selected, and then pasting works again.

running WAYLAND_DEBUG=1 wl-paste gives the following

❯ WAYLAND_DEBUG=1 wl-paste
[2017284.754]  -> wl_display@1.get_registry(new id wl_registry@2)
[2017284.763]  -> wl_display@1.sync(new id wl_callback@3)
[2017288.747] wl_display@1.delete_id(3)
[2017288.753] wl_registry@2.global(1, "wl_compositor", 5)
[2017288.757]  -> wl_registry@2.bind(1, "wl_compositor", 2, new id [unknown]@4)
[2017288.759] wl_registry@2.global(3, "zwp_tablet_manager_v2", 1)
[2017288.761] wl_registry@2.global(4, "zwp_keyboard_shortcuts_inhibit_manager_v1", 1)
[2017288.763] wl_registry@2.global(6, "xdg_wm_base", 4)
[2017288.767]  -> wl_registry@2.bind(6, "xdg_wm_base", 1, new id [unknown]@5)
[2017288.770] wl_registry@2.global(7, "zwlr_layer_shell_v1", 3)
[2017288.773] wl_registry@2.global(8, "zxdg_decoration_manager_v1", 1)
[2017288.775] wl_registry@2.global(9, "wp_viewporter", 1)
[2017288.778] wl_registry@2.global(10, "wp_fractional_scale_manager_v1", 1)
[2017288.780] wl_registry@2.global(11, "wl_shm", 1)
[2017288.783]  -> wl_registry@2.bind(11, "wl_shm", 1, new id [unknown]@6)
[2017288.785] wl_registry@2.global(12, "wl_seat", 8)
[2017288.788]  -> wl_registry@2.bind(12, "wl_seat", 2, new id [unknown]@7)
[2017288.791] wl_registry@2.global(13, "zwp_pointer_gestures_v1", 3)
[2017288.793] wl_registry@2.global(14, "zwp_pointer_constraints_v1", 1)
[2017288.796] wl_registry@2.global(15, "zwp_relative_pointer_manager_v1", 1)
[2017288.798] wl_registry@2.global(16, "wl_data_device_manager", 3)
[2017288.801]  -> wl_registry@2.bind(16, "wl_data_device_manager", 1, new id [unknown]@8)
[2017288.804] wl_registry@2.global(17, "zwlr_data_control_manager_v1", 2)
[2017288.807]  -> wl_registry@2.bind(17, "zwlr_data_control_manager_v1", 2, new id [unknown]@9)
[2017288.809] wl_registry@2.global(18, "zwp_primary_selection_device_manager_v1", 1)
[2017288.812]  -> wl_registry@2.bind(18, "zwp_primary_selection_device_manager_v1", 1, new id [unknown]@10)
[2017288.815] wl_registry@2.global(19, "org_kde_kwin_idle", 1)
[2017288.818] wl_registry@2.global(20, "zwp_idle_inhibit_manager_v1", 1)
[2017288.820] wl_registry@2.global(21, "ext_idle_notifier_v1", 1)
[2017288.823] wl_registry@2.global(22, "org_kde_plasma_shell", 8)
[2017288.825] wl_registry@2.global(23, "org_kde_kwin_appmenu_manager", 1)
[2017288.828] wl_registry@2.global(24, "org_kde_kwin_server_decoration_palette_manager", 1)
[2017288.830] wl_registry@2.global(26, "org_kde_plasma_virtual_desktop_management", 2)
[2017288.833] wl_registry@2.global(28, "org_kde_kwin_shadow_manager", 2)
[2017288.835] wl_registry@2.global(29, "org_kde_kwin_dpms_manager", 1)
[2017288.838] wl_registry@2.global(30, "org_kde_kwin_server_decoration_manager", 1)
[2017288.840] wl_registry@2.global(31, "kde_output_management_v2", 3)
[2017288.843] wl_registry@2.global(32, "zxdg_output_manager_v1", 3)
[2017288.846] wl_registry@2.global(33, "wl_subcompositor", 1)
[2017288.848] wl_registry@2.global(34, "zxdg_exporter_v2", 1)
[2017288.851] wl_registry@2.global(35, "zxdg_importer_v2", 1)
[2017288.853] wl_registry@2.global(38, "xdg_activation_v1", 1)
[2017288.856] wl_registry@2.global(40, "wp_content_type_manager_v1", 1)
[2017288.859] wl_registry@2.global(41, "wp_tearing_control_manager_v1", 1)
[2017288.862] wl_registry@2.global(43, "wl_eglstream_display", 1)
[2017288.864] wl_registry@2.global(44, "wl_drm", 2)
[2017288.867] wl_registry@2.global(45, "zwp_linux_dmabuf_v1", 4)
[2017288.869] wl_registry@2.global(47, "kde_output_device_v2", 2)
[2017288.872] wl_registry@2.global(49, "wl_output", 4)
[2017288.874] wl_registry@2.global(51, "wp_drm_lease_device_v1", 1)
[2017288.877] wl_registry@2.global(52, "kde_output_order_v1", 1)
[2017288.880] wl_registry@2.global(53, "zwp_text_input_manager_v1", 1)
[2017288.882] wl_registry@2.global(54, "zwp_text_input_manager_v2", 1)
[2017288.885] wl_registry@2.global(55, "zwp_text_input_manager_v3", 1)
[2017288.888] wl_registry@2.global(57, "org_kde_kwin_contrast_manager", 2)
[2017288.890] wl_registry@2.global(58, "org_kde_kwin_blur_manager", 1)
[2017288.893] wl_registry@2.global(59, "org_kde_kwin_slide_manager", 1)
[2017288.895] wl_registry@2.global(60, "kde_output_device_v2", 2)
[2017288.898] wl_registry@2.global(61, "wl_output", 4)
[2017288.900] wl_callback@3.done(25623)
[2017288.903]  -> wl_display@1.sync(new id wl_callback@3)
[2017291.546] wl_display@1.delete_id(3)
[2017291.551] wl_seat@7.capabilities(7)
[2017291.553] wl_seat@7.name("")
[2017291.555] wl_callback@3.done(25623)
[2017291.559]  -> zwlr_data_control_manager_v1@9.get_data_device(new id zwlr_data_control_device_v1@3, wl_seat@7)
[2017291.597] zwlr_data_control_device_v1@3.data_offer(new id zwlr_data_control_offer_v1@4278190080)
[2017291.600] zwlr_data_control_offer_v1@4278190080.offer("text/plain")
[2017291.602] zwlr_data_control_offer_v1@4278190080.offer("text/plain;charset=utf-8")
[2017291.604] zwlr_data_control_offer_v1@4278190080.offer("STRING")
[2017291.607] zwlr_data_control_offer_v1@4278190080.offer("text/plain")
[2017291.610] zwlr_data_control_offer_v1@4278190080.offer("TARGETS")
[2017291.613] zwlr_data_control_offer_v1@4278190080.offer("MULTIPLE")
[2017291.615] zwlr_data_control_offer_v1@4278190080.offer("TIMESTAMP")
[2017291.618] zwlr_data_control_offer_v1@4278190080.offer("SAVE_TARGETS")
[2017291.621] zwlr_data_control_device_v1@3.selection(zwlr_data_control_offer_v1@4278190080)
[2017291.631]  -> zwlr_data_control_offer_v1@4278190080.receive("text/plain;charset=utf-8", fd 6)
[2017291.635]  -> wl_display@1.sync(new id wl_callback@11)
[2017291.638] zwlr_data_control_device_v1@3.data_offer(new id zwlr_data_control_offer_v1@4278190081)
[2017291.641] zwlr_data_control_offer_v1@4278190081.offer("text/plain")
[2017291.644] zwlr_data_control_offer_v1@4278190081.offer("application/x-kde-onlyReplaceEmpty")
[2017291.647] zwlr_data_control_offer_v1@4278190081.offer("text/plain;charset=utf-8")
[2017291.650] zwlr_data_control_device_v1@3.primary_selection(zwlr_data_control_offer_v1@4278190081)
[2017291.653]  -> zwlr_data_control_offer_v1@4278190081.destroy()
[2017291.685] wl_display@1.delete_id(11)
[2017291.687] wl_callback@11.done(25623)
^C

My specs
Arch Linux
wl-clipboard 2.1.0
KDE Plasma 5.27.6
xorg-xwayland 23.1.2-1

NOTE: This is probably closely related to #149

Edit: The root cause is this Klipper bug https://bugs.kde.org/show_bug.cgi?id=449909. But still, wl-paste should not hang.

[bugaevc]

Hello and thanks for the clear report!

Edit: The root cause is this Klipper bug https://bugs.kde.org/show_bug.cgi?id=449909. But still, wl-paste should not hang.

(Please don't ever add important information in an edit 🙁 People who are subscribed to the issue, myself included, do not get notified when you edit your posts, new emails don't get sent, etc. I happened to see the edit, but I could have just as easily missed it. Edits are for fixing typos, not for adding new content.)

The reason KWrite doesn't hang is likely that it uses asynchronous I/O to receive clipboard contents without blocking the main thread. This very much makes sense for a GUI app, it must do that to keep the UI responsive while transferring the data.

wl-paste is not a GUI app, it's a Unix command line tool, so it uses blocking I/O like Unix command line tools are supposed to. For command line tools, it makes sense that they run/block for as long as it takes to perform their function (in this case, reading the data form the clipboard and writing it to stdout), and then exit. You'd get the same blocking behavior if you use another command-line Unix tool (for instance, wc) on some file that can block on reading (such as a socket or a pipe with no data buffered). Try sleep 100 | wc -- see, wc "hangs"!

The real bug here is not the wl-clipboard hangs, but that the data that wl-clipboard waits for never comes. This must either be the Klipper bug you've linked to, or perhaps something in KWin and XWayland integration.

In the issue report you've linked to, KDE developers were unable to reproduce the bug, perhaps you could help them with that?

[fredizzimo]

wl-paste is not a GUI app, it's a Unix command line tool, so it uses blocking I/O like Unix command line tools are supposed to. For command line tools, it makes sense that they run/block for as long as it takes to perform their function (in this case, reading the data form the clipboard and writing it to stdout), and then exit. You'd get the same blocking behavior if you use another command-line Unix tool (for instance, wc) on some file that can block on reading (such as a socket or a pipe with no data buffered). Try sleep 100 | wc -- see, wc "hangs"!

I disagree, the help for wl-paste says

Paste content from the Wayland clipboard

So it should either paste the content, or fail if it can't do that, not hang forever. What does it wait for here? Copying a valid content to the clipboard does not wake it up.

In the issue report you've linked to, KDE developers were unable to reproduce the bug, perhaps you could help them with that?

In this comment https://bugs.kde.org/show_bug.cgi?id=449909#c13, Nate Graham, which I assume is a developer was able to reproduce it. But I fear the bug does not have a very high priority from KDE's perspective, you can easily select the entry from Klipper to make it valid and paste it again.

With wl-paste and Neovim (a terminal application), which uses wl-clipboard as the default on Wayland, since that's as far as I know the only alternative available, this is an absolute showstopper though. The only way to recover in most cases is to kill Neovim, and possibly loose your work, in some rare cases ctrl-c works.

[fredizzimo]

I disabled Klipper, and installed CopyQ instead, but the same issue persists. But the root cause could be in kwin, which would mean it's still a KDE issue in the end, but other applications are able to deal with it, so I don't see why wl-clipboard shouldn't.

I furthermore, found out that pasting in zsh with oh-my-zsh also hangs until you press ctrl-c to abort it, and indeed that one also uses wl-paste for pasting on Wayland.

[bugaevc]

I disagree, the help for wl-paste says

Paste content from the Wayland clipboard

So it should either paste the content, or fail if it can't do that, not hang forever.

Well, I'm sure help for wc also says that it counts lines/words/bytes, not hangs forever 🙂

What does it wait for here?

I think I might understand why you're thinking of it this way. If the Wayland clipboard was just a "place" maintained by the compositor where you could put things into (copy) and get things from (paste), and wl-clipboard would indeed just wait for who-knows-what in this weird case, then it'd be appropriate to call it a hang, and indeed, why would wl-clipboard even do that, why wouldn't it just report an error and exit if it cannot paste. Is that what you thinking?

But that's not how clipboard works in Wayland (nor in X11). Clipboard is not a place, it's the interaction between the two clients (apps), the one who has copied the data and the one that's pasting.

When you copy (with Ctrl-C, or with wl-copy), you don't actually send the data to the compositor, all you do is you tell the compositor: hey, I have some data here, and I can provide it in this and that MIME types on request. When you want to paste, you don't just get the data from the compositor, instead you ask the compositor: I want to paste; who has the current clipboard contents? The compositor only acts as a broker, it connects the two clients, telling the first client to send the data to a pipe, and the second client to read the data from the pipe. It's the first client who actually sends the data to the second client, not the compositor. If the first client sends the data slowly, the second client will get the data slowly. If the first client hangs for three minutes and then sends the data, the second client will only get the data after three minutes.

wl-paste, in fact, quite literally spawns a cat process to copy the data from the pipe to its stdout. Naturally, the cat will "hang" (or rather: block on reading) if the other client sends it nothing, but keeps the pipe open. This is exactly like sleep 10000 | cat: the cat "hanging" is not an issue with cat, it's just that the sleep is sleeping instead of sending any data (except that sleep is not supposed to send any data, but the copying client is, and not sending it is a bug).

To reiterate: you cannot "just get" the clipboard contents, you have to wait for the other client to transfer them to you. That's just how it works. It only makes sense that wl-paste waits for the data transfer to complete (i.e. for the other client to write all the data and close the pipe). It could, in theory, add a timeout here ("it's been 2 minutes and you still haven't sent me a single byte, something must be wrong!"), but that doesn't sound very useful.

Copying a valid content to the clipboard does not wake it up.

That's because it's still trying to paste from the old client. By that point wl-paste is essentially done interacting with the compositor, it's connected (by a pipe) directly to the other client, and is waiting for the other client to transfer the data.

In this comment https://bugs.kde.org/show_bug.cgi?id=449909#c13, Nate Graham, which I assume is a developer was able to reproduce it.

Ah, I see, great. Yes, Nate is a KDE developer.

this is an absolute showstopper though. The only way to recover in most cases is to kill Neovim, and possibly loose your work, in some rare cases ctrl-c works.

I imagine you could just interrupt wl-paste (try pkill wl-paste; perhaps suspend Neovim first with Ctrl-Z if you don't have another console), but I don't know any Neovim so I can't exactly help you with that.

I disabled Klipper, and installed CopyQ instead, but the same issue persists. But the root cause could be in kwin, which would mean it's still a KDE issue in the end, but other applications are able to deal with it, so I don't see why wl-clipboard shouldn't.

As I said, the other applications likely "deal with it" by doing the data transfer in parallel (or rather, concurrently) with continuing to update the UI, reacting to mouse events, etc. If you peek under the hood, you should see that the data transfer (the attempt to paste) is still ongoing. (Perhaps they also have a way to cancel the transfer, either on timeout or if new a new data offer appears.)

wl-paste doesn't have a UI to maintain in addition to pasting; pasting is its only task. Moreover, it clearly differentiates between could not paste (exiting with an error) and waiting for data transfer ("hanging"), which GUI apps typically do not (you just see that no new content appears when you hit Ctrl-V).

The relevant code in KWin seems to be src/xwayland/clipboard.cpp and src/xwayland/selection.cpp. Perhaps they're not handling the XFixes SelectionWindowDestroy & SelectionClientClose events properly? (They clearly do register to receive them, though). It looks like it calls Selection::handleXfixesNotify() -> Clipboard::doHandleXfixesNotify() -> Selection::createX11Source(nullptr) -> setWlSource(nullptr). But they don't seem to ever call waylandServer()->seat()->setSelection(nullptr). I might be wrong about this part though.

[bugaevc]

Two more things:

1. In case of Neovim, Neovim should accept that wl-paste may block for some time (if the other client is slow), and run wl-paste asynchronously, without locking up the rest of Neovim (basically, don't do a synchronous wait() without WNOHANG on the main thread). It's just like those other GUI apps do the data transfer asynchronously, without blocking the UI; Neovim should do the same thing (except that its way of reading the clipboard is spawning wl-paste, not talking Wayland directly to compositor). The place to add asynchrony here is Neovim, not wl-paste.

2. While the model I described above presents clipboard as an interaction between two clients, the compositor might actually play the role of one of these clients (unbeknown to the other client), and either provide or request the data itself. In this specific case, the actual data is supposed to be provided to KWin by an X11 client (that has gone away), but it's KWin who sits at the other end of the pipe, not another Wayland client. This changes nothing in the overall picture though.