Implement vsock support for FreeBSD

[ziggythehamster]

FreeBSD supports something very nearly the same as Linux's vsock with Hyper-V, and this patch adds support for it to xrdp.

Practically speaking, this allows you to use enhanced session mode in Hyper-V, if you run Set-VM -EnhancedSessionTransportType HvSocket on the VM in PowerShell. That said, the performance is bad because it is fully software rendered, because the Hyper-V enhanced DRM/DRI driver hasn't been ported to FreeBSD yet (there's a pending change to implement the enhanced framebuffer, but I doubt that makes xorgxrdp able to use hardware accelerated rendering).

One bug is that when g_sck_close tries to run getsockname, EINVAL is returned and the socket is closed normally, but looking at the code, I think that also happens with Linux because sock_info.sa rather than sock_info.sa_vm or sock_info.sa_hvs are passed in, and sa is not the correct structure for the open vsock/hv_sock socket. Fixing this feels like it's probably out of scope for this change, since both Unix sockets and Linux vsocks would suffer from the same problem.

Here's a screenshot showing it in action, forgive the Windows DPI being messed up (I need to reboot, I think, to fix it):

[metalefty]

Awesome! Could you fix the formatting and cppcheck?

[ziggythehamster]

I believe I've resolved both issues. I also changed the configure script to check for AF_HYPERV, since it looks like this is only available in FreeBSD 13+, and someone may try this in an older FreeBSD.

[matt335672]

Code looks clean to me. I can't test, but I'd be happy to merge.

[metalefty]

I will merge after some other PRs.

[metalefty]

@ziggythehamster I have updated FreeBSD net/xrdp-devel port and defaulted --enable-vsock.

[ziggythehamster]

@metalefty oh, awesome. I was expecting to have to hunt down the port maintainer to get them to change the Makefile :D

[metalefty]

I am the maintainer 👍