WSLg in a VM - GUI app not showing when dedicated GPU is added

[cave35]

I am running Windows 11 22H2 in a Proxmox VM. RTX 3060 is passthrough to the VM. Windows recognized the GPU and install 461 version of driver automatically. After installing wsl with default Ubuntu distribution and install xterm. Everything works.
Then I update the driver to the latest from nVidia, 527.56, running xterm shows nothing:

Install mesa-utils and run glxinfo -B also shows nothing:

On the same PC, instead of Proxmox, I install windows 11 directly, everything works with the latest drivers. So it may have to do with VMs only. Has anyone able to run WSLg in a VM and able to run GUI apps with a GPU?

One more info in case that helps, if I install the standard drivers from nVidia, 472 from 2021, it works if only the drivers are installed, meaning no audio, no physx and no GeForce Experience. If any of those features are installed it will not work. The latest nVidia drivers are DCH and none of them works in the VM.

Thanks