Linux KVM (x86 virtualization) support

[joncampbell123]

Is your feature request related to a problem? Please describe.
Add a core to DOSBox-X if possible that runs the guest OS using the virtualization extensions of the processor.

Linux has a KVM virtualization interface for this purpose.

[joncampbell123]

Here's a good example of a hypervisor, to base Linux KVM support on.

https://github.com/david942j/kvm-kernel-example/blob/master/hypervisor/hypervisor.c

[jschwartzenberg]

DOSEMU2 also supports KVM: https://github.com/stsp/dosemu2/blob/42bc36c8adee504db92e5de9c6db95633c76813d/src/base/emu-i386/kvm.c

It would be very interesting to see this supported in DOSBox-X. It would probably enable very good Windows 9x support. DOSEMU2 runs up to Windows 3.x and QEMU with KVM has difficulty with Windows 9x (was quite unstable when I tried it).

[joncampbell123]

I want to see KVM support in DOSBox-X too, though I don't know how to program it yet and I don't have the free time to try right now. If implemented it would probably far surpass the dynamic core.

Then for Windows users it would have to support the same through HyperV.

[jschwartzenberg]

Maybe you could ask @stsp or @bartoldeman for info on KVM. There is probably more info to exchange regarding DOSEMU2 and DOSBox-X development :)

[AlbertHamik]

I'm curious if any progress was ever made into figuring this out. I've actually been using DOSEMU2 to speedrun a certain game, 1998's Gunmetal, that is well documented for it's myriad of technical issues but chief among them is it's problems with CPU timing on modern systems. I've had nothing but terrible luck trying to get it to properly run in any version of DOSBox, SVN or not.

After getting a lead from someone else to try out CPU virtualization for VM software, I finally settled on DOSEMU2, which has generally been fine though I'm limited to just my one game with it. Trying to run other games in DOSEMU2 is random chance and I really lucked out with the one game I was desperately needing something like DOSEMU2 for.

It would be amazing if not only did DOSBox-X support KVM, but also whatever comparative options exist in Windows (WHPX, HAXM, etc.). I'm looking for a way of making Gunmetal easier for others to check out, and currently if I can see if they fixed the issue with PS/2 input, QEMU looks to be my current best option for multiplatform support, to make it foolproof for others. But also yeah, CPU virtualization on top of DOSBox-X's accuracy and performance focused feature set would be great, far superior to stuff like QEMU that is very limited, or PCem which is insanely demanding.

[jschwartzenberg]

@AlbertHamik did you report the issues with other games in dosemu2? Reports to the @dosemu2 issue tracker would be useful!

[AlbertHamik]

Well one game I know isn't dosemu2's fault. Will get to that in time. I've been appreciative of the help from those developers over in the dosemu2 community.

[Torinde]

#1184 (comment) discussion to consider virt86

[Torinde]

@joncampbell123 @StrikerX3 it seems virt86 is exactly what's needed to get hypervisor core in DOSbox-X and it supports most OS supported by DOSbox-X - Windows, macOS, Linux.

What will take to plumb those together? Can you please give some guidance?

[StrikerX3]

I'm not too familiar with DOSBox-X's codebase, however it should be reasonably straightforward to integrate. Since DOSBox-X doesn't use CMake, you have two options: either use the binary distribution (following the build instructions in the README; the 1.1.0 release is outdated) or add virt86 as a submodule and include the source files manually into your build system.

virt86 is composed of the following modules:

• core: Contains the base classes and common definitions for all platforms
• sys: OS-specific code:
  • sys/darwin (macOS)
  • sys/linux
  • sys/windows
• Hypervisor-specific code:
  • haxm: Intel HAXM
  • hvf: Apple's Hypervisor.Framework (currently a stub, unimplemented)
  • kvm: KVM
  • whpx: Windows Hypervisor Platform
• virt86: The main entrypoint of the library. Really only contains a single header file which includes the rest of the library and defines an array of Platform objects based on what hypervisors are available.

Note that the modules are conditionally compiled based on the target operating system, as described in the modules/CMakeLists.txt file:

• core, virt86 and haxm are included on all builds
• sys/windows and whpx are Windows-only
• sys/linux and kvm are Linux-only
• sys/darwin and hvf are macOS-only (though hvf is not included because it is unimplemented)

virt86 makes use of a few C++17 features such as std::optional and nested namespaces.

For general usage guidance, you can check virt86's Getting Started wiki page and the demo projects in virt86-demos. The basic demo covers nearly every feature in the library.

[Torinde]

Thank you, @StrikerX3 !
Agree it sounds straightforward... although I'll need to learn a lot more before I can embark on that. Hopefully somebody more experienced gives it a try as well...

[stsp]

If implemented it would probably far surpass the dynamic core.

In our experience with dosemu2, this
is not always the case.

1. I haven't found a way to accelerate
   planar VGA modes with kvm. Likely
   there is no such way.
2. Some work-loads Speed Comparison dosemu2/dosemu2#1875
   just generate too many vmexits for kvm to
   stay competitive.
3. Many AMD chipsets appear to have some
   "firmware bug" that forces linux to refuse
   TSC as a system clock. Linux then falls
   back to the (very slow?) hpet, and KVM
   for some reason reads system time on
   every invocation. And in that case things
   really slow down. I happened to have the
   notebook with that "firmware bug", so to
   get the good performance out of KVM I
   need to reboot with "tsc=reliable" in kernel's
   command line.

This is JFYI. Of course KVM is still very
much worth adding. But its not a silver bullet,
very unfortunately.

[Torinde]

@volkertb, I saw your post about the same option - can you implement (part of) the above?

I was actually about to open a feature request here for an optional hypervisor-backed CPU core that would leverage hardware-assisted virtualization (if available) instead of emulating a CPU. After all, on an x86-based machine, how could complete CPU emulation ever be faster than virtualization, with only stufff like VGA and sound cards having to be emulated?

And please see QEMU-3dfx mod and #3405, which are related to the other idea in your post:

(And in the case of VESA mode, even the former could be paravirtualized, leaving basically only the sound card to be emulated in software.)

[volkertb]

@Torinde I assume you're quoting me on a message I posted in a thread on the dosbox-staging project, right?

Anyway, I merely requested such functionality. I currently lack the skills and experience to implement something like that myself, if that was your question. 😅

Also, @AlbertHamik and @StrikerX3, I saw you both mentioning HAXM here, but that project has been discontinued by Intel, likely because each still-supported major OS already comes with its own integrated type-2 hypervisor these days. Best to focus on KVM on Linux, HVF on macOS, WHPX on Windows, and maybe bhyve on FreeBSD. (Perhaps also Xen as a type-1 solution, if there is enough added value to support that too?)

I would really love to see an open-source hardware-assisted virtualization solution that was optimized for old x86 games (booter, DOS and early Windows). Basically a combination of VirtualBox and DOSBox. VirtualDOSBox, perhaps? 😉

The closest we have right now, as others have mentioned here, is dosemu2, but that one doesn't offer sound card emulation that is as good as that in DOSBox and its derivatives. Also, its run-time compatibility isn't as good.

There is the VMusic project by javispedro, which is a VirtualBox extension that adds emulation support for popular DOS-era sound hardware (OPL3 FM, MPU-401 MIDI). However, I noticed some performance issues while running some games under VirtualBox with that extension pack enabled. Nonetheless, it seems fairly promising. But of course, it doesn't address the emulation quality and performance of legacy VGA emulation in VirtualBox.

An actual gaming-optimized hypervisor solution would still be desirable. That is, at least as long as the x86 architecture remains dominant in the market, and modern x86 CPUs retain their ability to natively run 16-bit and 32-bit code, at least within hardware-assisted guest VMs. I agree that in the long run, this will cease to be the case, at which point all of this will no longer be relevant. I guess many developers of DOSBox and derivative projects are already disinterested in hardware-assisted x86 virtualization support for exactly this reason. 🤷‍♂️

[StrikerX3]

I saw you both mentioning HAXM here, but that project has been discontinued by Intel, likely because each still-supported major OS already comes with its own integrated type-2 hypervisor these days. Best to focus on KVM on Linux, HVF on macOS, WHPX on Windows, and maybe bhyve on FreeBSD. (Perhaps also Xen as a type-1 solution, if there is enough added value to support that too?)

Yeah, I'm aware of that. When virt86 was initially developed, HAXM was still relevant and was the main hypervisor platform used by the official Android Emulator. virt86 does support KVM and WHPX as well, but it lacks support for HVF as I don't have a macOS system. Help would be greatly appreciated there. Also, virt86 still lacks support for some important hypervisor functions, such as APIC emulation, IOMMU (direct hardware access) and TSC scaling.

I would really love to see an open-source hardware-assisted virtualization solution that was optimized for old x86 games (booter, DOS and early Windows). Basically a combination of VirtualBox and DOSBox. VirtualDOSBox, perhaps? 😉

I've been thinking of developing a simpler version of that (mostly focusing on simple command-line-based DOS apps) under the virt86-demos project as a demonstration of the library's capabilities, though if there's real interest it would probably make sense to turn that into a real project. However, I currently lack the free time to manage a project of this scale.

[Torinde]

SoftGPU driver requires high performance in the guest, so may benefit from virt86 core.