Nested Virtualization on Mac OS 15.1

[chodorenko]

Continuation of #3405

MacOS 15.1 Sequoia (24B83)
MacBook Air M3
UTM 4.6.0 (102)
Clean install Fedora Server 41 aarch64 from Fedora-Server-netinst-aarch64-41-1.4.iso
Maybe i do not correct understand howe to enable nested virtualization ?
config file config.plist.zip

[MoltenArmor]

Same on MacBook Pro M3, MacOS 15.1 Sequoia, UTM 4.6.0 (102) and Alpine Linux 3.20.

[MoltenArmor]

Same on MacBook Pro M3, MacOS 15.1 Sequoia, UTM 4.6.0 (102) and Alpine Linux 3.20.

Also tried Fedora 41. Same.

[aaradhak]

I have been trying to get nested VM in mac m3 (Mac OS 15.1), tried to launch fedora silverblue 41 multiple times (heard that it is flaky at the moment and works in one of the attempts), but I have not been able to get nested virtualization in any of the VM spins.

@fedora:~/fcos$ ls -lash /dev/kvm
ls: cannot access '/dev/kvm': No such file or directory

[neverpanic]

Here's the config.plist that gives me working nested virtualization on 15.1 with UTM 4.6.0, guest OS Fedora 41:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>Backend</key>
	<string>Apple</string>
	<key>ConfigurationVersion</key>
	<integer>4</integer>
	<key>Display</key>
	<array>
		<dict>
			<key>DynamicResolution</key>
			<true/>
			<key>HeightPixels</key>
			<integer>1200</integer>
			<key>PixelsPerInch</key>
			<integer>80</integer>
			<key>WidthPixels</key>
			<integer>1920</integer>
		</dict>
	</array>
	<key>Drive</key>
	<array>
		<dict>
			<key>Identifier</key>
			<string>94B9BBFF-1A69-4A2C-9CAE-27943A506ABD</string>
			<key>Nvme</key>
			<false/>
			<key>ReadOnly</key>
			<true/>
		</dict>
		<dict>
			<key>Identifier</key>
			<string>9809DEC4-9A11-4E1D-82AC-F4BDE8EA0E7C</string>
			<key>ImageName</key>
			<string>9809DEC4-9A11-4E1D-82AC-F4BDE8EA0E7C.img</string>
			<key>Nvme</key>
			<false/>
			<key>ReadOnly</key>
			<false/>
		</dict>
	</array>
	<key>Information</key>
	<dict>
		<key>Icon</key>
		<string>fedora</string>
		<key>IconCustom</key>
		<false/>
		<key>Name</key>
		<string>Fedora</string>
		<key>UUID</key>
		<string>5FC313D8-F2BA-4FC2-9287-4E065BDF6C50</string>
	</dict>
	<key>Network</key>
	<array>
		<dict>
			<key>MacAddress</key>
			<string>72:e4:99:70:8f:d5</string>
			<key>Mode</key>
			<string>Shared</string>
		</dict>
	</array>
	<key>Serial</key>
	<array/>
	<key>System</key>
	<dict>
		<key>Architecture</key>
		<string>aarch64</string>
		<key>Boot</key>
		<dict>
			<key>EfiVariableStoragePath</key>
			<string>efi_vars.fd</string>
			<key>OperatingSystem</key>
			<string>Linux</string>
			<key>UEFIBoot</key>
			<true/>
		</dict>
		<key>CPUCount</key>
		<integer>10</integer>
		<key>GenericPlatform</key>
		<dict>
			<key>machineIdentifier</key>
			<data>
			YnBsaXN0MDDRAQJUVVVJRE8QEDeWlFjsnkhKlOlHjO7qQRsICxAA
			AAAAAAABAQAAAAAAAAADAAAAAAAAAAAAAAAAAAAAIw==
			</data>
		</dict>
		<key>MemorySize</key>
		<integer>12288</integer>
	</dict>
	<key>Virtualization</key>
	<dict>
		<key>Audio</key>
		<true/>
		<key>Balloon</key>
		<true/>
		<key>ClipboardSharing</key>
		<true/>
		<key>Entropy</key>
		<true/>
		<key>Keyboard</key>
		<string>Generic</string>
		<key>Pointer</key>
		<string>Mouse</string>
		<key>Rosetta</key>
		<false/>
	</dict>
</dict>
</plist>

[chodorenko]

The difference is in this block

		<key>GenericPlatform</key>
		<dict>
			<key>machineIdentifier</key>
			<data>
			YnBsaXN0MDDRAQJUVVVJRE8QEDeWlFjsnkhKlOlHjO7qQRsICxAA
			AAAAAAABAQAAAAAAAAADAAAAAAAAAAAAAAAAAAAAIw==
			</data>
		</dict>

in my config its block not present. After i add Your block and Nested Virtualization work on vm
@osy May be You can answer why its block do not generated on wizard of create wm ? May be need add some option in wizard ?

[MoltenArmor]

The difference is in this block

		<key>GenericPlatform</key>
		<dict>
			<key>machineIdentifier</key>
			<data>
			YnBsaXN0MDDRAQJUVVVJRE8QEDeWlFjsnkhKlOlHjO7qQRsICxAA
			AAAAAAABAQAAAAAAAAADAAAAAAAAAAAAAAAAAAAAIw==
			</data>
		</dict>

in my config its block not present. After i add Your block and Nested Virtualization work on vm @osy May be You can answer why its block do not generated on wizard of create wm ? May be need add some option in wizard ?

It seems there might be some issues in this section of code in Platform/Shared/VMWizardState.swift:

I guess if you don't select 'Boot from Kernel Image', then config.system.genericPlatform will not be set, and nested virtualization will not be available:

I've never done macOS programming, so I might be wrong. @neverpanic Did you select this option, or did you add this config section manually?

[neverpanic]

I did not set this option, I just booted from ISO with Apple Virtualization enabled. I'm not sure whether UTM does some magic auto-detection to extract the kernel from the ISO to boot from that? In any case, I went through the normal (in my case OpenSuSE) installer when creating the VM, then copied the disk from an existing Fedora VM I had.

The base64 in the <machineIdentifier> section is just an Apple binary plist, that, when converted to XML, contains a machine UUID:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>UUID</key>
	<data>
	N5aUWOyeSEqU6UeM7upBGw==
	</data>
</dict>
</plist>

I'm not sure why and how that would trigger enabling of nested virtualization.

[osy]

I think there is a bug in the wizard. In the meantime, I think a workaround is to go into Boot settings, change Bootloader to Linux, select any random file, then select UEFI again.

[aaradhak]

I just tried the workaround, it seems to work.

[c4rt0]

I can also confirm that after using the above workaround, I was able to run nested FCOS VM on Fedora 41 VM (Macbook Air M3). 🎉