macOS Ventura Testing

[DrDonk]

It looks like there is a common issue for systems wih AMD and older Intel CPUS. There is no library available for the system.

I will use this thread to track it, but currently do not have an answer to the problem.

Library not loaded: /usr/lib/libSystem.B.dylib

  Referenced from: <2D7AC05B-8AF0-3676-A40A-E40B77ACA459> /sbin/launchd

  Reason: tried: '/usr/lib/libSystem.B.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/usr/lib/libSystem.B.dylib' (no such file), '/usr/lib/libSystem.B.dylib' (no such file, no dyld cache)

AMFI: Denying core dump for pid 1 (launchd)pid 1 exited -- exit reason namespace 6 subcode 0x1, description Library not loaded: /usr/lib/libSystem.B.dylib
  Referenced from: <2D7AC05B-8AF0-3676-A40A-E40B77ACA459> /sbin/launchd
  Reason: tried: '/usr/lib/libSystem.B.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/usr/lib/libSystem.B.dylib' (no such file), '/usr/lib/libSystem.B.dylib' (no such file, no dyld cache)
Debugger called: <panic>

panic(cpu 3 caller 0xffffff800b30a796):  initproc failed to start -- exit reason namespace 6 subcode 0x1 description: Library not loaded: /usr/lib/libSystem.B.dylib
  Referenced from: <2D7AC05B-8AF0-3676-A40A-E40B77ACA459> /sbin/launchd
  Reason: tried: '/usr/lib/libSystem.B.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/usr/lib/libSystem.B.dylib' (no such file), '/usr/lib/libSystem.B.dylib' (no such file, no dyld cache)

uuid info:
 0x10585e000	uuid = <2d7ac05b-8af0-3676-a40a-e40b77aca459>
 0x10f0e4000	uuid = <0f050705-2258-3d40-b7bc-f3b35a44bbea>

Thread 0 crashed

RAX: 0x0000000002000209, RBX: 0x0000000000000000, RCX: 0x00007ff7ba6a0738, RDX: 0x00007ff7ba6a0ba0
RSP: 0x00007ff7ba6a0738, RBP: 0x00007ff7ba6a0780, RSI: 0x0000000000000001, RDI: 0x0000000000000006
R8:  0x00007ff7ba6a07a0, R9:  0x0000000000000000, R10: 0x000000000000003d, R11: 0x0000000000000246
R12: 0x000000000000003d, R13: 0x00007ff7ba6a0ba0, R14: 0x0000000000000001, R15: 0x0000000000000006
RFL: 0x0000000000000246, RIP: 0x000000010f14d83a, CS:  0x0000000000000007, SS:  0x0000000000000023

Thread 0: 0xffffff86e5063598
	0x000000010f14d83a
	0x000000010f1669f9
	0x000000010f0ee1e1
	0x000000010f0eb660
	0x000000010f0ea281

[amagerard]

Debian 11 Bullseye
VMware® Workstation Technology Preview 22H2 Pro
Unlocker 4.2.3
processor : E5-2697 v2
Motherboard : X79
ram: 32G ECC for server
Macosx Ventura makes a loop.
I have the same problem .
With Windows 10 it's exactly the problem.
Maybe my Xeon processor is already too old !
I get the same result with VirtualBox.

It's all good with MacOSX Monterey

[DrDonk]

Ventura will not work if your CPU is pre-Kaby Lake with no AVX2.0. The only possible way forward is to create a form of OCLP patcher for VMware platform. There is no way to patch this in VMware.

Web Site: https://dortania.github.io/OpenCore-Legacy-Patcher/
GitHub Repo: https://github.com/dortania/OpenCore-Legacy-Patcher
Ventura details: https://dortania.github.io/OpenCore-Legacy-Patcher/VENTURA-DROP.html

I am not sure how easy this will be, but it would be worth trying for older Intel systems. AMD may well be another matter but I can't comment as not in a position to develop and test for AMD.

[DrDonk]

I have uploaded a utility I wrote to create bootable macOS recovery virtual disks directly from Apple servers. It's uploaded in the wiki page:

https://github.com/DrDonk/unlocker/wiki/Create-a-bootable-macOS-Recovery-vIrtual-disk

My thought process is we could all use the same Ventura test virtual disk that is common to everyone trying to help with the issue. It would allow a simple test of any VMX file changes to be done in a controlled way.

Thoughts?

Anyone up for helping, as I don't have a system that has a problem with Ventura?

[ashleyw-gh]

nice work! Is this intended for VMware desktop products only? Reason I ask is that I generated the vmdk file and uploaded it through to esxi. I was unable to add it to the config of a VM. I also tried to convert the vmdk to a type 4 vmdk for use with esxi;

"C:\Program Files (x86)\VMware\VMware Workstation\vmware-vdiskmanager.exe" -r ventura.vmdk -t 4 ventura-type4.vmdk

this resulted in ventura-type4.vmdk and ventura-type4-flat.vmdk files but I was still unable to add them to the esxi VM.

[DrDonk]

The created VMDK would be a type 0 file which is for the hosted products. That's a limitation of using qemu-img. Let me see what can be done to get it into ESXi.

BTW new version coming as don't need the dmg2img step, qemu-img can do it all.

[DrDonk]

I think vmware-vdiskmanager broke this type 4 stuff some years ago. Seems that the vmkfstools in ESXi can do the job https://kb.vmware.com/s/article/1028042. I've never tried it though.

I'm looking at it but it's lateish in UK so may be tomorrow before I have enough time.

UPDATE:

Try this, upload type-0 file to datastore and run:
vmkfstools -i ventura.vmdk -d thin ventura-esxi.vmdk

[ashleyw-gh]

thanks, I tried that but still no go when i try to add it to add the disk to an esxi VM (unsinf esxi8). I've tried a few different variants and tried to recreate the vmdk descriptor etc. I've tried with different virtual hardware levels on the vmx file.
Each time when I add the existing disk - it shows the maximum disk file as 0 bytes and will not let me save through the UI.

I think vmware-vdiskmanager broke this type 4 stuff some years ago. Seems that the vmkfstools in ESXi can do the job https://kb.vmware.com/s/article/1028042. I've never tried it though.

I'm looking at it but it's lateish in UK so may be tomorrow before I have enough time.

UPDATE:

Try this, upload type-0 file to datastore and run: vmkfstools -i ventura.vmdk -d thin ventura-esxi.vmdk

[ashleyw-gh]

rather than take this approach I tried the upload from VMware workstation, but it appears the upload to an esxi8 host is currently broken (just my luck eh!). However I tried the latest VMware converter standalone and was able to convert the VM that I created based on your vmdk.
Here are the results.
On Intel based machine the MacOS installer boot goes through fine and gets to the Language selection option on the install.

I then power off the VM.
VMware convertor>convert machine>source: VMware workstation>select mactest>desitnation esxi>convert

I then go to the esxi server. I change the OS type to Mac OS 13 (as the converter defaults to 32bit other).
I attempt to power on the VM - the VM hangs at the Apple logo.
I power off the VM, then add the folllowing lines into the vmx file;

cpuid.0.eax = "0000:0000:0000:0000:0000:0000:0000:1011"
cpuid.0.ebx = "0111:0101:0110:1110:0110:0101:0100:0111"
cpuid.0.ecx = "0110:1100:0110:0101:0111:0100:0110:1110"
cpuid.0.edx = "0100:1001:0110:0101:0110:1110:0110:1001"
cpuid.1.eax = "0000:0000:0000:0001:0000:0110:0111:0001"
cpuid.1.ebx = "0000:0010:0000:0001:0000:1000:0000:0000"
cpuid.1.ecx = "1000:0010:1001:1000:0010:0010:0000:0011"
cpuid.1.edx = "0000:0111:1000:1011:1111:1011:1111:1111"
smc.version = "0"

then when I power on the VM, it boot loops like before at the Apple logo (with or without a network adaptor).
esxi8 host = AMD 5950x
VMware workstation: Intel Core i7-4770 CPU

So this is good as I can reproduce the same boot loop issue based on your test on my AMD 5950x VMware server.

[DrDonk]

Good that this can replicate the issue, and thanks for testing. I have a few more checks that I would be grateful for you to try out.

1. Can you get a monterey recovery disk, and just prove that the the ESXi/Workstation works OK with a recovery disk? It did for me but a double check would be good.
2. Could you copy the CPUID to the Workstation VMX and see if that impacst both monterey and ventura recovery VMs?

I am also going to concoct some slightly different VMX file settings, and a drop in nvram file to enable debug output without having to boot to the shell and type it in each time.

[DrDonk]

OK quick update it boot loops for me running Monterey or Ventura on a nested ESXi on a real Mac so CPU should be OK.

Using the debug version of VMX causes an assertion:

2022-11-13T12:04:32.027Z Wa(03) vmx - smc: Index 432: Inconsistent read information.
2022-11-13T12:04:32.027Z Cr(01) vmx - PANIC: ASSERT bora/devices/misc/appleSMC.c:599

Checking the binaries and all appears OK, so looks like there is another check that needs patching, at least on ESXi.

It looks like patching SMC is not correct on ESXi8.

[ashleyw-gh]

great debugging! Are you in a position to test the same with a nested ESXi 7u3g hypervisor to see if the boot loop is specific to ESXi8 only?

[DrDonk]

Yes I have set it all up for ESXi7/8 and Monterey/Ventura, and will get on to it ASAP. However got a bad cold so the brain cells aren't firing on all cylinders.

[ashleyw-gh]

On my home esxi8 server, I created a nested esxi7u3g and an esxi8 nested hypervisors, and I tried the Ventura test - it boot loops on both nested esx7 and nested esxi8 hypervisors. The VM on nested esxi7 was set to hw level 19, The VM on nested esxi8 was set to hw level 20.

I then used your utility to generate a test vmdk for Monterey. I repeated the tests on esxi7 and esxi8 (by using vmware converter and taking the running workstation VM across and then adjusting the os ytype to match MacOS.
On esxi7u3g the Monterey recovery image goes through to the installer language screen.
On esxi8 the Monterey recovery image goes through a boot loop exactly like Ventura.

(note: as my hypervisor tin is an AMD 5950x, I had to apply the CPU masks to the vmx files of the Mac VMs, otherwise the Mac VMs hang on the Apple logo.).

happy to run any other tests - especially as I can run them nested now to prevent having to take down my home server each time.

[DrDonk]

That matches my tests. I will have some more VMX file tests coming but in the meantime could you remove the current cpuid. and the smc.version = 0 settings and try these settings on your real ESXi server please? Actually it would interesting to see what that does in Workstation as well on the Intel CPU.

These settings are from my real MacBook Pro which supports Ventura.

# >>> START <<<
# This spoofs MacBook Pro (15-inch, 2018) - MacBookPro15,1
cpuid.0.ebx = "0111:0101:0110:1110:0110:0101:0100:0111"
cpuid.0.ecx = "0110:1100:0110:0101:0111:0100:0110:1110"
cpuid.0.edx = "0100:1001:0110:0101:0110:1110:0110:1001"
featMask.vm.cpuid.FAMILY = "Val:0x6"
featMask.vm.cpuid.MODEL = "Val:0x9e"
featMask.vm.cpuid.STEPPING = "Val:0x0a"
cpuid.brandstring = "Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz"
cpuid.inhibitDarwinMasks = "TRUE"
board-id.reflectHost = "FALSE"
board-id = "Mac-937A206F2EE63C01"
# >>> END <<<

[amagerard]

I used all your VMDK from Catalina to Ventura on VMware Player and Esxi 7.U3. Everything is good except Ventura.
My processor is XEON E5-2697 V2 for VMware Player
My processor is XEON E5-2690 for ESXI
These 2 processors did not have AVX2.0. Which may explain why Ventura does not start.
I did a test with OpenCore Legacy Patcher and I obtained the same bad result only with Ventura.
I think I should change my equipment and upgrade to install Esxi8. It's not yet Christmas.

[DrDonk]

I believe I have bad news for ESXi8. The error is something to do with ACPI causing a panic when macOS kernel comes up. Sadly the serial logs are corrupted so cannot pinpoint the reason. However, as you are probably aware VMware have dropped support for macOS as host and guest in ESXi 8 and I'm thinking we're now stuck at ESXi 7.

I am going to concentrate on Ventura on Workstation/Player for now.

For reference, here's the backend of log with panic:

SNCoP IL oEcrarlo rV:a rMieatbhloeds  paarres ei/neixteicaultiizoend  ffaoirl emde t[h\o_dS B[._PSCT0AG].
2
2N.o_ SATrAg]u m(eNnotdse  afrfef fifnfi9t5i3a1l7iez4e5d8 0f)o,r  AmEe_tNhOoTd_ F[O_USNTDA ](
1
6A0C9P3I0 /Eprsrpoarr:s eM-e6t3h2o)d
parse/kPEDisableScreen -1
watchdog disabled by boot-arg
ACPI BIOS Error (bug): \_SB.PC08._OSC: Object (Alias) must be a control method with 4 argumentsACPI BIOS Error (bug): \_SB.PC0G._OSC: Object (Alias) must be a control method with 4 argumentsexecution fai l(e2d0 1[6\0_9S3B0./PnCs0aGr.gSu3m2eFn.t_sS-T2A2]5)
ACPI Warning:  (20160930/nsarguments-225)
 (Node ffffff95317e4580Null Node ptr (20160930/nsobject-383)
ACPI W)a,r nAiEn_gN:O T_FOUND (20160930/psparse-632)
[ PCI cMatched against SMC (started)
Generation from SMC report as 2
Null Node parameter (20160930/nsutils-200)
onfiguration begin ]
ACPI BIOS Error (bug): \_SB.PC08.De_bOuSgCg:e rA:C PUIn eBxIpOeSc tEerdr okre r(nbeulg )t:r a\p_ SnBu.mPbCer: 0xe, RIP: 0xffffff801697e360, CR2: 00Gx.a_
SC: Previous shutdown cause: 3
 ID: Legacy shiCPU 0 panic trap number 0mx e2,
rOibpj e0cxtf f(fAflfifa8s0)1 6m9u7set3 6b0e
ac rc0o n0tx000000008001003b cr2 0x000000000000000a cr3 0x0000000019ac8000 cr4 0x00000000003606e0
initialize_screen: b=F0000000, w=00000400, h=00000300, r=00001000, d=00000001
Debugger called: <panic>
panic(cpu 0 caller 0xffffff8015b4aee3): Kernel trap at 0xffffff801697e360, type 14=page fault, registers:
CR0: 0x000000008001003b, CR2: 0x000000000000000a, CR3: 0x0000000019ac8000, CR4: 0x00000000003606e0
RAX: 0x0000000000002140, RBX: 0x0000000000000000, RCX: 0x0000000000000009, RDX: 0xffffff80169b47b1
RSP: 0xfffffff2cd7c7bd0, RBP: 0xfffffff2cd7c7c10, RSI: 0xffffff80169af7cd, RDI: 0x00000000000000c9
R8:  0x0000000000000000, R9:  0x0000000000000000, R10: 0x0000000000000008, R11: 0x0000000000000010
R12: 0xffffff80169c0544, R13: 0x0000000000000000, R14: 0x0000000000000000, R15: 0xffffff9064a048c8
RFL: 0x0000000000010296, RIP: 0xffffff801697e360, CS:  0x0000000000000008, SS:  0x0000000000000000
Fault CR2: 0x000000000000000a, Error code: 0x0000000000000000, Fault CPU: 0x0 VMM, PL: 0, VF: 1

Panicked task 0xffffff8b97d27218: 28 threads: pid 0: 
Backtrace (CPU 0), panicked thread: 0xffffff90648bf0c8, Frame : Return Address
0xfffffff2cd7c75b0 : 0xffffff80159edf9d mach_kernel : _handle_debugger_trap + 0x4ad
0xfffffff2cd7c7600 : 0xffffff8015b5b786 mach_kernel : _kdp_i386_trap + 0x116
0xfffffff2cd7c7640 : 0xffffff8015b4aa10 mach_kernel : _kernel_trap + 0x3e0
0xfffffff2cd7c7690 : 0xffffff8015988951 mach_kernel : _return_from_trap + 0xc1
0xfffffff2cd7c76b0 : 0xffffff80159ee27d mach_kernel : _DebuggerTrapWithState + 0x5d
0xfffffff2cd7c77a0 : 0xffffff80159ed929 mach_kernel : _panic_trap_to_debugger + 0x1a9
0xfffffff2cd7c7800 : 0xffffff80161e0ecb mach_kernel : _panic + 0x84
0xfffffff2cd7c78f0 : 0xffffff8015b4aee3 mach_kernel : _sync_iss_to_iks + 0x2c3
0xfffffff2cd7c7a70 : 0xffffff8015b4abd2 mach_kernel : _kernel_trap + 0x5a2
0xfffffff2cd7c7ac0 : 0xffffff8015988951 mach_kernel : _return_from_trap + 0xc1
0xfffffff2cd7c7ae0 : 0xffffff801697e360 com.apple.driver.AppleACPIPlatform : _AcpiExResolveNodeToValue + 0x11c
0xfffffff2cd7c7c10 : 0xffffff801698659a com.apple.driver.AppleACPIPlatform : _AcpiNsEvaluate + 0x35b
0xfffffff2cd7c7c50 : 0xffffff801698ad46 com.apple.driver.AppleACPIPlatform : _AcpiEvaluateObject + 0x259
0xfffffff2cd7c7cb0 : 0xffffff8016944f2e com.apple.driver.AppleACPIPlatform : _AcpiEvaluateOSCMethod + 0xf7
0xfffffff2cd7c7d60 : 0xffffff801695a30b com.apple.driver.AppleACPIPlatform : __ZN12AppleACPIPCI20enableNativeFeaturesEj + 0x53
0xfffffff2cd7c7d90 : 0xffffff801695a262 com.apple.driver.AppleACPIPlatform : __ZN12AppleACPIPCI5startEP9IOService + 0x100
0xfffffff2cd7c7dd0 : 0xffffff80160e9002 mach_kernel : __ZN9IOService14startCandidateEPS_ + 0xd2
0xfffffff2cd7c7e30 : 0xffffff80160e8b7a mach_kernel : __ZN9IOService15probeCandidatesEP12OSOrderedSet + 0xe0a
0xfffffff2cd7c7ef0 : 0xffffff80160e7be0 mach_kernel : __ZN9IOService14doServiceMatchEj + 0x3b0
0xfffffff2cd7c7f50 : 0xffffff80160eace7 mach_kernel : __ZN15_IOConfigThread4mainEPvi + 0x157
0xfffffff2cd7c7fa0 : 0xffffff801598819e mach_kernel : _call_continuation + 0x2e
      Kernel Extensions in backtrace:
         com.apple.driver.AppleACPIPlatform(6.1)[FB7A70DE-6B50-30A3-AD8D-2036437B1A61]@0xffffff801693d000->0xffffff80169b4fff
            dependency: com.apple.driver.AppleSMC(3.1.9)[712419B7-1229-324C-A182-FA69C7844A2B]@0xffffff801706c000->0xffffff8017088fff
            dependency: com.apple.iokit.IOACPIFamily(1.4)[55E86E16-9FC5-3768-880B-ADFECF403215]@0xffffff8017f8f000->0xffffff8017f90fff
            dependency: com.apple.iokit.IOPCIFamily(2.9)[E2CE416B-59DA-3836-9985-FDC527C5E6C7]@0xffffff80183ff000->0xffffff801842efff

Process name corresponding to current thread (0xffffff90648bf0c8): Unknown
Boot args: -v serial=1 debug=0x108 keepsyms=1 wdt=-1 -no_panic_dialog -topo

Mac OS version:
Not yet set

Kernel version:
Darwin Kernel Version 22.1.0: Sun Oct  9 20:14:54 PDT 2022; root:xnu-8792.41.9~2/RELEASE_X86_64
Kernel UUID: BF7C9676-EF23-3E8D-A2E2-25DAC63091B6
roots installed: 0
KernelCache slide: 0x0000000015600000
KernelCache base:  0xffffff8015800000
Kernel slide:      0x00000000156dc000
Kernel text base:  0xffffff80158dc000
__HIB  text base: 0xffffff8015700000
System model name: VMware20,1 (Mac-937A206F2EE63C01)
System shutdown begun: NO
Panic diags file unavailable, panic occurred prior to initialization
Hibernation exit count: 0

System uptime in nanoseconds: 16224706247
Last Sleep:           absolute           base_tsc          base_nano
  Uptime  : 0x00000003c8408939
  Sleep   : 0x0000000000000000 0x0000000000000000 0x0000000000000000
  Wake    : 0x0000000000000000 0x00000002a3173649 0x0000000000000000
Compressor Info: 0% of compressed pages limit (OK) and 0% of segments limit (OK) with 0 swapfiles and OK swap space
Zone info:
  Zone map: 0xffffff80643da000 - 0xffffffa0643da000
  . PGZ   : 0xffffff80643da000 - 0xffffff8064bdb000
  . VM    : 0xffffff8064bdb000 - 0xffffff8531774000
  . RO    : 0xffffff8531774000 - 0xffffff86cb0a7000
  . GEN0  : 0xffffff86cb0a7000 - 0xffffff8b97c40000
  . GEN1  : 0xffffff8b97c40000 - 0xffffff90647d9000
  . GEN2  : 0xffffff90647d9000 - 0xffffff9531372000
  . GEN3  : 0xffffff9531372000 - 0xffffff99fdf0c000
  . DATA  : 0xffffff99fdf0c000 - 0xffffffa0643da000
  Metadata: 0xffffffbdb8f9f000 - 0xffffffbdd8f9f000
  Bitmaps : 0xffffffbdd8f9f000 - 0xffffffbdda79f000

last started kext at 15302368695: @private.KextAudit	1.0 (addr 0xffffff80189a4000, size 4096)
loaded kexts:
@private.KextAudit	1.0
>!AACPIButtons	6.1
>!AHPET	1.8
>!ARTC	2.0.1
>!ASMBIOS	2.1
>!AAPIC	1.7
$!AUserConsent	1
@!ASystemPolicy	2.0.0
@nke.applicationfirewall	403
|IOKitRegistryCompatibility	1
|EndpointSecurity	1
@Dont_Steal_Mac_OS_X	7.0.0
@kec.Compression	1
@kec.!AEncryptedArchive	1
>!AEFINVRAM	2.1
>!AEFIRuntime	2.1
|IOHID!F	2.0.0
|IOTimeSync!F	1110.13
|IOSkywalk!F	1.0
>mDNSOffloadUserClient	1.0.1b8
|IONetworking!F	3.4
>DiskImages	493.0.0
|IO!B!F	9.0.0
|IOReport!F	47
$quarantine	4
$sandbox	300.0
@kext.!AMatch	1.0.0d1
>!ASSE	1.0
>!AKeyStore	2
>!UTDM	547
|IOUSBMass!SDriver	232
|IOSCSIBlockCommandsDevice	476
|IO!S!F	2.1
|IOSCSIArchitectureModel!F	476
>!AMobileFileIntegrity	1.0.5
$!AImage4	5.0.0
@kext.CoreTrust	1
>!AFDEKeyStore	28.30
>!AEffaceable!S	1.0
>!ACredentialManager	1.0
|CoreAnalytics!F	1
>KernelRelayHost	1
|IOUSBHost!F	1.2
>!UHostMergeProperties	1.2
>usb.!UCommon	1.0
>!ABusPower!C	1.0
>!ASEPManager	1.0.1
>IOSlaveProcessor	1
>!AACPIPlatform	6.1
>!ASMC	3.1.9
|IOPCI!F	2.9
|IOACPI!F	1.4
>watchdog	1
@kec.pthread	1
@kec.Libm	1
@kec.corecrypto	12.0
Attempting to commit panic log to NVRAM

** In Memory Panic Stackshot Succeeded ** Bytes Traced 5375 (Uncompressed 12576) **
Attempting to commit panic log to NVRAM

Please go to https://panic.apple.com to report this panic

[DrDonk]

@amagerard Could you try something for me on VMware Player please? It's not a fix but something I want to check regarding the AVX2.0 instructions.

Can you add the following line to both your Montery and Ventura VMs and then boot? Do you get a message box telling you that AVX is not present on the host?

featMask.vm.cpuid.AVX2 = "Min:1

[AdrianEddy]

I tried adding that line to VMware Player on my AMD Ryzen 5950x, I didn't get any message box about AVX and boot loop is still happening as before

[DrDonk]

Thanks. Ryzen 5950x has AVX2 in which case it would not throw a message. Bottom line is I think we're at the end of the line for some setups due to the new CryptEx system in Ventura.

[amagerard]

The addition of:
featMask.vm.cpuid.AVX2 = "Min:1"
in the vmx file has the result that I cannot start the virtual machine.
I have this message:

Unable to change virtual machine power state: Feature 'cpuid.avx2' was absent, but must be present.
Module 'FeatureCompatLate' power on failed.
Failed to start the virtual machine.

[DrDonk]

OK. That's good in one way because I can tell everyone to use that setting and it will immediately flag up that their CPU is not compatible with Ventura.

Sorry not a fix and from unlocker perspective there never will be one. Best can hope for is someone works it out using something like Opencore.

Could you attach the vmware.log for the Ventursa VM please? I want to double check the CPUID settings?

[cppmyjob]

Maybe this article https://www.nicksherlock.com/2022/10/installing-macos-13-ventura-on-proxmox will help you understand what is wrong with processors for Venture on Vmware?

[jerrywoo96]

Based on wikipedia, it requires cpuid.7.ebx bit 5 to be enabled.

But we need to figure out what the rest of the other bits need to be set for cpuid.7.ebx

[DrDonk]

Thanks for the links on Proxmox but it uses Qemu under the hood in emulation mode so this won't work for virtualised CPU. (I've actually used Qemu to test this out.)

[DrDonk]

@ashleyw-gh I think this may be correct but I'm still not certain. Bottom line is I don't have any AMD kit and this was best endeavours on my part to use my knowledge to help out.

Now I have bare-bones Opencore boot VMDK that I worked on over the summer that can boot macOS with and without unlocker on Intel. I can zip up and post them as a starting point for other to try. I was thinking of starting a new repo with it in. Open to thoughts. I can help with Opencore just not able to test.

[DrDonk]

For now can I suggest if you want to test Ventura on both Intel and CPU you add these settings as defaults.

# >>> START <<<
# This spoofs CPUID as MacBook Pro (15-inch, 2018) - MacBookPro15,1
cpuid.0.ebx = "0111:0101:0110:1110:0110:0101:0100:0111"
cpuid.0.ecx = "0110:1100:0110:0101:0111:0100:0110:1110"
cpuid.0.edx = "0100:1001:0110:0101:0110:1110:0110:1001"
featMask.vm.cpuid.FAMILY = "Val:0x6"
featMask.vm.cpuid.MODEL = "Val:0x9e"
featMask.vm.cpuid.STEPPING = "Val:0x0a"
cpuid.brandstring = "Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz"
cpuid.inhibitDarwinMasks = "TRUE"
board-id.reflectHost = "FALSE"
board-id = "Mac-937A206F2EE63C01"

# Throw VMware error if no AVX2 support in CPU
featMask.vm.cpuid.AVX2 = "Min:1"

# Ensure no boot loop due to macOS unsupported Intel e1000 NIC
ethernet0.virtualDev = "vmxnet3"
# >>> END <<<

It emulates my real MacBook Pro 15,1 for the basic cpuid info & will throw an VMware error on boot if your CPU does not have AVX2, rather than boot loop.

UPDATE: Added Ethernet setting required to stop some boot loops.

[ashleyw-gh]

thanks - I think there is a small typo above; following line is missing a trailing double quote (just in case anyone is copy/pasting)

featMask.vm.cpuid.AVX2 = "Min:1:"

I also saw VMware workstation 17 has just launched and unlocker works as before on Workstation 17.

test1: VMwareWorkstation 17 pro, on Win11 with unlocker, Ventura test on "intel Core i7-4770" CPU, using setttings above. Test gets to region selection. test=success.

test2: Use VMware Convertor standalone and target ESXi 7u3 nested hypervisor on AMD 5950x using settings above. change the OS Type to MacOS 12 (this is the highest MAC OS verison listed for 7u3) after convertor as convertor defaults VM to 32bit other which is incorrect identifier. On power on with settings above, the boot hangs at the apple logo. test=fail

test3: as test2 except try with different combinations of the following enabled in addition to the settings above;

cpuid.0.eax = "0000:0000:0000:0000:0000:0000:0000:1011"
cpuid.1.eax = "0000:0000:0000:0001:0000:0110:0111:0001"
cpuid.1.ebx = "0000:0010:0000:0001:0000:1000:0000:0000"
cpuid.1.ecx = "1000:0010:1001:1000:0010:0010:0000:0011"
cpuid.1.edx = "0000:0111:1000:1011:1111:1011:1111:1111" 

either hangs on boot, or "your computer restarted because of a problem". test=fail

[DrDonk]

@ashleyw-gh Thanks for spotting the typo fixed in the post now.

I can confirm WKS 11 on Windows (not tried Linux yet) exhibits same behaviours as WKS 16.

Good tests, thanks. I'm not sure what the converter is doing so will download and test it out as well as building a new VM on ESXi 7 without the converter. Assume you are using the Recovery VMDKs we built earlier.

[ashleyw-gh]

thanks, yes I've been using your Recovery VMDKs and I've run some more tests.
test1: Dell PowerEdge R740xd2 - dual socket Intel Xeon Silver 4214R CPU @ 2.40GHz, I created a nested ESXi7 update3g host and installed the unlocker. I used the same VMware convertor trick to take your Ventura recovery image across. I used the recovery image to format a 2nd 80GB thin provisioned drive, and then installed Mac OSx13 onto it without any problems and this runs fine. Test successful.

test2: Dell PowerEdge R740xd2 - dual socket Intel Xeon Silver 4214R CPU @ 2.40GHz, I created a nested ESXi8 host and installed the unlocker. I used the same VMware convertor trick to take your Ventura recovery image across. Starting up the recovery image causes a boot loop. Test fail.

Conclusions;

• On supported Intel CPUs, there is no issue with the Ventura 13 install on ESXi 7 (tested with 7u3g).
• On AMD Ryzen CPUs (eg. 5950x), ESXi is not compatible currently with Ventura, despite AVX2 support etc. and leads to a boot loop, despite CPU masking to match known working Intel CPUs.
• On ESXi 8 the unlocker appears to correctly install but Ventura recovery image leads to a boot loop for both AMD and Intel CPUs.
• Don't forget to UNCLICK the secure boot option on the nested hypervisor; >"edit settings">GuestVM>"VM Options">"Boot options">"Secure Boot>"enable secure boot" on the nested hypervsior otherwise you won't be able to install the unlocker.
• Unlocker continues to work as expected on Workstation 17 on Intel on Win11, but I was unable to do a metal test on AMD to compare.

Some of the confusion could probably be avoided if the unlocker could be modified to correctly work with ESXi8.
One of the beauties of unlocker is the ease with which Mac VMs can be spun up.
OpenCore on the other hand requires significant time investment, so any documented approach to using OpenCore for both AMD and Intel specifically with VMware workstation/ESXi (assuming that's the only option going forwards) would greatly benefit the community.

My whole motivation for using Mac VMs is for the occasional functional test on my home lab when I need to reproduce an issue across platforms or generate cross platform documentation with minimal equipment and for this Mac VMs are an incredibly useful tool. I've never understood why Apple make it so hard to do this sort of stuff when it actually helps to grow their user base - its the complete opposite to where Linux is at.

[DrDonk]

OK my tests today and success on ESXi 8 on a supported CPU. Read on...

1. Done on a supported CPU Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz
2. You do need to ensure vmxnet3 is used for NIC (known issue now acknowledged by VMware)
3. Don't use Workstation/Fusion upload does not create correct VMX file
4. Don't use Workstation/Fusion connected to ESXi host to create VM on the ESXi host does not create correct VMX file
5. Don't use VMware Converter also does not work so avoid

Only reliable way is create VM using the ESXi Web UI

ESXi 7 with real SMC:
Monterey YES
Ventura YES

ESXi 7 no SMC and unlocked:
Monterey YES
Ventura YES

ESXi 8 with real SMC & ESXi 8 virtual machine:
Monterey NO
Ventura NO

ESXi 8 with real SMC & ESXi 7.0 U2 virtual machine
Monterey YES
Ventura YES

ESXi 8 no SMC and unlocked & ESXi 8 virtual machine:
Monterey NO
Ventura NO

ESXi 8 no SMC and unlocked & ESXi 7.0 U2 virtual machine
Monterey YES
Ventura YES

The bottom line is macOS support is probably gone from ESXi 8 virtual hardware VMs but it still supports ESXi 7 VMs. This seems to have been confirmed in a roundabout way some VMware KBs and Twitter posts.

I will get back to Workstation soon.

[DrDonk]

When creating a new VM in ESXi 8 UI make sure to select "Compatbility" and select "ESXi 7.0 U2 virtual machine"

[ashleyw-gh]

Awesome, thanks for narrowing that down.

I can confirm on my Dell Intel based server running nested ESXi8, I now have a running Ventura VM I took the following approach;

• run up converter standalone: source VMware Workstation VM (that has your recovery vmdk attached).
• target esxi8 host (that already has unlocker installed).
• make sure virtual hardware is level 19 (ESXi 7.0U2)
• After conversion;
• modify VM to OS type MAC OS, 12 (as converter defaults it to 32bit other, and Mac OS12 is last OS version listed for hw level 19).
• Add a second thin provisioned disk to VM (e.g. 80GB)
• make sure vmxnet3 NIC is on the VM (as converter by default seems to drop the nic by default).
• Boot VM on esxi8 host
• disk utilities>select the 80GB drive>Format: Mac OS Extended (Journaled), Scheme: GUID Partition Map>Erase>quit Disk Utility
• Reinstall macOS Ventura>continue>select the 80GB disk that was partitioned above.

When I try the exact same process and target an esxi8 host (both bare metal and nested hypervisor) on an AMD 5950x, then the VM starts but hangs on the apple logo.

so I believe the issue is narrowed down specifically to AMD CPUs and Ventura. (and older Intel CPUs not supporting some of the AVX extensions).

I don't have any issues with Ventura and the unlocker on Intel CPU using Workstation 17 (or 16) (provided the Intel CPU is recent enough to support AVX2).

[DrDonk]

Looks like we’re closing in on things. I have idea on a pre-formatted VMDK which has Opencore on it and cryptex fixe from OCLP.

I had previously built minimal Opencore VMDK as there’s a bug in VMware EFI which mean cannot boot to Recovery on an installed VM or change SIP.

I have some other bits to add and my intended outcome is a complete test VM to allow standards tests.

[DrDonk]

I have created 2 test VMs for Workstation/Player. Please download and test on pre-Haswell Intel and AMD CPUs. There is a lot of debugging info and so they are not particularly fast. It should boot to the Ventura Recovery OS.

https://github.com/DrDonk/VMCorePkg/releases/tag/0.0.1

When done can you attach the serial.log and vmware.log files from the VM folder on a post please.

ESXi ones to come.

[AdrianEddy]

AMD 5950X, Windows 11, VMware Workstation 16.2.4
logs.zip

[DrDonk]

@AdrianEddy Many thanks. From the logs neither booted correctly on your setup. This has given a different error regarding CPUID, which may well be different from the pre-Haswell issue and AVX 2.0 instructions.

[DrDonk]

All testers - Another couple of AMD traces from other systems would be useful.

[plamen-i]

Intel(R) Core(TM) i7-2670QM CPU @ 2.20GHz (Sandy Bridge), Windows 10, VMware Workstation 16.2.4
logs.zip

[DrDonk]

@plamen-i @AdrianEddy Could you re-test please with the new 0.0.2 downloads. I made 2 mistakes which may have affected the results.

These are now compatible with ESX and WKS/PLY.

https://github.com/DrDonk/VMCorePkg/releases/tag/0.0.2

[DrDonk]

I'm going to be honest here in that I'm not going to keep this issue open if it cannot be fixed with simple CPUID hacks. Issue tracker is really for fixes to unlocker code and I have always stated that things like AMD support are not really in scope for the unlocker. The same goes for older Intel chips now that Apple has deprecated their usage in Ventura. We all knew this would be coming down the line after the Apple Silicon announcement and VMware's dropping of macOS guests. This means the unlocker will have a definite shelf life.

The bottom line is this would become a duplicate of efforts such as OCLP and other groups who do "Ryzentosh" work. Their solutions should work in a VM, just substitute the VM for the hardware.

I will run these additional tests for now to see if anything can be done, but I'm not confident based on what I've seen so far.

[AdrianEddy]

logs.zip

[DrDonk]

For reference I have put together a spreadsheet of the CPUID values that can be altered in VMware. This was compiled from the VMware open VM tools source code:

https://github.com/vmware/open-vm-tools/blob/4d6147487ecc4bdf5c49bd0f78504669e8c76a20/open-vm-tools/lib/include/x86cpuid.h

It can be found here:

https://docs.google.com/spreadsheets/d/1QV4MReO_UgEisdFSW7apfVxNIn2K_0Du6VmRHh8WaHw/edit?usp=sharing

If MONSUPP column says ANY or YES then you can change the value in a VM. HWV is the earliest version of virtual hardware in which the value is supported.

[DrDonk]

@AdrianEddy Thanks for the logs and it proves that the Ventura kernel would need patching to run on AMD. It is failing on reading CPUID leaf 4 which cannot be overridden in the VMX file.

https://github.com/apple-oss-distributions/xnu/blob/5c2921b07a2480ab43ec66f5b9e41cb872bc554f/osfmk/i386/cpuid.c#L328

So for AMD CPUs you will need to look into Opencore and Kernel patches.

So that means it is closed from an unlocker perspective.

[plamen-i]

It starts now and reaches Opencore screen. When select "OPENCORE (dmg)" it runs some more time then reboots and again reaches Opencore screen.
logs.zip

[DrDonk]

@plamen-i Thanks for the logs. It is as I suspected in that OCLP must be used to install and boot on pre-Haswell CPU. Just adding the CryptexFix.kext does not fix the problem.

So unlocker cannot patch or fix AMD and pre-Haswell CPUs with Ventura.

In summary:

If you have a supported cpu you should be good to go so long as you have this in your VMX file.

# Ensure no boot loop due to macOS unsupported Intel e1000 NIC
ethernet0.virtualDev = "vmxnet3"

If you have a pre-Haswell CPU add this to your VMX file and it will confirm if by throwing a VMware error at boot.

# Throw VMware error if no AVX2 support in CPU
featMask.vm.cpuid.AVX2 = "Min:1"

[EmilAlipiev]

I made a mistake and upgraded from Monterey to Ventura and having this problem on my Ryzen 7 now. Is there a way to go back to monterey without losing data in the VM?

[DrDonk]

Create a Monterey VM attach the Ventura VM and copy data across.

[ashleyw-gh]

@DrDonk thank-you for all your efforts on this. Much appreciated. As a test I followed the notes for OCLP and used an existing running Catalina VM on my AMD home server and created a virtual 16GB USB device using the style;
https://williamlam.com/2022/07/emulating-a-virtual-usb-storage-device-using-nested-esxi.html
and then used the OCLP installer to create a Ventura install image on the virtual USB device and then shut down the virtual Catalina mac. I then copied the USB file into the directory of a new Mac VM with an 80GB thin provisioned disk and then added the USB config to the vmx file on the new VM.
The installer booted and all was looking good (I got past all the install settings etc) and I got to about the 5th reboot stage of the install, but then the install boot loops - so I suspect in the case of 5950x there is still work to be done by the OCLP team.

[DrDonk]

@ashleyw-gh
The virtual USB drive is useful and I've used it for testing. You can also set device type to "cdrom" though questionable value.

OCLP Ventura support is still in beta, but getting better. I have some minimal OpenCore (not OCLP) bootable VMDKs I'll upload as can be used to see the minimum needed for VMware. There are 4 versions; Unlocker required for SMC and no unlocker as it uses VirtualSMC, in debug and release versions. These aren't something I'd support but could help the community move forward and add extracted bits from OCLP and other Ryzentosh projects.

https://github.com/DrDonk/VMCorePkg/releases/tag/0.0.1

I'm drafting some updates to various Markdown documents in repo to explain the situation.

I am going to close this issue however I can add a wiki page if someone writes up a good guide on how to use OCLP for problem CPUs.

Thanks to everyone who helped out and it's a pity there wasn't an easy fix. Back to being retired and fixing my vintage Macs now.