Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vz: add support for nested virtualization #2530

Merged
merged 1 commit into from
Oct 9, 2024

Conversation

abiosoft
Copy link
Contributor

@abiosoft abiosoft commented Aug 5, 2024

This PR adds support for nested virtualization for the VZ driver via the nestedVirtualization yaml config.

It is dependent on upstream PR Code-Hex/vz#159 getting merged. VZ has been forked lima-vm/vz#1.

Nested virtualization is currently limited to m3 devices running macOS 15 or newer.

@abiosoft abiosoft marked this pull request as draft August 5, 2024 14:12
@abiosoft
Copy link
Contributor Author

abiosoft commented Sep 7, 2024

Testing Instructions

⚠️ This only works on macOS 15 (currently beta) with VZ driver and on M3 devices.

For those willing to try this out, these are the steps.

1. Clone the source

Clone my fork of Lima and checkout vz-nested-virtualization branch.

git clone -b vz-nested-virtualization github.com/abiosoft/lima

2. Build Lima

From the root directory of Lima, perform a build.

PS: You might need to unlink Brew version if previously installed with Brew. brew unlink lima

make && sudo make install

3. Start Lima with nested virtualization enabled

Use VZ driver and enabled nested virtualization.

The lima.yaml config file should contain the following.

vmType: vz
nestedVirtualization: true

4. Validate nested virtualization in Lima VM

Enter the Lima VM with lima (or similar command) and validate the existence of /dev/kvm.

$ ls -lh /dev/kvm
crw-rw---- 1 user kvm 10, 232 Aug 29 09:06 /dev/kvm

If the VM is running Ubuntu, you can install cpu-checker package and run kvm-ok to further ascertain.

$ kvm-ok
INFO: /dev/kvm exists
KVM acceleration can be used


if macOSProductVersion.LessThan(*semver.New("15.0.0")) {
return fmt.Errorf("nested virtualization requires macOS 15 or newer")
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will be little nicer to add a helper like isNestedVirtualiztionAvailable() with these checks. This can also be implemented for linux, so it can be a common interface for all platforms.

@asauber
Copy link

asauber commented Sep 18, 2024

I was able to create a Lima VM with KVM support with your testing instructions.

$ limactl shell ubuntunested
WARN[0000] treating lima version "bc39aed.m" from "/Users/asauber/.lima/ubuntunested/lima-version" as very latest release
asauber@lima-ubuntunested:/Users/asauber$ kvm-ok
INFO: /dev/kvm exists
KVM acceleration can be used

Very cool. Thank you for working on this.

@AkihiroSuda
Copy link
Member

CI is failing https://github.com/lima-vm/lima/actions/runs/10903645853/job/30258186834?pr=2530

# github.com/lima-vm/lima/pkg/vz
Error: pkg/vz/vm_darwin.go:256:18: platformConfig.SetNestedVirtualizationEnabled undefined (type *"github.com/Code-Hex/vz/v3".GenericPlatformConfiguration has no field or method SetNestedVirtualizationEnabled)
make[1]: *** [_output/bin/limactl] Error 1

@abiosoft
Copy link
Contributor Author

CI is failing https://github.com/lima-vm/lima/actions/runs/10903645853/job/30258186834?pr=2530

Yes, that is because the upstream PR is not yet merged and that is why I made it a draft PR.

@abiosoft abiosoft force-pushed the vz-nested-virtualization branch 4 times, most recently from b5f9998 to eb784b8 Compare September 21, 2024 13:30
@AkihiroSuda
Copy link
Member

Code-Hex/vz#159 remains still unreviewed, maybe the repo should be forked?

@abiosoft
Copy link
Contributor Author

abiosoft commented Oct 4, 2024

Code-Hex/vz#159 remains still unreviewed, maybe the repo should be forked?

lima-vm/vz#1

@abiosoft abiosoft force-pushed the vz-nested-virtualization branch 2 times, most recently from c574b0b to d0a8131 Compare October 8, 2024 08:25
@abiosoft abiosoft marked this pull request as ready for review October 8, 2024 08:30
Signed-off-by: Abiola Ibrahim <git@abiosoft.com>
Copy link
Member

@AkihiroSuda AkihiroSuda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@AkihiroSuda AkihiroSuda added this to the v1.0 milestone Oct 8, 2024
@abiosoft
Copy link
Contributor Author

abiosoft commented Oct 8, 2024

The failing test does not appear related to the code change and it appears to run Lima successfully https://github.com/lima-vm/lima/actions/runs/11231643603/job/31221517661?pr=2530#step:10:1133. 😕

@jandubois
Copy link
Member

The failing test does not appear related to the code change and it appears to run Lima successfully

This unfortunately happens all the time, and we are retriggering failed test many times each day. 😞

@AkihiroSuda
Copy link
Member

CI is green on the master, but this PR seems failing 4 times.

Maybe the failure is caused by another recent change in vz?

@AkihiroSuda
Copy link
Member

CI is now green, let's merge it

@AkihiroSuda AkihiroSuda merged commit 7ab02dc into lima-vm:master Oct 9, 2024
27 checks passed
@abiosoft abiosoft deleted the vz-nested-virtualization branch October 9, 2024 07:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants