Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

nested virtualization does not work on multiarch #1657

Open
jbtrystram opened this issue Jan 26, 2024 · 5 comments
Open

nested virtualization does not work on multiarch #1657

jbtrystram opened this issue Jan 26, 2024 · 5 comments
Labels
kind/enhancement

Comments

@jbtrystram
Copy link
Contributor

jbtrystram commented Jan 26, 2024 •
edited
Loading

Describe the enhancement

I want to create a VM within a coreOS VM, on aarch64 (s390x and ppc64le as well)

Steps to reproduce the issue :

# Build fedora coreOS as usual

$ cosa run 
 # Now we are in coreOS VM
cd /mnt/workdir 
podman pull quay.io/coreos-assembler/coreos-assembler
 # make sure we import cosa alias
$ cosa run 
+ podman run --rm -ti --security-opt=label=disable --privileged --uidmap=1000:0:1 --uidmap=0:1:1000 --uidmap=1001:1001:64536 -v=/mnt/workdir:/srv/ --device=/dev/kvm --device=/dev/fuse --tmpfs=/tmp -v=/var/tmp:/var/tmp --name=cosa quay.io/coreos-assembler/coreos-assembler:latest run
Could not access KVM kernel module: No such file or directory
qemu-system-aarch64: failed to initialize kvm: No such file or directory
```

### System details

Debug bod from FCOS pipeline on aarch64

### Additional information

S390X appears to be the same : 
Could not access KVM kernel module: No such device
qemu-system-s390x: failed to initialize kvm: No such device

Same on ppc64le
@jbtrystram jbtrystram changed the title nested virtualization does not work on aarch64 nested virtualization does not work on multiarch Jan 26, 2024
@jlebon
Copy link
Member

jlebon commented Jan 26, 2024

Can you confirm the outer VM is running with -cpu host?

Short-term WDYT about denylisting the test on !x86_64 for now (using this issue as the tracker)?

@jbtrystram
Copy link
Contributor Author

Can you confirm the outer VM is running with -cpu host?

I will look into that

Short-term WDYT about denylisting the test on !x86_64 for now (using this issue as the tracker)?

I was looking for the proper syntax and it looks like testiso ignore that setting
https://github.com/openshift/os/blob/9ec57e837968e55a695ae12af5d90e4ce52e1fdf/kola-denylist.yaml#L57C3-L59C14

jbtrystram added a commit to jbtrystram/fedora-coreos-config that referenced this issue Jan 29, 2024
@jbtrystram
Re enabling iiso-offline-install-iscsi.bios
7a440ca 
This requires further work still, tracked in coreos/fedora-coreos-tracker#1639
The iscsi test fails on multiarch because multi arch does not support
nested virtualization. See coreos/fedora-coreos-tracker#1657

This reverts commit f736ef7.
jbtrystram added a commit to jbtrystram/coreos-assembler that referenced this issue Jan 30, 2024
@jbtrystram
Disable the iscsi test on other arches than x86
8164418 
Nested virt does not work through cosa
ref coreos/fedora-coreos-tracker#1657
jbtrystram added a commit to jbtrystram/coreos-assembler that referenced this issue Jan 30, 2024
@jbtrystram
Disable the iscsi test on other arches than x86
27a4712 
Nested virt does not work through cosa
ref coreos/fedora-coreos-tracker#1657
@jbtrystram jbtrystram mentioned this issue Jan 30, 2024
Merged
jbtrystram added a commit to coreos/coreos-assembler that referenced this issue Jan 31, 2024
@jbtrystram
Disable the iscsi test on other arches than x86
bd3a6aa 
Nested virt does not work through cosa
ref coreos/fedora-coreos-tracker#1657
jbtrystram added a commit to coreos/coreos-assembler that referenced this issue Jan 31, 2024
@jbtrystram
Disable the iscsi test on other arches than x86
72a85b1 
Nested virt does not work through cosa
ref coreos/fedora-coreos-tracker#1657
jbtrystram added a commit to jbtrystram/fedora-coreos-config that referenced this issue Feb 2, 2024
@jbtrystram
Enable iscsi test
6187602 
I have found that nested virtualization does not work for
other arches than x86, so the test is disabled for those
arches in the kola testiso code
We can re-enable this so it runs on x86 at least in the meantime

Tracker issue: coreos/fedora-coreos-tracker#1657
jbtrystram added a commit to jbtrystram/fedora-coreos-config that referenced this issue Feb 2, 2024
@jbtrystram
Enable iscsi test
be2b868 
I have found that nested virtualization does not work for
other arches than x86, so the test is disabled for those
arches in the kola testiso code
We can re-enable this so it runs on x86 at least in the meantime

Tracker issue: coreos/fedora-coreos-tracker#1657
jbtrystram added a commit to jbtrystram/fedora-coreos-config that referenced this issue Feb 5, 2024
@jbtrystram
Enable iscsi test
6b2a636 
I have found that nested virtualization does not work for
other arches than x86, so the test is disabled for those
arches in the kola testiso code
We can re-enable this so it runs on x86 at least in the meantime

Tracker issue: coreos/fedora-coreos-tracker#1657
jbtrystram added a commit to jbtrystram/fedora-coreos-config that referenced this issue Feb 5, 2024
@jbtrystram
Enable iscsi test
499b19d 
I have found that nested virtualization does not work for
other arches than x86, so the test is disabled for those
arches in the kola testiso code
We can re-enable this so it runs on x86 at least in the meantime

Tracker issue: coreos/fedora-coreos-tracker#1657
jbtrystram added a commit to jbtrystram/fedora-coreos-config that referenced this issue Feb 5, 2024
@jbtrystram
Enable iscsi test
3c1ab73 
I have found that nested virtualization does not work for
other arches than x86, so the test is disabled for those
arches in the kola testiso code
We can re-enable this so it runs on x86 at least in the meantime

Tracker issue: coreos/fedora-coreos-tracker#1657
jlebon pushed a commit to coreos/fedora-coreos-config that referenced this issue Feb 6, 2024
@jbtrystram @jlebon
Enable iscsi test
c2efb86 
I have found that nested virtualization does not work for
other arches than x86, so the test is disabled for those
arches in the kola testiso code
We can re-enable this so it runs on x86 at least in the meantime

Tracker issue: coreos/fedora-coreos-tracker#1657
aaradhak pushed a commit to aaradhak/fedora-coreos-config that referenced this issue Mar 18, 2024
@jbtrystram @aaradhak
Enable iscsi test
f031707 
I have found that nested virtualization does not work for
other arches than x86, so the test is disabled for those
arches in the kola testiso code
We can re-enable this so it runs on x86 at least in the meantime

Tracker issue: coreos/fedora-coreos-tracker#1657
@dustymabe
Copy link
Member

@jbtrystram can you update this? does this issue need to stay open?

@jbtrystram
Copy link
Contributor Author

The iscsi test are still not enabled due to this.
I haven't put any work on this yet, so this is still an issue

@dustymabe dustymabe mentioned this issue May 2, 2024
Open
@jlebon
Copy link
Member

jlebon commented Oct 15, 2024 •
edited
Loading

Related to this, apparently the iBFT kernel module isn't enabled in the Fedora/RHEL kernel on non-x86_64 currently:

Internal request to change that: https://issues.redhat.com/browse/RHEL-62740

Edit: apparently this is by design; that kernel module isn't used on aarch64 for iBFT.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@dustymabe @jlebon @jbtrystram