-
Notifications
You must be signed in to change notification settings - Fork 195
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement compatibility with DKMS (Nvidia, etc.) #1091
Comments
Yeah, there's a huge world of stuff here. Supporting |
@cgwalters Would this also cover supported |
I have no idea honestly without diving in a lot. I suspect they're going to be mostly equivalent but it's really just a wild guess. |
@cgwalters The Akmods mechanism makes kmod RPMs for the kernel packages installed and installs them, rather than building kmods for the running kernel and just slotting them in. This was recently integrated into Fedora proper. |
I said this elsewhere but to repeat here; I think we could pretty easily implement a generic hook in Where things bifurcate a lot here is - do you install the equivalent of Option What would mix both |
Perhaps it's not quite the right issue to discuss this in, but I'd just like to raise the concern that ideally whatever solution for is proposed for this issue should not make it too difficult for an end-user to sign the resulting akmods-built modules for Secure Boot using their own keypair. As this issue identifies, losing easy access to ZFS, VirtualBox and the NVidia Drivers is a major concern for new users to Atomic/Silverblue, but I think it's also a concern if a user is only able to access the above at the cost of disabling Secure Boot. |
@alexhaydock Not just new users! I've been a Linux workstation / laptop user since Red Hat 6.2 and if a distro won't support my AMD GCN 1.1 card or WiFi hotspot or HP Omen laptop with an NVidia 1050Ti, I'll run a different distro. Secure Boot is over-rated; if I have to disable it to use my machine, that's exactly what I'll do. |
So, I took a short initial look at this from the perspective of supporting nvidia in silverblue. There are two major suppliers of the nvidia driver in rpm form, rpmfusion and negativo17. rpmfusion seem to only support akmod, whereas negativo17 does both akmod and dkms. I didn't know any details about dkms or akmod other than the fact that they auto-built drivers before this, so i took a quick look at them: akmodYou install akmod-nvidia, it depends on akmods (in fedora) and contains just:
Then akmods itself has a few hooks (boot service, rpm transaction hook, optionally shutdown service) that gets called such that we can rebuild the srpm whenever there is a new kernel, generating a kernel-specific version of the bundled src.rpm. For example, the above srpm + kernel 4.18.10-200 generates the built rpm
This seems very nice, simple and rpm-focused, and the akmods program is a 500 line shellscript. dkmsdkms is a more generic framework and works on multiple distros. As such it has its own database of stuff in /var/lib/dkms, matched with sources in /usr/src which is updated with the "dkms" helper. The dkms-nvidia package contains the sources for the module extracted in /usr/src/nvidia-396.54, as well as a dkms.conf file telling dkms how to build the sources. The %post of the dkms-nvidia rpm then does:
The first one sets up a symlink from Additionally, dkms has a hooks similar to akmods (boot service, rpm transaction hook) that runs the build and install parts for the new kernel. what works for rpm-ostreedkms is not really a great fit for rpm-ostree with its reliance of stuff in /var, and non-rpm-tracked module files. akmods seems like a pretty clean to me, and fits the overall rpm based scheme of rpm-ostree, but building on the live system or in the rpm transaction hook clearly doesn't work. However, they way akmods work is that you create a kmod-nvidia srpm with full sources, but when built normally just generates a akmod-nvidia rpm (containing a copy of the srpm, which is later rebuild targeting a specific kernel). This means that the yum repo for the akmod has a .src.rpm for the driver which is easy for rpm-ostree to get at via dnf. So, the way I propose this would work is that you can layer srpms as well as rpms:
This would mean the same as There are some special things we need to to when building the srpm. For instance we need to set the |
I guess the question is, do we take a dependency on podman & co for the build container, or do we use rpm-ostree itself to construct the image for building the srpm, deploy it to a termporary location and spawn it via bwrap? @cgwalters ? |
Thanks for the analysis; you went a bit deeper into the details of both akmod/dkms than I had before. But some of this was already noted in #1091 (comment) right? I like the idea, a whole lot of implications. Actually in general...I would really love to also support a more "build from source" model on top of rpm-ostree (e.g. "apply this patch to systemd", or "build systemd from this git commit"). There's a lot of prior art here; obviously libostree was split out of gnome-continuous which has such a model. Such a system could be built on top of something that built srpms, although I lean a bit towards skipping the srpm path and orienting more towards at least dist-git, as well as direct from upstream git. But even this though opens up a question as whether we would really want the build tools on the host or in a container.
This would probably block on #1180 The "build using container" was already prototyped out here #1091 (comment) Big picture...I lean a bit towards the container path. But I am not likely to hack on this myself in the near future (even though my laptop has an nvidia card, I don't play games and nouveau is OK for me). |
Yeah, I just have a primary interest in the specific nvidia case, so i wanted to dump my research here.
I agree that this would be nice. However, there are two complications to this. First of all rpm-ostree needs to have a way to specify how to build the modifications and store these in the ostree metadata next to where the package layer is stored. Punting this to srpm means all we need is to store the srpm name. Of course, one could punt specifiying this to some other highlevel method, like "run the container image named foo", then all you need to do is store the image name in the metadata. Secondly, there needs to be a way to extract the modifications of the new build into the final ostree image. With rpm the build and the install are automatically separated, whereas in a container situation they might not be. For example, you will be building in a container that has a /usr with compilers, etc, but then you want to install into a different /usr. I can imagine solving this. For example, you could have the newly composed ostree image checked out somewhere, and then you can use rofiles-fuse to get a safe version of that, which you mount as a volume in the build container, and then when you build the app you set DESTDIR to the volume during install. Should work, but it is a bunch of extra work you get for free from rpmbuild.
One complexity of using the container path here is that you have to somehow ensure that the build container matches the ABI of the final ostree image. For example, if we're building kernel modules we need to have the right kernel-devel header. However, if you're building arbitrary userspace code you need to match the full userspace ABI. I.e. if you build against a library it needs to be the same version of the library and built in the same way, you need same c++ compiler ABIs, etc. If we automatically compose the build environment from the same packages image as the ostree image this is a lot easier to guarantee. |
This is problematic for things like akmods, which rely on being able to build from a source package. In addition, you can't guarantee git. You can, however, guarantee a srpm.
Could we do something similar to the btrfs seed+sprout thing to support a transparent layer that is invoked as a container to do these things? The other, more practical issue is that we can't guarantee that the matching kernel packages are going to be present in the repo at the time this happens. So what do we do then? |
@cgwalters "Big picture...I lean a bit towards the container path. But I am not likely to hack on this myself in the near future (even though my laptop has an nvidia card, I don't play games and nouveau is OK for me)." I have an HP Omen with Intel graphics and an NVidia 1050Ti. How the drivers get built doesn't matter to me - if it takes a container that only occupies resources during an install and has to do a moderate-sized compile, that's no big deal. Not having the NVidia drivers is a show-stopper. So I like the source RPM install idea a lot. ;-) |
An alternative solution would be to provide |
@mskarbek kmod packages are not special in any way and probably works already. However, the problem with them is that they need to be updated in lock-step with new kernel updates, and they stop working the second you run a non-standard kernel. In practice this means that people need something like dkms to be guaranteed to have an up-to-date nvidia driver. |
This should be working in F30 Silverblue |
@matthiasclasen Great news! How do I get F30 Silverblue to test? I have a pretty short window of availability this coming week but can squeeze this in. |
I’m not sure its working automatically yet, the nvidia-kmod rpm needs to be
rebuilt with the new kmodtools rpm. I’ll check it out tomorrow and write a
blog post about it.
…On Tue, 5 Mar 2019 at 20:14, M. Edward (Ed) Borasky < ***@***.***> wrote:
@matthiasclasen <https://github.com/matthiasclasen> Great news! How do I
get F30 Silverblue to test? I have a pretty short window of availability
this coming week but can squeeze this in.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1091 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AA8eECrwsNIuHNGMPuT9Dsq3NikNURdAks5vTsIGgaJpZM4QVkDc>
.
|
Would be awesome! This is what forced me back to the traditional RPM setup after trying out Silverblue. |
I wrote up how to test this:
https://blogs.gnome.org/alexl/2019/03/06/nvidia-drivers-in-fedora-silverblue/
…On Wed, Mar 6, 2019 at 2:35 AM James Cassell ***@***.***> wrote:
Would be awesome! This is what forced me back to the traditional RPM setup
after trying out Silverblue.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1091 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AA8eEMfB_UoTI_EfG5qnsBhYD2p-EYeWks5vTxtFgaJpZM4QVkDc>
.
|
Hi @matthiasclasen, could you kindly explain why? |
Not sure I understand the question. I should be working because the necessary changes were merged |
FWIW, DKMS support is required for VirtualBox and ZFS. In addition, a ZFS root filesystem install requires the ability to run grub2-mkconfig to generate a new grub.conf, and to run dracut to generate a new initramfs. Running grub2-install to install grub is also needed on systems with legacy (non-EFI) BIOS. |
I'm on F32 Silverblue and I seem to be unable to modprobe a dkms module. By "working" do you mean that in F30 Silverblue there's a I installed it from https://copr.fedorainfracloud.org/coprs/sentry/v4l2loopback/, then following its instructions, I rebooted and executed: ➤ sudo modprobe v4l2loopback
modprobe: FATAL: Module v4l2loopback not found in directory /lib/modules/5.6.16-300.fc32.x86_64 If I'm not wrong, I should be able to modprobe it. 🤷 |
Unfortunately you can't do this. rpm-ostree based distros e.g Silverblue do not support dkms kernel modules. Any third-party modules should be compiled as a kmod module against the running kernel and packaged into an rpm package so you can install it with rpm-ostree. In the meantime you can install kernel-devel and and "Development tools" group in a toolbox container and compile the module there. Then you can load the .ko file in the host kernel with insmod. The downside is that you need to do this every time you get a kernel update. There is also the possibility of v4l2loopback to be included in the mainline kernel in future: umlaeute/v4l2loopback#268 |
Thanks for the explanation. If you manage to package that module, it'd be awesome, as in COVID-19 times, being able to use a phone or camera as webcam is more needed. |
@fouladi Thanks for sharing, and I'd also love to see v4l2loopback if not in the mainline kernel then on rpmfusion! However, following your pointers.. I can't seem to load the toolbox compiled .ko file into the host kernel using insmod. Any other pointers? |
My guess is that you have secure boot enabled. In that case, there is more to do to make it work. In short you need to sign the modules with your gpg private key (generate a public/private key pair if you don't have any) and then add the key to the UEFI boot loader. This might help but I guess you need to research a little too: https://docs.fedoraproject.org/en-US/Fedora/22/html/System_Administrators_Guide/sect-signing-kernel-modules-for-secure-boot.html |
Ah yes .. makes sense! Thanks.. I'll have a play. |
Not sure if this would be appropriate, but if there is someone who wanted to seriously take this on with intent to address allowing for ZFS to be utilized we would be interested and capable in funding that effort. We've used btrfs as well as kept an eye on stratis, but for some deployments it's still very comfortable to use ZFS. Given how ZFS charts to kernel releases it's typically better to stick to a distribution that is LTS focused, but it'd still be very interesting to see rpm-ostree allow for the use of ZFS. |
Well, I got the same message initially. I don't have secure boot, but I was dumb and was trying to add the module from inside the toolbox container. On the host, I got:
Just did a fresh rpm-ostree update and created the toolbox. Both are using:
There's just something slightly different about the toolbox vs the host... |
FWIW I got v4l2loopback working in silverblue with the following steps:
|
So, @Ramblurr has it right. However, the "Unknown symbol in module" problem (for me) stems from the fact that I tried to |
would really appreciate support for this for the sake of using zfs |
There is a discussion over here which poses the question:
Any insights? |
Is that from a checkout of the v4l2loopback repo? I was hoping there was a way without having to build it yourself Seems the simple task of creating a virtual device for a mirrorless/DSLR input and a virtual device for OBS to output to is difficult for silverblue users. These instructions apparently do the trick for non-immutable installations: obsproject/obs-studio#3929 (comment) |
rpm-ostree version info:
When installing nvidia kmod it fails:
From the journal:
The scriptlets are:
The text was updated successfully, but these errors were encountered: