Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DNF/yum removes ZFS on kernel updates (Fedora Core 38) #15188

Open
tomchiverton opened this issue Aug 18, 2023 · 14 comments
Open

DNF/yum removes ZFS on kernel updates (Fedora Core 38) #15188

tomchiverton opened this issue Aug 18, 2023 · 14 comments
Labels
Type: Feature Feature request or new feature

Comments

@tomchiverton
Copy link

dnf.log

2023-08-11T06:05:48+0100 DEBUG Installed: kernel-6.4.9-200.fc38.x86_64
2023-08-11T06:05:48+0100 DEBUG Installed: kernel-core-6.4.9-200.fc38.x86_64
2023-08-11T06:05:48+0100 DEBUG Installed: kernel-devel-6.4.9-200.fc38.x86_64
2023-08-11T06:05:48+0100 DEBUG Installed: kernel-modules-6.4.9-200.fc38.x86_64
2023-08-11T06:05:48+0100 DEBUG Installed: kernel-modules-core-6.4.9-200.fc38.x86_64 
2023-08-11T06:05:48+0100 DEBUG Removed: kernel-6.3.12-200.fc38.x86_64
2023-08-11T06:05:48+0100 DEBUG Removed: kernel-core-6.3.12-200.fc38.x86_64 
2023-08-11T06:05:48+0100 DEBUG Removed: kernel-devel-6.3.12-200.fc38.x86_64
2023-08-11T06:05:48+0100 DEBUG Removed: kernel-modules-6.3.12-200.fc38.x86_64
2023-08-11T06:05:48+0100 DEBUG Removed: kernel-modules-core-6.3.12-200.fc38.x86_64
2023-08-11T06:05:48+0100 DEBUG Removed: kmod-xtables-addons-6.3.12-200.fc38.x86_64-3.24-1.fc38.x86_64
2023-08-11T06:05:48+0100 DEBUG Removed: zfs-2.1.12-1.fc38.x86_64
2023-08-11T06:05:48+0100 DEBUG Removed: zfs-dkms-2.1.12-1.fc38.noarch

This should never be allowed to happen.

echo 'zfs' > /etc/dnf/protected.d/zfs.conf

Should be added to the RPM install script (and removed on removal)

@tomchiverton tomchiverton added the Type: Feature Feature request or new feature label Aug 18, 2023
@AllKind
Copy link
Contributor

AllKind commented Aug 18, 2023

zfs-2.1.12
Supported Platforms

Linux: compatible with 3.10 - 6.3 kernels

@tomchiverton
Copy link
Author

Note this isn't about what is or is not currently supported, though the above is a concrete example.

Adding two lines to the install/uninstall script would remove this footgun for any future incompatibility as well as the current one.

I'd submit a PR if I understood anything about how the RPMs are built.

@k-korn
Copy link

k-korn commented Sep 3, 2023

Adding two lines to the install/uninstall script would remove this footgun for any future incompatibility as well as the current one.

This will add the opposite footgun, e.g. "Having ZFS installed breaks updates on my Fedora!!11":

# dnf update
...
Error:
 Problem: The operation would result in removing the following protected packages: zfs-dkms
(try to add '--skip-broken' to skip uninstallable packages)

So yes, system administrator should be wary if they need ZFS non-removable or they need the latest updates w/o ZFS.

@tomchiverton
Copy link
Author

tomchiverton commented Sep 4, 2023 via email

@AllKind
Copy link
Contributor

AllKind commented Sep 4, 2023

Problem: The operation would result in removing the following protected packages: zfs-dkms
(try to add '--skip-broken' to skip uninstallable packages)

Then people would start using --skip-broken. -> The kernel gets updated to a version which is not compatible with the installed zfs version. Which could lead to disastrous results, if the dkms modules nevertheless successfully compile. If not, the result would be the same as you want to avoid. The zfs filesystem would not be accessible.

As a zfs user on a bleeding edge distro, you should know, that it always takes a few weeks until zfs catches up with the latest kernel versions.
This could only be solved, if zfs gets included in the kernel, which will likely never happen because of licensing reasons.

Edit: btw... never install updates unattended! ;-)

@tomchiverton
Copy link
Author

tomchiverton commented Sep 4, 2023 via email

@mskarbek
Copy link
Contributor

mskarbek commented Sep 4, 2023

It's been months now, not weeks, since ZFS from the repo stopped working with Fedora's latest kernel -- †øღ Sent from a super computer that fits in my pocket and is connected to the sum total of all human knowledge

@tomchiverton are you paying for OpenZFS support or something like that? If not, what justifies your demanding behavior? And it's been one month, not months. Fedora released the latest 6.3 kernel in July.
It's your responsibility to track and match compatible kernel versions and skip unsupported updates until OpenZFS releases a new version.

@Gibson85
Copy link

Gibson85 commented Sep 4, 2023

I agree with @tomchiverton. I know it is or was vacation time, but these delays are annoying. Especially often one gets not a single hint at all that after update ZFS won't work anymore. With every shutdown I must be aware to uncheck the "install updates" combobox otherwise I would have a broken system again. And I had dozen broken systems in the last years because of ZFS. Fedora is one of the most used distributions if not the most used at all. There exists testing version for e.g. Fedora. So why is it so hard to implemented future kernels right in time? I don't understand it. And this has nothing to do whether it's free software or not. Its only a thing of "we don't want it to work better". And "more stable" distributions are not a solution also. E.g. I used Debian for years before. This "rock solid" (laughing) outdated distro had horrible bugs and even did not support encryption when I used it last.

@m-ueberall
Copy link

m-ueberall commented Sep 4, 2023

Especially often one gets not a single hint at all that after update ZFS won't work anymore. With every shutdown I must be aware to uncheck the "install updates" combobox otherwise I would have a broken system again.

As @mskarbek pointed out, in general, as long as you use software (packages) from different sources/repositories, it is your responsibility to track and match available versions.

@Gibson85: I'm using Ubuntu instead of Fedora on a daily basis, but the dnf documentation explicitly explains how to exclude (here: kernel) packages from transactions which is supposed to cover every direct/indirect interaction with dnf (including the combo box you mentioned) similar to what apt offers on Ubuntu/Debian. If this does not work as documented, this should IMHO be reported against dnf ASAP. OTOH, if your environment stops working because you cannot update certain sofware components for a couple of weeks due to the lack of matching available versions or whatever reason, maybe it makes sense to look into options how to circumvent this problem in the first place (this can/should be discussed elsewhere, though.)

Fedora might be one of the most used distributions, but it is after all but one distribution among all those distributions, operating systems that the OpenZFS project addresses. If a sufficient number of Fedora users need to use both current kernels and current OpenZFS versions, then why not convert this issue into a discussion, look for fellow campaigners, fork this repository, and establish/maintain a workflow that fits your needs? If this proves feasible over a certain period of time, maybe merging said fork with the main repository might be an option (if need be—that's debatable).

@tomchiverton
Copy link
Author

tomchiverton commented Sep 4, 2023 via email

@ElCoyote27
Copy link

ElCoyote27 commented Sep 7, 2023

@tomchiverton to rebuild the rpms, I usually use this:
https://openzfs.github.io/openzfs-docs/Developer%20Resources/Custom%20Packages.html#get-the-source-code
the pre/post stuff in the rpm spec file.

@tomchiverton
Copy link
Author

I looked into that, and that'd be how I'd test any change in the pre/post script or add an extra file to be installed to provide /etc/dnf/protected.d/zfs.conf

What I can't find in the repo is where those (or their sources) actually are.

@jkool702
Copy link

I have a good bit of experience with dnf and building custom RPM's from source. The behavior you describe that prompted this issue is sorta not a bug.

The logic the dnf uses here is that it will keep around RPM's that depend on a specific kernel version so long as you have at least 1 kernel installed that is a valid version for that RPM. You'll notice that dnf didnt try to uninstall zfs when you installed your first 6.4.x kernel. It did, however, [try to] uninstall it when you uninstalled your last remaining 6.3.x kernel.

If you think about it, this logic makes sense - it will keep zfs around so long as it can be used somewhere in the installed system (even if it cant be used with your current kernel), but as soon as there is no valid possible way to use it (because you dont have a kernel installed that it is compatible with) it automatically uninstalls it.

Funnily enough, I use Zfs on root and this problem is automatically a non-issue for me, since dnf will never remove the running kernel and i cant get into the system at all on a unsupported kernel, so I stay on the latest supported kernel until zfs catches up (since...well...I dont really have a choice in the matter), and as such dnf never tries to remove fs since there is always at least 1 zfs-supported kernel around. Its pretty well foolproof...youd have to do an offline dnf update from a live usb to accidentally delete zfs if you staty on using the last zfs-supported kernel.

As such, my advice would be to not change anything and either

  1. keep on using the last zfs-supported kernel until a new zfs version is released, or
  2. add the last zfs-supported kernel as a protected package until a new zfs version is released

Alternatively, if you dont care about not accessing zfs for a while, let dnf uninstall it and install it again when a new version comes out. uninstalling zfs shouldnt effect your pools...they will still be there when you reinstall. And its not like you could (or should) use it with an incompatible kernel (mean you sometimes can if you tweak the specfile and recompile it, and it actually compiles, but in this case you really shouldnt...assuming you dont want data corruption).

I see no reason to try and change this behavior...it seems fairly optimal; as-is.

@tomchiverton
Copy link
Author

it will keep zfs around so long as it can be used somewhere in the installed system (even if it cant be used with your current kernel), but as soon as there is no valid possible way to use it (because you dont have a kernel installed that it is compatible with) it automatically uninstalls it.

OK, so this behaviour, though undesirable, is expected because ZFS says it only requires kernel 6.2.x or older, so as soon as the last 6.2.x is evicted, because ZFS doesn't depend on the remaining kernels, it's removed ?

Apparently ZFS 2.2 is nearly ready, which should hopefully resolve this ? https://www.theregister.com/2023/08/16/openzfs_zfsbootmenu_2_2/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Feature Feature request or new feature
Projects
None yet
Development

No branches or pull requests

8 participants