-
Notifications
You must be signed in to change notification settings - Fork 153
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
addfileprovides_queue: return queues containing all added provides #531
Conversation
The queues returned can be stored in repository meta section REPOSITORY_ADDEDFILEPROVIDES: https://github.com/openSUSE/libsolv/blob/master/examples/solv/fileprovides.c#L73 Storing file dependencies can lead to problems: - first load metadata without filelists - call pool_addfileprovides_queue(...) - and store queues in REPOSITORY_ADDEDFILEPROVIDES then on the next run filelists are loaded but during adding fileprovides none are added because stored file dependecies match required file dependencies. This can be solved by returning added file provides instead. This also matches with the REPOSITORY_ADDEDFILEPROVIDES name.
I don't really understand the problem you're trying to solv. The REPOSITORY_ADDEDFILEPROVIDES list should contain all the file list provides that have been added to the package provides. I.e. they need to include all the old entries. I don't understand why dnf5 repoclosure does not work correctly in the dnf5 issue. If a file is in the addedfileprovides list of a repository, it has been added to the provides of the packages that contain it and thus should not be reported as missing. So the old on-disk solv file seems to be in an inconsistent state. (Btw you shouldn't need to mess with the considered map in make_provides_ready() as you set the POOL_FLAG_WHATPROVIDESWITHDISABLED flag. Just saying...) |
It is quite possible we have somehow messed up the solvfile which is causing the issues. <?xml version="1.0" encoding="UTF-8"?>
<metadata xmlns="http://linux.duke.edu/metadata/common" xmlns:rpm="http://linux.duke.edu/metadata/rpm" packages="4">
<package type="rpm">
<name>agua</name>
<arch>x86_64</arch>
<version epoch="0" ver="1.0" rel="1.fc29"/>
<checksum type="sha256" pkgid="YES">ade558f47b9c4ca4de9a53d90c48a84ef6c6007908e86bfb71f2e576b055ddb9</checksum>
<summary>Made up package</summary>
<description>planta description</description>
<packager></packager>
<url>None</url>
<time file="1690201478" build="1690201478"/>
<size package="6627" installed="0" archive="252"/>
<location href="x86_64/agua-1.0-1.fc29.x86_64.rpm"/>
<format>
<rpm:license>GPLv3+</rpm:license>
<rpm:vendor></rpm:vendor>
<rpm:group>Unspecified</rpm:group>
<rpm:buildhost>7e6d7c9478f2</rpm:buildhost>
<rpm:sourcerpm>agua-1.0-1.fc29.src.rpm</rpm:sourcerpm>
<rpm:header-range start="4504" end="6525"/>
<rpm:provides>
<rpm:entry name="agua" flags="EQ" epoch="0" ver="1.0" rel="1.fc29"/>
<rpm:entry name="agua(x86-64)" flags="EQ" epoch="0" ver="1.0" rel="1.fc29"/>
</rpm:provides>
</format>
</package>
<package type="rpm">
<name>planta</name>
<arch>x86_64</arch>
<version epoch="0" ver="1.0" rel="1.fc29"/>
<checksum type="sha256" pkgid="YES">7686cdfd416a33058736d4887ef7632dd653133f1643efe90967866c05849a5d</checksum>
<summary>Made up package</summary>
<description>planta description</description>
<packager></packager>
<url>None</url>
<time file="1690201478" build="1690201478"/>
<size package="6146" installed="0" archive="124"/>
<location href="x86_64/planta-1.0-1.fc29.x86_64.rpm"/>
<format>
<rpm:license>GPLv3+</rpm:license>
<rpm:vendor></rpm:vendor>
<rpm:group>Unspecified</rpm:group>
<rpm:buildhost>7e6d7c9478f2</rpm:buildhost>
<rpm:sourcerpm>planta-1.0-1.fc29.src.rpm</rpm:sourcerpm>
<rpm:header-range start="4504" end="6097"/>
<rpm:provides>
<rpm:entry name="planta" flags="EQ" epoch="0" ver="1.0" rel="1.fc29"/>
<rpm:entry name="planta(x86-64)" flags="EQ" epoch="0" ver="1.0" rel="1.fc29"/>
</rpm:provides>
<rpm:requires>
<rpm:entry name="/a/path/to/water"/>
</rpm:requires>
</format>
</package>
</metadata> During loading we call Then on the next run we also load filelists: <?xml version="1.0" encoding="UTF-8"?>
<filelists xmlns="http://linux.duke.edu/metadata/filelists" packages="4">
<package pkgid="ade558f47b9c4ca4de9a53d90c48a84ef6c6007908e86bfb71f2e576b055ddb9" name="agua" arch="x86_64">
<version epoch="0" ver="1.0" rel="1.fc29"/>
<file>/a/path/to/water</file>
</package>
<package pkgid="7686cdfd416a33058736d4887ef7632dd653133f1643efe90967866c05849a5d" name="planta" arch="x86_64">
<version epoch="0" ver="1.0" rel="1.fc29"/>
</package> Here is the missing file provide but once
Thanks, I will look into it. |
So it may happen that the full filelist is not available when you call pool_addfileprovides()? That won't work. You have to options:
Option 2 makes sense if the filelist is only needed in rare cases. I don't know if this is true for your use cases. |
May I ask you what you mean by Additionally I have to mention that DNF5 does not even download file lists by default. The download of file lists is optional and configurable. |
On demand means when libsolv needs to know what packages contain the file provides. I.e. when it is asked to resolve a file dependency. The same mechanism is used when something like We could do that all the time for file provides, but it is much faster to first go through all packages and collect the needed file dependencies and then do one search through the filelist. That's exactly what pool_addfileprovides() does. Now if only a few packages contain file dependencies outside of the primary data, it makes sense to use a hybrid scheme: resolve all file dependencies from primary right away and do searches for the missing ones. That way the complete filelist is not needed most of the time. This can be used to solve the problem you have, because in this mode it does not matter if the complete filelist is available or not when pool_addfileprovides() is called. |
@mlschroe Thank you for the nice explanation |
Yes, thanks for the help! |
For: #520 Additional details in: openSUSE/libsolv#531 Test: rpm-software-management/ci-dnf-stack#1346
Since dnf4 now also conditionally load filelists it ran into the same problem as dnf5 here: rpm-software-management/dnf5#520 Additional details in: openSUSE/libsolv#531
Since dnf4 now also conditionally load filelists it ran into the same problem as dnf5 here: rpm-software-management/dnf5#520 Additional details in: openSUSE/libsolv#531
Since dnf4 now also conditionally load filelists it ran into the same problem as dnf5 here: rpm-software-management/dnf5#520 Additional details in: openSUSE/libsolv#531
Since dnf4 now also conditionally load filelists it ran into the same problem as dnf5 here: rpm-software-management/dnf5#520 Additional details in: openSUSE/libsolv#531
Change
addfileprovides_queue()
to return queues containing all added provides.(They can be stored in repository meta section REPOSITORY_ADDEDFILEPROVIDES like usual:
https://github.com/openSUSE/libsolv/blob/master/examples/solv/fileprovides.c#L73)
Before this change it returned file dependencies and storing those can lead to problems:
This can be solved by returning added file provides instead. This also matches with the REPOSITORY_ADDEDFILEPROVIDES name.
This is changing API.
I am not sure if it is a good solution to the problem.