Add overrides for multi-arch #374
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@jcajka, can you test this? |
|
(And yeah, we need to pick up discussions again about making it easier to add overrides so they're not mostly duplicated across 4 files.) |
Testing now. Looks good so far on ppc64le. |
|
s390x kind of too(I have too start issue about initscripts there, but that is other issue ;)) and aarch64 is ok too. Thanks for quick fix :). |
|
Actually, that is new failure on ppc64le build(fetch is ok). |
|
Same for build on aarch64. I assume that might be related to the new packages, rpm-ostree? |
Hmm, that's odd. What version of As a sanity-check, do you get a successful compose if you only have the |
Checking it now. With manifest lock it fails as noted: |
|
Keeping just the crypto-policies in the lock file leads to the fallowing failure during build, fetch is ok: Time to move to f32 base image? ;) |
OK, so the issue here is that this is pulling in a version of container-selinux that is much older than what's in the updates repo, and it doesn't have What happens if you add: container-selinux:
evra: 2:2.124.0-3.fc31.noarch? That should force libsolv to print out the conflict preventing it from picking the latest version.
Yup, we're planning to move there in two releases (three releases for stable). :) |
I can confirm that this is the fix here. On topic of f32, I can say that at least for the cosa images it works just fine on multiarch(not used here). |
jlebon
force-pushed
the
pr/arches-lockfiles
branch
from
9e76772
to
390751d
Compare
|
OK, I added a second commit to this PR: |
|
(To clarify, that latter issue has nothing to do with multi-arch. It just happened to come up at the same time. :) ) |
With coreos#355, the pool is now used for fetching lockfiles only. This however broke multi-arch builds because although there are no lockfiles, it's currently not possible to get a depsolve without at least locking crypto-policies: coreos#335 (comment) While we can't yet maintain base lockfiles for other arches, we can at least start maintaining overrides. This will also in the process fix builds there.
Latest version has a hard dep on selinux-policy-minimal, which wants python: https://src.fedoraproject.org/rpms/container-selinux/pull-request/3
jlebon
force-pushed
the
pr/arches-lockfiles
branch
from
390751d
to
c9d69bb
Compare
|
Seems sane to me though we clearly have two problems:
|
I think that also boils down if and how much COS wants to integrate in to the current release processes for packages.
IMO locks/pin/overrides should be at NVR level not NVRA(as currently) for most cases and packages as I don't believe there will be arch dependent overrides that will be that common(as they shouldn't be, to discourage any short term hacky workarounds) and ideally managed via some automated tool(-ing)(IMHO, as packager, bodhi/koji could be used/repurposed for this, but really rel-eng, infra will be able to answer that). One of my current main motivations is that I really do not have cycles to manage the multi-arch lock files on top of dealing with breakages and general fixes to the forming multi-arch. |
Right. And actually, one thing @dustymabe and I were discussing recently was that we should try to get FCOS CI directly into Bodhi so that packages don't get pushed to the updates repo if CI fails.
As mentioned in #355 (comment), base lockfiles should not be manually maintained. Only overrides. See e.g. coreos/fedora-coreos-pipeline#229, which we just merged to (re-)automate this again. We could make this blindly use NEVR instead of NEVRA, but the whole point of lockfiles is to get reproducible builds which passed validation. Using NEVR to lock multi-arch builds without having actually sanity-checked it is way less meaningful. So I think we just need to sort out coreos/fedora-coreos-tracker#262 and get proper multi-arch builds orchestration. |
Not sure I follow; !x86_64 isn't locked today; in the interim it'd help them test the exact state that matches the main build. |
It's not the exact state though. It's an almost completely different set of RPMs (and doesn't capture all the non-x86_64 arch-specific packages like If it helps, we can work towards this. But I think we should instead spend our energy fleshing out and working on coreos/fedora-coreos-tracker#262 instead. |
|
I agree for overrides though we probably want an NEVR lockfile to make it easier to maintain. Something like |
How so? Aren't we using koji to get the same source built across all the arches?
Sure, I doubt those are problematic very often though (and when they are it's something the arch team is in a position to debug and fix).
Yes we definitely need that. Though as you know I am increasingly convinced the lockfiles should manage repos: coreos/rpm-ostree#1858 (review) |
Yeah, but binary files themselves are different per architecture. So we're locking inputs we haven't technically tested, let alone composed.
The design though is strongly geared towards having the full OS locked. Not locking those means it wouldn't compose with I know these points might seem like nitpicking, and I don't disagree there's still some value to doing it anyway, but the original idea of lockfiles was locking the OS to a known green state and only allowing validated bumps at all steps. I'm wary of doing lots of work here (both for us and the arch team) that doesn't end up aligning with where we want to go for coreos/fedora-coreos-tracker#262. |
jlebon
added a commit
to jlebon/fedora-coreos-config
that referenced
this pull request
May 22, 2020
Let's start maintaining overrides for all arches in prep for promoting testing-devel to f32 (which already has multi-arc overrides -- see coreos#374). Stay tuned for a more generic `manifest-lock.overrides.yaml` file to help dedupe this stuff.
jlebon
added a commit
to jlebon/fedora-coreos-config
that referenced
this pull request
May 22, 2020
Let's start maintaining overrides for all arches in prep for promoting testing-devel to f32 (which already has multi-arc overrides -- see coreos#374). Stay tuned for a more generic `manifest-lock.overrides.yaml` file to help dedupe this stuff.
jlebon
added a commit
to jlebon/fedora-coreos-config
that referenced
this pull request
May 22, 2020
Let's start maintaining overrides for all arches in prep for promoting testing-devel to f32 (which already has multi-arc overrides -- see coreos#374). Stay tuned for a more generic `manifest-lock.overrides.yaml` file to help dedupe this stuff.
dustymabe
pushed a commit
that referenced
this pull request
May 22, 2020
Let's start maintaining overrides for all arches in prep for promoting testing-devel to f32 (which already has multi-arc overrides -- see #374). Stay tuned for a more generic `manifest-lock.overrides.yaml` file to help dedupe this stuff.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
With #355, the pool
is now used for fetching lockfiles only. This however broke multi-arch
builds because although there are no lockfiles, it's currently not
possible to get a depsolve without at least locking
crypto-policies:#335 (comment)
While we can't yet maintain base lockfiles for other arches, we can at
least start maintaining overrides. This will also in the process fix
builds there.