Upgrade to Catalyst 4 and don't use repo snapshots for stage1 #2115
Conversation
|
I have made associated documentation changes in flatcar/flatcar-website#351. |
Test report for 4027.0.0+nightly-20240710-2100 / amd64 arm64Platforms tested : qemu_uefi-amd64 qemu_update-amd64 qemu_uefi-arm64 qemu_update-arm64
Diagnostic output for qemu_uefi-arm64, run 1
Diagnostic output for qemu_uefi-arm64, run 1
Diagnostic output for qemu_uefi-arm64, run 2Diagnostic output for qemu_uefi-arm64, run 1
Diagnostic output for qemu_uefi-arm64, run 1
Diagnostic output for qemu_update-arm64, run 1
|
|
I ran a two-phase build in Jenkins, and it looks good. It doesn't show the changes in the SDK, but the production image is barely changed besides packages changing from ::coreos to ::coreos-overlay. Good to go then? I don't think it needs a changelog entry as its not user-facing. |
|
As it affects the end user image package full name, maybe it is worth a changes changelog? |
It's going to change to ::flatcar-overlay in the PR after this one, but I guess I should still add this in the meantime and update it later. |
This is needed by Catalyst 4. Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
This is needed by Catalyst 4. Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
This hasn't been needed for a while, since Gentoo started handling Python modules according to PEP 517. We now need util-linux with Python support for Catalyst 4, and this hack erroneously causes the module to be installed under /usr/lib64 rather than /usr/lib. Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
Catalyst 4 has totally changed the way repositories are handled. It only works when the name of the directory containing the repository matches the configured name of that repository. This was not the case for us, with the coreos repository residing in the coreos-overlay directory. We wanted to move and rename our repositories anyway, but this is a big change, so we'll do separately. For now, this just renames coreos to coreos-overlay. Catalyst 4 also ingests the main repository snapshot as a squashfs rather than a tarball. It features a utility to generate such a snapshot, but it doesn't fit Flatcar well, particularly because it expects each ebuild repository to reside at the top level of its own git repository. It was very easy to call tar2sqfs manually though. Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
This is what upstream Gentoo does. They would previously update the entire seed, but this took a long time. Our seeds are much bigger, so we kept repo snapshots to build stage1 against these instead. The new method of only rebuilding packages with changed sub-slots is a good compromise and removes the need to write stage1 hooks that selectively catch the repository up. This also avoids some conflicts by adding the `--ignore-world` option. Gentoo seeds have nothing in @world. We have much more, but none of that is needed for stage1. This continues to exclude cross-*-cros-linux-gnu/* as that is not needed for stage1. It now also excludes dev-lang/rust, because it is never a DEPEND, so it would not break other packages in this way. It may fail to run due to a sub-slot change in one of its own dependencies, but it is also unlikely to be needed in stage1 and it is not configured to use the system LLVM. If needs be, we could improve the behaviour of Portage's @changed-subslot to respect `--with-bdeps`. In my testing, it was unable to handle an SDK from 17 months ago, but one from 7 months ago did work. In practise, we will always use a much more recent one, which is far more likely to work. Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
The changes to support Catalyst 4 are not backwards compatible and we need a seamless transition for builds in CI. Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
sdk_container/src/third_party/portage-stable/app-arch/pixz/pixz-1.0.7-r1.ebuild
Show resolved
Hide resolved
...ainer/src/third_party/portage-stable/sys-fs/squashfs-tools-ng/squashfs-tools-ng-1.3.0.ebuild
Show resolved
Hide resolved
Upgrade to Catalyst 4 and don't use repo snapshots for stage1
Split out from #2093.
Although Catalyst 4 still only has release candidates, Gentoo has been using it for its releases for years. Catalyst 3 was masked a while ago, so we should not linger on this version.
Catalyst 4 has totally changed the way repositories are handled. It only works when the name of the directory containing the repository matches the configured name of that repository. This was not the case for us, with the coreos repository residing in the coreos-overlay directory. We wanted to move and rename our repositories anyway, but this is a big change, so we'll do separately. For now, this just renames coreos to coreos-overlay.
This also stops using repo snapshots for stage1, which avoids the need to fix up the coreos repo name in the seed with a hook script and also renders all the existing hooks scripts obsolete. This approach is in line with what Gentoo does.
How to use
Simply enter a recent SDK and run
sudo ./bootstrap_sdk. Note that this will automatically modify the SDK container to rename the coreos repo and upgrade Catalyst, so you will need to recreate the container to go back to an older branch.Testing done
I have manually rebuilt the SDK and am currently also rebuilding it in Jenkins using 4012.0.0 as the seed.
changelog/directory (user-facing change, bug fix, security fix, update)/bootand/usrsize, packages, list files for any missing binaries, kernel modules, config files, kernel modules, etc.