Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build ZFS sysext with each release #1742

Merged
merged 19 commits into from
Mar 13, 2024
Merged

Build ZFS sysext with each release #1742

merged 19 commits into from
Mar 13, 2024

Commits on Mar 11, 2024

  1. build_sysext: Use parallel jobs

    Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
    jepio committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    fa41889 View commit details
    Browse the repository at this point in the history
  2. build_sysext: Set SOURCE_DATE_EPOCH for reproducibility

    Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
    jepio committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    54c8545 View commit details
    Browse the repository at this point in the history
  3. build_image: Build sysext store after image when requested

    This function is meant to prebuild certain sysexts to be released along
    with each release. These will not be built into the image, but instead
    can be fetched by the user on demand.
    
    The command to build sysexts would be:
    
      ./build_image prod sysext
    
    Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
    jepio committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    5d5dd59 View commit details
    Browse the repository at this point in the history
  4. build_image: Allow building sysexts for an existing image

    by skipping the check for an existing image directory if we're not
    building an image. This makes './build_image sysext' work.
    
    Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
    jepio committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    1bb108e View commit details
    Browse the repository at this point in the history
  5. update_ebuilds: Fix support for rsync of eclass

    For eclasses we need to skip the mkdir and trailing slash.
    
    Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
    jepio committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    601adeb View commit details
    Browse the repository at this point in the history
  6. coreos-kernel.eclass: Adapt install_build_source for compatibility

    build/source, which is accidentally an empty directory, needs to be a
    symlink so that Gentoo kmod ebuilds can build when setting
    KERNEL_DIR=/lib/modules/.../build. They detect the proper layout with
    the symlink.
    
    The other issue is building the Nvidia driver with
    SYSSRC=/lib/modules/.../build. This works on Ubuntu but fails on
    Flatcar. Ubuntus build directory contains symlinks to includes from the
    source tree so recreate the same layout.
    
    Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
    jepio committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    a134b02 View commit details
    Browse the repository at this point in the history
  7. coreos-base/coreos: Switch to app-alternatives/awk

    We explicitly install gawk but our profile explicitly forces the gawk
    useflag for app-alternatives/awk.  Some packages, like zfs, depend on
    app-alternatives/awk and it also installs the awk -> gawk that the gawk
    ebuild creates through pkg_postinst. So switch to app-alternatives/awk
    to make the implementation cleaner.
    
    Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
    jepio committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    1b4ca8a View commit details
    Browse the repository at this point in the history
  8. profiles/coreos: Add zfs useflags

    and a package.provided entry for a dependency that we don't need.
    
    Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
    jepio committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    a4da571 View commit details
    Browse the repository at this point in the history
  9. portage-stable: Add zfs & zfs-kmod ebuilds

    From Gentoo commit df182f2891606e757be3e8406a69f4a0e53ee324. Also import
    dist-kernel-utils.eclass.
    
    Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
    jepio committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    8e1b151 View commit details
    Browse the repository at this point in the history
  10. .github: Add zfs ebuilds and related eclasses to sync list

    Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
    jepio committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    bda8148 View commit details
    Browse the repository at this point in the history
  11. coreos/config: Add hooks for zfs ebuilds

    The zfs-kmod ebuild needs KERNEL_DIR to point to the correct
    /lib/modules directory.
    
    The zfs ebuild installs two systemd unit masks to /usr/lib/systemd which
    result in "dangling symlink" errors during the image build. These
    systemd unit masks are only necessary for old Ubuntu systems that have
    sysv-init-systemd compat wrappers.
    
    Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
    jepio committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    e8b4b30 View commit details
    Browse the repository at this point in the history
  12. board-packages: Add zfs dep

    We need zfs build as a board package so that we can provide a zfs
    sysext.
    
    Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
    jepio committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    6ccd5b6 View commit details
    Browse the repository at this point in the history
  13. build_library: Add mangle script for zfs sysext

    There are two challenges with the sysext: it needs config files in /etc
    and it needs udev rules for mounting during boot to work. The etc files
    are placed in the standard flatcar etc overlay path but the overlay is
    mounted from the initrd. So instead, we create a tmpfiles.d rule that
    symlinks the best important files over. For the udev issue, we create a
    drop-in in /etc that ensures udev runs after systemd-sysext.
    
    We also can't rely on systemd presets to work, so instead parse the
    preset file and statically create the service dependencies. For the
    primary zfs.target we rely on an Upholds entry. Users can still disabled
    unwanted services if they want.
    
    We also removed unnecessary files:
    - development files
    - initramfs related scripts
    
    Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
    jepio committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    87262e4 View commit details
    Browse the repository at this point in the history
  14. ci: Build sysexts together with image

    Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
    jepio committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    53e1649 View commit details
    Browse the repository at this point in the history
  15. build_library: Place EXTRA_SYSEXTS in a separate file

    This allows it to be sourced from other scripts and used e.g. in
    generating an image changes summary.
    
    Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
    jepio committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    7929434 View commit details
    Browse the repository at this point in the history
  16. coreos-kernel.eclass: Install host arch Makefiles into source dir

    Gentoo's linux-info.eclass tries to check the kernel version by
    including /lib/modules/.../source/Makefile and printing $(VERSION).  It
    unsets ARCH= before doing this (no clue why) which works with a full
    source tree but not with our minimized one. To fix this we need to
    archive arch/x86/Makefile also for arm64.
    
    Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
    jepio committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    42ed2d9 View commit details
    Browse the repository at this point in the history
  17. build_library: create_prod_sysexts: Move artifacts to BUILD_DIR

    Putting things in a sysexts subdir results in the same layout on
    bincache and does not follow the expected url schema for fetching the
    sysext.
    
    Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
    jepio committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    1ec40b2 View commit details
    Browse the repository at this point in the history

Commits on Mar 13, 2024

  1. sys-kernel/bootengine: Bump commit for sysext mkdir fix

    Fix a missing mkdir of /sysroot/etc/flatcar/sysext from the initrd before
    attempting to move sysext files there. Also explicitly omit dracut zfs module
    from initrd generation.
    
    Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
    jepio committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    333e722 View commit details
    Browse the repository at this point in the history
  2. changelog: Add entry for zfs sysext

    Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>
    jepio authored and pothos committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    3aa9cd2 View commit details
    Browse the repository at this point in the history