Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

github/workflows/centos: Also push zstd:chunked compressed images #129

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

travier
Copy link
Member

@travier travier commented Jul 10, 2024

github/workflows: Use Ubuntu 24.04 runners


github/workflows/centos: Also push zstd:chunked compressed images

See: #128
See: https://fedoraproject.org/wiki/Changes/zstd:chunked
See: https://docs.podman.io/en/latest/markdown/podman-push.1.html#compression-format-gzip-zstd-zstd-chunked
See: coreos/fedora-coreos-tracker#1660

@travier
Copy link
Member Author

travier commented Jul 10, 2024

CC @castrojo :)

@travier
Copy link
Member Author

travier commented Jul 10, 2024

Let's test this format with one image first and then copy it for all the others.

@travier
Copy link
Member Author

travier commented Jul 10, 2024

Processing triggers for libc-bin (2.31-13+deb11u10) ...
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault (core dumped)
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault (core dumped)
dpkg: error processing package libc-bin (--configure):
 installed libc-bin package post-installation script subprocess returned error exit status 139
Processing triggers for ca-certificates (20210119) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
Processing triggers for dbus (1.12.28-0+deb11u1) ...
Processing triggers for libgdk-pixbuf-2.0-0:arm64 (2.42.2+dfsg-1+deb11u2) ...
Errors were encountered while processing:
 libc-bin
E: Sub-process /usr/bin/dpkg returned an error code (1)
Error: building at STEP "RUN apt-get update &&     DEBIAN_FRONTEND=noninteractive apt-get -y install         libnss-myhostname         $(cat extra-packages | xargs) &&     rm -rd /var/lib/apt/lists/*": while running runtime: exit status 100

😞

@castrojo
Copy link
Member

Thanks for the tag! cc @p5

One thing I learned is that many tutorials recommend ubuntu-latest, which is currently pointing to 22.04, so when I turned this on someone immediately filed an issue since their downstream build broke due to the old podman version in 22.04.

The github public roadmap mentions that 24.04 builders should go out of beta this summer, so we figured it would just be best to roll it out the day after those go live or something after confirming that ubuntu-latest is pointing to 24.04.

Copy link

@p5 p5 Jul 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome!

Is the plan going to be to make both these images accessible under a single manifest (tag)?

This will allow a user to run podman pull <image>:stream9 and receive a zstd:chunked image if their container tooling supports it, or a gzipped image if it does not.

Additional required steps would be to (some details excluded):

  1. Push the gzipped image to a different tag
  2. Create a manifest (podman manifest create <name>)
  3. Add each images to the manifest (podman manifest add <name> <image>)
  4. Push the manifest to the registry under a stable tag (podman manifest push <name> <image>:stream9)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using a single tag would be #128 (comment)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't know this was possible until today. I'll take a look. PRs also welcomed :)

Copy link

@p5 p5 Jul 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this would be an awesome feature of redhat-actions/push-to-registry, although I seem to remember somebody saying this action isn't really something managed by RedHat themselves, but rather one or two people who happen to be at RedHat.

Docker requires you to supply the format as a build arg, so it's far more work.

Podman has it under push args, so it would be cool if the action could push these compression-specific images to a registry and build/push the manifest file to the tags we define as inputs. It would make it so much easier if everyone could just say compression-formats: gzip zstd:chunked

Copy link

@p5 p5 Jul 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have created an action that does just this for ublue. The implementation is quite possibly one of the worst things I have ever created, but it does the job.
https://github.com/ublue-os/push-to-registry/tree/scaffold-repository

You define compression-formats: gzip zstd:chunked, it pushes <image>:<tag>-gzip and <image>:<tag>-zstd-chunked to the registry. It then builds the manifest with the required annotations before pushing it to the registry

I'm not suggesting you use my action. Quite the opposite - it would be best if there was a "supported" way of doing this in the existing push-to-registry action. And my module can be an example of the complexity required to DIY this, hopefully justifying the need for improved support.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not opposed to using your action until we find the time to push this upstream in redhat-actions/push-to-registry. Maybe I can take look as I have made some patches to that project before redhat-actions/push-to-registry#90 that are still pending.

travier added a commit to travier/toolbx-images that referenced this pull request Aug 22, 2024
@travier travier changed the title github/workflows/centos: Also push zstd:chunked compressed images & Use Ubuntu 24.04 runners github/workflows/centos: Also push zstd:chunked compressed images Aug 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants