Introduce Ubuntu LTS and latest non-LTS images #483

Jmennius · 2020-06-27T15:04:57Z

This adds Ubuntu 16.04, 18.04, 20.04, 21.10 and 22.04 images that play well with toolbox:
password-less sudo, able to resolve its own hostname (so that sudo does not complain), SELinux is masked off, etc.
Since 18.04 Ubuntu images on the Docker Hub are 'minimized', so we have to undo that.
Pushed images to https://hub.docker.com/r/jmennius/ubuntu-toolbox for the time being.

Users may want to generate their locale after toolbox creation with sudo sed -i "/${LANG}/s/^# //g" /etc/locale.gen && sudo locale-gen (assuming that you have LANG set) to avoid perl and others complain about it.

I've been using an 18.04 image for ~~several months~~ 1.5yrs now with great success.

Some things I'd like to resolve or discuss at least (need help/guidance/decisionmaking/PRs):

Use actual quay toolbx robot credentials.
READMEs are not symlinked. Didn't get this to work as symlink target is out of build context. images: Use a regular file, not a symbolic link, for README.md #723
Moreover, I'd also like to share extra-packages between images. Maybe we can store all Containerfiles in the ubuntu and a single readme and extra-packages alongside?
align READMEs with the rest of the repo before merging
Needs Drop unnecessary and wrong group detection on host #401 to function
I can mention myself as the image maintainer if that's the way to go.
Any CI/CD integration? Yes, see Consider building and shipping toolbx images using Quay.io #1019
~~I've only tested this with a shell-based toolbox.~~ Works perfectly with golang implementation.
Did not use NAME and VERSION variables (discussed in NAME and VERSION leak into toolbox container env #188)

I'd be grateful for any testing and feedback on their use-case and environment.

Alternative-to: #298

softwarefactory-project-zuul · 2020-06-27T15:10:24Z

Build failed.

shellcheck : SUCCESS in 1m 56s
test-with-podman-stable-f30 : FAILURE in 2m 45s
test-with-podman-stable-f31 : FAILURE in 3m 03s
test-with-podman-rawhide : FAILURE in 3m 53s

Jmennius · 2020-07-07T15:59:16Z

I'd really appreciate some feedback ;) @debarshiray ?

HarryMichal · 2020-07-08T11:12:21Z

I'd really appreciate some feedback ;) @debarshiray ?

I'd love to give this a spin but I'm currently working on something else.. :(

Jmennius · 2020-07-08T13:21:40Z

@HarryMichal no problem :)

What do you think about sharing extra-packages and READMEs? Any nice way to do it?
Maybe we can drop READMEs from images themselves?

HarryMichal · 2020-07-08T15:23:08Z

In my opinion, sharing of those files mainly depends on the way the images will be built. fedora-toolbox images are built in Fedora's build system and their definitions are maintained there (meaning the definitions in this repository could be called a mirror - or is it the other way @debarshiray?). If the container definitions are tracked in a got repository where every version is in a separate branch than it's not really possible to share. If they are in a single branch but separated in their own directories, than a single file shared by all definitions could be placed in the root directory. But that approach may be dangerous a bit if the package is renamed in some of those version (though in some cases very unlikely).

I myself never worked with Docker Hub and other tools so I can't really help much with this.

Dropping READMEs seems okay-ish to me. But on the other hand having some kind of file that holds some info about the image/tool is nice. Maybe some specialized file that will be defined once, will be shorter than the project's README that will not require to be updated with every tweak to the project's README?

mfocko · 2021-02-07T15:24:10Z

used the Dockerfile for 20.10, seems to be running OK 👍

mtalexan · 2021-02-17T00:28:35Z

Probably out-of-scope for the intended changes here, but support for ubuntu as a distribution under the (-d|--distribution) and the related releases under the (-r|--release) options would require updating the src/pkg/utils/utils.go supportedDistros map to add a Distro structure (and the related functions to fill it) for an ubuntu key. Otherwise the results of these builds are limited to use of the --image option.

Jmennius · 2021-03-21T18:01:58Z

cc @debarshiray

softwarefactory-project-zuul · 2021-03-21T18:09:49Z

Build succeeded.

shellcheck : SUCCESS in 1m 56s
system-test-fedora-32 : SUCCESS in 5m 10s
system-test-fedora-33 : SUCCESS in 5m 37s
system-test-fedora-34 : SUCCESS in 5m 34s
system-test-fedora-rawhide : SUCCESS in 10m 45s

Jmennius · 2021-03-21T19:02:15Z

I now get WARNING: libcap needs an update (cap=40 should have a name). when entering a 20.04 toolbox.
This is likely because I am using Linux 5.10 (Fedora 33) but Ubuntu 20.04 userspace ships with libcap2 2.32 which is not aware of CAP_CHECKPOINT_RESTORE (40).
Everyhting works fine though.

runfalk · 2021-05-19T20:54:53Z

I too would really like to see Ubuntu support being included one way or another. My use case is doing firmware development on Fedora when the rest of the team uses Ubuntu. They have set up install scripts and debs for installing the all the required cross compilation targets.

I tried the image in this project using:

$ toolbox create --image registry.hub.docker.com/jmennius/ubuntu-toolbox:20.04 firmaredev

The /README.md talks about using systemctl for managing services, but it doesn't seem like systemd is actually running in the container:

$ systemctl
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down

This seems to be documented in another issue as well: #176.

Other than some dbus errors (probably because systemd isn't running) and the libpcap error I don't see any other problems with these containers. I tried both 20.04 and 18.04.

I shouldn't hijack this MR, but is there a way of running services like SSH in the container?

Jmennius · 2021-07-31T17:15:54Z

Guys, any updates or feedback?

Jmennius · 2021-07-31T17:17:24Z

Probably out-of-scope for the intended changes here, but support for ubuntu as a distribution under the (-d|--distribution) and the related releases under the (-r|--release) options would require updating the src/pkg/utils/utils.go supportedDistros map to add a Distro structure (and the related functions to fill it) for an ubuntu key. Otherwise the results of these builds are limited to use of the --image option.

I've implemented this when I ported it to Go (back in April).

Jmennius · 2021-07-31T17:32:05Z

The /README.md talks about using systemctl for managing services, but it doesn't seem like systemd is actually running in the container:
$ systemctl
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down
This seems to be documented in another issue as well: #176.

Other than some dbus errors (probably because systemd isn't running) and the libpcap error I don't see any other problems with these containers. I tried both 20.04 and 18.04.

I shouldn't hijack this MR, but is there a way of running services like SSH in the container?

Well, I wouldn't expect any of this to really work, the README should probably be corrected.
I don't think that running sshd is feasible, but you can probably google that with regards to containers in general.

dabrain34 · 2021-10-28T08:21:26Z

I gave a try to 16.04 and 18.04 and both are failing for the same reason as:

level=debug msg="Binding /tmp to /run/host/tmp"
mount: wrong fs type, bad option, bad superblock on /run/host/tmp,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.
Error: failed to bind /tmp to /run/host/tmp

The 20.04 image from #878 is working fine.

I'm running podman and toolbox 0.0.99.3 on a Debian SID

dabrain34 · 2021-10-28T15:10:00Z

mounting a tmpfs on /tmp made it work !

mount -o mode=1777,nosuid,nodev -t tmpfs tmpfs /tmp

Jmennius · 2021-10-31T14:46:41Z

mounting a tmpfs on /tmp made it work !

mount -o mode=1777,nosuid,nodev -t tmpfs tmpfs /tmp

Interesting.. so you didn't have a tmpfs on /tmp on your host (Debian)?
If so - that probably deserves a separate issue to be fixed properly.

edit: I see you've opened #906

dabrain34 · 2021-11-03T09:27:37Z

For now i havent investigated into toolbox code to see how I could propose a patch but the other problem I faced was that on the tip, the binary was not exectutable.

Jmennius · 2021-11-14T23:28:02Z

Few updates:

decided to also provide the image for the latest non-LTS release - 21.10 image
I've noticed a delay when using sudo in new images (and that has probably been there forever... no idea how I lived with it for over a year!?) - fixed by moving myhostname nsswitch entry to the proper place manually instead of trusting package postinst with it.
few updates to align with fedora image changes
rebuilt and pushed all images

softwarefactory-project-zuul · 2021-11-14T23:41:21Z

Build succeeded.

unit-test : SUCCESS in 5m 05s
system-test-fedora-33 : SUCCESS in 8m 22s
system-test-fedora-34 : SUCCESS in 10m 13s
system-test-fedora-rawhide : SUCCESS in 20m 06s

travier · 2023-05-22T08:30:12Z

OK, I have access to the toolbx org on Quay.io now.

.github/workflows/ubuntu-images.yaml

travier · 2023-05-22T08:34:34Z

I've set up the robot and permissions on Quay and added the token in the repo here.

travier · 2023-05-22T08:37:01Z

Note that to clearly distinguish PR & commit workflows, I've used two distinct GitHub Action definition in the toolbx-images repo. This PR is missing the one for PRs (https://github.com/toolbx-images/images/blob/main/.github/workflows/ubuntu-pr.yaml) and thus it is not triggered here.

debarshiray · 2023-05-22T16:43:48Z

I've set up the robot and permissions on Quay and added the token in
the repo here.

Thanks for doing that, @travier !

What are the next steps? Are we waiting for @Jmennius to update the GitHub Action in this pull request?

Note that to clearly distinguish PR & commit workflows, I've used
two distinct GitHub Action definition in the toolbx-images repo. This
PR is missing the one for PRs
(https://github.com/toolbx-images/images/blob/main/.github/workflows/ubuntu-pr.yaml)
and thus it is not triggered here.

I will let @Jmennius respond to this one.

debarshiray · 2023-06-06T16:38:43Z

Ping, @travier @Jmennius : what's the next step here?

The most pressing issue right now for Ubuntu support is to get the ubuntu-toolbox images for 23.04 published.

travier · 2023-06-08T08:34:19Z

The missing bits are in my review #483 (review). Then we should be able to merge this one.

Build and push when there are changes in the `images/ubuntu` directory or in the GitHub workflow itself, as well as every other week (7th and 21st days of a month to be precise). Until now, the Ubuntu images (versions 16.04, 18.04, 20.04, 22.04 and 22.10) were published at quay.io/toolbx-images/ubuntu-toolbox:22.04, etc.. This changes the location to quay.io/toolbx/ubuntu-toolbox:22.04 and builds an image for Ubuntu 23.04 that was added recently [1]. Note that the changes to the toolbox(1) code and the system tests cannot be tested until the first round of images are available from the new location. [1] Commit 3cfb6bf containers#1292 containers#483 Signed-off-by: Ievgen Popovych <jmennius@gmail.com>

Until now, the Ubuntu images (versions 16.04, 18.04, 20.04, 22.04 and 22.10) were published at quay.io/toolbx-images/ubuntu-toolbox:22.04, etc.. This changes the location to quay.io/toolbx/ubuntu-toolbox:22.04 and builds an image for Ubuntu 23.04 that was added recently [1]. Build and push when there are changes in the `images/ubuntu` directory or in the GitHub workflow itself, as well as every other week (7th and 21st days of a month to be precise). The toolbox(1) code and the system tests will be switched to the new location after the first round of images are available. [1] Commit 3cfb6bf containers#1292 containers#483 Signed-off-by: Ievgen Popovych <jmennius@gmail.com>

softwarefactory-project-zuul · 2023-06-08T13:17:39Z

Build succeeded.
https://softwarefactory-project.io/zuul/t/local/buildset/074a7db07d22487f933b780c7317c0cb

✔️ unit-test SUCCESS in 8m 41s
✔️ unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 24s
✔️ unit-test-restricted SUCCESS in 8m 30s
✔️ system-test-fedora-rawhide SUCCESS in 21m 35s
✔️ system-test-fedora-38 SUCCESS in 20m 51s
✔️ system-test-fedora-37 SUCCESS in 21m 25s
✔️ system-test-fedora-36 SUCCESS in 20m 55s

debarshiray · 2023-06-08T13:24:46Z

The missing bits are in my review #483 (review). Then we should be able to merge this one.

Okay! I took the liberty to make those changes and merged. I hope I got it right.

debarshiray · 2023-06-08T13:37:03Z

I filed a pull request to switch the toolbox(1) code and the system tests to use the new location: #1306

Now that Toolbx offers built-in support for Ubuntu containers [1], it makes sense to test that it works well on Ubuntu hosts. Ubuntu 22.04 is the latest long term support (or LTS) release [2] from Ubuntu, and GitHub provides runners for GitHub workflows [3]. Ubuntu 22.04 only has Bats 1.2.1 [4], while Toolbx requires 1.7.0 [5]; and Shadow 4.8 [6], while libsubid.so was introduced in Shadow 4.9 [7]. Hence, newer versions of these dependencies need to be built to run the tests. Note that a separate sub-directory inside $GITHUB_WORKSPACE [8] is used for Toolbx itself to prevent codespell from getting triggered by problems in the dependencies themselves [9]. [1] Commit a84a358 containers#483 containers#1284 [2] https://wiki.ubuntu.com/Releases [3] https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners [4] https://packages.ubuntu.com/jammy/bats [5] Commit e22a82f containers#1273 [6] https://packages.ubuntu.com/source/jammy/shadow https://packages.ubuntu.com/source/jammy-updates/shadow [7] Shadow commit 0a7888b1fad613a0 shadow-maint/shadow@0a7888b1fad613a0 shadow-maint/shadow#154 [8] https://docs.github.com/en/actions/learn-github-actions/variables [9] bats-core/bats-core#743 containers#1319

Now that Toolbx offers built-in support for Ubuntu containers [1], adding an Ubuntu host to the upstream CI will help ensure that Toolbx continues to work well on Ubuntu. Ubuntu 22.04 is the latest long term support (or LTS) release [2] from Ubuntu, and GitHub provides runners for GitHub workflows [3]. Ubuntu 22.04 only has Bats 1.2.1 [4], while Toolbx requires 1.7.0 [5]; and Shadow 4.8 [6], while libsubid.so was introduced in Shadow 4.9 [7]. Hence, newer versions of these dependencies need to be built to run the tests. Note that a separate sub-directory inside $GITHUB_WORKSPACE [8] is used for Toolbx itself to prevent codespell from getting triggered by problems in the dependencies themselves [9]. [1] Commit a84a358 containers#483 containers#1284 [2] https://wiki.ubuntu.com/Releases [3] https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners [4] https://packages.ubuntu.com/jammy/bats [5] Commit e22a82f containers#1273 [6] https://packages.ubuntu.com/source/jammy/shadow https://packages.ubuntu.com/source/jammy-updates/shadow [7] Shadow commit 0a7888b1fad613a0 shadow-maint/shadow@0a7888b1fad613a0 shadow-maint/shadow#154 [8] https://docs.github.com/en/actions/learn-github-actions/variables [9] bats-core/bats-core#743 containers#1319

Now that Toolbx offers built-in support for Ubuntu containers [1], adding an Ubuntu host to the upstream CI will help ensure that Toolbx continues to work well on Ubuntu. Ubuntu 22.04 is the latest long term support (or LTS) release [2] from Ubuntu, and is the latest Ubuntu version that GitHub provides runners for [3]. Ubuntu 22.04 only has Bats 1.2.1 [4], while Toolbx requires 1.7.0 [5]; and Shadow 4.8 [6], while Toolbx requires 4.9 because it needs libsubid.so [7,8]. Hence, newer versions of these dependencies need to be built to run the tests. The build flags for Shadow were taken from the Debian package [9]. A separate sub-directory inside $GITHUB_WORKSPACE [10] is used for Toolbx itself to prevent codespell from getting triggered by spelling mistakes in these dependencies themselves [11]. Unfortunately, the SHELL environment variable goes mysteriously missing from the runtime environment of the GitHub Actions workflow [12]. This breaks the 'create' and 'enter' commands, and therefore tests involving them can't be run until this is resolved. Meanwhile, running the CI on Ubuntu with a subset of the tests, is still better than not running the CI on Ubuntu at all. [1] Commit a84a358 containers#483 containers#1284 [2] https://wiki.ubuntu.com/Releases [3] https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners [4] https://packages.ubuntu.com/jammy/bats [5] Commit e22a82f containers#1273 [6] https://packages.ubuntu.com/source/jammy/shadow https://packages.ubuntu.com/source/jammy-updates/shadow [7] Shadow commit 0a7888b1fad613a0 shadow-maint/shadow@0a7888b1fad613a0 shadow-maint/shadow#154 [8] Commit ca8007c containers#1074 [9] https://salsa.debian.org/debian/shadow/ [10] https://docs.github.com/en/actions/learn-github-actions/variables [11] bats-core/bats-core#743 [12] https://github.com/orgs/community/discussions/59413 containers#1319

See: containers/toolbox@a84a358b3b90dac9 containers/toolbox#483 containers/toolbox#1284

HarryMichal added the 2. Images Images for creating toolbox containers label Jul 8, 2020

Jmennius mentioned this pull request Sep 10, 2020

Define image test suite - functionality that's expected to work in toolbox images #385

Open

Jmennius force-pushed the introduce-ubuntu-lts-images branch from cd5eacc to 475a1dd Compare March 21, 2021 17:56

Base automatically changed from master to main March 25, 2021 22:25

castrojo mentioned this pull request Aug 5, 2021

Doesn't work with standard ubuntu image castrojo/build-ubuntu-toolbox#1

Open

erazemk mentioned this pull request Aug 11, 2021

Toolbox - any linux distro inside. #789

Closed

Jmennius mentioned this pull request Oct 31, 2021

images: add Ubuntu 20.04 image #878

Closed

Jmennius mentioned this pull request Oct 31, 2021

First attempt at an Ubuntu toolbox container image. #298

Closed

Jmennius changed the title ~~Introduce Ubuntu LTS images~~ Introduce Ubuntu LTS and latest non-LTS images Nov 14, 2021

debarshiray mentioned this pull request Dec 1, 2021

Some success running toolbox on Ubuntu 18.04 (first experiment) #197

Closed

This was referenced May 18, 2023

images: Add Arch Linux image #861

Merged

Withdraw distribution: Ubuntu toolbx-images/images#81

Closed

travier reviewed May 22, 2023

View reviewed changes

.github/workflows/ubuntu-images.yaml Outdated Show resolved Hide resolved

travier reviewed May 22, 2023

View reviewed changes

.github/workflows/ubuntu-images.yaml Outdated Show resolved Hide resolved

debarshiray mentioned this pull request Jun 7, 2023

Withdraw the Arch Linux image (renamed) toolbx-images/images#82

Merged

debarshiray force-pushed the introduce-ubuntu-lts-images branch from b58a774 to 8e912ee Compare June 8, 2023 12:49

debarshiray force-pushed the introduce-ubuntu-lts-images branch from 8e912ee to 8dcd5d7 Compare June 8, 2023 12:54

debarshiray approved these changes Jun 8, 2023

View reviewed changes

debarshiray merged commit 8dcd5d7 into containers:main Jun 8, 2023
1 check passed

This was referenced Jun 8, 2023

Withdraw the Ubuntu images toolbx-images/images#84

Merged

.github/workflows: Publish the Arch Linux image at quay.io/toolbx/... #1308

Merged

debarshiray mentioned this pull request Jun 27, 2023

.github/workflows: Run the CI on Ubuntu 22.04 #1319

Merged

debarshiray added a commit to containers/containertoolbx.org that referenced this pull request Sep 27, 2023

Use the supported quay.io/toolbx/ubuntu-toolbox:22.04 image in examples

e6d6ea1

See: containers/toolbox@a84a358b3b90dac9 containers/toolbox#483 containers/toolbox#1284

Jmennius deleted the introduce-ubuntu-lts-images branch December 10, 2023 19:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce Ubuntu LTS and latest non-LTS images #483

Introduce Ubuntu LTS and latest non-LTS images #483

Jmennius commented Jun 27, 2020 •

edited

Loading

softwarefactory-project-zuul bot commented Jun 27, 2020

Jmennius commented Jul 7, 2020

HarryMichal commented Jul 8, 2020

Jmennius commented Jul 8, 2020

HarryMichal commented Jul 8, 2020

mfocko commented Feb 7, 2021

mtalexan commented Feb 17, 2021

Jmennius commented Mar 21, 2021

softwarefactory-project-zuul bot commented Mar 21, 2021

Jmennius commented Mar 21, 2021

runfalk commented May 19, 2021

Jmennius commented Jul 31, 2021

Jmennius commented Jul 31, 2021

Jmennius commented Jul 31, 2021

dabrain34 commented Oct 28, 2021 •

edited

Loading

dabrain34 commented Oct 28, 2021 •

edited

Loading

Jmennius commented Oct 31, 2021 •

edited

Loading

dabrain34 commented Nov 3, 2021

Jmennius commented Nov 14, 2021

softwarefactory-project-zuul bot commented Nov 14, 2021

travier commented May 22, 2023

travier commented May 22, 2023

travier commented May 22, 2023

debarshiray commented May 22, 2023

debarshiray commented Jun 6, 2023

travier commented Jun 8, 2023 •

edited

Loading

softwarefactory-project-zuul bot commented Jun 8, 2023

debarshiray commented Jun 8, 2023

debarshiray commented Jun 8, 2023

Introduce Ubuntu LTS and latest non-LTS images #483

Introduce Ubuntu LTS and latest non-LTS images #483

Conversation

Jmennius commented Jun 27, 2020 • edited Loading

softwarefactory-project-zuul bot commented Jun 27, 2020

Jmennius commented Jul 7, 2020

HarryMichal commented Jul 8, 2020

Jmennius commented Jul 8, 2020

HarryMichal commented Jul 8, 2020

mfocko commented Feb 7, 2021

mtalexan commented Feb 17, 2021

Jmennius commented Mar 21, 2021

softwarefactory-project-zuul bot commented Mar 21, 2021

Jmennius commented Mar 21, 2021

runfalk commented May 19, 2021

Jmennius commented Jul 31, 2021

Jmennius commented Jul 31, 2021

Jmennius commented Jul 31, 2021

dabrain34 commented Oct 28, 2021 • edited Loading

dabrain34 commented Oct 28, 2021 • edited Loading

Jmennius commented Oct 31, 2021 • edited Loading

dabrain34 commented Nov 3, 2021

Jmennius commented Nov 14, 2021

softwarefactory-project-zuul bot commented Nov 14, 2021

travier commented May 22, 2023

travier commented May 22, 2023

travier commented May 22, 2023

debarshiray commented May 22, 2023

debarshiray commented Jun 6, 2023

travier commented Jun 8, 2023 • edited Loading

softwarefactory-project-zuul bot commented Jun 8, 2023

debarshiray commented Jun 8, 2023

debarshiray commented Jun 8, 2023

Jmennius commented Jun 27, 2020 •

edited

Loading

dabrain34 commented Oct 28, 2021 •

edited

Loading

dabrain34 commented Oct 28, 2021 •

edited

Loading

Jmennius commented Oct 31, 2021 •

edited

Loading

travier commented Jun 8, 2023 •

edited

Loading