Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'systemd-tmpfiles --create' is not called when installing #955

Closed
antonc42 opened this issue Dec 13, 2021 · 9 comments · Fixed by #965
Closed

'systemd-tmpfiles --create' is not called when installing #955

antonc42 opened this issue Dec 13, 2021 · 9 comments · Fixed by #965
Labels
1. Bug Something isn't working 2. Build System Issue is related to the build system 2. Host Realm The issue is related to what happens on the host machine where Toolbox is executed
Milestone

Comments

@antonc42
Copy link

Describe the bug
On Ubuntu 20.04, toolbox version 0.0.99.3 builds and installs but fails to execute with:

Failed to execute process '/usr/local/bin/toolbox'. Reason:
The file '/usr/local/bin/toolbox' does not exist or could not be executed.

File exists:

ll /usr/local/bin/toolbox
-rwxr-xr-x 1 root root 8.3M Dec 13 10:15 /usr/local/bin/toolbox

The interpreter for the binary seems to be incorrect

file /usr/local/bin/toolbox
/usr/local/bin/toolbox: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /run/host/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2, BuildID[sha1]=51a9844e89fe0d1aea45dd776804f298516fad68, for GNU/Linux 3.2.0, not stripped

The interpreter file doesn't exist on my system:

ll /run/host/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
ls: cannot access '/run/host/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2': No such file or directory

Steps how to reproduce the behaviour

  1. Download https://github.com/containers/toolbox/releases/download/0.0.99.3/toolbox-0.0.99.3.tar.xz
  2. Extract
  3. Enter into extracted directory
  4. Activate virtualenv
  5. meson setup -Dprofile_dir=/etc/profile.d builddir meson-setup.log
  6. meson compile -C builddir meson-compile.log
  7. sudo meson install -C builddir meson-install.log
  8. toolbox --help

Expected behaviour
Toolbox builds, installs, and runs correctly.

Actual behaviour
Toolbox builds and installs correctly but does not run.

toolbox --help
Failed to execute process '/usr/local/bin/toolbox'. Reason:
The file '/usr/local/bin/toolbox' does not exist or could not be executed.

Output of toolbox --version (v0.0.90+)

toolbox --version
Failed to execute process '/usr/local/bin/toolbox'. Reason:
The file '/usr/local/bin/toolbox' does not exist or could not be executed.

Toolbox package info (rpm -q toolbox)
Installed from source.

Output of podman version

Version:      3.4.2
API Version:  3.4.2
Go Version:   go1.16.6
Built:        Wed Dec 31 18:00:00 1969
OS/Arch:      linux/amd64

Podman package info (rpm -q podman)

apt info podman
Package: podman
Version: 100:3.4.2-1
Priority: optional
Section: devel
Maintainer: Lokesh Mandvekar <lsm5@fedoraproject.org>
Installed-Size: 80.2 MB
Depends: libseccomp2 (>= 2.4.3-1), libdevmapper1.02.1, libgpgme11, catatonit, conmon (>= 100:2.0.25~2), containers-common (>= 100:1-21), dbus-user-session, iptables, podman-plugins (>= 100:1.2.0-1), podman-machine-cni (>= 100:0.0.0-1), crun (>= 100:0.19.1-1)
Recommends: slirp4netns (>= 100:1.1.8-3), containernetworking-plugins (>= 100:1.0.0-1), uidmap, fuse-overlayfs
Conflicts: podman-rootless
Homepage: https://github.com/containers/podman.git
Download-Size: 17.7 MB
APT-Manual-Installed: yes
APT-Sources: https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_20.04  Packages
Description: Manage pods, containers and container images.

Info about your OS
Ubuntu 20.04

Additional context
Toolbox 0.0.99.2 builds, installs, and runs correctly with the same build process.

Interpreter for 0.0.99.2 appears to be correct.

file /usr/local/bin/toolbox
/usr/local/bin/toolbox: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=28adb3f0a94d52af1e29d3ad2642bca06a5cbd3e, for GNU/Linux 3.2.0, not stripped

Meson installed in virtualenv via pip

meson -v
0.60.2

Python3 from package manager

apt info python3
Package: python3
Version: 3.8.2-0ubuntu2
Priority: important
Section: python
Source: python3-defaults
Origin: Ubuntu
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Matthias Klose <doko@debian.org>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 194 kB
Provides: python3-profiler
Pre-Depends: python3-minimal (= 3.8.2-0ubuntu2)
Depends: python3.8 (>= 3.8.2-1~), libpython3-stdlib (= 3.8.2-0ubuntu2)
Suggests: python3-doc (>= 3.8.2-0ubuntu2), python3-tk (>= 3.8.2-1~), python3-venv (>= 3.8.2-0ubuntu2)
Replaces: python3-minimal (<< 3.1.2-2)
Homepage: https://www.python.org/
Task: minimal, ubuntu-core
Download-Size: 47.6 kB
APT-Manual-Installed: yes
APT-Sources: http://mirrors.gigenet.com/ubuntuarchive focal/main amd64 Packages
Description: interactive high-level object-oriented language (default python3 version)
 Python, the high-level, interactive object oriented language,
 includes an extensive class library with lots of goodies for
 network programming, system administration, sounds and graphics.
 .
 This package is a dependency package, which depends on Debian's default
 Python 3 version (currently v3.8).
python3 -V
Python 3.8.10

Ninja from package manager

apt info ninja-build
Package: ninja-build
Version: 1.10.0-1build1
Priority: optional
Section: universe/devel
Origin: Ubuntu
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Felix Geyer <fgeyer@debian.org>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 338 kB
Depends: libc6 (>= 2.15), libstdc++6 (>= 5.2)
Suggests: python3
Homepage: https://ninja-build.org/
Download-Size: 107 kB
APT-Manual-Installed: yes
APT-Sources: http://mirrors.gigenet.com/ubuntuarchive focal/universe amd64 Packages
Description: small build system closest in spirit to Make
 Ninja is yet another build system. It takes as input the interdependencies of
 files (typically source code and output executables) and orchestrates
 building them, quickly.
 .
 Ninja joins a sea of other build systems. Its distinguishing goal is to be
 fast. It is born from the Chromium browser project, which has over 30,000
 source files and whose other build systems can take ten seconds to start
 building after changing one file. Ninja is under a second.
ninja --version
1.10.0

Go from package manager
(using newer version with update-alternatives: sudo update-alternatives --install /usr/bin/go go /usr/lib/go-1.16/bin/go 100 --slave /usr/bin/gofmt gofmt /usr/lib/go-1.16/bin/gofmt)

Package: golang-1.16
Version: 1.16.6-5
Priority: optional
Section: golang
Maintainer: Go Compiler Team <team+go-compiler@tracker.debian.org>
Installed-Size: 59.4 kB
Depends: golang-1.16-doc (>= 1.16.6-5), golang-1.16-go (>= 1.16.6-5), golang-1.16-src (>= 1.16.6-5)
Homepage: https://golang.org
Download-Size: 27.8 kB
APT-Manual-Installed: yes
APT-Sources: https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_20.04  Packages
Description: Go programming language compiler - metapackage
 The Go programming language is an open source project to make
 programmers more productive. Go is expressive, concise, clean, and
 efficient. Its concurrency mechanisms make it easy to write programs
 that get the most out of multicore and networked machines, while its
 novel type system enables flexible and modular program construction.
 Go compiles quickly to machine code yet has the convenience of
 garbage collection and the power of run-time reflection. It's a
 fast, statically typed, compiled language that feels like a
 dynamically typed, interpreted language.
 .
 This package is a metapackage that, when installed, guarantees
 that (most of) a full Go development environment is installed.
 .
 To use this version, instead of the default one provided by golang-go
 package, add /usr/lib/go-1.16/bin/ to PATH, or invoke /usr/lib/go-1.16/bin/go
 directly.

go version
go version go1.16.6 linux/amd64
@antonc42 antonc42 added the 1. Bug Something isn't working label Dec 13, 2021
@debarshiray
Copy link
Member

The interpreter for the binary seems to be incorrect

[...]

The interpreter file doesn't exist on my system:

I think the binary was built as expected and the problem is that the /run/host symbolic link didn't get created on your host.

Do you have systemd-tmpfiles(8) ?

For further details see #897 which fixed #821

@HarryMichal
Copy link
Member

meson install only puts files in the right places but any further actions (e.g. triggering the creation of tmpfiles) have to be done manually. Adding a message about this in the build system could prove helpful.

@HarryMichal HarryMichal added 2. Build System Issue is related to the build system 2. Host Realm The issue is related to what happens on the host machine where Toolbox is executed labels Dec 14, 2021
@antonc42
Copy link
Author

Yes, systemd-tmpfiles is installed. It appears to be part of the default systemd package on Ubuntu 20.04, so it should be available on all installs.

Running the command sudo systemd-tmpfiles --create after sudo meson install -C builddir fixes the problem. toolbox is now able to run.

So, if I understand correctly, meson install is installing data/tmpfiles.d/toolbox.conf to /usr/lib/tmpfiles.d/toolbox.conf, but it cannot run the sudo systemd-tmpfiles --create command? I'm not familiar with meson. Does it not have the capability of doing actions like that? If not, then a documentation update would be greatly appreciated. I was following the build directions on https://containertoolbx.org/install/.

Thanks for the help!

@HarryMichal
Copy link
Member

It actually should be possible to run the command from the build system, yes. Is that something we want to do? @debarshiray

@debarshiray
Copy link
Member

meson install only puts files in the right places but any further actions
(e.g. triggering the creation of tmpfiles) have to be done manually.
Adding a message about this in the build system could prove helpful.

We only need to invoke systemd-tmpfiles separately when building and installing from source, not when using a downstream package, because:

  • When meson install is called as part of building the package, that's not when the temporary files need to be created. They need to be created when the package is later downloaded and installed by the user.
  • Downstream tools, like RPM, can sometimes handle it automatically.

So, yes, we should invoke systemd-tmpfiles --create as part of our build, but only when the DESTDIR environment variable is not set (because it denotes that we are not building a downstream package). We might have to use meson.add_install_script for this.

@HarryMichal
Copy link
Member

We might have to use meson.add_install_script for this.

That will solve this exact problem but won't be compatible with #840. There the final binary is used to create the completions scripts. But maybe some other way could be found for that particular problem. I'll put together a PR for this.

@debarshiray
Copy link
Member

That will solve this exact problem but won't be compatible with #840.

Let's not worry about #840 There are other ways to solve that problem. :)

@HarryMichal HarryMichal added this to the Release 0.1.0 milestone Dec 17, 2021
debarshiray added a commit to debarshiray/toolbox that referenced this issue Dec 17, 2021
It's only necessary to call 'systemd-tmpfiles --create' when building
and installing from source, and not when using a downstream package,
because:

  * When 'meson install' is called as part of building the package,
    that's not when the temporary files need to be created. They need
    to be created when the package is later downloaded and installed
    by the user.

  * Downstream tools can sometimes handle it automatically. eg., on
    Fedora, the systemd RPM installs a trigger that tells RPM to call
    'systemd-tmpfiles --create' automatically when a tmpfiles.d snippet
    is installed.

containers#955
@debarshiray
Copy link
Member

Does #965 help?

@antonc42
Copy link
Author

Yes, I tried it and it works. It creates the symlink on the install step. Thanks!

@HarryMichal HarryMichal linked a pull request Jan 8, 2022 that will close this issue
mulles added a commit to mulles/containertoolbx.org that referenced this issue Jan 10, 2022
Add solution  mentioned in  containers/toolbox#955. 

Addeed version of meson build as most system package manager (apt) come with older meson version.
debarshiray added a commit to debarshiray/toolbox that referenced this issue Jan 10, 2022
It's only necessary to call 'systemd-tmpfiles --create' when building
and installing from source, and not when using a pre-built binary
downstream package, because:

  * When 'meson install' is called as part of building the package,
    that's not when the temporary files need to be created. They need
    to be created when the binary package is later downloaded and
    installed by the user.

  * Downstream tools can sometimes handle it automatically. eg., on
    Fedora, the systemd RPM installs a trigger that tells RPM to call
    'systemd-tmpfiles --create' automatically when a tmpfiles.d snippet
    is installed.

Downstream distributors set the DESTDIR environment variable when
building their packages. Therefore, it's used to detect when a
downstream package is being built.

containers#955
debarshiray added a commit to debarshiray/toolbox that referenced this issue Jan 10, 2022
It's only necessary to call 'systemd-tmpfiles --create' when building
and installing from source on the host operating system.

It's not needed when using a pre-built binary downstream package,
because:

  * When 'meson install' is called as part of building the package,
    that's not when the temporary files need to be created. They need
    to be created when the binary package is later downloaded and
    installed by the user.

  * Downstream tools can sometimes handle it automatically. eg., on
    Fedora, the systemd RPM installs a trigger that tells RPM to call
    'systemd-tmpfiles --create' automatically when a tmpfiles.d snippet
    is installed.

It's also not needed when installing inside a toolbox container because
the files that 'systemd-tmpfiles --create' is supposed to create are
meant to be on the host.

Downstream distributors set the DESTDIR environment variable when
building their packages. Therefore, it's used to detect when a
downstream package is being built.

Unfortunately, environment variables are messy and, generally, Meson
doesn't support accessing them inside its scripts [1]. Therefore, this
adds a spurious build-time dependency on systemd for downstream
distributors. However, that's probably not a big problem because all
supported downstream operating systems are already expected to use
systemd for the tmpfiles.d(5) snippets to work.

[1] mesonbuild/meson#9

containers#955
debarshiray added a commit to debarshiray/toolbox that referenced this issue Jan 10, 2022
It's only necessary to call 'systemd-tmpfiles --create' when building
and installing from source on the host operating system.

It's not needed when using a pre-built binary downstream package,
because:

  * When 'meson install' is called as part of building the package,
    that's not when the temporary files need to be created. They need
    to be created when the binary package is later downloaded and
    installed by the user.

  * Downstream tools can sometimes handle it automatically. eg., on
    Fedora, the systemd RPM installs a trigger that tells RPM to call
    'systemd-tmpfiles --create' automatically when a tmpfiles.d snippet
    is installed.

It's also not needed when installing inside a toolbox container because
the files that 'systemd-tmpfiles --create' is supposed to create are
meant to be on the host.

Downstream distributors set the DESTDIR environment variable when
building their packages. Therefore, it's used to detect when a
downstream package is being built.

Unfortunately, environment variables are messy and, generally, Meson
doesn't support accessing them inside its scripts [1]. Therefore, this
adds a spurious build-time dependency on systemd for downstream
distributors. However, that's probably not a big problem because all
supported downstream operating systems are already expected to use
systemd for the tmpfiles.d(5) snippets to work.

[1] mesonbuild/meson#9

containers#955
@debarshiray debarshiray changed the title v0.0.99.3 builds but fails to execute on Ubuntu 20.04 'systemd-tmpfiles --create' is not called when installing Jan 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1. Bug Something isn't working 2. Build System Issue is related to the build system 2. Host Realm The issue is related to what happens on the host machine where Toolbox is executed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants