Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support building OSI images outside of container #111

Closed
rezib opened this issue May 23, 2023 · 0 comments · Fixed by #115
Closed

Support building OSI images outside of container #111

rezib opened this issue May 23, 2023 · 0 comments · Fixed by #115
Assignees
Labels
feature New feature or request
Milestone

Comments

@rezib
Copy link
Contributor

rezib commented May 23, 2023

For building some OSI output formats (eg. gpt_ext4 or gpt_btrfs), mkosi needs to allocate loop device with losetup. Unfortunately, this cannot work in containers due to missing virtualization support of loop devices in Linux. This fails with the following error:

$ fatbuildrctl --uri dbus://system/hpckit build -a node-debian11 -s firehpc/envs --watch
Submitted node-debian11 build task 831c1f39-eb1e-4aae-8ab1-61962ccb5bac
 ⚬ INFO: Running build 831c1f39-eb1e-4aae-8ab1-61962ccb5bac
 ⚬ INFO: Extracting tarball /run/fatbuildr/fatbuildr-artifact-b9aedccu.tar.xz in destination /var/lib/fatbuildr/workspaces/hpckit/831c1f39-eb1e-4aae-8ab1-61962ccb5bac
 ⚬ INFO: Building the OS image based node-debian11
‣ Changing ownership of output file /var/lib/fatbuildr/workspaces/hpckit/831c1f39-eb1e-4aae-8ab1-61962ccb5bac/debian~bullseye to user fatbuildr…
‣ Changed ownership of /var/lib/fatbuildr/workspaces/hpckit/831c1f39-eb1e-4aae-8ab1-61962ccb5bac/debian~bullseye
‣ Removing output files…
‣ Detaching namespace
‣ Setting up temporary workspace.
‣ Temporary workspace set up in /var/tmp/mkosi-hf5j_7l3
‣ Running second (final) stage…
‣  Creating image with partition table…
Disk /var/lib/fatbuildr/workspaces/hpckit/831c1f39-eb1e-4aae-8ab1-61962ccb5bac/debian~bullseye/.mkosi-9jfpbe27: 3 GiB, 3221266432 bytes, 6291536 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

>>> Script header accepted.
>>> Script header accepted.
>>> Script header accepted.
>>> Created a new GPT disklabel (GUID: CC8A1A7E-B84A-A640-A914-8F4F2E3ABBA5).
/var/lib/fatbuildr/workspaces/hpckit/831c1f39-eb1e-4aae-8ab1-61962ccb5bac/debian~bullseye/.mkosi-9jfpbe27p1: Created a new partition 1 of type 'Linux root (x86-64)' and of size 3 GiB.
/var/lib/fatbuildr/workspaces/hpckit/831c1f39-eb1e-4aae-8ab1-61962ccb5bac/debian~bullseye/.mkosi-9jfpbe27p2: Done.

New situation:
Disklabel type: gpt
Disk identifier: CC8A1A7E-B84A-A640-A914-8F4F2E3ABBA5

Device                                                                                                      Start     End Sectors Size Type
/var/lib/fatbuildr/workspaces/hpckit/831c1f39-eb1e-4aae-8ab1-61962ccb5bac/debian~bullseye/.mkosi-9jfpbe27p1    40 6291495 6291456   3G Linu

The partition table has been altered.
‣  Created image with partition table as /var/lib/fatbuildr/workspaces/hpckit/831c1f39-eb1e-4aae-8ab1-61962ccb5bac/debian~bullseye/.mkosi-9jfpbe27
‣  Attaching /var/lib/fatbuildr/workspaces/hpckit/831c1f39-eb1e-4aae-8ab1-61962ccb5bac/debian~bullseye/.mkosi-9jfpbe27 as loopback…
losetup: cannot find an unused loop device
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/lib/python3.11/site-packages/mkosi/__main__.py", line 45, in <module>
    main()
  File "/usr/lib64/python3.11/contextlib.py", line 81, in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/mkosi/__main__.py", line 41, in main
    run_verb(a)
  File "/usr/lib/python3.11/site-packages/mkosi/__init__.py", line 8254, in run_verb
    manifest = build_stuff(config)
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/mkosi/__init__.py", line 7559, in build_stuff
    image = build_image(state, manifest=manifest)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/mkosi/__init__.py", line 7236, in build_image
    with attach_base_image(state.config.base_image, state.partition_table) as base_image, \
  File "/usr/lib64/python3.11/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/mkosi/__init__.py", line 982, in attach_image_loopback
    with get_loopdev(image) as loopdev, flock(loopdev):
  File "/usr/lib64/python3.11/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/mkosi/__init__.py", line 963, in get_loopdev
    c = run(["losetup", "--find", "--show", "--partscan", f.name], stdout=subprocess.PIPE, text=True)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/mkosi/backend.py", line 860, in run
    return subprocess.run(cmdline, check=check, stdout=stdout, stderr=stderr, env={**os.environ, **env}, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['losetup', '--find', '--show', '--partscan', '/var/lib/fatbuildr/workspaces/hpckit/831c1f39-eb1e-4aae-8ab1-61962ccb5bac/debian~bullseye/.mkosi-9jfpbe27']' returned non-zero exit status 1.
 ⚬ ERROR: error while running task 831c1f39-eb1e-4aae-8ab1-61962ccb5bac: Command /usr/libexec/fatbuildr/u-nspawn --directory /var/lib/fatbuildr/images/hpckit/osi.img --bind /usr/share/fatbuildr/images/common --quiet --register=no --keep-unit --bind /var/lib/fatbuildr/workspaces/hpckit/831c1f39-eb1e-4aae-8ab1-61962ccb5bac --bind /var/cache/fatbuildr/hpckit/node-debian11 --bind /var/lib/fatbuildr/registry/hpckit/osi --setenv SUDO_UID=998 /usr/bin/mkosi --nspawn-keep-unit --default /var/lib/fatbuildr/workspaces/hpckit/831c1f39-eb1e-4aae-8ab1-61962ccb5bac/osi/node-debian11.mkosi --output-dir /var/lib/fatbuildr/workspaces/hpckit/831c1f39-eb1e-4aae-8ab1-61962ccb5bac --image-id node-debian11 --image-version 1 --checksum failed with exit code 1
 ⚬ INFO: Task failed

The solution would be to support building OSI images outside containers, by running mkosi directly on the host system.

@rezib rezib added the feature New feature or request label May 23, 2023
@rezib rezib self-assigned this May 23, 2023
@rezib rezib added this to the v2.1.0 milestone May 25, 2023
@rezib rezib closed this as completed in #115 Jun 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant