Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New debootstrap is unable to bootstrap pre-bookworm debian derivatives #361

Open
sjoerdsimons opened this issue Sep 20, 2022 · 10 comments · May be fixed by #424
Open

New debootstrap is unable to bootstrap pre-bookworm debian derivatives #361

sjoerdsimons opened this issue Sep 20, 2022 · 10 comments · May be fixed by #424

Comments

@sjoerdsimons
Copy link
Member

Debian unstable/testing as well as the latest debian stable release have a patch to the debian-common script which requires
usr-is-merged on anything that's not an older known Debian release..

Which breaks using that debootstrap for any derivative distribution with a different codename, which is rather unfortunate as that's a reasonable common usage.

Without changing debootstrap we have the following options:

  • Make the debootstrap script argument configurable #16 - Making the debootstrap script configurable.. While that would give an escape hatch, it will require using a different
    script entirely as the offending code is in debian-common
  • RFE: support using mmdebstrap (instead of debootstrap) #127 - Support another bootstrapping tool; That won't fix current recipes unless it's done as a drop-in replacement. But switching
    the bootstrapping tool underneath the user opens a whole other can of worms. And really i'd like to avoid having to
    support multiple tools, even with all its issues debootstrap is still the official bootstrapping tools
  • actions: debootstrap: Expose include and exclude arguments #358 - Expose debootstraps include and exclude options; Which we avoided thusfar as debootstrap really isn't the right location
    to add more packages (use apt actions for that) or to remove package as that's risky as well. When using a minbase
    variant the latter really shouldn't be needed. Not unsurprising it was only proposed due to this issue. And ofcourse, this
    still requires users to switch their recipes so it's quite annoying.

TL;DR we can't have nice things; and in particular all these options require users to chagne their recipes which is really not great

@obbardc
Copy link
Member

obbardc commented Sep 20, 2022

One attempted modification to debootstrap was to only install usr-is-merged if the suite is testing or unstable; it wasn't accepted and the developer's suggestion was to exclude usr-is-merge by debootstrap's --exclude argument.

@sjoerdsimons sjoerdsimons changed the title New debootstrap is unstable to bootstrap pre-bookworm debian derivatives New debootstrap is unable to bootstrap pre-bookworm debian derivatives Sep 20, 2022
@sjoerdsimons
Copy link
Member Author

Digging into this further with some pointers from @smcv; The current overview is:

  • usr-is-merged is a dummy package to say:
  • This system was created in a merged usr form
  • This system expclitly opted out from merged-usr (flag file in etc: unsupported-skip-usrmerge-conversion); Which is
    mostly intended for QA reasons.

New debootstrap (stable and onwards) will on codenames that it doesn't recognize as an older debian version always require this package and if --no-merged-usr is given will create the flag file mentioned above. Ideally with the goal of avoiding the installation of usrmerge. Though when testing with debootstrapping sid for the new init-system-helpers (with updated dependencies) it seems that still pulls in both usrmerge and usr-is-merged.

Given that as a quick fix i'm planning to simply hardcode --exclude=usr-is-merged which will mostly get us back to the behaviour we had before. The impact on non-debian suites will be the flag file gets dropped in etc when running a non-usr-merged setup and no changes otherwise. For sid (and soon bookworm) based systems it will mean that usr-is-merged doesn't get installed, but usrmerge is (as it would anyway as things stand at the moment).

For a longer term solution on the debos side we probably want to have a field in the debootstrap action to indicate what "base" distribution the bootstrapping should be done for, which will then also be used for the debootstrap script and potentially to do workarounds like the above only for older debian releases.. That said to avoid dropping the flag file that would still need some fixes to debootstrap as it bases its decision on the codename of the release file rather then the debootstrap script being used

sjoerdsimons added a commit that referenced this issue Sep 21, 2022
Work around new debootstrap behaviour by always excluding the
usr-is-merged package as this allows derivatives based on older debian
versions to build again. See #361 for more details

Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>
sjoerdsimons added a commit that referenced this issue Sep 22, 2022
Work around new debootstrap behaviour by always excluding the
usr-is-merged package as this allows derivatives based on older debian
versions to build again. See #361 for more details

Signed-off-by: Sjoerd Simons <sjoerd@collabora.com>
bors bot added a commit that referenced this issue Sep 22, 2022
362: fix non-debian builds based on current debian releases r=obbardc a=sjoerdsimons

 Work around new debootstrap behaviour by always excluding the
 usr-is-merged package as this allows derivatives based on older debian
 versions to build again. See #361 for more details


Co-authored-by: Sjoerd Simons <sjoerd@collabora.com>
@obbardc
Copy link
Member

obbardc commented Sep 22, 2022

This has been fixed in #362 but let's keep this open to track the longer-term solution.

jadams pushed a commit to jadams/kali-vm that referenced this issue Nov 30, 2022
Somehow apt chokes while trying to resolve the set of standard packages.
This seems to be due to the following dependency chain:

  dbus -> init-system-helpers -> usrmerge | usr-is-merged.

I couldn't understand why apt refuses to install usrmerge (installing it
manually works, but it requires listing all the dependencies that it
complains about on the command-line). I really couldn't understand what
was wrong with those deps.

In any case, it turns out that usr-is-merged *should have been
installed*, that's what debootstrap is supposed to do. However, this has
been disabled in a recent debos commit, cf:
* go-debos/debos#361
* go-debos/debos@b3c1f76b

So let's install usr-is-merged manually (as the debos workaround is not
for our use-case anyway), that brings us back in line with a standard
debootstrap chroot, which is surely a good thing.

From this point the problem disappears.

Fixes: #23
@Penlect
Copy link

Penlect commented Dec 13, 2022

Dear maintainers, I would like to bring something to your attention that I discovered today when using debos (in Debian bookworm, version 1.1.1-2) that seems related to this issue.

When using bookworm as suit in the debootstrap action

  - action: debootstrap
    suite: bookworm
    components:
      - main
      - contrib
      - non-free
    mirror: https://deb.debian.org/debian
    variant: minbase

any following apt-action step will fail with the following message:

2022/12/13 09:01:26 ==== Install system packages ====
Hit:1 https://deb.debian.org/debian bookworm InRelease
Get:2 https://deb.debian.org/debian bookworm/main Translation-en [5941 kB]
Get:3 https://deb.debian.org/debian bookworm/contrib Translation-en [46.9 kB]
Get:4 https://deb.debian.org/debian bookworm/non-free Translation-en [92.3 kB]
Fetched 6080 kB in 1s (9733 kB/s)                                      
Reading package lists... Done
Reading package lists... Done
Building dependency tree... Done
2022/12/13 09:01:27 apt | You might want to run 'apt --fix-broken install' to correct these.
2022/12/13 09:01:27 apt | The following packages have unmet dependencies:
2022/12/13 09:01:27 apt |  init-system-helpers : Depends: usrmerge but it is not going to be installed or
2022/12/13 09:01:27 apt |                                 usr-is-merged
2022/12/13 09:01:27 apt |  systemd : Depends: libcryptsetup12 (>= 2:2.4) but it is not going to be installed
2022/12/13 09:01:27 apt |            Depends: libfdisk1 (>= 2.33) but it is not going to be installed
2022/12/13 09:01:27 apt |            Depends: libkmod2 (>= 15) but it is not going to be installed
2022/12/13 09:01:27 apt |            Depends: libsystemd-shared (= 252.2-2) but it is not going to be installed
2022/12/13 09:01:27 apt | E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).
2022/12/13 09:01:27 Action `Install system packages` failed at stage Run, error: exit status 100

However, installing usr-is-merged immediately after the deboostrap-action seems to be a workaround

  - action: apt
    packages:
      - usr-is-merged

andhe added a commit to andhe/debos that referenced this issue Jan 3, 2023
The workaround for go-debos#361
that was applied in go-debos@b3c1f76
breaks recipes for bookworm and newer.
andhe added a commit to andhe/debos that referenced this issue Jan 3, 2023
The workaround for go-debos#361
that was applied in go-debos@b3c1f76
breaks recipes for bookworm and newer.

Signed-off-by: Andreas Henriksson <andreas@fatal.se>
andhe added a commit to andhe/debos that referenced this issue Jan 3, 2023
The workaround for go-debos#361
that was applied in go-debos@b3c1f76
breaks recipes for bookworm and newer.

Signed-off-by: Andreas Henriksson <andreas@fatal.se>
andhe added a commit to andhe/debos that referenced this issue Jan 3, 2023
The workaround for go-debos#361
that was applied in go-debos@b3c1f76
breaks recipes for bookworm and newer.

Signed-off-by: Andreas Henriksson <andreas@fatal.se>
bors bot added a commit that referenced this issue Jan 3, 2023
390: Limit old suite workaround r=obbardc a=andhe

The workaround for #361 that was applied in b3c1f76 breaks recipes for bookworm and newer.

This change makes the workaround less damaging (should atleast work with official debian releases for the forseeable future), while still far from perfect (any derivate distribution that is bookworm-based will fail).

Co-authored-by: Andreas Henriksson <andreas@fatal.se>
@elboulangero
Copy link
Contributor

elboulangero commented Jun 29, 2023

@obbardc The commit 18998ff doesn't work for derivatives based on recent Debian's suites (eg. Kali Linux based on testing). But I guess you already know it and there's no simple solution.

The workaround is simple though, just this step after debootstrap:

  - description: "Install usr-is-merged (cf. debos #361 and #362)"
    action: apt
    packages: [ usr-is-merged ]

@obbardc
Copy link
Member

obbardc commented Aug 4, 2023

@obbardc The commit 18998ff doesn't work for derivatives based on recent Debian's suites (eg. Kali Linux based on testing).

Hi @elboulangero sorry for the late response, it got lost in my backlog. Can you please check the implementation under #424 ?

@elboulangero
Copy link
Contributor

Hello @obbardc, I just tried #424 on my side, it works great for Kali. Outstanding work! Thank you very much!

@fortysixandtwo
Copy link

fortysixandtwo commented Aug 8, 2024

@obbardc The commit 18998ff doesn't work for derivatives based on recent Debian's suites (eg. Kali Linux based on testing). But I guess you already know it and there's no simple solution.

The workaround is simple though, just this step after debootstrap:

  - description: "Install usr-is-merged (cf. debos #361 and #362)"
    action: apt
    packages: [ usr-is-merged ]

Just a datapoint:
This workaround did not work for me on arm64 (tried with PureOS crimson, which is based on bookworm)
unless I also set variant: minbase on the debootstrap step.

I haven't checked in detail what the difference is between amd64 and arm64 in this case.

Anyway, personally my preferred solution would be #358 as this gives users control (and get rid of the logic that adds --exclude=usr-is-merged). I prefer it over #424 because that extra layer effectively does the same thing for this case.

@obbardc
Copy link
Member

obbardc commented Aug 8, 2024

@fortysixandtwo #352 is not related. Can you check which PR you mean ? thanks.

@fortysixandtwo
Copy link

My bad, I meant #358 (edited in the above post as well)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants