Failed to build on debian:sid-slim

[wojiushixiaobai]

#8 1832.1 + find /usr/local -type f -executable -not ( -name *tkinter* ) -exec ldd {} ;
#8 1832.1 + sort -u
#8 1832.1 + awk /=>/ { so = $(NF-1); if (index(so, "/usr/local/") == 1) { next }; gsub("^/(usr/)?", "", so); printf "*%s\n", so }
#8 1832.1 + xargs -r apt-mark manual
#8 1832.1 + cut -d: -f1
#8 1832.1 + sort -u
#8 1832.1 + xargs -r dpkg-query --search
#8 1832.2 	not a dynamic executable
#8 1832.3 	not a dynamic executable
#8 1832.4 	not a dynamic executable
#8 1832.5 	not a dynamic executable
#8 1832.6 	not a dynamic executable
#8 1832.7 	not a dynamic executable
#8 1832.8 	not a dynamic executable
#8 1832.9 	not a dynamic executable
#8 1833.0 	not a dynamic executable
#8 1833.1 	not a dynamic executable
#8 1833.2 	not a dynamic executable
#8 1833.3 	not a dynamic executable
#8 1833.4 	not a dynamic executable
#8 1833.5 	not a dynamic executable
#8 1833.6 	not a dynamic executable
#8 1833.7 	not a dynamic executable
#8 1833.8 	not a dynamic executable
#8 1833.9 	not a dynamic executable
#8 1834.0 	not a dynamic executable
#8 1834.1 	not a dynamic executable
#8 1834.2 	not a dynamic executable
#8 1834.3 	not a dynamic executable
#8 1834.4 	not a dynamic executable
#8 1834.5 	not a dynamic executable
#8 1834.6 	not a dynamic executable
#8 1834.7 	not a dynamic executable
#8 1834.8 	not a dynamic executable
#8 1834.9 	not a dynamic executable
#8 1835.0 	not a dynamic executable
#8 1835.1 	not a dynamic executable
#8 1835.2 	not a dynamic executable
#8 1844.5 	not a dynamic executable
#8 1844.6 	not a dynamic executable
#8 1844.7 	not a dynamic executable
#8 1844.8 	not a dynamic executable
#8 1844.9 	not a dynamic executable
#8 1845.0 	not a dynamic executable
#8 1845.4 	not a dynamic executable
#8 1845.5 	not a dynamic executable
#8 1845.6 	not a dynamic executable
#8 1845.8 	not a dynamic executable
#8 1850.6 libreadline8t64 was already set to manually installed.
#8 1850.6 libreadline8t64 was already set to manually installed.
#8 1850.6 libreadline8t64 set to manually installed.
#8 1850.6 libbz2-1.0 set to manually installed.
#8 1850.6 libc6 set to manually installed.
#8 1850.6 libcrypt1 set to manually installed.
#8 1850.6 libdb5.3t64 set to manually installed.
#8 1850.6 libexpat1 set to manually installed.
#8 1850.6 libffi8 set to manually installed.
#8 1850.6 libgdbm6t64 set to manually installed.
#8 1850.6 liblzma5 set to manually installed.
#8 1850.6 libncursesw6 set to manually installed.
#8 1850.6 libsqlite3-0 set to manually installed.
#8 1850.6 libssl3t64 set to manually installed.
#8 1850.6 libtinfo6 set to manually installed.
#8 1850.6 libuuid1 set to manually installed.
#8 1850.6 libzstd1 set to manually installed.
#8 1850.6 zlib1g set to manually installed.
#8 1850.6 E: Unable to locate package diversion
#8 1850.6 E: Unable to locate package by
#8 1850.6 E: Unable to locate package from
#8 1850.6 E: Unable to locate package diversion
#8 1850.6 E: Unable to locate package by
#8 1850.6 E: Unable to locate package to
#8 ERROR: process "/bin/sh -c set -eux; \t\tsavedAptMark=\"$(apt-mark showmanual)\"; \tapt-get update; \tapt-get install -y --no-install-recommends \t\tdpkg-dev \t\tgcc \t\tgnupg \t\tlibbluetooth-dev \t\tlibbz2-dev \t\tlibc6-dev \t\tlibdb-dev \t\tlibexpat1-dev \t\tlibffi-dev \t\tlibgdbm-dev \t\tliblzma-dev \t\tlibncursesw5-dev \t\tlibreadline-dev \t\tlibsqlite3-dev \t\tlibssl-dev \t\tmake \t\ttk-dev \t\tuuid-dev \t\twget \t\txz-utils \t\tzlib1g-dev \t; \t\twget -O python.tar.xz \"[https://www.python.org/ftp/python/${PYTHON_VERSION%%[a-z]*}/Python-$PYTHON_VERSION.tar.xz\](https://www.python.org/ftp/python/$%7BPYTHON_VERSION%%[a-z]*%7D/Python-$PYTHON_VERSION.tar.xz/)"; \twget -O python.tar.xz.asc \"[https://www.python.org/ftp/python/${PYTHON_VERSION%%[a-z]*}/Python-$PYTHON_VERSION.tar.xz.asc\](https://www.python.org/ftp/python/$%7BPYTHON_VERSION%%[a-z]*%7D/Python-$PYTHON_VERSION.tar.xz.asc/)"; \tGNUPGHOME=\"$(mktemp -d)\"; export GNUPGHOME; \tgpg --batch --keyserver hkps://keys.openpgp.org --recv-keys \"$GPG_KEY\"; \tgpg --batch --verify python.tar.xz.asc python.tar.xz; \tgpgconf --kill all; \trm -rf \"$GNUPGHOME\" python.tar.xz.asc; \tmkdir -p /usr/src/python; \ttar --extract --directory /usr/src/python --strip-components=1 --file python.tar.xz; \trm python.tar.xz; \t\tcd /usr/src/python; \tgnuArch=\"$(dpkg-architecture --query DEB_BUILD_GNU_TYPE)\"; \t./configure \t\t--build=\"$gnuArch\" \t\t--enable-loadable-sqlite-extensions \t\t--enable-optimizations \t\t--enable-option-checking=fatal \t\t--enable-shared \t\t--with-system-expat \t\t--without-ensurepip \t; \tnproc=\"$(nproc)\"; \tEXTRA_CFLAGS=\"$(dpkg-buildflags --get CFLAGS)\"; \tLDFLAGS=\"$(dpkg-buildflags --get LDFLAGS)\"; \tLDFLAGS=\"${LDFLAGS:--Wl},--strip-all\"; \tmake -j \"$nproc\" \t\t\"EXTRA_CFLAGS=${EXTRA_CFLAGS:-}\" \t\t\"LDFLAGS=${LDFLAGS:-}\" \t\t\"PROFILE_TASK=${PROFILE_TASK:-}\" \t; \trm python; \tmake -j \"$nproc\" \t\t\"EXTRA_CFLAGS=${EXTRA_CFLAGS:-}\" \t\t\"LDFLAGS=${LDFLAGS:--Wl},-rpath='\\$\\$ORIGIN/../lib'\" \t\t\"PROFILE_TASK=${PROFILE_TASK:-}\" \t\tpython \t; \tmake install; \t\tcd /; \trm -rf /usr/src/python; \t\tfind /usr/local -depth \t\t\\( \t\t\t\\( -type d -a \\( -name test -o -name tests -o -name idle_test \\) \\) \t\t\t-o \\( -type f -a \\( -name '*.pyc' -o -name '*.pyo' -o -name 'libpython*.a' \\) \\) \t\t\\) -exec rm -rf '{}' + \t; \t\tldconfig; \t\tapt-mark auto '.*' > /dev/null; \tapt-mark manual $savedAptMark; \tfind /usr/local -type f -executable -not \\( -name '*tkinter*' \\) -exec ldd '{}' ';' \t\t| awk '/=>/ { so = $(NF-1); if (index(so, \"/usr/local/\") == 1) { next }; gsub(\"^/(usr/)?\", \"\", so); printf \"*%s\\n\", so }' \t\t| sort -u \t\t| xargs -r dpkg-query --search \t\t| cut -d: -f1 \t\t| sort -u \t\t| xargs -r apt-mark manual \t; \tapt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false; \trm -rf /var/lib/apt/lists/*; \t\tpython3 --version" did not complete successfully: exit code: 123

	find /usr/local -type f -executable -not \( -name '*tkinter*' \) -exec ldd '{}' ';' \
		| awk '/=>/ { so = $(NF-1); if (index(so, "/usr/local/") == 1) { next }; gsub("^/(usr/)?", "", so); printf "*%s\n", so }' \
		| sort -u \
		| xargs -r dpkg-query --search \
		| cut -d: -f1 \
		| sort -u \
		| xargs -r apt-mark manual \
	; \

...
libreadline8t64:loong64: /usr/lib/loongarch64-linux-gnu/libreadline.so.8
diversion by libreadline8t64 from: /lib/loongarch64-linux-gnu/libreadline.so.8
diversion by libreadline8t64 to: /lib/loongarch64-linux-gnu/libreadline.so.8.usr-is-merged
...

# fix
	find /usr/local -type f -executable -not \( -name '*tkinter*' \) -exec ldd '{}' ';' \
		| awk '/=>/ { so = $(NF-1); if (index(so, "/usr/local/") == 1) { next }; gsub("^/(usr/)?", "", so); printf "*%s\n", so }' \
		| sort -u \
		| xargs -r dpkg-query --search \
		| grep -P '^[a-z0-9][a-z0-9\.\+-]*:' \
		| cut -d: -f1 \
		| sort -u \
		| xargs -r apt-mark manual

[tianon]

Hmm, we don't officially support building FROM debian:sid-slim, and diverted files at this point in the Dockerfile is definitely strange (we haven't done any intentional dpkg-divert calls, so they'd have to be something in the packages themselves, which is weird).

Is this something specific to a build against loong64, or can you reproduce on a stock amd64 build as well?

[wojiushixiaobai]

@tianon
Yes, as long as the debian:sid-slim image is used, any architecture can reproduce this problem.

• debian:sid build successfully.
• debian:trixie build successfully.
• debian:sid-slim build failed.
• debian:trixie-slim build failed.

[tianon]

Interesting 🤔 (thanks for the detailed testing!)

I'm not sure why we're seeing diversions that are clearly related to usrmerge, but the resulting output format from dpkg-query that we're choking on is not unexpected/is documented so I guess we ought to put more thought into this:

The output format consists of one line per matching pattern, with a list of packages owning the pathname separated by a comma (U+002C ‘,’) and a space (U+0020 ‘ ’), followed by a colon (U+003A ‘:’) and a space, followed by the pathname. As in:

pkgname1, pkgname2: pathname1
pkgname3: pathname2

File diversions are printed with the following localized strings:

diversion by pkgname from: diverted-from
diversion by pkgname to: diverted-to

or for local diversions:

local diversion from: diverted-from
local diversion to: diverted-to

https://manpages.debian.org/bookworm/dpkg/dpkg-query.1.en.html