python3/host: build install race condition with uboot/scripts/dtc/pylibfdt #19241
Labels
Comments
ynezz
changed the title
/builder/shared-workdir/build/staging_dir/host/bin/install -c -m 644 ./Lib/encodings/shift_jis_2004.py /builder/shared-workdir/build/staging_dir/hostpkg/lib/python3.10/encodings
Python path configuration:
PYTHONHOME = (not set)
PYTHONPATH = 'scripts/dtc/pylibfdt'
program name = 'python3'
isolated = 0
environment = 1
user site = 1
import site = 1
sys._base_executable = '/builder/shared-workdir/build/staging_dir/hostpkg/bin/python3'
sys.base_prefix = '/builder/shared-workdir/build/staging_dir/hostpkg'
sys.base_exec_prefix = '/builder/shared-workdir/build/staging_dir/hostpkg'
sys.platlibdir = 'lib'
sys.executable = '/builder/shared-workdir/build/staging_dir/hostpkg/bin/python3'
sys.prefix = '/builder/shared-workdir/build/staging_dir/hostpkg'
sys.exec_prefix = '/builder/shared-workdir/build/staging_dir/hostpkg'
sys.path = [
'scripts/dtc/pylibfdt',
'/builder/shared-workdir/build/staging_dir/hostpkg/lib/python310.zip',
'/builder/shared-workdir/build/staging_dir/hostpkg/lib/python3.10',
'/builder/shared-workdir/build/staging_dir/hostpkg/lib/python3.10/lib-dynload',
]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
LookupError: unknown encoding: utf-8
Current thread 0x00007fc2d9cb0740 (most recent call first):
<no Python frame>
/builder/shared-workdir/build/staging_dir/host/bin/install -c -m 644 ./Lib/encodings/shift_jisx0213.py /builder/shared-workdir/build/staging_dir/hostpkg/lib/python3.10/encodings
make[4]: *** [scripts/Makefile.spl:355: include/generated/dt-decl.h] Error 1
make[4]: *** Deleting file 'include/generated/dt-decl.h' |
/builder/shared-workdir/build/staging_dir/host/bin/install -c -m 644 ./Lib/ensurepip/_bundled/setuptools-58.1.0-py3-none-any.whl /builder/shared-workdir/build/staging_dir/hostpkg/lib/python3.10/ensurepip/_bundle
COPY spl/u-boot-spl.dtb
DTOC spl/dts/dt-plat.c
/builder/shared-workdir/build/staging_dir/host/bin/install -c -m 644 ./Lib/html/__init__.py /builder/shared-workdir/build/staging_dir/hostpkg/lib/python3.10/html
...
builder/shared-workdir/build/staging_dir/host/bin/install -c -m 644 ./Lib/idlelib/statusbar.py /builder/shared-workdir/build/staging_dir/hostpkg/lib/python3.10/idlelib
/builder/shared-wo
rkdir/build/staging_dir/host/bin/install -c -m 644 ./Lib/idlelib/textview.py /builder/shared-workdir/build/staging_dir/hostpkg/lib/python3.10/idlelib
make[1]: *** [package/Makefile:119: package/boot/uboot-imx/compile] Error 1
make[1]: *** Waiting for unfinished jobs....
...
/builder/shared-workdir/build/staging_dir/host/bin/install -c -m 644 ./Lib/lib2to3/fixes/fix_long.py /builder/shared-workdir/build/staging_dir/hostpkg/lib/python3.10/lib2to3/fixes
Traceback (most recent call last):
File "/builder/shared-workdir/build/build_dir/target-arm_cortex-a9+neon_musl_eabi/u-boot-mx6cuboxi/u-boot-2022.01/./tools/dtoc/dtoc", line 38, in <module>
from dtoc import dtb_platdata
File "/builder/shared-workdir/build/build_dir/target-arm_cortex-a9+neon_musl_eabi/u-boot-mx6cuboxi/u-boot-2022.01/tools/dtoc/../dtoc/dtb_platdata.py", line 23, in <module>
from dtoc import fdt
File "/builder/shared-workdir/build/build_dir/target-arm_cortex-a9+neon_musl_eabi/u-boot-mx6cuboxi/u-boot-2022.01/tools/dtoc/../dtoc/fdt.py", line 12, in <module>
from dtoc import fdt_util
File "/builder/shared-workdir/build/build_dir/target-arm_cortex-a9+neon_musl_eabi/u-boot-mx6cuboxi/u-boot-2022.01/tools/dtoc/../dtoc/fdt_util.py", line 16, in <module>
from patman import command
File "/builder/shared-workdir/build/build_dir/target-arm_cortex-a9+neon_musl_eabi/u-boot-mx6cuboxi/u-boot-2022.01/tools/dtoc/../patman/command.py", line 7, in <module>
from patman import cros_subprocess
File "/builder/shared-workdir/build/build_dir/target-arm_cortex-a9+neon_musl_eabi/u-boot-mx6cuboxi/u-boot-2022.01/tools/dtoc/../patman/cros_subprocess.py", line 262, in <module>
class TestSubprocess(unittest.TestCase):
AttributeError: module 'unittest' has no attribute 'TestCase'
/builder/shared-workdir/build/staging_dir/host/bin/install -c -m 644 ./Lib/lib2to3/fixes/fix_map.py /builder/shared-workdir/build/staging_dir/hostpkg/lib/python3.10/lib2to3/fixes
...
make[4]: *** [scripts/Makefile.spl:355: spl/dts/dt-plat.c] Error 1 |
ynezz
referenced
this issue
Oct 27, 2022
Using https://gitlab.freedesktop.org/mobile-broadband/libmbim.git to download the source code. Enabled lto and additional gcc flags for perfomance and less size. Modified to use meson as upstream has abandoned autotools. Removed BUILD_PARALLEL options. These are default with ninja/meson. Signed-off-by: Maxim Anisimov <maxim.anisimov.ua@gmail.com> (cherry picked from commit fea19c2)
jefferyto
added a commit
to jefferyto/openwrt-packages
that referenced
this issue
May 20, 2023
When doing parallel builds, host Python can install the python3 symlink before the Python standard library is installed completely. When this occurs, it is possible for other packages to detect the python3 symlink and try to use host Python before it is fully installed. This adds a patch to make commoninstall (where the standard library is installed) a prerequisite of bininstall (where the python3 symlink is installed), so that commoninstall is fully completed before bininstall begins. Fixes: openwrt#19241 Signed-off-by: Jeffery To <jeffery.to@gmail.com>
jefferyto
added a commit
to jefferyto/openwrt-packages
that referenced
this issue
May 20, 2023
When doing parallel builds, host Python can install the python3 symlink before the Python standard library is installed completely. When this occurs, it is possible for other packages to detect the python3 symlink and try to use host Python before it is fully installed. This adds a patch to make commoninstall (where the standard library is installed) a prerequisite of bininstall (where the python3 symlink is installed), so that commoninstall is fully completed before bininstall begins. Fixes: openwrt#19241 Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This was referenced May 20, 2023
jefferyto
added a commit
to jefferyto/openwrt-packages
that referenced
this issue
May 20, 2023
When doing parallel builds, host Python can install the python3 symlink before the Python standard library is installed completely. When this occurs, it is possible for other packages to detect the python3 symlink and try to use host Python before it is fully installed. This adds a patch to make commoninstall (where the standard library is installed) a prerequisite of bininstall (where the python3 symlink is installed), so that commoninstall is fully completed before bininstall begins. Patch has been submitted upstream: python/cpython#104693 Fixes: openwrt#19241 Signed-off-by: Jeffery To <jeffery.to@gmail.com>
jefferyto
added a commit
to jefferyto/openwrt-packages
that referenced
this issue
May 22, 2023
When doing parallel builds, host Python can install the python3 symlink before the Python standard library is installed completely. When this occurs, it is possible for other packages to detect the python3 symlink and try to use host Python before it is fully installed. This adds a patch to make commoninstall (where the standard library is installed) a prerequisite of bininstall (where the python3 symlink is installed), so that commoninstall is fully completed before bininstall begins. Patch has been submitted upstream: python/cpython#104693 Fixes: openwrt#19241 Signed-off-by: Jeffery To <jeffery.to@gmail.com>
jefferyto
added a commit
to jefferyto/openwrt-packages
that referenced
this issue
May 23, 2023
When doing parallel builds, host Python can install the python3 symlink before the Python standard library is installed completely. When this occurs, it is possible for other packages to detect the python3 symlink and try to use host Python before it is fully installed. This adds a patch to make commoninstall (where the standard library is installed) a prerequisite of bininstall (where the python3 symlink is installed), so that commoninstall is fully completed before bininstall begins. Patch has been submitted upstream: python/cpython#104693 Fixes: openwrt#19241 Signed-off-by: Jeffery To <jeffery.to@gmail.com>
neheb
pushed a commit
that referenced
this issue
May 24, 2023
When doing parallel builds, host Python can install the python3 symlink before the Python standard library is installed completely. When this occurs, it is possible for other packages to detect the python3 symlink and try to use host Python before it is fully installed. This adds a patch to make commoninstall (where the standard library is installed) a prerequisite of bininstall (where the python3 symlink is installed), so that commoninstall is fully completed before bininstall begins. Patch has been submitted upstream: python/cpython#104693 Fixes: #19241 Signed-off-by: Jeffery To <jeffery.to@gmail.com>
jefferyto
added a commit
to jefferyto/openwrt-packages
that referenced
this issue
May 24, 2023
When doing parallel builds, host Python can install the python3 symlink before the Python standard library is installed completely. When this occurs, it is possible for other packages to detect the python3 symlink and try to use host Python before it is fully installed. This adds a patch to make commoninstall (where the standard library is installed) a prerequisite of bininstall (where the python3 symlink is installed), so that commoninstall is fully completed before bininstall begins. Patch has been submitted upstream: python/cpython#104693 Fixes: openwrt#19241 Signed-off-by: Jeffery To <jeffery.to@gmail.com> (cherry picked from commit 67e47f1)
jefferyto
added a commit
to jefferyto/openwrt-packages
that referenced
this issue
May 24, 2023
When doing parallel builds, host Python can install the python3 symlink before the Python standard library is installed completely. When this occurs, it is possible for other packages to detect the python3 symlink and try to use host Python before it is fully installed. This adds a patch to make commoninstall (where the standard library is installed) a prerequisite of bininstall (where the python3 symlink is installed), so that commoninstall is fully completed before bininstall begins. Patch has been submitted upstream: python/cpython#104693 Fixes: openwrt#19241 Signed-off-by: Jeffery To <jeffery.to@gmail.com> (cherry picked from commit 67e47f1) Signed-off-by: Jeffery To <jeffery.to@gmail.com>
graysky2
pushed a commit
to graysky2/packages
that referenced
this issue
May 25, 2023
When doing parallel builds, host Python can install the python3 symlink before the Python standard library is installed completely. When this occurs, it is possible for other packages to detect the python3 symlink and try to use host Python before it is fully installed. This adds a patch to make commoninstall (where the standard library is installed) a prerequisite of bininstall (where the python3 symlink is installed), so that commoninstall is fully completed before bininstall begins. Patch has been submitted upstream: python/cpython#104693 Fixes: openwrt#19241 Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Maintainer: @jefferyto
Environment: imx/cortexa9, OpenWrt buildbot for snapshots
Description:
We're experiencing following occasional build failures:
It looks like
python3/hostshould make this symlink/builder/shared-workdir/build/staging_dir/hostpkg/bin/python3available once the installation of python3/host is complete, not sooner, otherwise some Python3 scripts might start using incomplete Python3 interpreter installation which likely results in this failure.The text was updated successfully, but these errors were encountered: