0.27.0 build failure on fedora: python3 not detected #8665

vbatts · 2019-06-18T14:41:43Z

Attempting to build the fedora release rpms for v0.27.0 (https://copr.fedorainfracloud.org/coprs/vbatts/bazel) using the spec file at https://github.com/vbatts/copr-build-bazel

Description of the problem / feature request:

full build log bazel-f30.log
relevant snip:

+ /usr/bin/mkdir -p bazel-0.27.0
+ cd bazel-0.27.0
+ /usr/bin/unzip -qq /builddir/build/SOURCES/bazel-0.27.0-dist.zip
+ STATUS=0
+ '[' 0 -ne 0 ']'
+ /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w .
+ exit 0
Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.ovaoCj
+ umask 022
+ cd /builddir/build/BUILD
+ cd bazel-0.27.0
+ export CC=gcc
+ CC=gcc
+ export CXX=g++
+ CXX=g++
+ export 'EXTRA_BAZEL_ARGS= --host_javabase=@local_jdk//:jdk --verbose_failures'
+ EXTRA_BAZEL_ARGS=' --host_javabase=@local_jdk//:jdk --verbose_failures'
+ ./compile.sh
Building Bazel from scratch..find: 'src/tools/xcode-common/java/com/google/devtools/build/xcode/common': No such file or directory
find: 'src/tools/xcode-common/java/com/google/devtools/build/xcode/util': No such file or directory
....
Building Bazel with Bazel.
.WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.protobuf.UnsafeUtil (file:/tmp/bazel_JcPgQJ9V/archive/libblaze.jar) to field java.nio.Buffer.address
WARNING: Please consider reporting this to the maintainers of com.google.protobuf.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
Loading:
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
    Fetching @io_bazel_skydoc; fetching
Loading: 0 packages loaded
    Fetching @build_bazel_rules_nodejs; fetching
Loading: 0 packages loaded
Loading: 0 packages loaded
DEBUG: /tmp/bazel_JcPgQJ9V/out/external/build_bazel_rules_nodejs/internal/common/check_bazel_version.bzl:49:5:
Current Bazel is not a release version, cannot check for compatibility.
Loading: 0 packages loaded
DEBUG: /tmp/bazel_JcPgQJ9V/out/external/build_bazel_rules_nodejs/internal/common/check_bazel_version.bzl:51:5: Make sure that you are running at least Bazel 0.17.1.
Loading: 0 packages loaded
Loading: 1 packages loaded
[...]
[1,304 / 1,791] 5 actions, 1 running
    JavacBootstrap .../buildjar/libskylark-deps.jar [for host]; 30s local
    [Scann] Compiling src/main/cpp/global_variables.cc
    [Scann] Compiling src/main/cpp/main.cc
    [Scann] Compiling src/main/cpp/blaze.cc
    [Prepa] Executing genrule //src:embedded_tools_nojdk
ERROR: /builddir/build/BUILD/bazel-0.27.0/src/BUILD:311:2: Executing genrule //src:embedded_tools_nojdk failed (Exit 127): bash failed: error executing command 
  (cd /tmp/bazel_JcPgQJ9V/out/execroot/io_bazel && \
  exec env - \
    PATH=/builddir/.local/bin:/builddir/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin \
  /bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; bazel-out/host/bin/src/create_embedded_tools "bazel-out/k8-opt/bin/src/embedded_tools_nojdk.zip" bazel-out/k8-opt/bin/src/embedded_tools_nojdk.params')
Execution platform: @bazel_tools//platforms:host_platform
[1,305 / 1,791] 4 actions, 1 running
    JavacBootstrap .../buildjar/libskylark-deps.jar [for host]; 30s local
    [Scann] Compiling src/main/cpp/global_variables.cc
    [Scann] Compiling src/main/cpp/main.cc
    [Scann] Compiling src/main/cpp/blaze.cc
/usr/bin/env: 'python': No such file or directory
[1,305 / 1,791] 4 actions, 1 running
    JavacBootstrap .../buildjar/libskylark-deps.jar [for host]; 30s local
    [Scann] Compiling src/main/cpp/global_variables.cc
    [Scann] Compiling src/main/cpp/main.cc
    [Scann] Compiling src/main/cpp/blaze.cc
Target //src:bazel_nojdk failed to build
[1,306 / 1,791] 3 actions, 0 running
    [Scann] Compiling src/main/cpp/global_variables.cc
    [Scann] Compiling src/main/cpp/main.cc
    [Scann] Compiling src/main/cpp/blaze.cc
INFO: Elapsed time: 819.724s, Critical Path: 30.61s
[1,306 / 1,791] 3 actions, 0 running
    [Scann] Compiling src/main/cpp/global_variables.cc
    [Scann] Compiling src/main/cpp/main.cc
    [Scann] Compiling src/main/cpp/blaze.cc
INFO: 1260 processes: 1260 local.
[1,306 / 1,791] 3 actions, 0 running
    [Scann] Compiling src/main/cpp/global_variables.cc
    [Scann] Compiling src/main/cpp/main.cc
    [Scann] Compiling src/main/cpp/blaze.cc
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

building the fedora-30-x86_64 rpm on copr

What operating system are you running Bazel on?

Fedora 30 x86_64

What's the output of `bazel info release`?

n/a

What's the output of `git remote get-url origin ; git rev-parse master ; git rev-parse HEAD` ?

n/a

The text was updated successfully, but these errors were encountered:

meisterT · 2019-06-18T14:46:08Z

From the log:

/usr/bin/env: 'python': No such file or directory

@brandjon

bazelbuild/bazel#8665 Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>

vbatts · 2019-06-18T14:57:43Z

it ignored the --python3_path flag, so i had dropped that argument. And it says it assumes PY3 in the python detection.

[root@f2f16cce4a3b copr-build-bazel]# python3^T^t
python3     python3.7   python3.7m

There is python3 present, but it is not being detected.

brandjon · 2019-06-18T16:44:07Z

So previously you passed --python3_path? That flag's deprecated (and I believe a no-op) since before 0.27.

The failure comes from the stub script's shebang, before Python 2 vs Python 3 differences come into play (the stub script is compatible with both). This part of how Python programs are launched shouldn't have changed in 0.27.

Is python on the PATH passed to the action? Which according to the log is:

PATH=/builddir/.local/bin:/builddir/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin

I see this recent change in the spec file. I suspect you need the BuildRequires: python line, unconditionally, for there to be a python command found by the /usr/bin/env shebang.

vbatts · 2019-06-18T17:10:10Z

So previously you passed --python3_path? That flag's deprecated (and I believe a no-op) since before 0.27.

It worked up to 0.26.0 to manually point to /usr/bin/python3

The failure comes from the stub script's shebang, before Python 2 vs Python 3 differences come into play (the stub script is compatible with both). This part of how Python programs are launched shouldn't have changed in 0.27.

Is python on the PATH passed to the action? Which according to the log is:
PATH=/builddir/.local/bin:/builddir/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin

python is not in the build chroot.

I see this recent change in the spec file. I suspect you need the BuildRequires: python line, unconditionally, for there to be a python command found by the /usr/bin/env shebang.

This is something in rpm dependency magic of what provides that name (or string), which use to be python 2.7, but Fedora is pushing everything to be python 3 now.

vbatts · 2019-06-18T18:23:57Z

@brandjon I just found that if I also install python 2.7, such that there is a python in the PATH, that this issue is no longer failing. This seems broken. It may be using python3, but expecting python to just exist?

bazelbuild/bazel#8665

brandjon · 2019-06-18T18:41:05Z

It worked up to 0.26.0 to manually point to /usr/bin/python3

I strongly suspect it wasn't doing anything and could've been removed prior to 0.27 without affecting the build.

python is not in the build chroot. [...] I just found that if I also install python 2.7, such that there is a python in the PATH, that this issue is no longer failing. This seems broken.

Nope, it's working as intended. python (whether it's Python 2 or Python 3) must be on PATH for the execution of any py_binary or py_test target. This requirement has existed long before 0.27 (it might've been since the beginning of Python support in Bazel).

The python command is used to run the stub script, which is a simple one-file program that launches the real payload program using the specific requested Python interpreter (which may be python3). Nothing you set in python_version = ... or Python toolchains will affect the fact that the stub needs #!/usr/bin/env python to resolve to a Python interpreter.

My recommendation is to ensure that python is within the chroot of the build.

brandjon · 2019-06-18T18:41:44Z

See also #8446.

vbatts · 2019-06-18T19:15:01Z

sorry, this is not making sense.
You're telling me this is not the case, but I am looking at the fact that only python3 being there is insufficient, and when I install a /usr/bin/python which is 2.7, it does in fact build.
I'm truly not being contrary, but just doing a best-effort to keep a community build of bazel for fedora and rhel/centos community.

brandjon · 2019-06-18T19:31:31Z

Sorry for not being clear -- there definitely must be a Python interpreter available on PATH that can be invoked as the command python. On some distros python is an alias for python2, on others it is an alias for python3. Bazel doesn't care which it is, either will work fine, so long as a python command exists.

From the point of view of the Bazel 0.27 release, the important changes are:

Some builds -- including I presume this build that packages Bazel itself -- now require --host_force_python=PY2 in order to keep the old behavior of using Python 2 in certain places.
The flag --python3_path must be deleted. This is safe because it wasn't doing anything before.

But Bazel 0.27 doesn't change what versions of Python are actually needed on the system. If you needed Python 2 before, that's still true, and same for if you needed Python 3. So I'm not sure why that part of the spec file was changed for 0.27.

aehlig · 2019-06-19T14:04:23Z

Sorry for not being clear -- there definitely must be a Python interpreter available on PATH that can be invoked as the command python.

Unless you fix the shebang line, which is a standard step in many packaging frameworks. For the FreeBSD ports, the SHEBANG_REGEX .*(sh|txt|_stub|stub_.*|bazel|get_workspace_status|protobuf_support|_so) is used. I'm not sure, how the corresponding framework works for fedora, but in the end, it's just a single find(1) command anyway.

(For 0.27.0 I also had to patch in the path of the python 3 interpreter to avoid having to depend on the generic python3 symlink during the build, but since you have a python3 executable on PATH that is probably not necessary in this case.)

Thanks @vbatts for maintaining a bazel package and good luck porting.

vbatts · 2019-06-19T20:32:57Z

@aehlig good pointers. I've just tried several permutations of that to no avail.
Something like has gotten closest:

find . -type f -regextype posix-extended -iregex '.*(sh|txt|py|_stub|stub_.*|bazel|get_workspace_status|protobuf_support|_so)' -exec %{__sed} -i -e '1s|^#!/usr/bin/env python$|#!/usr/bin/env python3|' "{}" \;
export EXTRA_BAZEL_ARGS="${EXTRA_BAZEL_ARGS} --python_path=/usr/bin/python3"

but still /usr/bin/env: 'python': No such file or directory

vbatts · 2019-06-19T20:52:54Z

It looks like the hacki'est winner is simply forcing a python into the path.

# horrible of horribles, just to have `python` in the PATH
mkdir -p ./bin-hack
ln -sf /usr/bin/python3 ./bin-hack/python
export PATH=$(pwd)/bin-hack:$PATH

vbatts · 2019-06-20T13:24:34Z

I've worked around this issue for now, but requiring python just to be in the path, when python3 is detected does not seem like working as intended.

brandjon · 2019-06-20T14:42:08Z

Since this problem was worked around I'll close this issue. I filed #8685 to track not requiring a python command for PY3 targets. The general feature of not being so dependent on the target system's Python interpreter for the stub script is tracked in #8446.

brandjon · 2019-06-20T14:47:57Z

(For 0.27.0 I also had to patch in the path of the python 3 interpreter to avoid having to depend on the generic python3 symlink during the build, but since you have a python3 executable on PATH that is probably not necessary in this case.)

@aehlig, it looks like you patched the wrapper script that invokes Python on the payload code in order to locate a Python 3 interpreter for the payload. That shouldn't be necessary -- you can define a Python toolchain that knows the exact path to the Python 2 and Python 3 interpreters, and use that in place of the autodetecting toolchain (so the wrapper script isn't called at all).

The issue with shebang is trickier because we don't support injecting a custom shebang via a toolchain, and it's not clear that we want to add support for that (as opposed to just rewriting the stub in C++).

aehlig · 2019-06-21T09:07:31Z

@aehlig, it looks like you patched the wrapper script that invokes Python on the payload code in order to locate a Python 3 interpreter for the payload. That shouldn't be necessary -- you can define a Python toolchain that knows the exact path to the Python 2 and Python 3 interpreters, and use that in place of the autodetecting toolchain (so the wrapper script isn't called at all).

That script is also embedded into bazel, so it has to work also in the installed bazel, not just when boostrapping bazel. And here, Python 3 is special in that I know the path of a good Python 3 interpreter—the one that is a RUN_DEPENDS of bazel. So I can improve the "auto-detection" of a working Python 3 interpreter to work even if no python3 is on PATH (which will be the typical situation if you just install bazel); for detecting Python 2, the script works unchanged. As bazel can be reasonably used without a Python 2 interpreter installed on the system, I can't know ahead of time if such an interpreter is available (and here we're not even talking about custom PREFIX/LOCALBASE), let alone embed such a declaration in the user's WORKSPACE file.

mkandes · 2020-05-20T21:34:15Z

Symbolic link method seems to have worked for me ...

# Download, build, and install Bazel
    mkdir -p bazel-0.29.1
    cd bazel-0.29.1
    wget https://github.com/bazelbuild/bazel/releases/download/0.29.1/bazel-0.29.1-dist.zip
    unzip -o bazel-0.29.1-dist.zip
    export EXTRA_BAZEL_ARGS="--host_javabase=@local_jdk//:jdk"
    ln -s /usr/bin/python3 /usr/bin/python
    ./compile.sh

eokeeffe · 2021-08-17T15:06:53Z

symbolic link also worked for me

vbatts mentioned this issue Jun 18, 2019

Release 0.27 - June 2019 #7816

Closed

vbatts added a commit to vbatts/copr-build-bazel that referenced this issue Jun 18, 2019

fix the epel7 python build (all fedora's are still broken)

88b7585

bazelbuild/bazel#8665 Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>

brandjon added P2 We'll consider working on this in future. (Assignee optional) team-Rules-Python Native rules for Python type: support / not a bug (process) labels Jun 18, 2019

vbatts changed the title ~~0.27.0 build failure on fedora: nojdk~~ 0.27.0 build failure on fedora: python3 not detected Jun 18, 2019

vbatts added a commit to vbatts/copr-build-bazel that referenced this issue Jun 18, 2019

workaround python3 detection

f481f4e

bazelbuild/bazel#8665

brandjon mentioned this issue Jun 20, 2019

Allow customizing Python stub script's shebang in the Python toolchain #8685

Closed

brandjon closed this as completed Jun 20, 2019

coryan mentioned this issue Nov 9, 2020

test(storage): add emulator unit tests and cleanup googleapis/google-cloud-cpp#5453

Merged

stefanbucur mentioned this issue Jan 15, 2021

Add test project for the Bazel fuzzing rules. google/oss-fuzz#4936

Merged

ted-xie mentioned this issue Apr 20, 2021

Cannot run Bazel without /usr/bin/python, even if /usr/bin/python3 exists #13379

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

0.27.0 build failure on fedora: python3 not detected #8665

0.27.0 build failure on fedora: python3 not detected #8665

vbatts commented Jun 18, 2019 •

edited

Loading

meisterT commented Jun 18, 2019

vbatts commented Jun 18, 2019 •

edited

Loading

brandjon commented Jun 18, 2019

vbatts commented Jun 18, 2019

vbatts commented Jun 18, 2019

brandjon commented Jun 18, 2019

brandjon commented Jun 18, 2019

vbatts commented Jun 18, 2019

brandjon commented Jun 18, 2019

aehlig commented Jun 19, 2019

vbatts commented Jun 19, 2019

vbatts commented Jun 19, 2019

vbatts commented Jun 20, 2019

brandjon commented Jun 20, 2019

brandjon commented Jun 20, 2019

aehlig commented Jun 21, 2019

mkandes commented May 20, 2020

eokeeffe commented Aug 17, 2021

0.27.0 build failure on fedora: python3 not detected #8665

0.27.0 build failure on fedora: python3 not detected #8665

Comments

vbatts commented Jun 18, 2019 • edited Loading

Description of the problem / feature request:

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

What operating system are you running Bazel on?

What's the output of bazel info release?

What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

meisterT commented Jun 18, 2019

vbatts commented Jun 18, 2019 • edited Loading

brandjon commented Jun 18, 2019

vbatts commented Jun 18, 2019

vbatts commented Jun 18, 2019

brandjon commented Jun 18, 2019

brandjon commented Jun 18, 2019

vbatts commented Jun 18, 2019

brandjon commented Jun 18, 2019

aehlig commented Jun 19, 2019

vbatts commented Jun 19, 2019

vbatts commented Jun 19, 2019

vbatts commented Jun 20, 2019

brandjon commented Jun 20, 2019

brandjon commented Jun 20, 2019

aehlig commented Jun 21, 2019

mkandes commented May 20, 2020

eokeeffe commented Aug 17, 2021

vbatts commented Jun 18, 2019 •

edited

Loading

What's the output of `bazel info release`?

What's the output of `git remote get-url origin ; git rev-parse master ; git rev-parse HEAD` ?

vbatts commented Jun 18, 2019 •

edited

Loading