Skip to content

Commit

Permalink
PR #17319: Fixes XLA build with numpy>=2.1.0
Browse files Browse the repository at this point in the history
Imported from GitHub PR #17319

When building XLA using the command from dev guide: docs/developer_guide.md
```bash
./configure.py --backend=CPU
bazel build --test_output=all --spawn_strategy=sandboxed //xla/...
```

Using numpy==2.1.0 we can have the following linking error:
```
ERROR: /xla/xla/python/BUILD:1453:11: Linking xla/python/libnb_numpy.so failed: (Exit 1): clang failed: error executing command (from target //xla/python:nb_numpy) /usr/lib/llvm-14/bin/clang @bazel-out/k8-opt/bin/xla/python/libnb_numpy.so-2.params

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
ld.lld: error: undefined hidden symbol: _xla_numpy_api
>>> referenced by nb_numpy.cc
>>>               bazel-out/k8-opt/bin/xla/python/_objs/nb_numpy/nb_numpy.pic.o:(xla::nb_dtype::from_args(nanobind::object const&))
>>> referenced by nb_numpy.cc
>>>               bazel-out/k8-opt/bin/xla/python/_objs/nb_numpy/nb_numpy.pic.o:(xla::nb_numpy_ndarray::nb_numpy_ndarray(xla::nb_dtype, absl::lts_20230802::Span<long const>, std::optional<absl::lts_20230802::Span<long const> >, void const*, nanobind::handle))
>>> referenced by nb_numpy.cc
>>>               bazel-out/k8-opt/bin/xla/python/_objs/nb_numpy/nb_numpy.pic.o:(xla::nb_numpy_ndarray::nb_numpy_ndarray(xla::nb_dtype, absl::lts_20230802::Span<long const>, std::optional<absl::lts_20230802::Span<long const> >, void const*, nanobind::handle))
>>> referenced 4 more times

ld.lld: error: undefined hidden symbol: _xla_numpy_apiPyArray_RUNTIME_VERSION
>>> referenced by nb_numpy.cc
>>>               bazel-out/k8-opt/bin/xla/python/_objs/nb_numpy/nb_numpy.pic.o:(xla::nb_numpy_ndarray::itemsize() const)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```

Which should be related to https://github.com/numpy/numpy/blob/main/doc/source/release/2.1.0-notes.rst#api-symbols-now-hidden-but-customizable

This PR fixes the build issue
Copybara import of the project:

--
2f6e1b3 by vfdev-5 <vfdev.5@gmail.com>:

Fixes XLA build with numpy>=2.1.0

When building XLA using the command from dev guide: docs/developer_guide.md
```bash
./configure.py --backend=CPU
bazel build --test_output=all --spawn_strategy=sandboxed //xla/...
```

Using numpy==2.1.0 we can have the following linking error:
```
ERROR: /xla/xla/python/BUILD:1453:11: Linking xla/python/libnb_numpy.so failed: (Exit 1): clang failed: error executing command (from target //xla/python:nb_numpy) /usr/lib/llvm-14/bin/clang @bazel-out/k8-opt/bin/xla/python/libnb_numpy.so-2.params

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
ld.lld: error: undefined hidden symbol: _xla_numpy_api
>>> referenced by nb_numpy.cc
>>>               bazel-out/k8-opt/bin/xla/python/_objs/nb_numpy/nb_numpy.pic.o:(xla::nb_dtype::from_args(nanobind::object const&))
>>> referenced by nb_numpy.cc
>>>               bazel-out/k8-opt/bin/xla/python/_objs/nb_numpy/nb_numpy.pic.o:(xla::nb_numpy_ndarray::nb_numpy_ndarray(xla::nb_dtype, absl::lts_20230802::Span<long const>, std::optional<absl::lts_20230802::Span<long const> >, void const*, nanobind::handle))
>>> referenced by nb_numpy.cc
>>>               bazel-out/k8-opt/bin/xla/python/_objs/nb_numpy/nb_numpy.pic.o:(xla::nb_numpy_ndarray::nb_numpy_ndarray(xla::nb_dtype, absl::lts_20230802::Span<long const>, std::optional<absl::lts_20230802::Span<long const> >, void const*, nanobind::handle))
>>> referenced 4 more times

ld.lld: error: undefined hidden symbol: _xla_numpy_apiPyArray_RUNTIME_VERSION
>>> referenced by nb_numpy.cc
>>>               bazel-out/k8-opt/bin/xla/python/_objs/nb_numpy/nb_numpy.pic.o:(xla::nb_numpy_ndarray::itemsize() const)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```

Which should be related to https://github.com/numpy/numpy/blob/main/doc/source/release/2.1.0-notes.rst#api-symbols-now-hidden-but-customizable

This PR fixes the build issue

Merging this change closes #17319

COPYBARA_INTEGRATE_REVIEW=#17319 from vfdev-5:fix-build-numpy-2.1.0-and-later 2f6e1b3
PiperOrigin-RevId: 679594145
  • Loading branch information
vfdev-5 authored and Google-ML-Automation committed Sep 27, 2024
1 parent 7d3676c commit 3ccf821
Showing 1 changed file with 7 additions and 0 deletions.
7 changes: 7 additions & 0 deletions xla/tsl/python/lib/core/numpy.h
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,13 @@ limitations under the License.
#define NO_IMPORT_ARRAY
#endif

// Prevent linking error with numpy>=2.1.0
// error: undefined hidden symbol: _xla_numpy_apiPyArray_RUNTIME_VERSION
// Without this define, Numpy's API symbols will have hidden symbol visibility,
// which may break things if Bazel chooses to build a cc_library target into
// its own .so file. Bazel typically does this for debug builds.
#define NPY_API_SYMBOL_ATTRIBUTE

// clang-format off
// Place `<locale>` before <Python.h> to avoid build failure in macOS.
#include <locale>
Expand Down

0 comments on commit 3ccf821

Please sign in to comment.