python polars 0.14.11 #4848
Merged
python polars 0.14.11 #4848
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
|
Any idea how we can fix this @messense? I think the changes we did since latest release was using |
|
The problem is that 1021f23 introduced |
|
You can try my [patch.crates-io]
cmake = { git = "https://github.com/messense/cmake-rs.git", branch = "cross-compile" }Or you can add some CMake toolchain files for these cross compiling targets, then set |
Right, thanks for the help as always! 🙌 I will patch to your branch and keep an eye out for the issue upstream. |
|
🤦♂️ I was messing around with a potential default cmake toolchain file in |
|
Should I point to another branch? |
|
I'm afraid GCC 4.8.5 might not be able to build zlib-ng. |
|
Alright.. I think we'll just ship zlib for arm then. 👍 |
|
Can't we build zlib-ng with |
I'm experimenting with maturin build --target aarch64-unknown-linux-gnu --release --zigusing maturin v0.13.3, but I haven't tested the wheel on arm64 Linux yet.
Some references in case you're interested: |
Unfortunately it does not work Python 3.10.7 (main, Sep 13 2022, 16:13:35) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from polars import polars
/usr/local/lib/python3.10/site-packages/polars/__init__.py:11: UserWarning: polars binary missing!
warnings.warn("polars binary missing!")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: /usr/local/lib/python3.10/site-packages/polars/polars.abi3.so: cannot allocate memory in static TLS blockError message is |
|
I think it just failed to load the polars library, probably due to wrong minimum glibc version. Edit:
|
root@c3b9f52db868:/io# ldd /usr/local/lib/python3.10/site-packages/polars/polars.abi3.so
libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x00000055039f0000)
libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000005503a9d000)
libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000005503ace000)
libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000005503c43000)
/lib/ld-linux-aarch64.so.1 (0x0000005500000000)
root@c3b9f52db868:/io#
root@c3b9f52db868:/io# ls /usr/lib/aarch64-linux-gnu/libgomp.so.1
/usr/lib/aarch64-linux-gnu/libgomp.so.1
root@c3b9f52db868:/io# LD_PRELOAD=/usr/lib/aarch64-linux-gnu/libgomp.so.1 python
Python 3.10.7 (main, Sep 13 2022, 16:13:35) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from polars import polars
/usr/local/lib/python3.10/site-packages/polars/__init__.py:11: UserWarning: polars binary missing!
warnings.warn("polars binary missing!")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: /usr/local/lib/python3.10/site-packages/polars/polars.abi3.so: cannot allocate memory in static TLS block
>>> |
|
When I try to build Polars with zig, it fails to compile zlib-ng ❯ maturin build --zig
📦 Including license file "LICENSE"
🍹 Building a mixed python/rust project
🔗 Found pyo3 bindings with abi3 support for Python ≥ 3.7
🐍 Not using a specific python interpreter
Compiling cfg-if v1.0.0
Compiling scopeguard v1.1.0
Compiling once_cell v1.14.0
Compiling either v1.8.0
Compiling static_assertions v1.1.0
Compiling futures-sink v0.3.24
Compiling ryu v1.0.11
Compiling futures-io v0.3.24
Compiling pin-project-lite v0.2.9
Compiling smallvec v1.9.0
Compiling pin-utils v0.1.0
Compiling itoa v1.0.3
Compiling alloc-no-stdlib v2.0.3
Compiling rle-decode-fast v1.0.3
Compiling siphasher v0.3.10
Compiling adler v1.0.2
Compiling fallible-streaming-iterator v0.1.9
Compiling iana-time-zone v0.1.47
Compiling array-init-cursor v0.2.0
Compiling crc-catalog v1.1.1
Compiling adler32 v1.2.0
Compiling simdutf8 v0.1.4
Compiling rawpointer v0.2.1
Compiling ppv-lite86 v0.2.16
Compiling base64 v0.13.0
Compiling hash_hasher v2.0.3
Compiling foreign_vec v0.1.0
Compiling streaming-iterator v0.1.7
Compiling dyn-clone v1.0.9
Compiling strength_reduce v0.2.3
Compiling bitflags v1.3.2
Compiling strum v0.23.0
Compiling unicode-width v0.1.9
Compiling regex-syntax v0.6.27
Compiling byteorder v1.4.3
Compiling hex v0.4.3
Compiling unindent v0.1.10
Compiling glob v0.3.0
Compiling libc v0.2.132
Compiling crossbeam-utils v0.8.11
Compiling libm v0.2.5
Compiling rayon-core v1.9.3
Compiling serde v1.0.144
Compiling memchr v2.5.0
Compiling futures-core v0.3.24
Compiling futures-channel v0.3.24
Compiling futures-task v0.3.24
Compiling parking_lot_core v0.9.3
Compiling futures-util v0.3.24
Compiling crc32fast v1.3.2
Compiling serde_json v1.0.85
Compiling log v0.4.17
Compiling lexical-util v0.8.5
Compiling snap v1.0.5
Compiling signal-hook v0.3.14
Compiling libflate_lz77 v1.1.0
Compiling alloc-stdlib v0.2.1
Compiling miniz_oxide v0.5.4
Compiling planus v0.3.1
Compiling streaming-decompression v0.1.2
Compiling crc v2.1.0
Compiling matrixmultiply v0.3.2
Compiling anyhow v1.0.64
Compiling num-traits v0.2.15
Compiling memoffset v0.6.5
Compiling crossbeam-epoch v0.9.10
Compiling num-integer v0.1.45
Compiling slab v0.4.7
Compiling indexmap v1.9.1
Compiling lock_api v0.4.8
Compiling num-bigint v0.4.3
Compiling num-iter v0.1.43
Compiling num-rational v0.4.1
Compiling uncased v0.9.7
Compiling rayon v1.5.3
Compiling ahash v0.7.6
Compiling smartstring v1.0.1
Compiling fxhash v0.2.1
Compiling brotli-decompressor v2.3.2
Compiling lexical-write-integer v0.8.5
Compiling lexical-parse-integer v0.8.6
Compiling crossbeam-channel v0.5.6
Compiling libflate v1.2.0
Compiling aho-corasick v0.7.19
Compiling csv-core v0.1.10
Compiling phf_shared v0.11.1
Compiling getrandom v0.2.7
Compiling num_cpus v1.13.1
Compiling time v0.1.44
Compiling signal-hook-registry v1.4.0
Compiling mio v0.8.4
Compiling dirs-sys v0.3.7
Compiling memmap2 v0.5.7
Compiling lexical-write-float v0.8.5
Compiling lexical-parse-float v0.8.5
Compiling crossbeam-deque v0.8.2
Compiling phf v0.11.1
Compiling rand_core v0.6.3
Compiling parking_lot v0.12.1
Compiling brotli v3.3.4
Compiling dirs v4.0.0
Compiling async-stream v0.3.3
Compiling bytemuck v1.12.1
Compiling multiversion v0.6.1
Compiling thiserror v1.0.34
Compiling inventory v0.2.3
Compiling zstd-sys v2.0.1+zstd.1.5.2
Compiling lz4-sys v1.9.4
Compiling jemalloc-sys v0.5.1+5.3.0-patched
Compiling num-complex v0.4.2
Compiling float-cmp v0.9.0
Compiling regex v1.6.0
Compiling rand_chacha v0.3.1
Compiling lexical-core v0.8.5
Compiling signal-hook-mio v0.2.3
Compiling libz-ng-sys v1.1.8
Compiling pyo3-ffi v0.16.6
Compiling chrono v0.4.22
Compiling lexical v6.1.1
Compiling rand v0.8.5
Compiling crossterm v0.23.2
Compiling pyo3 v0.16.6
Compiling ndarray v0.15.6
Compiling comfy-table v5.0.1
Compiling rand_distr v0.4.3
Compiling chrono-tz v0.6.3
Compiling futures-executor v0.3.24
Compiling arrow-format v0.7.0
Compiling bincode v1.3.3
Compiling num v0.4.0
Compiling futures v0.3.24
Compiling parquet-format-safe v0.2.4
Compiling hashbrown v0.12.3
Compiling polars-utils v0.23.0 (/home/luna.kuleuven.be/u0079808/software/polars/polars/polars-utils)
Compiling halfbrown v0.1.15
Compiling json-deserializer v0.4.0
Compiling value-trait v0.4.0
Compiling numpy v0.16.2
Compiling avro-schema v0.3.0
Compiling simd-json v0.6.0
Compiling jsonpath_lib v0.3.0 (https://github.com/ritchie46/jsonpath?branch=improve_compiled#24eaf0b4)
Compiling lz4 v1.24.0
Compiling zstd-safe v5.0.2+zstd.1.5.2
Compiling zstd v0.11.2+zstd.1.5.2
error: failed to run custom build command for `libz-ng-sys v1.1.8`
Caused by:
process didn't exit successfully: `/home/luna.kuleuven.be/u0079808/software/polars/py-polars/target/debug/build/libz-ng-sys-568e90ca8c20e909/build-script-build_zng` (exit status: 101)
--- stdout
CMAKE_TOOLCHAIN_FILE_x86_64-unknown-linux-gnu = None
CMAKE_TOOLCHAIN_FILE_x86_64_unknown_linux_gnu = None
HOST_CMAKE_TOOLCHAIN_FILE = None
CMAKE_TOOLCHAIN_FILE = None
CMAKE_GENERATOR_x86_64-unknown-linux-gnu = None
CMAKE_GENERATOR_x86_64_unknown_linux_gnu = None
HOST_CMAKE_GENERATOR = None
CMAKE_GENERATOR = None
CMAKE_PREFIX_PATH_x86_64-unknown-linux-gnu = None
CMAKE_PREFIX_PATH_x86_64_unknown_linux_gnu = None
HOST_CMAKE_PREFIX_PATH = None
CMAKE_PREFIX_PATH = None
CMAKE_x86_64-unknown-linux-gnu = None
CMAKE_x86_64_unknown_linux_gnu = None
HOST_CMAKE = None
CMAKE = None
running: "cmake" "/home/luna.kuleuven.be/u0079808/.cargo/registry/src/git.luolix.top-1ecc6299db9ec823/libz-ng-sys-1.1.8/src/zlib-ng" "-DBUILD_SHARED_LIBS=OFF" "-DZLIB_COMPAT=OFF" "-DZLIB_ENABLE_TESTS=OFF" "-DWITH_GZFILEOP=ON" "-DCMAKE_INSTALL_PREFIX=/home/luna.kuleuven.be/u0079808/software/polars/py-polars/target/x86_64-unknown-linux-gnu/debug/build/libz-ng-sys-1f0805334c6c82d3/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_C_COMPILER=/home/luna.kuleuven.be/u0079808/.cache/cargo-zigbuild/0.12.0/zigcc-x86_64-unknown-linux-gnu.2.17.sh" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_CXX_COMPILER=/home/luna.kuleuven.be/u0079808/.cache/cargo-zigbuild/0.12.0/zigcxx-x86_64-unknown-linux-gnu.2.17.sh" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_ASM_COMPILER=/home/luna.kuleuven.be/u0079808/.cache/cargo-zigbuild/0.12.0/zigcc-x86_64-unknown-linux-gnu.2.17.sh" "-DCMAKE_BUILD_TYPE=Debug"
-- Using CMake version 3.16.3
-- ZLIB_HEADER_VERSION: 1.2.11
-- ZLIBNG_HEADER_VERSION: 2.1.0.devel
-- The C compiler identification is Clang 13.0.1
-- Check for working C compiler: /home/luna.kuleuven.be/u0079808/.cache/cargo-zigbuild/0.12.0/zigcc-x86_64-unknown-linux-gnu.2.17.sh
-- Check for working C compiler: /home/luna.kuleuven.be/u0079808/.cache/cargo-zigbuild/0.12.0/zigcc-x86_64-unknown-linux-gnu.2.17.sh -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Arch detected: 'x86_64'
-- Basearch of 'x86_64' has been detected as: 'x86'
-- Performing Test FNO_LTO_AVAILABLE
-- Performing Test FNO_LTO_AVAILABLE - Success
-- Looking for sys/auxv.h
-- Looking for sys/auxv.h - found
-- Looking for sys/sdt.h
-- Looking for sys/sdt.h - not found
-- Looking for unistd.h
-- Looking for unistd.h - found
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of off64_t
-- Check size of off64_t - done
-- Looking for fseeko
-- Looking for fseeko - found
-- Looking for strerror
-- Looking for strerror - found
-- Looking for posix_memalign
-- Looking for posix_memalign - found
-- Performing Test HAVE_NO_INTERPOSITION
-- Performing Test HAVE_NO_INTERPOSITION - Success
-- Performing Test HAVE_ATTRIBUTE_VISIBILITY_HIDDEN
-- Performing Test HAVE_ATTRIBUTE_VISIBILITY_HIDDEN - Success
-- Performing Test HAVE_ATTRIBUTE_VISIBILITY_INTERNAL
-- Performing Test HAVE_ATTRIBUTE_VISIBILITY_INTERNAL - Success
-- Performing Test HAVE_BUILTIN_CTZ
-- Performing Test HAVE_BUILTIN_CTZ - Success
-- Performing Test HAVE_BUILTIN_CTZLL
-- Performing Test HAVE_BUILTIN_CTZLL - Success
-- Performing Test HAVE_PTRDIFF_T
-- Performing Test HAVE_PTRDIFF_T - Success
-- Performing Test HAVE_AVX2_INTRIN
-- Performing Test HAVE_AVX2_INTRIN - Success
-- Performing Test HAVE_AVX512_INTRIN
-- Performing Test HAVE_AVX512_INTRIN - Success
-- Performing Test HAVE_MASK_INTRIN
-- Performing Test HAVE_MASK_INTRIN - Success
-- Performing Test HAVE_AVX512VNNI_INTRIN
-- Performing Test HAVE_AVX512VNNI_INTRIN - Success
-- Performing Test HAVE_SSE41_INTRIN
-- Performing Test HAVE_SSE41_INTRIN - Success
-- Performing Test HAVE_SSE42CRC_INLINE_ASM
-- Performing Test HAVE_SSE42CRC_INLINE_ASM - Success
-- Performing Test HAVE_SSE42CRC_INTRIN
-- Performing Test HAVE_SSE42CRC_INTRIN - Success
-- Performing Test HAVE_SSE42CMPSTR_INTRIN
-- Performing Test HAVE_SSE42CMPSTR_INTRIN - Success
-- Performing Test HAVE_SSE2_INTRIN
-- Performing Test HAVE_SSE2_INTRIN - Success
-- Performing Test HAVE_SSSE3_INTRIN
-- Performing Test HAVE_SSSE3_INTRIN - Success
-- Performing Test HAVE_PCLMULQDQ_INTRIN
-- Performing Test HAVE_PCLMULQDQ_INTRIN - Success
-- Performing Test HAVE_VPCLMULQDQ_INTRIN
-- Performing Test HAVE_VPCLMULQDQ_INTRIN - Failed
-- Architecture-specific source files: arch/x86/x86_features.c;arch/x86/slide_hash_avx2.c;arch/x86/chunkset_avx.c;arch/x86/compare256_avx2.c;arch/x86/adler32_avx2.c;arch/x86/adler32_avx512.c;arch/x86/adler32_avx512_vnni.c;arch/x86/chunkset_sse41.c;arch/x86/adler32_sse42.c;arch/x86/insert_string_sse42.c;arch/x86/chunkset_sse2.c;arch/x86/compare256_sse2.c;arch/x86/slide_hash_sse2.c;arch/x86/adler32_ssse3.c;arch/x86/crc32_fold_pclmulqdq.c
-- The following features have been enabled:
* CMAKE_BUILD_TYPE, Build type: Debug (selected)
* AVX2_SLIDEHASH, Support AVX2 optimized slide_hash, using "-mavx2"
* AVX_CHUNKSET, Support AVX optimized chunkset, using "-mavx2"
* AVX2_COMPARE256, Support AVX2 optimized compare256, using "-mavx2"
* AVX2_ADLER32, Support AVX2-accelerated adler32, using "-mavx2"
* AVX512_ADLER32, Support AVX512-accelerated adler32, using "-mavx512f -mavx512dq -mavx512bw -mavx512vl -mtune=cascadelake"
* AVX512VNNI_ADLER32, Support AVX512VNNI adler32, using "-mavx512f -mavx512dq -mavx512bw -mavx512vl -mavx512vnni -mtune=cascadelake"
* SSE42_CRC, Support SSE4.2 optimized CRC hash generation, using "-msse4.2"
* SSSE3_ADLER32, Support SSSE3-accelerated adler32, using "-mssse3"
* PCLMUL_CRC, Support CRC hash generation using PCLMULQDQ, using "-mssse3 -msse4.2 -mpclmul"
* WITH_GZFILEOP, Compile with support for gzFile related functions
* WITH_SANITIZER, Enable sanitizer support
* WITH_OPTIM, Build with optimisation
* WITH_NEW_STRATEGIES, Use new strategies
* WITH_AVX2, Build with AVX2
* WITH_AVX512, Build with AVX512
* WITH_AVX512VNNI, Build with AVX512 VNNI
* WITH_SSE2, Build with SSE2
* WITH_SSSE3, Build with SSSE3
* WITH_SSE41, Build with SSE41
* WITH_SSE42, Build with SSE42
* WITH_PCLMULQDQ, Build with PCLMULQDQ
-- The following features have been disabled:
* FORCE_TZCNT, Assume CPU is TZCNT capable
* ZLIB_SYMBOL_PREFIX, Publicly exported symbols DO NOT have a custom prefix
* ZLIB_COMPAT, Compile with zlib compatible API
* ZLIB_ENABLE_TESTS, Build test binaries
* ZLIB_DUAL_LINK, Dual link tests against system zlib
* WITH_FUZZERS, Build test/fuzz
* WITH_BENCHMARKS, Build test/benchmarks
* WITH_BENCHMARK_APPS, Build application benchmarks
* WITH_NATIVE_INSTRUCTIONS, Instruct the compiler to use the full instruction set on this host (gcc/clang -march=native)
* WITH_MAINTAINER_WARNINGS, Build with project maintainer warnings
* WITH_CODE_COVERAGE, Enable code coverage reporting
* WITH_INFLATE_STRICT, Build with strict inflate distance checking
* WITH_INFLATE_ALLOW_INVALID_DIST, Build with zero fill for inflate invalid distances
* WITH_VPCLMULQDQ, Build with VPCLMULQDQ
* INSTALL_UTILS, Copy minigzip and minideflate during install
-- Configuring done
-- Generating done
-- Build files have been written to: /home/luna.kuleuven.be/u0079808/software/polars/py-polars/target/x86_64-unknown-linux-gnu/debug/build/libz-ng-sys-1f0805334c6c82d3/out/build
running: "cmake" "--build" "." "--target" "install" "--config" "Debug" "--parallel" "12"
Scanning dependencies of target zlib
[ 2%] Building C object CMakeFiles/zlib.dir/adler32.c.o
[ 4%] Building C object CMakeFiles/zlib.dir/adler32_fold.c.o
[ 6%] Building C object CMakeFiles/zlib.dir/chunkset.c.o
[ 8%] Building C object CMakeFiles/zlib.dir/compare256.c.o
[ 10%] Building C object CMakeFiles/zlib.dir/compress.c.o
[ 12%] Building C object CMakeFiles/zlib.dir/crc32_braid.c.o
[ 14%] Building C object CMakeFiles/zlib.dir/crc32_braid_comb.c.o
[ 17%] Building C object CMakeFiles/zlib.dir/cpu_features.c.o
[ 19%] Building C object CMakeFiles/zlib.dir/crc32_fold.c.o
[ 21%] Building C object CMakeFiles/zlib.dir/deflate.c.o
[ 23%] Building C object CMakeFiles/zlib.dir/deflate_fast.c.o
[ 25%] Building C object CMakeFiles/zlib.dir/deflate_huff.c.o
[ 27%] Building C object CMakeFiles/zlib.dir/deflate_medium.c.o
[ 29%] Building C object CMakeFiles/zlib.dir/deflate_quick.c.o
[ 31%] Building C object CMakeFiles/zlib.dir/deflate_rle.c.o
[ 34%] Building C object CMakeFiles/zlib.dir/deflate_slow.c.o
[ 36%] Building C object CMakeFiles/zlib.dir/deflate_stored.c.o
[ 38%] Building C object CMakeFiles/zlib.dir/functable.c.o
[ 40%] Building C object CMakeFiles/zlib.dir/infback.c.o
[ 42%] Building C object CMakeFiles/zlib.dir/inffast.c.o
[ 44%] Building C object CMakeFiles/zlib.dir/inflate.c.o
[ 46%] Building C object CMakeFiles/zlib.dir/inftrees.c.o
[ 48%] Building C object CMakeFiles/zlib.dir/insert_string.c.o
[ 51%] Building C object CMakeFiles/zlib.dir/slide_hash.c.o
[ 53%] Building C object CMakeFiles/zlib.dir/insert_string_roll.c.o
[ 55%] Building C object CMakeFiles/zlib.dir/trees.c.o
[ 57%] Building C object CMakeFiles/zlib.dir/uncompr.c.o
[ 59%] Building C object CMakeFiles/zlib.dir/zutil.c.o
[ 61%] Building C object CMakeFiles/zlib.dir/arch/x86/x86_features.c.o
[ 63%] Building C object CMakeFiles/zlib.dir/arch/x86/slide_hash_avx2.c.o
[ 65%] Building C object CMakeFiles/zlib.dir/arch/x86/chunkset_avx.c.o
[ 68%] Building C object CMakeFiles/zlib.dir/arch/x86/compare256_avx2.c.o
[ 70%] Building C object CMakeFiles/zlib.dir/arch/x86/adler32_avx2.c.o
[ 72%] Building C object CMakeFiles/zlib.dir/arch/x86/adler32_avx512.c.o
[ 74%] Building C object CMakeFiles/zlib.dir/arch/x86/adler32_avx512_vnni.c.o
[ 76%] Building C object CMakeFiles/zlib.dir/arch/x86/chunkset_sse41.c.o
[ 78%] Building C object CMakeFiles/zlib.dir/arch/x86/adler32_sse42.c.o
[ 80%] Building C object CMakeFiles/zlib.dir/arch/x86/insert_string_sse42.c.o
[ 82%] Building C object CMakeFiles/zlib.dir/arch/x86/chunkset_sse2.c.o
[ 85%] Building C object CMakeFiles/zlib.dir/arch/x86/compare256_sse2.c.o
[ 87%] Building C object CMakeFiles/zlib.dir/arch/x86/slide_hash_sse2.c.o
[ 89%] Building C object CMakeFiles/zlib.dir/arch/x86/adler32_ssse3.c.o
[ 91%] Building C object CMakeFiles/zlib.dir/arch/x86/crc32_fold_pclmulqdq.c.o
[ 93%] Building C object CMakeFiles/zlib.dir/gzread.c.o
[ 95%] Building C object CMakeFiles/zlib.dir/gzlib.c.o
[ 97%] Building C object CMakeFiles/zlib.dir/gzwrite.c.o
[100%] Linking C static library libz-ng.a
--- stderr
CMake Warning:
Manually-specified variables were not used by the project:
CMAKE_ASM_COMPILER
CMAKE_ASM_FLAGS
CMAKE_CXX_COMPILER
CMAKE_CXX_FLAGS
make: warning: -j12 forced in submake: resetting jobserver mode.
error: Found argument 'qc' which wasn't expected, or isn't valid in this context
USAGE:
ar <SUBCOMMAND>
For more information try --help
make[2]: *** [CMakeFiles/zlib.dir/build.make:760: libz-ng.a] Error 2
make[1]: *** [CMakeFiles/Makefile2:76: CMakeFiles/zlib.dir/all] Error 2
make: *** [Makefile:130: all] Error 2
thread 'main' panicked at '
command did not execute successfully, got: exit status: 2
build script failed, must exit now', /home/luna.kuleuven.be/u0079808/.cargo/registry/src/github.com-1ecc6299db9ec823/cmake-0.1.48/src/lib.rs:975:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
💥 maturin failed
Caused by: Failed to build a native library through cargo
Caused by: Cargo build finished with "exit status: 101": `"cargo" "rustc" "--target" "x86_64-unknown-linux-gnu" "--message-format" "json" "--lib"`With manually running So it looks to me that maturin Edit: This was still with maturin 0.13.2 not 0.13.3. |
|
@ghuls Can you try maturin 0.13.3 and aarch64-unknown-linux-gnu target? |
Is there an easy way to run aarch64 binaries on x86_64 with python installed? |
|
You can use Docker: Full command: |
|
To run docker containers with arm64 binaries on sudo apt-get install qemu binfmt-support qemu-user-static # Install the qemu packages
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes # This step will execute the registering scriptshttps://www.stereolabs.com/docs/docker/building-arm-container-on-x86/ I got Polars running for arm64 by builidng diff --git a/py-polars/Cargo.toml b/py-polars/Cargo.toml
index aaf29df69..597183e9e 100644
--- a/py-polars/Cargo.toml
+++ b/py-polars/Cargo.toml
@@ -19,7 +19,8 @@ description = "Blazingly fast DataFrame library"
mimalloc = { version = "*", default-features = false }
[target.'cfg(target_os="linux")'.dependencies]
-jemallocator = { version = "0.5", features = ["disable_initial_exec_tls"] }
+#jemallocator = { version = "0.5", features = ["disable_initial_exec_tls"] }
+mimalloc = { version = "*", default-features = false }
[dependencies]
ahash = "0.7"
diff --git a/py-polars/src/lib.rs b/py-polars/src/lib.rs
index 1159339a6..e25d49e41 100644
--- a/py-polars/src/lib.rs
+++ b/py-polars/src/lib.rs
@@ -21,7 +21,8 @@ mod set;
pub mod utils;
#[cfg(target_os = "linux")]
-use jemallocator::Jemalloc;
+//use jemallocator::Jemalloc;
+use mimalloc::MiMalloc;
use lazy::ToExprs;
#[cfg(not(target_os = "linux"))]
use mimalloc::MiMalloc;
@@ -53,7 +54,8 @@ use crate::series::PySeries;
#[global_allocator]
#[cfg(target_os = "linux")]
-static ALLOC: Jemalloc = Jemalloc;
+//static ALLOC: Jemalloc = Jemalloc;
+static ALLOC: MiMalloc = MiMalloc;
#[global_allocator]
#[cfg(not(target_os = "linuxHere is a similar issue: We already have jemallocator with It could also be that it only happens under qemu emulation as I also managed to get a jemalloc warning while trying to get it to work: root@238e1e9be22f:/# LD_PRELOAD=/usr/local/lib/python3.10/site-packages/polars/polars.abi3.so python
<jemalloc>: MADV_DONTNEED does not work (memset will be used instead)
<jemalloc>: (This is the expected behaviour if you are running under QEMU)
Python 3.10.7 (main, Sep 13 2022, 16:13:35) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import polars.polars
qemu: uncaught target signal 5 (Trace/breakpoint trap) - core dumped
Trace/breakpoint trap (core dumped)
root@238e1e9be22f:/# ldd /usr/local/lib/python3.10/site-packages/polars/polars.abi3.so
libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000005509cd6000)
libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000005509d83000)
libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000005509db4000)
libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000005509f29000)
/lib/ld-linux-aarch64.so.1 (0x0000005500000000) |
|
It's the same error on native arm64 container root@fd07873e3328:/io# python3
Python 3.10.4 (main, Jun 29 2022, 12:14:53) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import polars.polars
/usr/local/lib/python3.10/dist-packages/polars/__init__.py:11: UserWarning: polars binary missing!
warnings.warn("polars binary missing!")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: /usr/local/lib/python3.10/dist-packages/polars/polars.abi3.so: cannot allocate memory in static TLS block
>>> exit()
root@fd07873e3328:/io# ldd /usr/local/lib/python3.10/dist-packages/polars/polars.abi3.so
linux-vdso.so.1 (0x0000ffff9c1db000)
libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000ffff99f40000)
libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000ffff99f20000)
libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000ffff99d70000)
libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000ffff99d50000)
/lib/ld-linux-aarch64.so.1 (0x0000ffff9c1a2000)
root@fd07873e3328:/io# uname -a
Linux fd07873e3328 5.15.0-47-generic #51-Ubuntu SMP Fri Aug 12 08:18:32 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux |
|
@messense and with mimalloc as allocator? |
|
mimalloc works fine root@bfc5a3974df3:/io# python3
Python 3.10.4 (main, Jun 29 2022, 12:14:53) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import polars.polars
>>> polars.polars
<module 'polars.polars' from '/usr/local/lib/python3.10/dist-packages/polars/polars.abi3.so'>
>>> exit() |
It also works with jemalloc if we root@9beb31be6257:/io# LD_PRELOAD=/usr/local/lib/python3.10/dist-packages/polars/polars.abi3.so python3
Python 3.10.4 (main, Jun 29 2022, 12:14:53) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import polars.polars
>>> polars.polars
<module 'polars.polars' from '/usr/local/lib/python3.10/dist-packages/polars/polars.abi3.so'> |
|
So the latter is with Jemalloc as allocator? Do you compile polars on aarch64 itself or is it a cross-compile? |
Yes.
Cross compiled from a x86_64 Linux. The build log shows that |
3 tasks
2 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.