Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

0.5.4 fails to install on m1 macbook (2021) #536

Closed
sa- opened this issue Jan 8, 2022 · 3 comments · Fixed by #541
Closed

0.5.4 fails to install on m1 macbook (2021) #536

sa- opened this issue Jan 8, 2022 · 3 comments · Fixed by #541
Labels
bug Something isn't working good first issue Good for newcomers help wanted Extra attention is needed

Comments

@sa-
Copy link
Contributor

sa- commented Jan 8, 2022

Environment

Delta-rs version: 0.5.3

Binding: python

Environment:

  • Cloud provider:
  • OS: macos monterey
  • Other:

Bug

▶ python --version
Python 3.8.12

▶ pip install deltalake
Collecting deltalake
  Using cached deltalake-0.5.4.tar.gz (4.3 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting pyarrow>=4
  Using cached pyarrow-6.0.1-cp38-cp38-macosx_11_0_arm64.whl (13.7 MB)
Requirement already satisfied: numpy>=1.16.6 in ./.pyenv/versions/3.8.12/lib/python3.8/site-packages (from pyarrow>=4->deltalake) (1.22.0)
Building wheels for collected packages: deltalake
  Building wheel for deltalake (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/sa/.pyenv/versions/3.8.12/bin/python3.8 /Users/sa/.pyenv/versions/3.8.12/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /var/folders/hh/2s1wcfgd29x8dkcv39jdwm580000gn/T/tmppwe0zwz6
       cwd: /private/var/folders/hh/2s1wcfgd29x8dkcv39jdwm580000gn/T/pip-install-9tskg1dr/deltalake_326893990f544bbcab6cb6037b482153
  Complete output (255 lines):
  Running `maturin pep517 build-wheel -i /Users/sa/.pyenv/versions/3.8.12/bin/python3.8`
     Compiling libc v0.2.112
     Compiling proc-macro2 v1.0.36
     Compiling unicode-xid v0.2.2
     Compiling syn v1.0.85
     Compiling autocfg v1.0.1
     Compiling cfg-if v1.0.0
     Compiling memchr v2.4.1
     Compiling log v0.4.14
     Compiling version_check v0.9.4
     Compiling serde_derive v1.0.133
     Compiling serde v1.0.133
     Compiling pin-project-lite v0.2.8
     Compiling futures-core v0.3.19
     Compiling bytes v1.1.0
     Compiling lazy_static v1.4.0
     Compiling itoa v1.0.1
     Compiling hashbrown v0.11.2
     Compiling futures-sink v0.3.19
     Compiling futures-channel v0.3.19
     Compiling itoa v0.4.8
     Compiling once_cell v1.9.0
     Compiling futures-task v0.3.19
     Compiling fnv v1.0.7
     Compiling futures-util v0.3.19
     Compiling pin-utils v0.1.0
     Compiling futures-io v0.3.19
     Compiling ryu v1.0.9
     Compiling slab v0.4.5
     Compiling typenum v1.15.0
     Compiling bitflags v1.3.2
     Compiling core-foundation-sys v0.8.3
     Compiling ppv-lite86 v0.2.16
     Compiling serde_json v1.0.74
     Compiling httparse v1.5.1
     Compiling percent-encoding v2.1.0
     Compiling try-lock v0.2.3
     Compiling proc-macro-hack v0.5.19
     Compiling base64 v0.13.0
     Compiling tower-service v0.3.1
     Compiling httpdate v1.0.2
     Compiling native-tls v0.2.8
     Compiling remove_dir_all v0.5.3
     Compiling matches v0.1.9
     Compiling tinyvec_macros v0.1.0
     Compiling untrusted v0.7.1
     Compiling async-trait v0.1.52
     Compiling opaque-debug v0.3.0
     Compiling unicode-bidi v0.3.7
     Compiling static_assertions v1.1.0
     Compiling crc32fast v1.3.0
     Compiling semver-parser v0.7.0
     Compiling const_fn v0.4.9
     Compiling hex v0.4.3
     Compiling semver v1.0.4
     Compiling subtle v2.4.1
     Compiling zeroize v1.4.3
     Compiling shlex v0.1.1
     Compiling xml-rs v0.8.4
     Compiling encoding_rs v0.8.30
     Compiling smallvec v1.7.0
     Compiling md5 v0.7.0
     Compiling pyo3-build-config v0.14.5
     Compiling mime v0.3.16
     Compiling parking_lot_core v0.8.5
     Compiling ipnet v2.3.1
     Compiling zstd-safe v4.1.3+zstd.1.5.1
     Compiling alloc-no-stdlib v2.0.3
     Compiling regex-syntax v0.6.25
     Compiling scopeguard v1.1.0
     Compiling regex-automata v0.1.10
     Compiling async-timer v1.0.0-beta.8
     Compiling integer-encoding v1.1.7
     Compiling byteorder v1.4.3
     Compiling adler v1.0.2
     Compiling snap v1.0.5
     Compiling dyn-clone v1.0.4
     Compiling str-buf v1.0.5
     Compiling termcolor v1.1.2
     Compiling unindent v0.1.7
     Compiling parquet v6.5.0
     Compiling anyhow v1.0.52
     Compiling dtoa v0.4.8
     Compiling humantime v2.1.0
     Compiling RustyXML v0.3.0
     Compiling strsim v0.10.0
     Compiling textwrap v0.14.2
     Compiling maplit v1.0.2
     Compiling instant v0.1.12
     Compiling generic-array v0.14.5
     Compiling standback v0.2.17
     Compiling time v0.2.27
     Compiling num-traits v0.2.14
     Compiling indexmap v1.8.0
     Compiling num-integer v0.1.44
     Compiling num-bigint v0.4.3
     Compiling num-rational v0.4.0
     Compiling num-iter v0.1.42
     Compiling miniz_oxide v0.4.4
     Compiling tracing-core v0.1.21
     Compiling http v0.2.6
     Compiling form_urlencoded v1.0.1
     Compiling rustls-pemfile v0.2.1
     Compiling tinyvec v1.5.1
     Compiling lexical-util v0.8.1
     Compiling twox-hash v1.6.2
     Compiling semver v0.9.0
     Compiling alloc-stdlib v0.2.1
     Compiling lock_api v0.4.5
     Compiling tracing v0.1.29
     Compiling http-body v0.4.4
     Compiling lexical-parse-integer v0.8.0
     Compiling lexical-write-integer v0.8.0
     Compiling rustc_version v0.2.3
     Compiling unicode-normalization v0.1.19
     Compiling brotli-decompressor v2.3.2
     Compiling want v0.3.0
     Compiling aho-corasick v0.7.18
     Compiling csv-core v0.1.10
     Compiling os_str_bytes v6.0.0
     Compiling num_cpus v1.13.1
     Compiling mio v0.7.14
     Compiling signal-hook-registry v1.4.0
     Compiling getrandom v0.2.3
     Compiling time v0.1.43
     Compiling socket2 v0.4.2
     Compiling cpufeatures v0.2.1
     Compiling dirs-sys-next v0.1.2
     Compiling atty v0.2.14
     Compiling error-code v2.3.0
     Compiling errno v0.2.8
     Compiling jobserver v0.1.24
     Compiling quote v1.0.14
     Compiling core-foundation v0.9.2
     Compiling security-framework-sys v2.4.2
     Compiling lexical-write-float v0.8.2
     Compiling lexical-parse-float v0.8.2
     Compiling rusoto_core v0.46.0
     Compiling rustc_version v0.4.0
     Compiling idna v0.2.3
     Compiling paste-impl v0.1.18
     Compiling regex v1.5.4
     Compiling brotli v3.3.2
     Compiling threadpool v1.8.1
     Compiling rand_core v0.6.3
     Compiling pyo3 v0.14.5
     Compiling dirs-next v2.0.0
     Compiling parking_lot v0.11.2
     Compiling cc v1.0.72
     Compiling security-framework v2.4.2
     Compiling ordered-float v1.1.1
     Compiling num-complex v0.4.0
     Compiling clap v3.0.5
     Compiling lexical-core v0.8.2
     Compiling digest v0.9.0
     Compiling block-buffer v0.9.0
     Compiling crypto-mac v0.10.1
     Compiling azure_core v0.1.0 (https://github.com/Azure/azure-sdk-for-rust?rev=2f2caff49b3e1ba08f430f82e16994670dddda95#2f2caff4)
     Compiling flate2 v1.0.22
     Compiling rand_chacha v0.3.1
     Compiling env_logger v0.9.0
     Compiling paste v0.1.18
     Compiling glibc_version v0.1.2
     Compiling ring v0.16.20
     Compiling zstd-sys v1.6.2+zstd.1.5.1
     Compiling lz4-sys v1.9.2
     Compiling thrift v0.13.0
     Compiling sha2 v0.9.9
     Compiling hmac v0.10.1
     Compiling rand v0.8.4
     Compiling deltalake v0.4.1 (/private/var/folders/hh/2s1wcfgd29x8dkcv39jdwm580000gn/T/pip-install-9tskg1dr/deltalake_326893990f544bbcab6cb6037b482153/local_dependencies/deltalake)
     Compiling pyo3-macros-backend v0.14.5
     Compiling parquet-format v2.6.1
     Compiling tempfile v3.2.0
     Compiling tokio-macros v1.7.0
     Compiling futures-macro v0.3.19
     Compiling thiserror-impl v1.0.30
     Compiling time-macros-impl v0.1.2
     Compiling multiversion-macros v0.6.1
     Compiling async-stream-impl v0.3.2
     Compiling indoc-impl v0.3.6
     Compiling num v0.4.0
     Compiling pyo3-macros v0.14.5
     Compiling tokio v1.15.0
     Compiling time-macros v0.1.1
     Compiling thiserror v1.0.30
     Compiling indoc v0.3.6
     Compiling async-stream v0.3.2
     Compiling multiversion v0.6.1
     Compiling flatbuffers v2.0.0
     Compiling futures-executor v0.3.19
     Compiling webpki v0.22.0
     Compiling sct v0.7.0
     Compiling futures v0.3.19
     Compiling rustls v0.20.2
     Compiling webpki-roots v0.22.2
     Compiling tokio-util v0.6.9
     Compiling tokio-native-tls v0.3.0
     Compiling tokio-stream v0.1.8
     Compiling lz4 v1.23.2
     Compiling chrono v0.4.19
     Compiling url v2.2.2
     Compiling serde_urlencoded v0.7.0
     Compiling serde_path_to_error v0.1.6
     Compiling uuid v0.8.2
     Compiling bstr v0.2.17
     Compiling serde-xml-rs v0.4.1
     Compiling h2 v0.3.10
     Compiling tokio-rustls v0.23.2
     Compiling serde_urlencoded v0.6.1
     Compiling tame-oauth v0.4.7
     Compiling tame-gcs v0.10.0
     Compiling csv v1.1.6
     Compiling arrow v6.5.0
     Compiling hyper v0.14.16
     Compiling zstd v0.9.2+zstd.1.5.1
     Compiling hyper-tls v0.5.0
     Compiling rusoto_credential v0.46.0
     Compiling hyper-rustls v0.23.0
     Compiling rusoto_signature v0.46.0
     Compiling reqwest v0.11.8
     Compiling oauth2 v4.1.0
     Compiling rusoto_s3 v0.46.0
     Compiling rusoto_glue v0.46.0
     Compiling rusoto_dynamodb v0.46.0
     Compiling rusoto_sts v0.46.0
     Compiling azure_identity v0.1.0 (https://github.com/Azure/azure-sdk-for-rust?rev=2f2caff49b3e1ba08f430f82e16994670dddda95#2f2caff4)
     Compiling azure_storage v0.1.0 (https://github.com/Azure/azure-sdk-for-rust?rev=2f2caff49b3e1ba08f430f82e16994670dddda95#2f2caff4)
  error: could not compile `deltalake` due to 2 previous errors
  warning: build failed, waiting for other jobs to finish...
  error: build failed
  💥 maturin failed
    Caused by: Failed to build a native library through cargo
    Caused by: Cargo build finished with "exit status: 101": `cargo rustc --message-format json --manifest-path Cargo.toml --release --lib -- -C link-arg=-undefined -C link-arg=dynamic_lookup -C link-args=-Wl,-install_name,@rpath/deltalake.abi3.so`
  🍹 Building a mixed python/rust project
  🔗 Found pyo3 bindings with abi3 support for Python ≥ 3.6
  🐍 Not using a specific python interpreter (With abi3, an interpreter is only required on windows)
  error: field is never read: `root`
    --> local_dependencies/deltalake/src/storage/file/mod.rs:34:5
     |
  34 |     root: String,
     |     ^^^^^^^^^^^^
     |
  note: the lint level is defined here
    --> local_dependencies/deltalake/src/lib.rs:45:9
     |
  45 | #![deny(warnings)]
     |         ^^^^^^^^
     = note: `#[deny(dead_code)]` implied by `#[deny(warnings)]`


  error: aborting due to previous error


  Error: command ['maturin', 'pep517', 'build-wheel', '-i', '/Users/sa/.pyenv/versions/3.8.12/bin/python3.8'] returned non-zero exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for deltalake
Failed to build deltalake
ERROR: Could not build wheels for deltalake which use PEP 517 and cannot be installed directly
@sa- sa- added the bug Something isn't working label Jan 8, 2022
@houqp
Copy link
Member

houqp commented Jan 8, 2022

I think this is because we are only have binary releases for macos_10_07 uploaded to https://pypi.org/project/deltalake/#files. We need to update our python release pipeline to also include macosx_11_0 so users won't try to build it from source.

@sa- can you try to build it with the latest master branch and see if it works for monterey?

@houqp
Copy link
Member

houqp commented Jan 8, 2022

As for the proper fix, I think we can just add macos-11 to the list at

delta-rs/.github/workflows/python_release.yml

Line 33 in 3c83a1d

- macOS-10.15

@houqp houqp added good first issue Good for newcomers help wanted Extra attention is needed labels Jan 8, 2022
@sa-
Copy link
Contributor Author

sa- commented Jan 10, 2022

Deleted some irrelevant comments.

It works.

I ran make develop in the python folder, and voilà

>>> from deltalake import DeltaTable
>>>

@sa- sa- mentioned this issue Jan 10, 2022
Merged
This was referenced Dec 23, 2024
Closed
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

add macOS 11 sa-/delta-rs
2 participants
@houqp @sa-