Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

r-arrow 9.0 is broken on Linux #56

Closed
1 task done
jtilly opened this issue Sep 26, 2022 · 6 comments
Closed
1 task done

r-arrow 9.0 is broken on Linux #56

jtilly opened this issue Sep 26, 2022 · 6 comments
Labels
bug Something isn't working

Comments

@jtilly
Copy link

jtilly commented Sep 26, 2022

Solution to issue cannot be found in the documentation.

  • I checked the documentation.

Issue

Using the latest version from conda-forge on linux (Ubuntu 22.04.1):

mamba create -y -n r-arrow9 r-arrow
conda activate r-arrow9
R -e "library('arrow'); data(mtcars); write_parquet(mtcars, 'test.parquet')"

results in

R -e "library('arrow'); data(mtcars); write_parquet(mtcars, 'test.parquet')"

R version 4.1.3 (2022-03-10) -- "One Push-Up"
Copyright (C) 2022 The R Foundation for Statistical Computing
Platform: x86_64-conda-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library('arrow'); data(mtcars); write_parquet(mtcars, 'test.parquet')
Some features are not enabled in this build of Arrow. Run `arrow_info()` for more information.

Attaching package: ‘arrow’

The following object is masked from ‘package:utils’:

    timestamp

Error in parquet___WriterProperties___Builder__create() : 
  Cannot call parquet___WriterProperties___Builder__create(). See https://arrow.apache.org/docs/r/articles/install.html for help installing Arrow C++ libraries. 
Calls: write_parquet ... <Anonymous> -> parquet___WriterProperties___Builder__create
Execution halted

Things work fine with r-arrow<9.

Curiously, when I build r-arrow locally using

conda mambabuild -m .ci_support/linux_64_r_base4.1.yaml recipe/meta.yaml

and install the resulting tarball, things also just work fine.

Also, this is a conda packaging issue. When I install arrow from the binaries that are provided by rstudio, things work fine:

options(
  HTTPUserAgent =
    sprintf(
      "R/%s R (%s)",
      getRversion(),
      paste(getRversion(), R.version["platform"], R.version["arch"], R.version["os"])
    )
)

install.packages("arrow", repos = "https://packagemanager.rstudio.com/all/__linux__/focal/latest")
library('arrow'); data(mtcars); write_parquet(mtcars, 'test.parquet')

Installed packages

# packages in environment at /opt/conda/envs/r-arrow9:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
_r-mutex                  1.0.1               anacondar_1    conda-forge
arrow-cpp                 9.0.0           py310h028ac91_6_cpu    conda-forge
aws-c-cal                 0.5.11               h95a6274_0    conda-forge
aws-c-common              0.6.2                h7f98852_0    conda-forge
aws-c-event-stream        0.2.7               h3541f99_13    conda-forge
aws-c-io                  0.10.5               hfb6a706_0    conda-forge
aws-checksums             0.1.11               ha31a3da_7    conda-forge
aws-sdk-cpp               1.8.186              hb4091e7_3    conda-forge
binutils_impl_linux-64    2.36.1               h193b22a_2    conda-forge
bwidget                   1.9.14               ha770c72_1    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.18.1               h7f98852_0    conda-forge
ca-certificates           2022.9.24            ha878542_0    conda-forge
cairo                     1.16.0            ha61ee94_1014    conda-forge
curl                      7.83.1               h7bff187_0    conda-forge
expat                     2.4.9                h27087fc_0    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.14.0               hc2a2eb6_1    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
freetype                  2.12.1               hca18f0e_0    conda-forge
fribidi                   1.0.10               h36c2ea0_0    conda-forge
gcc_impl_linux-64         12.1.0              hea43390_16    conda-forge
gettext                   0.19.8.1          h27087fc_1009    conda-forge
gflags                    2.2.2             he1b5a44_1004    conda-forge
gfortran_impl_linux-64    12.1.0              h1db8e46_16    conda-forge
glog                      0.6.0                h6f12383_0    conda-forge
graphite2                 1.3.13            h58526e2_1001    conda-forge
grpc-cpp                  1.47.1               h05bd8bd_6    conda-forge
gsl                       2.7                  he838d99_0    conda-forge
gxx_impl_linux-64         12.1.0              hea43390_16    conda-forge
harfbuzz                  5.2.0                hf9f4e7c_0    conda-forge
icu                       70.1                 h27087fc_0    conda-forge
jpeg                      9e                   h166bdaf_2    conda-forge
kernel-headers_linux-64   2.6.32              he073ed8_15    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
krb5                      1.19.3               h3790be6_0    conda-forge
ld_impl_linux-64          2.36.1               hea4e1c9_2    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libabseil                 20220623.0      cxx17_h48a1fff_4    conda-forge
libblas                   3.9.0           16_linux64_openblas    conda-forge
libbrotlicommon           1.0.9                h166bdaf_7    conda-forge
libbrotlidec              1.0.9                h166bdaf_7    conda-forge
libbrotlienc              1.0.9                h166bdaf_7    conda-forge
libcblas                  3.9.0           16_linux64_openblas    conda-forge
libcrc32c                 1.1.2                h9c3ff4c_0    conda-forge
libcurl                   7.83.1               h7bff187_0    conda-forge
libdeflate                1.14                 h166bdaf_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libevent                  2.1.10               h9b69904_4    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-devel_linux-64     12.1.0              h1ec3361_16    conda-forge
libgcc-ng                 12.1.0              h8d9b700_16    conda-forge
libgfortran-ng            12.1.0              h69a702a_16    conda-forge
libgfortran5              12.1.0              hdcd56e2_16    conda-forge
libglib                   2.72.1               h2d90d5f_0    conda-forge
libgomp                   12.1.0              h8d9b700_16    conda-forge
libgoogle-cloud           2.2.0                h838d150_1    conda-forge
libiconv                  1.17                 h166bdaf_0    conda-forge
liblapack                 3.9.0           16_linux64_openblas    conda-forge
libnghttp2                1.47.0               hdcd2b5c_1    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libopenblas               0.3.21          pthreads_h78a6416_3    conda-forge
libpng                    1.6.38               h753d276_0    conda-forge
libprotobuf               3.21.6               h6239696_1    conda-forge
libsanitizer              12.1.0              ha89aaad_16    conda-forge
libsqlite                 3.39.3               h753d276_0    conda-forge
libssh2                   1.10.0               haa6b8db_3    conda-forge
libstdcxx-devel_linux-64  12.1.0              h1ec3361_16    conda-forge
libstdcxx-ng              12.1.0              ha89aaad_16    conda-forge
libthrift                 0.16.0               h491838f_2    conda-forge
libtiff                   4.4.0                h55922b4_4    conda-forge
libutf8proc               2.7.0                h7f98852_0    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libwebp-base              1.2.4                h166bdaf_0    conda-forge
libxcb                    1.13              h7f98852_1004    conda-forge
libxml2                   2.10.2               h4c7fe37_1    conda-forge
libzlib                   1.2.12               h166bdaf_3    conda-forge
lz4-c                     1.9.3                h9c3ff4c_1    conda-forge
make                      4.3                  hd18ef5c_1    conda-forge
ncurses                   6.3                  h27087fc_1    conda-forge
numpy                     1.23.3          py310h53a5b5f_0    conda-forge
openssl                   1.1.1q               h166bdaf_0    conda-forge
orc                       1.7.6                h09e0d61_1    conda-forge
pango                     1.50.10              hc4f8a73_0    conda-forge
pcre                      8.45                 h9c3ff4c_0    conda-forge
pcre2                     10.37                hc3806b6_1    conda-forge
pip                       22.2.2             pyhd8ed1ab_0    conda-forge
pixman                    0.40.0               h36c2ea0_0    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
python                    3.10.6          h582c2e5_0_cpython    conda-forge
python_abi                3.10                    2_cp310    conda-forge
r-arrow                   9.0.0             r41h27087fc_0    conda-forge
r-assertthat              0.2.1             r41hc72bb7e_2    conda-forge
r-base                    4.1.3                ha8c3e7c_2    conda-forge
r-bit                     4.0.4             r41hcfec24a_0    conda-forge
r-bit64                   4.0.5             r41hcfec24a_0    conda-forge
r-cli                     3.4.1             r41h7525677_0    conda-forge
r-ellipsis                0.3.2             r41hcfec24a_0    conda-forge
r-glue                    1.6.2             r41h06615bd_0    conda-forge
r-magrittr                2.0.3             r41h06615bd_0    conda-forge
r-purrr                   0.3.4             r41hcfec24a_1    conda-forge
r-r6                      2.5.1             r41hc72bb7e_0    conda-forge
r-rlang                   1.0.5             r41h7525677_0    conda-forge
r-tidyselect              1.1.2             r41hc72bb7e_0    conda-forge
r-vctrs                   0.4.1             r41h7525677_0    conda-forge
re2                       2022.06.01           h27087fc_0    conda-forge
readline                  8.1.2                h0f457ee_0    conda-forge
s2n                       1.0.10               h9b69904_0    conda-forge
sed                       4.8                  he412f7d_0    conda-forge
setuptools                65.3.0             pyhd8ed1ab_1    conda-forge
snappy                    1.1.9                hbd366e4_1    conda-forge
sysroot_linux-64          2.12                he073ed8_15    conda-forge
tk                        8.6.12               h27826a3_0    conda-forge
tktable                   2.10                 hb7b940f_3    conda-forge
tzdata                    2022c                h191b570_0    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.0.10               h7f98852_0    conda-forge
xorg-libsm                1.2.3             hd9c2040_1000    conda-forge
xorg-libx11               1.7.2                h7f98852_0    conda-forge
xorg-libxau               1.0.9                h7f98852_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h7f98852_1    conda-forge
xorg-libxrender           0.9.10            h7f98852_1003    conda-forge
xorg-libxt                1.2.1                h7f98852_2    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-xextproto            7.3.0             h7f98852_1002    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
zlib                      1.2.12               h166bdaf_3    conda-forge
zstd                      1.5.2                h6239696_4    conda-forge

Environment info

active environment : r-arrow9
    active env location : /opt/conda/envs/r-arrow9
            shell level : 1
       user config file : /home/jtilly/.condarc
 populated config files : /opt/conda/.condarc
          conda version : 4.14.0
    conda-build version : 3.22.0
         python version : 3.10.6.final.0
       virtual packages : __linux=5.15.0=0
                          __glibc=2.35=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /opt/conda  (writable)
      conda av data dir : /opt/conda/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /opt/conda/pkgs
                          /home/jtilly/.conda/pkgs
       envs directories : /opt/conda/envs
                          /home/jtilly/.conda/envs
               platform : linux-64
             user-agent : conda/4.14.0 requests/2.28.1 CPython/3.10.6 Linux/5.15.0-48-generic ubuntu/22.04.1 glibc/2.35
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False
@jtilly jtilly added the bug Something isn't working label Sep 26, 2022
@mlondschien
Copy link
Member

I get the same error for r-arrow=11.0.0 on osx-arm64:

(r-arrow) ~/code/icu-experiments $ R -e "library('arrow'); data(mtcars); write_parquet(mtcars, 'test.parquet')"

R version 4.2.3 (2023-03-15) -- "Shortstop Beagle"
Copyright (C) 2023 The R Foundation for Statistical Computing
Platform: aarch64-apple-darwin20.0.0 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

[Previously saved workspace restored]

> library('arrow'); data(mtcars); write_parquet(mtcars, 'test.parquet')
Some features are not enabled in this build of Arrow. Run `arrow_info()` for more information.

Attaching package: ‘arrow’

The following object is masked from ‘package:utils’:

    timestamp

Error in parquet___WriterProperties___Builder__create() : 
  Cannot call parquet___WriterProperties___Builder__create(). See https://arrow.apache.org/docs/r/articles/install.html for help installing Arrow C++ libraries. 
Calls: write_parquet ... <Anonymous> -> parquet___WriterProperties___Builder__create
Execution halted
`conda list` 
(r-arrow) ~/code/icu-experiments $ conda list
# packages in environment at /Users/mlondschien/mambaforge/envs/r-arrow:
#
# Name                    Version                   Build  Channel
_r-mutex                  1.0.1               anacondar_1    conda-forge
aws-c-auth                0.6.26               h9ec8a34_1    conda-forge
aws-c-cal                 0.5.21               h134b2d6_2    conda-forge
aws-c-common              0.8.14               h1a8c8d9_0    conda-forge
aws-c-compression         0.2.16               h59b28b2_5    conda-forge
aws-c-event-stream        0.2.20               ha370d96_4    conda-forge
aws-c-http                0.7.6                hca61f3c_0    conda-forge
aws-c-io                  0.13.19              ha2443f3_3    conda-forge
aws-c-mqtt                0.8.6               hcd3fa0e_12    conda-forge
aws-c-s3                  0.2.7                hc514de9_1    conda-forge
aws-c-sdkutils            0.1.8                h59b28b2_0    conda-forge
aws-checksums             0.1.14               h59b28b2_5    conda-forge
aws-crt-cpp               0.19.8              h6be2766_12    conda-forge
aws-sdk-cpp               1.10.57              h3daea3c_8    conda-forge
bwidget                   1.9.14               hce30654_1    conda-forge
bzip2                     1.0.8                h3422bc3_4    conda-forge
c-ares                    1.18.1               h3422bc3_0    conda-forge
ca-certificates           2022.12.7            h4653dfc_0    conda-forge
cairo                     1.16.0            h4741ed9_1015    conda-forge
cctools_osx-arm64         973.0.1             h7a0f565_13    conda-forge
clang                     16.0.1               h1e5164c_0    conda-forge
clang-16                  16.0.1          default_he7141eb_0    conda-forge
clang_osx-arm64           16.0.1               hdccda30_0    conda-forge
clangxx                   16.0.1          default_ha0f6327_0    conda-forge
clangxx_osx-arm64         16.0.1               h0c1d50e_0    conda-forge
compiler-rt               16.0.1               h30b49de_0    conda-forge
compiler-rt_osx-arm64     16.0.1               h30b49de_0    conda-forge
curl                      7.88.1               h9049daf_1    conda-forge
expat                     2.5.0                hb7217d7_1    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.14.2               h82840c6_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
freetype                  2.12.1               hd633e50_1    conda-forge
fribidi                   1.0.10               h27ca646_0    conda-forge
gettext                   0.21.1               h0186832_0    conda-forge
gflags                    2.2.2             hc88da5d_1004    conda-forge
gfortran_impl_osx-arm64   11.3.0              h2a9d086_31    conda-forge
gfortran_osx-arm64        11.3.0               h57527a5_1    conda-forge
glog                      0.6.0                h6da1cb0_0    conda-forge
gmp                       6.2.1                h9f76cd9_0    conda-forge
graphite2                 1.3.13            h9f76cd9_1001    conda-forge
gsl                       2.7                  h6e638da_0    conda-forge
harfbuzz                  6.0.0                hd7af201_1    conda-forge
icu                       72.1                 he12128b_0    conda-forge
isl                       0.25                 h9a09cb3_0    conda-forge
krb5                      1.20.1               h69eda48_0    conda-forge
ld64_osx-arm64            609                 h297e91e_13    conda-forge
lerc                      4.0.0                h9a09cb3_0    conda-forge
libabseil                 20230125.0      cxx17_hb7217d7_1    conda-forge
libarrow                  11.0.0          hdcd110a_13_cpu    conda-forge
libblas                   3.9.0           16_osxarm64_openblas    conda-forge
libbrotlicommon           1.0.9                h1a8c8d9_8    conda-forge
libbrotlidec              1.0.9                h1a8c8d9_8    conda-forge
libbrotlienc              1.0.9                h1a8c8d9_8    conda-forge
libcblas                  3.9.0           16_osxarm64_openblas    conda-forge
libclang-cpp16            16.0.1          default_he7141eb_0    conda-forge
libcrc32c                 1.1.2                hbdafb3b_0    conda-forge
libcurl                   7.88.1               h9049daf_1    conda-forge
libcxx                    16.0.1               h75e25f2_0    conda-forge
libdeflate                1.18                 h1a8c8d9_0    conda-forge
libedit                   3.1.20191231         hc8eb9b7_2    conda-forge
libev                     4.33                 h642e427_1    conda-forge
libevent                  2.1.10               h7673551_4    conda-forge
libexpat                  2.5.0                hb7217d7_1    conda-forge
libffi                    3.4.2                h3422bc3_5    conda-forge
libgfortran               5.0.0           12_2_0_hd922786_31    conda-forge
libgfortran-devel_osx-arm64 11.3.0              hfe9555d_31    conda-forge
libgfortran5              12.2.0              h0eea778_31    conda-forge
libglib                   2.74.1               h4646484_1    conda-forge
libgoogle-cloud           2.8.0                h7206bc8_1    conda-forge
libgrpc                   1.52.1               he98ff75_1    conda-forge
libiconv                  1.17                 he4db4b2_0    conda-forge
libjpeg-turbo             2.1.5.1              h1a8c8d9_0    conda-forge
liblapack                 3.9.0           16_osxarm64_openblas    conda-forge
libllvm16                 16.0.1               h1e5164c_0    conda-forge
libnghttp2                1.52.0               hae82a92_0    conda-forge
libopenblas               0.3.21          openmp_hc731615_3    conda-forge
libpng                    1.6.39               h76d750c_0    conda-forge
libprotobuf               3.21.12              hb5ab8b9_0    conda-forge
libssh2                   1.10.0               h7a5bd25_3    conda-forge
libthrift                 0.18.1               h6635e49_0    conda-forge
libtiff                   4.5.0                h4f7d55c_6    conda-forge
libutf8proc               2.8.0                h1a8c8d9_0    conda-forge
libwebp-base              1.3.0                h1a8c8d9_0    conda-forge
libxml2                   2.10.3               h2aff0a6_6    conda-forge
libzlib                   1.2.13               h03a7124_4    conda-forge
llvm-openmp               16.0.1               h7cfbb63_0    conda-forge
llvm-tools                16.0.1               h1e5164c_0    conda-forge
lz4-c                     1.9.4                hb7217d7_0    conda-forge
make                      4.3                  he57ea6c_1    conda-forge
mpc                       1.3.1                h91ba8db_0    conda-forge
mpfr                      4.2.0                he09a6ba_0    conda-forge
ncurses                   6.3                  h07bb92c_1    conda-forge
openssl                   3.1.0                h03a7124_0    conda-forge
orc                       1.8.3                hef0d403_0    conda-forge
pango                     1.50.14              h6c112b8_0    conda-forge
pcre2                     10.40                hb34f9b4_0    conda-forge
pixman                    0.40.0               h27ca646_0    conda-forge
r-arrow                   11.0.0            r42h7ea286d_0    conda-forge
r-assertthat              0.2.1             r42hc72bb7e_3    conda-forge
r-base                    4.2.3                h0794cc7_2    conda-forge
r-bit                     4.0.5             r42h5d63f41_0    conda-forge
r-bit64                   4.0.5             r42h5d63f41_1    conda-forge
r-cli                     3.6.1             r42h9a38a31_0    conda-forge
r-glue                    1.6.2             r42h5d63f41_1    conda-forge
r-lifecycle               1.0.3             r42hc72bb7e_1    conda-forge
r-magrittr                2.0.3             r42h5d63f41_1    conda-forge
r-purrr                   1.0.1             r42h5d63f41_0    conda-forge
r-r6                      2.5.1             r42hc72bb7e_1    conda-forge
r-rlang                   1.1.0             r42h9a38a31_0    conda-forge
r-tidyselect              1.2.0             r42h011812f_0    conda-forge
r-vctrs                   0.6.1             r42h9a38a31_0    conda-forge
r-withr                   2.5.0             r42hc72bb7e_1    conda-forge
re2                       2023.02.02           hb7217d7_0    conda-forge
readline                  8.2                  h92ec313_1    conda-forge
sigtool                   0.1.3                h44b9a77_0    conda-forge
snappy                    1.1.10               h17c5cce_0    conda-forge
tapi                      1100.0.11            he4954df_0    conda-forge
tk                        8.6.12               he1e0b03_0    conda-forge
tktable                   2.10                 h4161312_3    conda-forge
xz                        5.2.6                h57fd34a_0    conda-forge
zlib                      1.2.13               h03a7124_4    conda-forge
zstd                      1.5.2                hf913c23_6    conda-forge
`conda info` 
(r-arrow) ~/code/icu-experiments $ conda info

     active environment : r-arrow
    active env location : /Users/mlondschien/mambaforge/envs/r-arrow
            shell level : 6
       user config file : /Users/mlondschien/.condarc
 populated config files : /Users/mlondschien/mambaforge/.condarc
                          /Users/mlondschien/.config/conda/.condarc
          conda version : 4.11.0
    conda-build version : not installed
         python version : 3.9.15.final.0
       virtual packages : __osx=13.2.1=0
                          __unix=0=0
                          __archspec=1=arm64
       base environment : /Users/mlondschien/mambaforge  (writable)
      conda av data dir : /Users/mlondschien/mambaforge/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/osx-arm64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /Users/mlondschien/mambaforge/pkgs
                          /Users/mlondschien/.conda/pkgs
       envs directories : /Users/mlondschien/mambaforge/envs
                          /Users/mlondschien/.conda/envs
               platform : osx-arm64
             user-agent : conda/4.11.0 requests/2.27.1 CPython/3.9.15 Darwin/22.3.0 OSX/13.2.1
                UID:GID : 501:20
             netrc file : None
           offline mode : False

Downgrading to r-arrow=10.0.1 works.

@sebp
Copy link

sebp commented Apr 22, 2023 •
edited
Loading

@h-vetinari

I'm having the same issue with read_parquet from the r-arrow 11.0.0 package:

> read_parquet("data.parquet")
Error in parquet___arrow___ArrowReaderProperties__Make(isTRUE(use_threads)) : 
  Cannot call parquet___arrow___ArrowReaderProperties__Make(). See https://arrow.apache.org/docs/r/articles/install.html for help installing Arrow C++ libraries.

arrow_info() indicates that it has been compiled without parquet support:

> arrow_info()
Arrow package version: 11.0.0

Capabilities:
               
dataset   FALSE
substrait FALSE
parquet   FALSE
json      FALSE
s3        FALSE
gcs       FALSE
utf8proc   TRUE
re2        TRUE
snappy     TRUE
gzip       TRUE
brotli     TRUE
zstd       TRUE
lz4        TRUE
lz4_frame  TRUE
lzo       FALSE
bz2        TRUE
jemalloc   TRUE
mimalloc   TRUE

To reinstall with more optional capabilities enabled, see
   https://arrow.apache.org/docs/r/articles/install.html

Memory:
                  
Allocator jemalloc
Current    0 bytes
Max        0 bytes

Runtime:
                        
SIMD Level          avx2
Detected SIMD Level avx2

Build:
                                                             
C++ Library Version                                    11.0.0
C++ Compiler                                              GNU
C++ Compiler Version                                   11.3.0
Git ID               f967112e28d9c7b56d86fb786ae21fe1cd6742e7

Output of conda info:

# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
_r-mutex                  1.0.1               anacondar_1    conda-forge
arrow-cpp                 11.0.0          ha770c72_14_cpu    conda-forge
aws-c-auth                0.6.26               hf365957_1    conda-forge
aws-c-cal                 0.5.21               h48707d8_2    conda-forge
aws-c-common              0.8.14               h0b41bf4_0    conda-forge
aws-c-compression         0.2.16               h03acc5a_5    conda-forge
aws-c-event-stream        0.2.20               h00877a2_4    conda-forge
aws-c-http                0.7.6                hf342b9f_0    conda-forge
aws-c-io                  0.13.19              h5b20300_3    conda-forge
aws-c-mqtt                0.8.6               hc4349f7_12    conda-forge
aws-c-s3                  0.2.7                h909e904_1    conda-forge
aws-c-sdkutils            0.1.8                h03acc5a_0    conda-forge
aws-checksums             0.1.14               h03acc5a_5    conda-forge
aws-crt-cpp               0.19.8              hf7fbfca_12    conda-forge
aws-sdk-cpp               1.10.57              h17c43bd_8    conda-forge
binutils_impl_linux-64    2.40                 hf600244_0    conda-forge
bwidget                   1.9.14               ha770c72_1    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.18.1               h7f98852_0    conda-forge
ca-certificates           2022.12.7            ha878542_0    conda-forge
cairo                     1.16.0            h35add3b_1015    conda-forge
curl                      8.0.1                h588be90_0    conda-forge
expat                     2.5.0                hcb278e6_1    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.14.2               h14ed4e7_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
freetype                  2.12.1               hca18f0e_1    conda-forge
fribidi                   1.0.10               h36c2ea0_0    conda-forge
gcc_impl_linux-64         12.2.0              hcc96c02_19    conda-forge
gettext                   0.21.1               h27087fc_0    conda-forge
gflags                    2.2.2             he1b5a44_1004    conda-forge
gfortran_impl_linux-64    12.2.0              h55be85b_19    conda-forge
glog                      0.6.0                h6f12383_0    conda-forge
graphite2                 1.3.13            h58526e2_1001    conda-forge
gsl                       2.7                  he838d99_0    conda-forge
gxx_impl_linux-64         12.2.0              hcc96c02_19    conda-forge
harfbuzz                  7.1.0                h3ff4399_1    conda-forge
icu                       72.1                 hcb278e6_0    conda-forge
kernel-headers_linux-64   2.6.32              he073ed8_15    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
krb5                      1.20.1               h81ceb04_0    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libabseil                 20230125.0      cxx17_hcb278e6_1    conda-forge
libarrow                  11.0.0          h93537a5_14_cpu    conda-forge
libblas                   3.9.0           16_linux64_openblas    conda-forge
libbrotlicommon           1.0.9                h166bdaf_8    conda-forge
libbrotlidec              1.0.9                h166bdaf_8    conda-forge
libbrotlienc              1.0.9                h166bdaf_8    conda-forge
libcblas                  3.9.0           16_linux64_openblas    conda-forge
libcrc32c                 1.1.2                h9c3ff4c_0    conda-forge
libcurl                   8.0.1                h588be90_0    conda-forge
libdeflate                1.18                 h0b41bf4_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libevent                  2.1.10               h28343ad_4    conda-forge
libexpat                  2.5.0                hcb278e6_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-devel_linux-64     12.2.0              h3b97bd3_19    conda-forge
libgcc-ng                 12.2.0              h65d4601_19    conda-forge
libgfortran-ng            12.2.0              h69a702a_19    conda-forge
libgfortran5              12.2.0              h337968e_19    conda-forge
libglib                   2.76.2               hebfc3b9_0    conda-forge
libgomp                   12.2.0              h65d4601_19    conda-forge
libgoogle-cloud           2.8.0                h0bc5f78_1    conda-forge
libgrpc                   1.52.1               hcf146ea_1    conda-forge
libiconv                  1.17                 h166bdaf_0    conda-forge
libjpeg-turbo             2.1.5.1              h0b41bf4_0    conda-forge
liblapack                 3.9.0           16_linux64_openblas    conda-forge
libnghttp2                1.52.0               h61bc06f_0    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libnuma                   2.0.16               h0b41bf4_1    conda-forge
libopenblas               0.3.21          pthreads_h78a6416_3    conda-forge
libpng                    1.6.39               h753d276_0    conda-forge
libprotobuf               3.21.12              h3eb15da_0    conda-forge
libsanitizer              12.2.0              h46fd767_19    conda-forge
libsqlite                 3.40.0               h753d276_0    conda-forge
libssh2                   1.10.0               hf14f497_3    conda-forge
libstdcxx-devel_linux-64  12.2.0              h3b97bd3_19    conda-forge
libstdcxx-ng              12.2.0              h46fd767_19    conda-forge
libthrift                 0.18.1               h5e4af38_0    conda-forge
libtiff                   4.5.0                ha587672_6    conda-forge
libutf8proc               2.8.0                h166bdaf_0    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libwebp-base              1.3.0                h0b41bf4_0    conda-forge
libxcb                    1.13              h7f98852_1004    conda-forge
libxml2                   2.10.4               hfdac1af_0    conda-forge
libzlib                   1.2.13               h166bdaf_4    conda-forge
lz4-c                     1.9.4                hcb278e6_0    conda-forge
make                      4.3                  hd18ef5c_1    conda-forge
ncurses                   6.3                  h27087fc_1    conda-forge
numpy                     1.24.2           py38h10c12cc_0    conda-forge
openssl                   3.1.0                h0b41bf4_0    conda-forge
orc                       1.8.3                hfdbbad2_0    conda-forge
pandas                    2.0.0            py38hdc8b05c_0    conda-forge
pango                     1.50.14              heaa33ce_1    conda-forge
parquet-cpp               1.5.1                         2    conda-forge
pcre2                     10.40                hc3806b6_0    conda-forge
pip                       23.1.1             pyhd8ed1ab_0    conda-forge
pixman                    0.40.0               h36c2ea0_0    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
pyarrow                   11.0.0          py38h9c9c3a3_14_cpu    conda-forge
python                    3.8.16          he550d4f_1_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-tzdata             2023.3             pyhd8ed1ab_0    conda-forge
python_abi                3.8                      3_cp38    conda-forge
pytz                      2023.3             pyhd8ed1ab_0    conda-forge
r-arrow                   11.0.0            r42hcb278e6_0    conda-forge
r-assertthat              0.2.1             r42hc72bb7e_3    conda-forge
r-base                    4.2.3                h4a03800_2    conda-forge
r-bit                     4.0.5             r42h06615bd_0    conda-forge
r-bit64                   4.0.5             r42h06615bd_1    conda-forge
r-cli                     3.6.1             r42h38f115c_0    conda-forge
r-glue                    1.6.2             r42h06615bd_1    conda-forge
r-lifecycle               1.0.3             r42hc72bb7e_1    conda-forge
r-magrittr                2.0.3             r42h06615bd_1    conda-forge
r-purrr                   1.0.1             r42h133d619_0    conda-forge
r-r6                      2.5.1             r42hc72bb7e_1    conda-forge
r-rlang                   1.1.0             r42h38f115c_0    conda-forge
r-tidyselect              1.2.0             r42hc72bb7e_0    conda-forge
r-vctrs                   0.6.2             r42ha503ecb_0    conda-forge
r-withr                   2.5.0             r42hc72bb7e_1    conda-forge
re2                       2023.02.02           hcb278e6_0    conda-forge
readline                  8.2                  h8228510_1    conda-forge
s2n                       1.3.41               h3358134_0    conda-forge
sed                       4.8                  he412f7d_0    conda-forge
setuptools                67.7.1             pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
snappy                    1.1.10               h9fff704_0    conda-forge
sysroot_linux-64          2.12                he073ed8_15    conda-forge
tk                        8.6.12               h27826a3_0    conda-forge
tktable                   2.10                 hb7b940f_3    conda-forge
ucx                       1.14.0               h8c404fb_1    conda-forge
wheel                     0.40.0             pyhd8ed1ab_0    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.0.10               h7f98852_0    conda-forge
xorg-libsm                1.2.3             hd9c2040_1000    conda-forge
xorg-libx11               1.8.4                h0b41bf4_0    conda-forge
xorg-libxau               1.0.9                h7f98852_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h0b41bf4_2    conda-forge
xorg-libxrender           0.9.10            h7f98852_1003    conda-forge
xorg-libxt                1.2.1                h7f98852_2    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-xextproto            7.3.0             h0b41bf4_1003    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
zlib                      1.2.13               h166bdaf_4    conda-forge
zstd                      1.5.2                h3eb15da_6    conda-forge

Output of conda info:

     active environment : arrow
    active env location : /home/…/mambaforge/envs/arrow
            shell level : 2
       user config file : /home/…/.condarc
 populated config files : /home/…/mambaforge/.condarc
                          /home/…/.condarc
          conda version : 22.9.0
    conda-build version : 3.23.3
         python version : 3.10.8.final.0
       virtual packages : __cuda=12.1=0
                          __linux=6.2.11=0
                          __glibc=2.36=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /home/…/mambaforge  (writable)
      conda av data dir : /home/…/mambaforge/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /home/…/mambaforge/pkgs
                          /home/…/.conda/pkgs
       envs directories : /home/…/mambaforge/envs
                          /home/…/.conda/envs
               platform : linux-64
             user-agent : conda/22.9.0 requests/2.28.1 CPython/3.10.8 Linux/6.2.11-200.fc37.x86_64 fedora/37 glibc/2.36
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

@nealrichardson
Copy link
Contributor

I haven't looked into the details here, but I wonder if apache/arrow#35140 is the cause (not of the original report, which predates the issue identified there, but the more recent ones commented here). The symptoms would be the same.

@nealrichardson nealrichardson mentioned this issue Apr 22, 2023
Closed
@h-vetinari
Copy link
Member

Thanks for chiming in @nealrichardson!

I had missed this issue (and #62) because I missed switching on notifications for this feedstock after joining. AFAICT, nothing substantial about the build changed on conda-forge side between 10 & 11 (both for libarrow and r-arrow), so I suspect this comes from something upstream.

our libarrow is built with parquet & dataset support (& tested).

@nealrichardson
Copy link
Contributor

Yeah I think the timing for the change the reporter flagged on the apache/arrow issue would explain a regression between 10 and 11 for identifying the build features in some conditions.

I think we test the conda packaging nightly in arrow, but I doubt we do any extra tests that certain features are enabled. The test suite will skip e.g. parquet tests if the package was built without it, so the regular suite wouldn't tell us this. But that's solvable.

This was referenced Apr 23, 2023
Merged
Merged
@h-vetinari
Copy link
Member

Closing as fixed by #63; please let us know if something is still not working with r-arrow=11.0.0=*_1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@sebp @nealrichardson @jtilly @h-vetinari @mlondschien