Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] to_parquet using "cudf" engine fails #7074

Closed
madsbk opened this issue Jan 5, 2021 · 3 comments
Closed

[BUG] to_parquet using "cudf" engine fails #7074

madsbk opened this issue Jan 5, 2021 · 3 comments
Labels
0 - Waiting on Author Waiting for author to respond to review bug Something isn't working invalid This doesn't seem right libcudf Affects libcudf (C++/CUDA) code.

Comments

@madsbk
Copy link
Member

madsbk commented Jan 5, 2021 •
edited
Loading

Describe the bug
Calling cudf.io.parquet.to_parquet(), which uses the default engine "cudf" fails. The "pyarrow" engine works fine.

Steps/Code to reproduce bug

import cudf
from cudf.io.parquet import to_parquet
df = cudf.DataFrame({"a": [1, 1, 3]})
to_parquet(df, "test_cudf", engine='cudf')

Fails with:

*** stack smashing detected ***: terminated
Aborted

Expected behavior
Not crash :P

Environment overview (please complete the following information)

  • Environment location: Bare-metal (my laptop)
  • Method of cuDF install (today):
conda create -n rapids-0.18 -c rapidsai-nightly -c nvidia -c conda-forge \
  -c defaults rapids-blazing=0.18 python=3.7 cudatoolkit=10.2

Environment details

Click here to see environment details
 **git***
 commit 31c0d29a37f5073cdc8a6f78307d6ddfbb1f7144 (HEAD -> branch-0.18, upstream/branch-0.18, origin/branch-0.18)
 Author: Mike Wilson <hyperbolic2346@users.noreply.github.com>
 Date:   Mon Jan 4 19:52:25 2021 -0500
 
 Adding decimal writing support to parquet(#7017)
 
 Since cudf doesn't support precision, the precision must be passed in as a write option. This is handled as a vector of uint8's that indicates the precision of each flattened column in order to support nested types.
 
 Partially closes #6474
 
 Authors:
 - Mike Wilson <knobby@burntsheep.com>
 - Mike Wilson <hyperbolic2346@users.noreply.github.com>
 
 Approvers:
 - Vukasin Milovanovic
 - Mark Harris
 
 URL: https://github.com/rapidsai/cudf/pull/7017
 **git submodules***
 
 ***OS Information***
 DISTRIB_ID=Ubuntu
 DISTRIB_RELEASE=20.04
 DISTRIB_CODENAME=focal
 DISTRIB_DESCRIPTION="Ubuntu 20.04.1 LTS"
 NAME="Ubuntu"
 VERSION="20.04.1 LTS (Focal Fossa)"
 ID=ubuntu
 ID_LIKE=debian
 PRETTY_NAME="Ubuntu 20.04.1 LTS"
 VERSION_ID="20.04"
 HOME_URL="https://www.ubuntu.com/"
 SUPPORT_URL="https://help.ubuntu.com/"
 BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
 PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
 VERSION_CODENAME=focal
 UBUNTU_CODENAME=focal
 Linux madsbk-laptop 5.4.0-58-generic #64-Ubuntu SMP Wed Dec 9 08:16:25 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
 
 ***GPU Information***
 Tue Jan  5 15:15:19 2021
 +-----------------------------------------------------------------------------+
 | NVIDIA-SMI 450.80.02    Driver Version: 450.80.02    CUDA Version: 11.0     |
 |-------------------------------+----------------------+----------------------+
 | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
 | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
 |                               |                      |               MIG M. |
 |===============================+======================+======================|
 |   0  Quadro P1000        Off  | 00000000:01:00.0 Off |                  N/A |
 | N/A   51C    P0    N/A /  N/A |   1407MiB /  4006MiB |      3%      Default |
 |                               |                      |                  N/A |
 +-------------------------------+----------------------+----------------------+
 
 +-----------------------------------------------------------------------------+
 | Processes:                                                                  |
 |  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
 |        ID   ID                                                   Usage      |
 |=============================================================================|
 |    0   N/A  N/A      1224      G   /usr/lib/xorg/Xorg                810MiB |
 |    0   N/A  N/A      1702      G   xfwm4                               1MiB |
 |    0   N/A  N/A      3945      G   ...AAAAAAAAA= --shared-files      547MiB |
 |    0   N/A  N/A     10901      G   ...AAAAAAAA== --shared-files       44MiB |
 +-----------------------------------------------------------------------------+
 
 ***CPU***
 Architecture:                    x86_64
 CPU op-mode(s):                  32-bit, 64-bit
 Byte Order:                      Little Endian
 Address sizes:                   39 bits physical, 48 bits virtual
 CPU(s):                          12
 On-line CPU(s) list:             0-11
 Thread(s) per core:              2
 Core(s) per socket:              6
 Socket(s):                       1
 NUMA node(s):                    1
 Vendor ID:                       GenuineIntel
 CPU family:                      6
 Model:                           158
 Model name:                      Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
 Stepping:                        10
 CPU MHz:                         3368.420
 CPU max MHz:                     4100.0000
 CPU min MHz:                     800.0000
 BogoMIPS:                        4399.99
 Virtualization:                  VT-x
 L1d cache:                       192 KiB
 L1i cache:                       192 KiB
 L2 cache:                        1.5 MiB
 L3 cache:                        9 MiB
 NUMA node0 CPU(s):               0-11
 Vulnerability Itlb multihit:     KVM: Mitigation: Split huge pages
 Vulnerability L1tf:              Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable
 Vulnerability Mds:               Mitigation; Clear CPU buffers; SMT vulnerable
 Vulnerability Meltdown:          Mitigation; PTI
 Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
 Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
 Vulnerability Spectre v2:        Mitigation; Full generic retpoline, IBPB conditional, IBRS_FW, STIBP conditional, RSB filling
 Vulnerability Srbds:             Mitigation; Microcode
 Vulnerability Tsx async abort:   Not affected
 Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d
 
 ***CMake***
 
 ***g++***
 /usr/bin/g++
 g++ (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
 Copyright (C) 2019 Free Software Foundation, Inc.
 This is free software; see the source for copying conditions.  There is NO
 warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
 
 
 ***nvcc***
 
 ***Python***
 /home/mkristensen/apps/miniconda3/envs/rap-1118/bin/python
 Python 3.8.6
 
 ***Environment Variables***
 PATH                            : /home/mkristensen/apps/miniconda3/envs/rap-1118/bin:/usr/local/cuda-10.1/bin:/home/mkristensen/apps/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
 LD_LIBRARY_PATH                 : /home/mkristensen/.local/lib/
 NUMBAPRO_NVVM                   :
 NUMBAPRO_LIBDEVICE              :
 CONDA_PREFIX                    : /home/mkristensen/apps/miniconda3/envs/rap-1118
 PYTHON_PATH                     :
 
 ***conda packages***
 /home/mkristensen/apps/miniconda3/condabin/conda
 # packages in environment at /home/mkristensen/apps/miniconda3/envs/rap-1118:
 #
 # Name                    Version                   Build  Channel
 _libgcc_mutex             0.1                 conda_forge    conda-forge
 _openmp_mutex             4.5                       1_gnu    conda-forge
 abseil-cpp                20200225.2           he1b5a44_2    conda-forge
 aiohttp                   3.7.2            py38h1e0a361_0    conda-forge
 appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
 argon2-cffi               20.1.0           py38h25fe258_2    conda-forge
 arrow-cpp                 1.0.1           py38h9444e92_16_cuda    conda-forge
 arrow-cpp-proc            2.0.0                      cuda    conda-forge
 async-timeout             3.0.1                   py_1000    conda-forge
 async_generator           1.10                       py_0    conda-forge
 atk                       2.36.0               ha770c72_4    conda-forge
 atk-1.0                   2.36.0               h0d5b62e_4    conda-forge
 attrs                     20.3.0             pyhd3deb0d_0    conda-forge
 autoconf                  2.69            pl5320h36c2ea0_10    conda-forge
 automake                  1.16.2          pl5320ha770c72_3    conda-forge
 aws-c-common              0.4.59               h36c2ea0_1    conda-forge
 aws-c-event-stream        0.1.6                had2084c_6    conda-forge
 aws-checksums             0.1.10               h4e93380_0    conda-forge
 aws-sdk-cpp               1.8.70               h57dc084_1    conda-forge
 backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
 backports                 1.0                        py_2    conda-forge
 backports.functools_lru_cache 1.6.1                      py_0    conda-forge
 black                     19.10b0                  pypi_0    pypi
 bleach                    3.2.1              pyh9f0ad1d_0    conda-forge
 bokeh                     2.2.3            py38h32f6830_0    conda-forge
 boost                     1.72.0           py38h1e42940_1    conda-forge
 boost-cpp                 1.72.0               h8e57a91_0    conda-forge
 brotli                    1.0.9                he1b5a44_3    conda-forge
 brotlipy                  0.7.0           py38h8df0ef7_1001    conda-forge
 bzip2                     1.0.8                h516909a_3    conda-forge
 c-ares                    1.16.1               h516909a_3    conda-forge
 ca-certificates           2020.11.8            ha878542_0    conda-forge
 cairo                     1.16.0            hcf35c78_1003    conda-forge
 certifi                   2020.11.8        py38h578d9bd_0    conda-forge
 cffi                      1.14.3           py38h1bdcb99_1    conda-forge
 cfgv                      3.2.0                    pypi_0    pypi
 cfitsio                   3.470                hb418390_7    conda-forge
 chardet                   3.0.4           py38h924ce5b_1008    conda-forge
 click                     7.1.2              pyh9f0ad1d_0    conda-forge
 click-plugins             1.1.1                      py_0    conda-forge
 cligj                     0.7.0                      py_0    conda-forge
 cloudpickle               1.6.0                      py_0    conda-forge
 colorcet                  2.0.1                      py_0    conda-forge
 cryptography              3.2.1            py38h7699a38_0    conda-forge
 cudatoolkit               10.2.89              h6bb024c_0    nvidia
 cudf                      0.17.0a201118   cuda_10.2_py38_g3bf56068e9_280    rapidsai-nightly
 cudf_kafka                0.17.0a201118   py38_g3bf56068e9_280    rapidsai-nightly
 cudnn                     7.6.5                cuda10.2_0
 cugraph                   0.17.0a201117   py38_g4cf595fc_93    rapidsai-nightly
 cuml                      0.17.0a201117   cuda10.2_py38_g238a8decf_120    rapidsai-nightly
 cupy                      8.1.0            py38h0c20449_0    conda-forge
 curl                      7.71.1               he644dc0_8    conda-forge
 cusignal                  0.17.0a201117   py38_ga3e5293_29    rapidsai-nightly
 cuspatial                 0.17.0a201117   py38_gec8941b_8    rapidsai-nightly
 custreamz                 0.17.0a201118   py38_g3bf56068e9_280    rapidsai-nightly
 cuxfilter                 0.17.0a201117   py38_g01ee2c8_96    rapidsai-nightly
 cyrus-sasl                2.1.27               h063b49f_1    conda-forge
 cython                    0.29.21          py38h348cfbe_1    conda-forge
 cytoolz                   0.11.0           py38h1e0a361_1    conda-forge
 dask                      2020.12.0+8.gdbc65574           dev_0    <develop>
 dask-cuda                 0.18.0a0+103.gd1933d5           dev_0    <develop>
 dask-cudf                 0.17.0a201118   py38_g3bf56068e9_280    rapidsai-nightly
 dataclasses               0.7                pyhb2cacf7_7    conda-forge
 datashader                0.11.1             pyh9f0ad1d_0    conda-forge
 datashape                 0.5.4                      py_1    conda-forge
 decorator                 4.4.2                      py_0    conda-forge
 defusedxml                0.6.0                      py_0    conda-forge
 distlib                   0.3.1                    pypi_0    pypi
 distributed               2.30.0+56.gb4dfc925           dev_0    <develop>
 dlpack                    0.3                  he1b5a44_1    conda-forge
 entrypoints               0.3             pyhd8ed1ab_1003    conda-forge
 expat                     2.2.9                he1b5a44_2    conda-forge
 faiss-proc                1.0.0                      cuda    rapidsai-nightly
 fastavro                  1.1.1            py38h25fe258_0    conda-forge
 fastrlock                 0.5              py38h950e882_1    conda-forge
 filelock                  3.0.12                   pypi_0    pypi
 fiona                     1.8.13           py38h033e0f6_1    conda-forge
 flake8                    3.8.4                      py_0    conda-forge
 font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
 font-ttf-inconsolata      2.001                hab24e00_0    conda-forge
 font-ttf-source-code-pro  2.030                hab24e00_0    conda-forge
 font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
 fontconfig                2.13.1            h86ecdb6_1001    conda-forge
 fonts-conda-ecosystem     1                             0    conda-forge
 fonts-conda-forge         1                             0    conda-forge
 freetype                  2.10.4               h7ca028e_0    conda-forge
 freexl                    1.0.5             h516909a_1002    conda-forge
 fribidi                   1.0.10               h36c2ea0_0    conda-forge
 fsspec                    0.8.4                      py_0    conda-forge
 gdal                      3.0.4           py38h172510d_10    conda-forge
 gdk-pixbuf                2.42.0               h0536704_0    conda-forge
 geopandas                 0.8.1                      py_0    conda-forge
 geos                      3.8.1                he1b5a44_0    conda-forge
 geotiff                   1.6.0                h05acad5_0    conda-forge
 gettext                   0.19.8.1          hf34092f_1004    conda-forge
 gflags                    2.2.2             he1b5a44_1004    conda-forge
 gh                        1.2.0                h375a9b1_0    conda-forge
 giflib                    5.2.1                h36c2ea0_2    conda-forge
 glib                      2.66.2               h58526e2_0    conda-forge
 glog                      0.4.0                h49b9bf7_3    conda-forge
 gobject-introspection     1.66.1           py38h4eacb9c_3    conda-forge
 graphite2                 1.3.13            h58526e2_1001    conda-forge
 graphviz                  2.42.3               h6939c30_2    conda-forge
 grpc-cpp                  1.33.2               h1870a98_1    conda-forge
 gtk2                      2.24.32              h586f36d_1    conda-forge
 gts                       0.7.6                h17b2bb4_1    conda-forge
 harfbuzz                  2.4.0                h9f30f68_3    conda-forge
 hdf4                      4.2.13            hf30be14_1003    conda-forge
 hdf5                      1.10.6          nompi_h1022a3e_1110    conda-forge
 heapdict                  1.0.1                      py_0    conda-forge
 icu                       64.2                 he1b5a44_1    conda-forge
 identify                  1.5.10                   pypi_0    pypi
 idna                      2.10               pyh9f0ad1d_0    conda-forge
 importlib-metadata        2.0.0                      py_1    conda-forge
 importlib_metadata        2.0.0                         1    conda-forge
 iniconfig                 1.1.1              pyh9f0ad1d_0    conda-forge
 ipykernel                 5.3.4            py38h81c977d_1    conda-forge
 ipython                   7.19.0           py38h81c977d_0    conda-forge
 ipython_genutils          0.2.0                      py_1    conda-forge
 ipywidgets                7.5.1              pyh9f0ad1d_1    conda-forge
 isort                     5.0.9                    pypi_0    pypi
 jedi                      0.17.2           py38h578d9bd_1    conda-forge
 jinja2                    2.11.2             pyh9f0ad1d_0    conda-forge
 joblib                    0.17.0                     py_0    conda-forge
 jpeg                      9d                   h36c2ea0_0    conda-forge
 json-c                    0.13.1            hbfbb72e_1002    conda-forge
 jsonschema                3.2.0                      py_2    conda-forge
 jupyter-server-proxy      1.5.0                      py_0    conda-forge
 jupyter_client            6.1.7                      py_0    conda-forge
 jupyter_core              4.6.3            py38h578d9bd_2    conda-forge
 jupyterlab_pygments       0.1.2              pyh9f0ad1d_0    conda-forge
 kealib                    1.4.13               h33137a7_1    conda-forge
 krb5                      1.17.2               h926e7f8_0    conda-forge
 lcms2                     2.11                 hcbb858e_1    conda-forge
 ld_impl_linux-64          2.35.1               hed1e6ac_0    conda-forge
 libblas                   3.9.0                2_openblas    conda-forge
 libcblas                  3.9.0                2_openblas    conda-forge
 libcudf                   0.17.0a201118   cuda10.2_g3bf56068e9_280    rapidsai-nightly
 libcudf_kafka             0.17.0a201118   g3bf56068e9_280    rapidsai-nightly
 libcugraph                0.17.0a201117   cuda10.2_g4cf595fc_93    rapidsai-nightly
 libcuml                   0.17.0a201117   cuda10.2_g238a8decf_120    rapidsai-nightly
 libcumlprims              0.17.0a201030   cuda10.2_g1fa28a5_8    rapidsai-nightly
 libcurl                   7.71.1               hcdd3856_8    conda-forge
 libcuspatial              0.17.0a201117   cuda10.2_gec8941b_8    rapidsai-nightly
 libdap4                   3.20.6               h1d1bd15_1    conda-forge
 libedit                   3.1.20191231         he28a2e2_2    conda-forge
 libev                     4.33                 h516909a_1    conda-forge
 libevent                  2.1.10               hcdb4288_3    conda-forge
 libfaiss                  1.6.3           he61ee18_3_cuda    conda-forge
 libffi                    3.2.1             he1b5a44_1007    conda-forge
 libgcc-ng                 9.3.0               h5dbcf3e_17    conda-forge
 libgcrypt                 1.8.7                h36c2ea0_0    conda-forge
 libgdal                   3.0.4               he6a97d6_10    conda-forge
 libgfortran-ng            9.3.0               he4bcb1c_17    conda-forge
 libgfortran5              9.3.0               he4bcb1c_17    conda-forge
 libglib                   2.66.2               hbe7bbb4_0    conda-forge
 libgomp                   9.3.0               h5dbcf3e_17    conda-forge
 libgpg-error              1.39                 he1b5a44_0    conda-forge
 libgsasl                  1.8.0                         2    conda-forge
 libhwloc                  2.3.0                h3c4fd83_0    conda-forge
 libiconv                  1.16                 h516909a_0    conda-forge
 libkml                    1.3.0             hd79254b_1012    conda-forge
 liblapack                 3.9.0                2_openblas    conda-forge
 libllvm10                 10.0.1               he513fc3_3    conda-forge
 libnetcdf                 4.7.4           nompi_hefab0ff_106    conda-forge
 libnghttp2                1.41.0               h8cfc5f6_2    conda-forge
 libntlm                   1.4               h516909a_1002    conda-forge
 libopenblas               0.3.12          pthreads_h4812303_1    conda-forge
 libpng                    1.6.37               h21135ba_2    conda-forge
 libpq                     12.3                 h5513abc_2    conda-forge
 libprotobuf               3.13.0.1             h8b12597_0    conda-forge
 librdkafka                1.5.2                h40bdf00_0    conda-forge
 librmm                    0.17.0a201106   cuda10.2_gb1ac445_43    rapidsai-nightly
 libsodium                 1.0.18               h36c2ea0_1    conda-forge
 libspatialindex           1.9.3                he1b5a44_3    conda-forge
 libspatialite             4.3.0a            h2482549_1038    conda-forge
 libssh2                   1.9.0                hab1572f_5    conda-forge
 libstdcxx-ng              9.3.0               h2ae2ef3_17    conda-forge
 libthrift                 0.13.0               h5aa387f_6    conda-forge
 libtiff                   4.1.0                h4f3a223_6    conda-forge
 libtool                   2.4.6             h58526e2_1007    conda-forge
 libutf8proc               2.5.0                h516909a_2    conda-forge
 libuuid                   2.32.1            h14c3975_1000    conda-forge
 libuv                     1.34.0               h516909a_0    conda-forge
 libwebp                   1.1.0                h76fa15c_4    conda-forge
 libwebp-base              1.1.0                h36c2ea0_3    conda-forge
 libxcb                    1.13              h14c3975_1002    conda-forge
 libxgboost                1.2.0dev.rapidsai0.17      cuda10.2_0    rapidsai-nightly
 libxml2                   2.9.10               hee79883_0    conda-forge
 llvmlite                  0.34.0           py38h4f45e52_2    conda-forge
 locket                    0.2.0                      py_2    conda-forge
 lz4-c                     1.9.2                he1b5a44_3    conda-forge
 m4                        1.4.18            h516909a_1001    conda-forge
 make                      4.3                  hd18ef5c_1    conda-forge
 markdown                  3.3.3              pyh9f0ad1d_0    conda-forge
 markupsafe                1.1.1            py38h8df0ef7_2    conda-forge
 mccabe                    0.6.1                      py_1    conda-forge
 mistune                   0.8.4           py38h25fe258_1002    conda-forge
 more-itertools            8.6.0              pyhd8ed1ab_0    conda-forge
 msgpack-python            1.0.0            py38hbf85e49_2    conda-forge
 multidict                 4.7.5            py38h1e0a361_2    conda-forge
 multipledispatch          0.6.0                      py_0    conda-forge
 munch                     2.5.0                      py_0    conda-forge
 mypy_extensions           0.4.3            py38h32f6830_2    conda-forge
 nbclient                  0.5.1                      py_0    conda-forge
 nbconvert                 6.0.7            py38h578d9bd_3    conda-forge
 nbformat                  5.0.8                      py_0    conda-forge
 nccl                      2.8.3.1              h1a5f58c_0    conda-forge
 ncurses                   6.2                  h58526e2_4    conda-forge
 nest-asyncio              1.4.3              pyhd8ed1ab_0    conda-forge
 networkx                  2.5                        py_0    conda-forge
 nodeenv                   1.5.0                    pypi_0    pypi
 nodejs                    13.13.0              hf5d1a2b_0    conda-forge
 notebook                  6.1.5            py38h578d9bd_0    conda-forge
 numba                     0.51.2           py38hc5bc63f_0    conda-forge
 numpy                     1.19.4           py38hf0fd68c_1    conda-forge
 nvtx                      0.2.1            py38h1e0a361_2    conda-forge
 olefile                   0.46               pyh9f0ad1d_1    conda-forge
 openjpeg                  2.3.1                h981e76c_3    conda-forge
 openssl                   1.1.1h               h516909a_0    conda-forge
 orc                       1.6.5                hd3605a7_0    conda-forge
 packaging                 20.4               pyh9f0ad1d_0    conda-forge
 pandas                    1.1.4            py38h0ef3d22_0    conda-forge
 pandoc                    2.11.1.1             h36c2ea0_0    conda-forge
 pandocfilters             1.4.2                      py_1    conda-forge
 panel                     0.10.1             pyhd8ed1ab_1    conda-forge
 pango                     1.42.4               h7062337_4    conda-forge
 param                     1.10.0                     py_0    conda-forge
 parquet-cpp               1.5.1                         2    conda-forge
 parso                     0.7.1              pyh9f0ad1d_0    conda-forge
 partd                     1.1.0                      py_0    conda-forge
 pathspec                  0.8.1              pyhd3deb0d_0    conda-forge
 pcre                      8.44                 he1b5a44_0    conda-forge
 perl                      5.32.0               h36c2ea0_0    conda-forge
 pexpect                   4.8.0              pyh9f0ad1d_2    conda-forge
 pickleshare               0.7.5                   py_1003    conda-forge
 pillow                    8.0.1            py38h70fbd49_0    conda-forge
 pip                       20.2.4                     py_0    conda-forge
 pixman                    0.38.0            h516909a_1003    conda-forge
 pkg-config                0.29.2            h36c2ea0_1008    conda-forge
 pluggy                    0.13.1           py38h924ce5b_3    conda-forge
 poppler                   0.87.0               h4190859_1    conda-forge
 poppler-data              0.4.10                        0    conda-forge
 postgresql                12.3                 h8573dbc_2    conda-forge
 pre-commit                2.9.3                    pypi_0    pypi
 proj                      7.0.0                h966b41f_5    conda-forge
 prometheus_client         0.9.0              pyhd3deb0d_0    conda-forge
 prompt-toolkit            3.0.8              pyha770c72_0    conda-forge
 protobuf                  3.13.0.1         py38h950e882_1    conda-forge
 psutil                    5.7.3            py38h8df0ef7_0    conda-forge
 pthread-stubs             0.4               h14c3975_1001    conda-forge
 ptyprocess                0.6.0                   py_1001    conda-forge
 py                        1.9.0              pyh9f0ad1d_0    conda-forge
 py-xgboost                1.2.0dev.rapidsai0.17  cuda10.2py38_0    rapidsai-nightly
 pyarrow                   1.0.1           py38h3e2403a_16_cuda    conda-forge
 pycodestyle               2.6.0              pyh9f0ad1d_0    conda-forge
 pycparser                 2.20               pyh9f0ad1d_2    conda-forge
 pyct                      0.4.6                      py_0    conda-forge
 pyct-core                 0.4.6                      py_0    conda-forge
 pydeck                    0.5.0              pyh9f0ad1d_0    conda-forge
 pyee                      7.0.4              pyh9f0ad1d_0    conda-forge
 pyflakes                  2.2.0              pyh9f0ad1d_0    conda-forge
 pygments                  2.7.2                      py_0    conda-forge
 pynvml                    8.0.4                      py_1    conda-forge
 pyopenssl                 19.1.0                     py_1    conda-forge
 pyparsing                 2.4.7              pyh9f0ad1d_0    conda-forge
 pyppeteer                 0.2.2                      py_1    conda-forge
 pyproj                    2.6.1.post1      py38h7521cb9_0    conda-forge
 pyrsistent                0.17.3           py38h25fe258_1    conda-forge
 pysocks                   1.7.1            py38h924ce5b_2    conda-forge
 pytest                    6.1.2            py38h578d9bd_0    conda-forge
 pytest-asyncio            0.14.0           py38h578d9bd_0    conda-forge
 python                    3.8.6           h852b56e_0_cpython    conda-forge
 python-confluent-kafka    1.5.0            py38h1e0a361_0    conda-forge
 python-dateutil           2.8.1                      py_0    conda-forge
 python-graphviz           0.15               pyhd3deb0d_0    conda-forge
 python_abi                3.8                      1_cp38    conda-forge
 pytz                      2020.4             pyhd8ed1ab_0    conda-forge
 pyviz_comms               0.7.6              pyh9f0ad1d_0    conda-forge
 pyyaml                    5.3.1            py38h8df0ef7_1    conda-forge
 pyzmq                     20.0.0           py38h1d1b12f_1    conda-forge
 rapids                    0.17.0a201117   cuda10.2_py38_g5aced79_102    rapidsai-nightly
 rapids-xgboost            0.17.0a201117   cuda10.2_py38_g5aced79_102    rapidsai-nightly
 re2                       2020.11.01           h58526e2_0    conda-forge
 readline                  8.0                  he28a2e2_2    conda-forge
 regex                     2020.11.13       py38h25fe258_0    conda-forge
 requests                  2.25.0             pyhd3deb0d_0    conda-forge
 rmm                       0.17.0a201106   cuda_10.2_py38_gb1ac445_43    rapidsai-nightly
 rtree                     0.9.4            py38h02d302b_2    conda-forge
 scikit-learn              0.23.2           py38h5d63f67_2    conda-forge
 scipy                     1.5.3            py38hb2138dd_0    conda-forge
 send2trash                1.5.0                      py_0    conda-forge
 setuptools                49.6.0           py38h924ce5b_2    conda-forge
 shapely                   1.7.1            py38hc7361b7_1    conda-forge
 simpervisor               0.3                        py_1    conda-forge
 six                       1.15.0             pyh9f0ad1d_0    conda-forge
 snakeviz                  2.1.0                    pypi_0    pypi
 snappy                    1.1.8                he1b5a44_3    conda-forge
 sortedcontainers          2.3.0              pyhd8ed1ab_0    conda-forge
 spdlog                    1.7.0                hc9558a2_2    conda-forge
 sqlite                    3.33.0               h4cf870e_1    conda-forge
 streamz                   0.6.1              pyhd3deb0d_1    conda-forge
 tbb                       2020.2               hc9558a2_0    conda-forge
 tblib                     1.6.0                      py_0    conda-forge
 terminado                 0.9.1            py38h32f6830_1    conda-forge
 testpath                  0.4.4                      py_0    conda-forge
 threadpoolctl             2.1.0              pyh5ca1d4c_0    conda-forge
 tiledb                    1.7.7                h8efa9f0_3    conda-forge
 tk                        8.6.10               hed695b0_1    conda-forge
 toml                      0.10.2             pyhd8ed1ab_0    conda-forge
 toolz                     0.11.1                     py_0    conda-forge
 tornado                   6.1              py38h25fe258_0    conda-forge
 tqdm                      4.52.0             pyhd3deb0d_0    conda-forge
 traitlets                 5.0.5                      py_0    conda-forge
 treelite                  0.93             py38hadf7658_3    conda-forge
 treelite-runtime          0.93                     pypi_0    pypi
 typed-ast                 1.4.1            py38h25fe258_1    conda-forge
 typing-extensions         3.7.4.3                       0    conda-forge
 typing_extensions         3.7.4.3                    py_0    conda-forge
 tzcode                    2020a                h516909a_0    conda-forge
 ucx                       1.8.1+g6b29558       cuda10.2_0    rapidsai-nightly
 ucx-proc                  1.0.0                       gpu    rapidsai-nightly
 ucx-py                    0.17.0a201117   py38_g6b29558_17    rapidsai-nightly
 urllib3                   1.25.11                    py_0    conda-forge
 virtualenv                20.2.2                   pypi_0    pypi
 wcwidth                   0.2.5              pyh9f0ad1d_2    conda-forge
 webencodings              0.5.1                      py_1    conda-forge
 websockets                8.1              py38h1e0a361_2    conda-forge
 wheel                     0.35.1             pyh9f0ad1d_0    conda-forge
 widgetsnbextension        3.5.1            py38h578d9bd_4    conda-forge
 xarray                    0.16.1                     py_0    conda-forge
 xerces-c                  3.2.2             h8412b87_1004    conda-forge
 xgboost                   1.2.0dev.rapidsai0.17  cuda10.2py38_0    rapidsai-nightly
 xorg-kbproto              1.0.7             h14c3975_1002    conda-forge
 xorg-libice               1.0.10               h516909a_0    conda-forge
 xorg-libsm                1.2.3             h84519dc_1000    conda-forge
 xorg-libx11               1.6.12               h516909a_0    conda-forge
 xorg-libxau               1.0.9                h14c3975_0    conda-forge
 xorg-libxdmcp             1.1.3                h516909a_0    conda-forge
 xorg-libxext              1.3.4                h516909a_0    conda-forge
 xorg-libxpm               3.5.13               h516909a_0    conda-forge
 xorg-libxrender           0.9.10            h516909a_1002    conda-forge
 xorg-libxt                1.1.5             h516909a_1003    conda-forge
 xorg-renderproto          0.11.1            h14c3975_1002    conda-forge
 xorg-xextproto            7.3.0             h14c3975_1002    conda-forge
 xorg-xproto               7.0.31            h14c3975_1007    conda-forge
 xz                        5.2.5                h516909a_1    conda-forge
 yaml                      0.2.5                h516909a_0    conda-forge
 yarl                      1.6.3            py38h25fe258_0    conda-forge
 zeromq                    4.3.3                h58526e2_2    conda-forge
 zict                      2.0.0                      py_0    conda-forge
 zipp                      3.4.0                      py_0    conda-forge
 zlib                      1.2.11            h516909a_1010    conda-forge
 zstd                      1.4.5                h6597ccf_2    conda-forge

@madsbk madsbk added bug Something isn't working Needs Triage Need team to review and classify labels Jan 5, 2021
@madsbk madsbk mentioned this issue Jan 5, 2021
Merged
4 tasks
@devavret
Copy link
Contributor

devavret commented Jan 5, 2021

Couldn't repro on latest changes on 10.2.

@harrism harrism changed the title [BUG] to_parquet using "cudf" enigne fails [BUG] to_parquet using "cudf" engine fails Jan 5, 2021
@madsbk
Copy link
Member Author

madsbk commented Jan 6, 2021

Couldn't repro on latest changes on 10.2.

It seems to also work in CI: #7082
I will try again with a clean environment

@kkraus14 kkraus14 added invalid This doesn't seem right libcudf Affects libcudf (C++/CUDA) code. 0 - Waiting on Author Waiting for author to respond to review and removed Needs Triage Need team to review and classify labels Jan 8, 2021
@madsbk
Copy link
Member Author

madsbk commented Jan 8, 2021

It turns out it was a version mismatch issue. Installing the rapids conda meta package when new packages are being released, different version of cudf, libcudf, librmm, rmm, etc. might be installed.

@madsbk madsbk closed this as completed Jan 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
0 - Waiting on Author Waiting for author to respond to review bug Something isn't working invalid This doesn't seem right libcudf Affects libcudf (C++/CUDA) code.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@madsbk @devavret @kkraus14