[R-package] R package crashes on windows when loaded together with {fansi} or anything that depends on it #4464
Comments
|
Thanks for the report and for using LightGBM @dfalbel ! I'll look into this as soon as possible. |
jameslamb
changed the title
|
Thanks for reporting that! So I think more details about the versions of R, RTools can be helpful to identify the cause. |
|
Hi @shiyu1994 thanks for taking a look at this. Here's the sessionInfo() of the system I can reproduce the error: This is using master lightgbm too. |
|
I'm seeing the same issue. I'm guessing this may be related to #4007 and #4259. Some further details: No crashRunning a clean install of the script below in a new project with renv enabled works for 3.2.1.99. See library(lightgbm)
data(agaricus.train, package='lightgbm')
train <- agaricus.train
dtrain <- lgb.Dataset(train$data, label = train$label)
model <- lgb.cv(
params = list(
objective = "regression"
, metric = "l2"
)
, data = dtrain
) Session InfoInstalling parsnip and loading it after lightgbm likewise does not result in a crash. library(lightgbm)
library(parsnip)
data(agaricus.train, package='lightgbm')
train <- agaricus.train
dtrain <- lgb.Dataset(train$data, label = train$label)
model <- lgb.cv(
params = list(
objective = "regression"
, metric = "l2"
)
, data = dtrain
)
Session InfoCrashHowever, loading parsnip before lightgbm results in a crash at the library(parsnip)
library(lightgbm)
data(agaricus.train, package='lightgbm')
train <- agaricus.train
dtrain <- lgb.Dataset(train$data, label = train$label)
model <- lgb.cv(
params = list(
objective = "regression"
, metric = "l2"
)
, data = dtrain
) Session InfoNotes
EditDid a quick trip through the Imports of parsnip, loading each library before lightgbm 1-by-1. The following libraries cause crashes: While the following cause no issues: I then traveled through the dependencies of tibble and dplyr to find the lowest level library call that will cause a crash. Seems like fansi may be the actual culprit. The script below causes a crash for me in a fresh environment with lightgbm 3.2.1 (from CRAN) and 3.2.1.99 (from GitHub) library(fansi)
library(lightgbm)
data(agaricus.train, package='lightgbm')
train <- agaricus.train
dtrain <- lgb.Dataset(train$data, label = train$label)
model <- lgb.cv(
params = list(
objective = "regression"
, metric = "l2"
)
, data = dtrain
) Session Info |
|
Thanks to everyone participating for your help and investigation! I am planning to test some theories about this tomorrow when I have some time and easy access to a Windows environment.
@dfalbel , I looked at the definition of that GHA job (https://github.com/curso-r/treesnip/actions/runs/1049626089/workflow). I noticed that there's a call of Seen in the logs for that step: https://github.com/curso-r/treesnip/runs/3116229035?check_suite_focus=true.
If possible, could you try adding @dfalbel and @dfsnow if you have time, could you also confirm whether you are using RStudio, and if so, whether your examples also produce this issue when that code is stored in a script and run with I suspect that both of you are using RStudio but @shiyu1994 did not in #4464 (comment), so I'd like to see if that is relevant. |
|
Hi @jameslamb, thanks for looking at this! I have added the For the second question, I can confirm that error happens on both RStudio and on a vanilla R session: |
|
I was able to reproduce this issue today using the latest environment info and install instructions (click me)I installed git clone --recursive git@github.com:microsoft/LightGBM.git
cd LightGBM
Rscript --vanilla -e "remove.packages('lightgbm')"
Rscript --vanilla -e "install.packages(c('R6', 'data.table', 'jsonlite'), repos = 'https://cran.r-project.org')"
Rscript --vanilla -e "install.packages(c('fansi'), repos = 'https://cran.r-project.org')"
sh build-cran-package.sh
R CMD INSTALL lightgbm_3.2.1.99.tar.gzI'm using Rtools40 downloaded on May 9, 2020, so not the newest one. As far as I can tell from https://cran.r-project.org/bin/windows/Rtools/history.html, it isn't possible to access previous versions of Rtools. Output of Rscript -e "library(fansi); library(lightgbm); sessionInfo()"Thanks to the helpful contributions of @dfalbel and @dfsnow so far, I was able to reduce this to an even smaller reproducible example, cutting out Running the script below with library(fansi)
library(lightgbm)
data(agaricus.train, package='lightgbm')
train <- agaricus.train
dtrain <- lgb.Dataset(train$data, label = train$label)
dtrain$construct()Next, I'll add a ton of logging to dataset construction to try to narrow down the issue further. I also plan to inspect |
|
Alright I added a bunch of log statements and I think I've narrowed down the place where this segfault is being thrown. I'm able to reproduce the issue on library(fansi)
library(lightgbm)
dtrain <- lgb.Dataset(
data = matrix(rnorm(1000), nrow = 100)
, label = rnorm(100)
)
dtrain$construct()The segfault is being thrown from calls to LightGBM/src/io/dataset_loader.cpp Line 626 in fdc582e I pushed a branch with all the extra logging and with those calls skipped. https://github.com/jameslamb/LightGBM/tree/misc/investigating-dataset-segfault. On that branch, the reproducible example runs successfully and does not produce a segfault. Next, I'm going to try to figure out how the behavior of this code is changed by loading |
|
I'm convinced that the root of the problem is related to the way that R loads DLLs, and that @dfsnow is right that If library(dplyr)
dyn.unload(file.path(.libPaths()[1], "fansi", "libs", "x64", "fansi.dll"))
library(lightgbm)
dtrain <- lgb.Dataset(
data = matrix(rnorm(1000), nrow = 100)
, label = rnorm(100)
)
dtrain$construct()If This finding plus the finding from #4464 (comment) that commenting out
I'm going to investigate this more closely with Updates to follow! |
|
Just to rule out another possibility like "loading any other package with compiled code before I attempted |
jameslamb
changed the title
|
@dfalbel @dfsnow thanks very much for your patience. I think I found the problem and have a fix up. Could you please try installing from my branch and let me know if it seems to resolve the issue? git clone --recursive https://github.com/microsoft/LightGBM.git --branch fix/network-setup
cd LightGBM
sh build-cran-package.sh
R CMD INSTALL lightgbm_3.2.1.99.tar.gz |
|
@jameslamb Your branch works for me. Fixes both library(fansi)
library(lightgbm)
data(agaricus.train, package='lightgbm')
train <- agaricus.train
dtrain <- lgb.Dataset(train$data, label = train$label)
model <- lgb.cv(
params = list(
objective = "regression",
metric = "l2"
) ,
data = dtrain
)Also works with renv. Thanks for the quick turnaround! |
|
Hey @jameslamb ! Thanks very much for the investigation and fix. I can confirm that this works great! |
When I run this I get file not found |
|
Some versions of the unix tools for Windows might have slightly different behavior. Can you try commenting out the uses of |
|
I commented this out (if I understood you correctly) Then I ran I tried to run this |
That doesn't look specific to When using R 4.x on Windows, if you plan to install packages from source it's expected that you have installed Rtools (click here to get it) at |
|
Also, since I just noticed that error message is about building the 32-bit version of the library ( If you DO have Rtools installed but during the installation you chose to only install the 64-bit components, then use |
|
I do have Rtools 4.0 installed but there is no mingw_64 folder under C:/rtools40/usr/ /mingw_64/bin/g++ does exist but not under /usr/. It's located in the root /rtools40/ |
|
oh! I see now. I think you might have downloaded RTools35 and installed it in directory Rtools35 has folders (from the root of Rtools) named I know this for sure because we use those paths in this project's CI LightGBM/.ci/test_r_package_windows.ps1 Lines 68 to 73 in dc09d1b LightGBM/.ci/test_r_package_windows.ps1 Lines 76 to 79 in dc09d1b You might need to visit https://cran.r-project.org/bin/windows/Rtools/ and get the newest version of Rtools. And you might find some of the discussion about similar issues (a path for Rtools being assumed and hard-coded into some versions of R) at https://stackoverflow.com/questions/39090983/rcpp-rtools-installed-but-error-message-g-not-found. Can you also please try installing another package requiring compilation from source? Rscript -e "install.packages('data.table', type = 'source', repos = 'https://cran.r-project.org')"I expect you'll experience this same problem doing that, and if you do then I think that would confirm that this isn't an issue with |
|
After solving a problem related to rtools everything works now :) This was installed for R-4.0.x even though I have R-4.1.x installed. Is this not supported for R 4.1.x? |
I'm not sure what you mean by this statement, sorry. If you have multiple versions of R on your system, please examine the You might also try the following from a command prompt to inspect which version of R is first on your PATH. # version of R
Rscript --version
# where the R executables are
Rscript -e "print(R.home())"
# where packages will be installed to / loaded from
Rscript -e "print(.libPaths())" |
|
I have multiple versions. 4.0.1 was first on PATH. Didn't know about it. Thanks for the help. |
|
Now that #4496 has been merged, I believe this issue has been resolved. Thanks so much to everyone involved here for your help with reproducible examples and debugging ideas! |
|
This issue has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this. |
This is probably related to:
Description
Using lightgbm while
parsnipis loaded crashes the R session with: Exited with status -1073741819.Reproducible example
Calling:
Environment info
I am using the dev version of LightGBM as suggested in #4007 (comment)
The error only occurs on Windows.
Here's a GitHub actions run that shows the behavior.
This shows that it works fine if parsnip is not loaded: https://github.com/curso-r/treesnip/runs/3037580458?check_suite_focus=true#step:9:1
And this one shows the error message: https://github.com/curso-r/treesnip/runs/3037580458?check_suite_focus=true#step:10:21
I could also reproduce it locally on a Windows machine, but I am not sure what's the best way to get a stack trace.
Let me know if I can help with further debugging.
The text was updated successfully, but these errors were encountered: