WSL version of cmdstanr 0.5.3 is incompatible with brms::brm(backend = “cmdstanr”)

[CLRafaelR]

Describe the bug

I have two different version of cmdstanr 0.5.3 in my Windows 10 raptop: one runs in the native Windows environment and the other runs in the Windows Subsystem for Linux (WSL). Both versions of cmdstanr works fine when I run Cmdstanr's starting example (Getting started with CmdStanR). However, I found that brms::brm(backend = "cmdstanr") only works with the native Windows version of cmdstanr, not with the WSL version.

The issue has already been reported in the Stan forum, but I repost it here since it has not received any comment over a week.

To Reproduce

The example of brms::brm(count ~ zAge + zBase * Trt + (1 | patient), ...) came from the documentation of brm() {brms} 2.17.0.

library("brms")
library("cmdstanr")

data_list <- list(N = 10, y = c(0, 1, 0, 0, 0, 0, 0, 0, 0, 1))

#----------------------------------------------
# Set path to native windows' cmdstan
#----------------------------------------------

stringr::str_replace_all(
  string = Sys.getenv("HOME"),
  pattern = "\\\\",
  replacement = "/"
) |>
  paste0("/.cmdstan/cmdstan-2.30.1") |>
  set_cmdstan_path()
cmdstan_path()

#----------------------------------------------
# Brms' example run under native Windows
#----------------------------------------------

prior1 <- prior(normal(0, 10), class = b) +
  prior(cauchy(0, 2), class = sd)

fit_win_brm <- brm(
  count ~ zAge + zBase * Trt + (1 | patient),
  data = epilepsy,
  family = poisson(),
  prior = prior1,
  backend = "cmdstanr"
)

fit_win_brm

#----------------------------------------------
# Cmdstanr's example run under native Windows
#----------------------------------------------

file_win <- file.path(cmdstan_path(), "examples", "bernoulli", "bernoulli.stan")
mod_win <- cmdstan_model(file_win)

fit_win <- mod_win$sample(
  data = data_list,
  seed = 123,
  chains = 4,
  parallel_chains = 4,
  refresh = 500 # print update every 500 iters
)

fit_win

#----------------------------------------------
# Set path to native WSL's cmdstan
#----------------------------------------------

stringr::str_replace_all(
  string = Sys.getenv("HOME"),
  pattern = "\\\\",
  replacement = "/"
) |>
  paste0("/.cmdstan/wsl-cmdstan-2.30.1") |>
  set_cmdstan_path()
cmdstan_path()

#------------------------------------
# Brms' example run under WSL
#------------------------------------

fit_wsl_brm <- brm(
  count ~ zAge + zBase * Trt + (1 | patient),
  data = epilepsy,
  family = poisson(),
  prior = prior1,
  backend = "cmdstanr"
)

## Error in `process_initialize(self, private, command, args, stdin, stdout, …`:
## ! Native call to `processx_exec` failed
## Caused by error in `chain_call(c_processx_exec, command, c(command, args), pty, pty_options, …`:
## ! Command 'bin/stanc.exe' not found @win/processx.c:982 (processx_exec)
## Type .Last.error to see the more details.

## > .Last.error
## <c_error/rlib_error_3_0/rlib_error/error>
## Error in `process_initialize(self, private, command, args, stdin, stdout, …`:
## ! Native call to `processx_exec` failed
## Caused by error in `chain_call(c_processx_exec, command, c(command, args), pty, pty_options, …`:
## ! Command 'bin/stanc.exe' not found @win/processx.c:982 (processx_exec)
## ---
## Backtrace:
##  1. brms::brm(count ~ zAge + zBase * Trt + (1 | patient), data = epilepsy, …
##  2. brms:::.make_stancode(bterms, data = data, prior = prior, stanvars = stanvars, …
##  3. brms:::.canonicalize_stan_model(tmp_file, overwrite_file = FALSE)
##  4. processx::run(command = stanc_cmd, args = c(stan_file, stanc_flags), …
##  5. process$new(command, args, echo_cmd = echo_cmd, wd = wd, windows_verbatim_args = windows_verbatim_args, …
##  6. local initialize(...)
##  7. processx:::process_initialize(self, private, command, args, stdin, stdout, …
##  8. processx:::chain_call(c_processx_exec, command, c(command, args), pty, pty_options, …
##  9. | base::withCallingHandlers(do.call(".Call", list(.NAME, ...)), error = function(e) { …
## 10. | base::do.call(".Call", list(.NAME, ...))
## 11. | base::.handleSimpleError(function (e) …
## 12. | local h(simpleError(msg, call))
## 13. | processx:::throw_error(err, parent = e)

fit_wsl

## Error: object 'fit_wsl' not found

#------------------------------------
# Cmdstanr's example run under WSL
#------------------------------------

file_wsl <- file.path(cmdstan_path(), "examples", "bernoulli", "bernoulli.stan")
mod_wsl <- cmdstan_model(file_wsl)

fit_wsl <- mod_wsl$sample(
  data = data_list,
  seed = 123,
  chains = 4,
  parallel_chains = 4,
  refresh = 500 # print update every 500 iters
)

fit_wsl

Expected behavior

fit_wsl_brm returns the similar results as fit_win_brm.

Operating system

Windows 10 Pro

CmdStanR version number

0.5.3

Additional context

> sessionInfo()
R version 4.2.0 (2022-04-22 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19044)

Matrix products: default

locale:
[1] LC_COLLATE=Japanese_Japan.utf8  LC_CTYPE=Japanese_Japan.utf8
[3] LC_MONETARY=Japanese_Japan.utf8 LC_NUMERIC=C
[5] LC_TIME=Japanese_Japan.utf8

attached base packages:
[1] stats     graphics  grDevices datasets  utils     methods   base

other attached packages:
[1] cmdstanr_0.5.3 brms_2.17.0    Rcpp_1.0.9

loaded via a namespace (and not attached):
 [1] nlme_3.1-158         matrixStats_0.62.0   xts_0.12.1
 [4] threejs_0.3.3        rstan_2.26.13        tensorA_0.36.2
 [7] tools_4.2.0          backports_1.4.1      utf8_1.2.2
[10] R6_2.5.1             DT_0.23              DBI_1.1.3
[13] colorspace_2.0-3     withr_2.5.0          tidyselect_1.1.2
[16] gridExtra_2.3        prettyunits_1.1.1    processx_3.7.0
[19] Brobdingnag_1.2-7    curl_4.3.2           compiler_4.2.0
[22] cli_3.3.0            shinyjs_2.1.0        colourpicker_1.1.1
[25] posterior_1.2.2      scales_1.2.0         dygraphs_1.1.1.6
[28] checkmate_2.1.0      mvtnorm_1.1-3        ggridges_0.5.3
[31] callr_3.7.1          stringr_1.4.0        digest_0.6.29
[34] StanHeaders_2.26.13  base64enc_0.1-3      pkgconfig_2.0.3
[37] htmltools_0.5.3      fastmap_1.1.0        htmlwidgets_1.5.4
[40] rlang_1.0.4          shiny_1.7.2          farver_2.1.1
[43] generics_0.1.3       zoo_1.8-10           jsonlite_1.8.0
[46] crosstalk_1.2.0      gtools_3.9.3         dplyr_1.0.9
[49] distributional_0.3.0 inline_0.3.19        magrittr_2.0.3
[52] loo_2.5.1            bayesplot_1.9.0      Matrix_1.4-1
[55] munsell_0.5.0        fansi_1.0.3          abind_1.4-5
[58] lifecycle_1.0.1      stringi_1.7.8        pkgbuild_1.3.1
[61] plyr_1.8.7           grid_4.2.0           parallel_4.2.0
[64] promises_1.2.0.1     crayon_1.5.1         miniUI_0.1.1.1
[67] lattice_0.20-45      knitr_1.39           ps_1.7.1
[70] pillar_1.8.0         igraph_1.3.4         markdown_1.1
[73] shinystan_2.6.0      reshape2_1.4.4       codetools_0.2-18
[76] stats4_4.2.0         rstantools_2.2.0     glue_1.6.2
[79] V8_4.2.0             data.table_1.14.2    renv_0.15.5
[82] RcppParallel_5.1.5   vctrs_0.4.1          httpuv_1.6.5
[85] gtable_0.3.0         purrr_0.3.4          assertthat_0.2.1
[88] ggplot2_3.3.6        xfun_0.31            mime_0.12
[91] xtable_1.8-4         coda_0.19-4          later_1.3.0
[94] tibble_3.1.8         shinythemes_1.2.0    ellipsis_0.3.2
[97] bridgesampling_1.1-2

[andrjohns]

Thanks for catching this. The WSL backend is currently undergoing a major refactor, I'll make sure to have this addressed/compatible in the refactor

[andrjohns]

brms compatibility will be fixed with the merge of this PR, and the WSL backend will likely see noticeable improvements once #696 is ready to merge

[andrjohns]

@CLRafaelR If you install the github version of brms, it should now be compatible with WSL cmdstanr

[CLRafaelR]

@andrjohns

Thank you for letting me know and sorry for the late reply. Just one clarification question: should I only install the github version of brms (i.e. run only remotes::install_github("paul-buerkner/brms")) or should I install not only the brms but also the latest github version of cmdstanr and rebuild CmdStan via running cmdstanr::rebuild_cmdstan()?

[andrjohns]

Just brms, there haven't been any cmdstanr changes yet

[CLRafaelR]

@andrjohns I understand. Thank you for your reply.