Stuck at plan(multisession) #578

fnavarro94 · 2022-01-04T20:58:56Z

fnavarro94
Jan 4, 2022

I am having trouble running the command

plan(multisession, workers = 2)

when I do so I get the following warnings and the code never runs past that command. Help much appreciated!

  running command '"C:/PROGRA~1/R/R-41~1.2/bin/x64/Rscript" -e "try(suppressWarnings(cat(Sys.getpid(),file=\"C:/Users/Felipe/AppData/Local/Temp/RtmpuYzmcY/worker.rank=1.parallelly.parent=21780.55145ced6e18.pid\")), silent = TRUE)" -e "file.exists(\"C:/Users/Felipe/AppData/Local/Temp/RtmpuYzmcY/worker.rank=1.parallelly.parent=21780.55145ced6e18.pid\")"' had status 1
Warning in system(test_cmd, intern = TRUE, input = input) :
  running command '"C:/PROGRA~1/R/R-41~1.2/bin/x64/Rscript" -e "try(suppressWarnings(cat(Sys.getpid(),file=\"C:/Users/Felipe/AppData/Local/Temp/RtmpuYzmcY/worker.rank=2.parallelly.parent=21780.551494a13f1.pid\")), silent = TRUE)" -e "file.exists(\"C:/Users/Felipe/AppData/Local/Temp/RtmpuYzmcY/worker.rank=2.parallelly.parent=21780.551494a13f1.pid\")"' had status 1

 version  R version 4.1.2 (2021-11-01)
 os       Windows 10 x64 (build 19041)
 system   x86_64, mingw32
 ui       RStudio
 language (EN)
 collate  English_United States.1252
 ctype    English_United States.1252
 tz       America/Bogota
 date     2022-01-04
 rstudio  2021.09.1+372 Ghost Orchid (desktop)
 pandoc   2.14.0.3 @ C:/Program Files/RStudio/bin/pandoc/ (via rmarkdown)

future::futuresessioninfo() outputs:

*** Package versions
future 1.23.0, parallelly 1.30.0, parallel 4.1.2, globals 0.14.0, listenv 0.8.0

*** Allocations
availableCores():
system 
    12 
availableWorkers():
$system
 [1] "localhost" "localhost" "localhost" "localhost" "localhost" "localhost" "localhost" "localhost" "localhost"
[10] "localhost" "localhost" "localhost"

*** Settings
- future.plan=<no set>
- future.fork.multithreading.enable=<no set>
- future.globals.maxSize=<no set>
- future.globals.onReference=<no set>
- future.resolve.recursive=<no set>
- future.rng.onMisuse=<no set>
- future.wait.timeout=<no set>
- future.wait.interval=<no set>
- future.wait.alpha=<no set>
- future.startup.script=<no set>

*** Backends
Number of workers: 2List of future strategies:
1. multisession:
   - args: function (..., envir = parent.frame(), workers = 2)
   - tweaked: TRUE
   - call: plan(multisession, workers = 2)
*** Basic tests
Error in makeClusterPSOCK(workers, ...) : 
  Cluster setup failed. %d of %d workers failed to connect.22
In addition: Warning messages:
1: In system(test_cmd, intern = TRUE, input = input) :
  running command '"C:/PROGRA~1/R/R-41~1.2/bin/x64/Rscript" -e "try(suppressWarnings(cat(Sys.getpid(),file=\"C:/Users/Felipe/AppData/Local/Temp/RtmpuYzmcY/worker.rank=1.parallelly.parent=21780.551467c36865.pid\")), silent = TRUE)" -e "file.exists(\"C:/Users/Felipe/AppData/Local/Temp/RtmpuYzmcY/worker.rank=1.parallelly.parent=21780.551467c36865.pid\")"' had status 1
2: In system(test_cmd, intern = TRUE, input = input) :
  running command '"C:/PROGRA~1/R/R-41~1.2/bin/x64/Rscript" -e "try(suppressWarnings(cat(Sys.getpid(),file=\"C:/Users/Felipe/AppData/Local/Temp/RtmpuYzmcY/worker.rank=2.parallelly.parent=21780.551428d01d7a.pid\")), silent = TRUE)" -e "file.exists(\"C:/Users/Felipe/AppData/Local/Temp/RtmpuYzmcY/worker.rank=2.parallelly.parent=21780.551428d01d7a.pid\")"' had status 1

Answered by HenrikBengtsson

Jan 4, 2022

@fnavarro94 found the answer - it was a broken ~/.Rprofile setup;

Ok, so I ran
 system2(rscript, args = c("-e", "Sys.getpid()"))
#> Error in library(RPostgreSQL) : there is no package called 'RPostgreSQL'
#> Execution halted
RPostgreSQL is a library that I load in in my .Rprofile. Anyways, I manually installed the library and now I can run

plan(multisession, workers = 2)

So, great! that is solved. ...

View full answer

HenrikBengtsson · 2022-01-04T21:10:23Z

HenrikBengtsson
Jan 4, 2022
Maintainer

Hi. Can you please edit your post to use a code block for that warning output. After that, what does future::futureSessionInfo() output?

5 replies

fnavarro94 Jan 4, 2022
Author

Absolutely, done.

HenrikBengtsson Jan 4, 2022
Maintainer

Got it. All packages are up-to-date. For some reason, the internal initial tests (part of parallelly) used to check if you can even launch a background worker fails. Does the following work for you:

rscript <- file.path(R.home("bin"), "Rscript")
rscript
#> [1] "C:/PROGRA~1/R/R-41~1.2/bin/x64/Rscript"
system2(rscript, args = "--version")
#> R scripting front-end version 4.1.2 (2021-11-01)

If that works, what about:

system2(rscript, args = c("-e", "Sys.getpid()"))
#> [1] 2840 # you'll get another number

? And if that works, what about:

pidfile <- normalizePath(tempfile(), winslash = "/", mustWork = FALSE)
#> [1] "C:/Users/hb/AppData/Local/Temp/RtmpITmm6Q/file1eb8cae79a1"
cmd <- sprintf('cat(Sys.getpid(),file="%s")', pidfile)
cmd
#> [1] "cat(Sys.getpid(),file=\"C:/Users/hb/AppData/Local/Temp/RtmpITmm6Q/file1eb8cae79a1\")"
res <- system2(rscript, args = c("-e", shQuote(cmd)))
res
#> [1] 0

?

fnavarro94 Jan 4, 2022
Author

Ok, so I ran

 system2(rscript, args = c("-e", "Sys.getpid()"))
#> Error in library(RPostgreSQL) : there is no package called 'RPostgreSQL'
#> Execution halted

RPostgreSQL is a library that I load in in my .Rprofile. Anyways, I manually installed the library and now I can run

plan(multisession, workers = 2)

So, great! that is solved. Now I have a different issue. I am trying to run queries in parallel like this:

system.time({
q1 %<-% dbGetQuery(conn, q1_sql)
q2 %<-% dbGetQuery(conn, q2_sql)
q3 %<-% dbGetQuery(conn, q3_sql)

})

but I get the following error

q1
#>Error in (function (classes, fdef, mtable)  : 
#>unable to find an inherited method for function 'dbSendQuery' for signature '"Amazon Athena", "character"'

do you know what is the cause of this? Thanks for the quick responces!

HenrikBengtsson Jan 4, 2022
Maintainer

RPostgreSQL is a library that I load in in my .Rprofile. Anyways, I manually installed the library and now ...

I see. That explains it. So, it's always better to load/attach packages during the R startup conditionally on them being installed, e.g.

if (!require("RPostgreSQL")) warning("Tried to attach RPostgreSQL during startup, but it's not available")

This avoids producing an error during the startup process, which you never want. This being just one example. I would even go as far as saying you should not attach packages during the startup if it's just for convenience of not having to type library(...). My $.02

For the dbSendQuery() issue, please post a new discussion topic, because that one might expand to something longer. I'd to keep discussions here short and concise so that they can be useful for future visitors.

fnavarro94 Jan 5, 2022
Author

Will do. Thanks for the help!

HenrikBengtsson · 2022-01-04T23:40:25Z

HenrikBengtsson
Jan 4, 2022
Maintainer

@fnavarro94 found the answer - it was a broken ~/.Rprofile setup;

Ok, so I ran
 system2(rscript, args = c("-e", "Sys.getpid()"))
#> Error in library(RPostgreSQL) : there is no package called 'RPostgreSQL'
#> Execution halted
RPostgreSQL is a library that I load in in my .Rprofile. Anyways, I manually installed the library and now I can run

plan(multisession, workers = 2)

So, great! that is solved. ...

0 replies

angelini75 · 2023-11-20T12:54:49Z

angelini75
Nov 20, 2023

I had similar problem, but different output. Check it out please

system2(rscript, args = c("-e", "Sys.getpid()"))
R library paths: 
 C:/Users/angel/Documents/R/win-library/4.3
 C:/Program Files/R/R-4.3.1/library
Error in file(filename, "r", encoding = encoding) : 
  cannot open the connection
Calls: source -> file
In addition: Warning message:
In file(filename, "r", encoding = encoding) :
  cannot open file 'packrat/init.R': No such file or directory
Execution halted

In addition:

> future::futureSessionInfo()
*** Package versions
future 1.33.0, parallelly 1.36.0, parallel 4.3.1, globals 0.16.2, listenv 0.9.0

*** Allocations
availableCores():
system 
    16 
availableWorkers():
$system
 [1] "localhost" "localhost" "localhost" "localhost" "localhost" "localhost" "localhost" "localhost"
 [9] "localhost" "localhost" "localhost" "localhost" "localhost" "localhost" "localhost" "localhost"


*** Settings
- future.plan=<not set>
- future.fork.multithreading.enable=<not set>
- future.globals.maxSize=<not set>
- future.globals.onReference=<not set>
- future.resolve.recursive=<not set>
- future.rng.onMisuse=<not set>
- future.wait.timeout=<not set>
- future.wait.interval=<not set>
- future.wait.alpha=<not set>
- future.startup.script=<not set>

*** Backends
Number of workers: 14
List of future strategies:
1. multisession:
   - args: function (..., workers = 14, envir = parent.frame())
   - tweaked: TRUE
   - call: plan(multisession, workers = 14)

*** Basic tests
Main R session details:
    pid     r sysname release     version nodename machine   login    user effective_user
1 31448 4.3.1 Windows  10 x64 build 19044  host001  x86-64 user001 user001        user001
There were 14 warnings (use warnings() to see them)

1:
.
.
.
13:
14: In system(test_cmd, intern = TRUE, input = input) :
  running command '"C:/PROGRA~1/R/R-43~1.1/bin/x64/Rscript" -e "try(suppressWarnings(cat(Sys.getpid(),file=\"C:/Users/angel/AppData/Local/Temp/RtmpS6c5qR/worker.rank=14.parallelly.parent=31448.7ad87ea65e.pid\")), silent = TRUE)" -e "file.exists(\"C:/Users/angel/AppData/Local/Temp/RtmpS6c5qR/worker.rank=14.parallelly.parent=31448.7ad87ea65e.pid\")"' had status 1

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Stuck at plan(multisession) #578

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 3 comments 5 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

Select a reply

Stuck at plan(multisession) #578

fnavarro94 Jan 4, 2022

Replies: 3 comments · 5 replies

HenrikBengtsson Jan 4, 2022 Maintainer

fnavarro94 Jan 4, 2022 Author

HenrikBengtsson Jan 4, 2022 Maintainer

fnavarro94 Jan 4, 2022 Author

HenrikBengtsson Jan 4, 2022 Maintainer

fnavarro94 Jan 5, 2022 Author

HenrikBengtsson Jan 4, 2022 Maintainer

angelini75 Nov 20, 2023

fnavarro94
Jan 4, 2022

Replies: 3 comments 5 replies

HenrikBengtsson
Jan 4, 2022
Maintainer

fnavarro94 Jan 4, 2022
Author

HenrikBengtsson Jan 4, 2022
Maintainer

fnavarro94 Jan 4, 2022
Author

HenrikBengtsson Jan 4, 2022
Maintainer

fnavarro94 Jan 5, 2022
Author

HenrikBengtsson
Jan 4, 2022
Maintainer

angelini75
Nov 20, 2023