seed required when connecting to PostgreSQL in multisession plan?
#672
-
|
I'm getting warning about unexpected random numbers, when connecting to PostgreSQL (14) inside a future (and disconnecting immediately), when library(future)
plan(multisession) # plan(sequential)
f <- future::future({
con <- DBI::dbConnect(
drv = RPostgres::Postgres(),
dbname = "postgres",
user = "postgres",
password = "<password>"
)
DBI::dbDisconnect(con)
})
value(f)I skimmed through Session info: |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 9 replies
-
|
My best guess is that RPostgres, or one of its package dependencies, generates a random number when loaded or during initialization of the database, e.g. generate a random TCP port, a random temporary variable, or similar. It's not super common, but I've seen a few packages doing this. Sometimes this only happens once per R process, meaning if you call the same code multiple times, it only happens the first time, which could explain why you don't see it when using the 'sequential' backend; the RNG has been used before the future is evaluated. To help narrow in on code that generates random numbers (i.e. uses the RNG) when running interactively, see https://www.jottr.org/2020/09/21/detect-when-the-random-number-generator-was-used/. That has helped me, via trial and error, to identify where the RNG is used. |
Beta Was this translation helpful? Give feedback.
-
|
Interesting. Thanks for diving deeper into the details. I'm not aware of any best-practices around being conservative in using the RNG, and using it in stealth mode, but from a scientific point of view, it's an important issue because of reproducibility. AFAIK, there haven't been many active discussions on this in the R community, at least not in the forums I've followed. Either way, ... DBI is such an essential tools, so it might be worth reaching out to @krlmlr et al. to see if it is possible to restrict DBI (https://github.com/r-dbi/DBI/) and DBI backends from forwarding the RNG state - at least is some of the places and especially on package load. I suspect it doesn't need the RNG to be stochastically perfect, but just so it can draw some random integer, which then don't need the "fancy" RNG that R provides. If so, this might be something that could go into the 'DBI Specification' (https://dbi.r-dbi.org/articles/spec) and be validated via the DBItest (https://github.com/r-dbi/DBItest/). |
Beta Was this translation helpful? Give feedback.
-
|
Thanks. Does this still occur with the development version of RPostgres? |
Beta Was this translation helpful? Give feedback.
-
|
It does not! I checked couple of methods mentioned earlier. |
Beta Was this translation helpful? Give feedback.
-
|
I guess |
Beta Was this translation helpful? Give feedback.
-
|
No need, this is not a problem, I got into it more out of curiosity:) |
Beta Was this translation helpful? Give feedback.
My best guess is that RPostgres, or one of its package dependencies, generates a random number when loaded or during initialization of the database, e.g. generate a random TCP port, a random temporary variable, or similar. It's not super common, but I've seen a few packages doing this. Sometimes this only happens once per R process, meaning if you call the same code multiple times, it only happens the first time, which could explain why you don't see it when using the 'sequential' backend; the RNG has been used before the future is evaluated.
To help narrow in on code that generates random numbers (i.e. uses the RNG) when running interactively, see https://www.jottr.org/2020/09/21/detect-…