Set future to use parallel safe random number approach by default

[cdriveraus]

Is there an option to set future to always use the parallel safe rng approach, rather than having to set seed=TRUE for every future call? I searched and read a lot but somehow can't find this. I tried

options(future.seed=TRUE)

[scottkosty]

I do not believe there is an option (see, e.g., future options). As for why there's no option, although using an option for this might be convenient it could lead to code that can't be reproduced by others that don't set the option. In general, it's good practice to reduce global dependencies, especially ones that affect output.

An alternative that might work for you is to define a new function. The new function could be a wrapper, or it could be a copy of the function that changes the default argument. For example, suppose future_lapply is the function with default arguments that you want to change. You can do the following:

library("future.apply")

# ps = "parallel safe"
future_lapply_ps <- future_lapply
formals(future_lapply_ps)$future.seed <- TRUE

# this gives a warning because RNG is used without future.seed set:
future_lapply(X = c(1, 3), FUN = rnorm)
# no warning with the new function
future_lapply_ps(X = c(1, 3), FUN = rnorm)

[cdriveraus]

Thanks. I don't really see the reproducibility argument as it can be set in the script, and defining a new %<-% operator is more likely to confuse anyone looking in future, but good to know I wasn't just missing something obvious!

[scottkosty]

I don't really see the reproducibility argument as it can be set in the script

That's a fair point. Although it can be set in the script, users might accidentally set it in, e.g., ~/.Rprofile, as is done with a lot of R options. I don't think users expect R options to change output. Perhaps in this case they might think it's just a question of extra overhead and that output wouldn't change. In theory though, every user should read the documentation of the R option they set.

This of course is my personal point of view. And there are ways in standard R to affect output through global settings, e.g., by setting the seed or by setting RNGkind(), or options like na.omit which is used by lm.

[HenrikBengtsson]

Hi. @scottkosty has a good point about reproducibility issues if we introduce a global option for controlling the default value of seed argument. Assume there's a package OtherPkg with the following function:

run_it <- function(n, future.seed = getOption("future.seed", FALSE)) {
  future.apply::future_lapply(1:n, FUN = sqrt, future.seed = future.seed)
}

Then consider package MyPkg, that calls run_it() and then future_lapply() with random number generation, e.g.

my_fcn <- function(n) {
  y <- run_it(n)
  future.apply::future_lapply(y, FUN = function(x) {
    rnorm(1) + x
  }, future.seed = TRUE)
}

Now, the current, default behavior of future.seed (=FALSE), we get the following numerically reproducible values:

> set.seed(1); str(my_fcn(3))
List of 3
 $ : num 2.38
 $ : num -0.323
 $ : num 1.6

and again,

> set.seed(1); str(my_fcn(3))
List of 3
 $ : num 2.38
 $ : num -0.323
 $ : num 1.6

However, if we change the default behavior

> options(future.seed = TRUE)

we get other, numerically reproducible values:

> set.seed(1); str(my_fcn(3))
List of 3
 $ : num -1.3
 $ : num 1.15
 $ : num 3.6

and again,

> set.seed(1); str(my_fcn(3))
List of 3
 $ : num -1.3
 $ : num 1.15
 $ : num 3.6

So, the problem is that with a global variable, you're not only changing the behavior of the code you control in your script or in your package, but you're also changing the behavior in other packages that is out of your control.

So, we want to stay away from any global settings that affect the results elsewhere in R. FWIW, a famous example from the past was stringsAsFactors, which now has been removed. But, there are unfortunately other remaining in R, e.g. R options na.action and contrast.

The remaining discussion would be whether future.seed should default to FALSE (as now), or TRUE. This was deliberately set to FALSE, because it is expensive to set up parallel RNG streams, and we don't want to set them up if they're not used. In contrast, checking for use of RNG without declaring future.seed = TRUE, is cheap, which is why it throws a warning(*) about. For more background, see discussions in Issue #353 and #365.

(*) One could even argue that it should be an error rather than a warning, because in some statistical analyses it's a severe problem to not use proper RNGs.

[cdriveraus]

Fair point re other package behaviour. I ended up here after ignoring the warning when running some simulations, thinking that while there might be some issue it was not too critical a case -- instead every sim was exactly the same. So I guess I am in the either default to TRUE, or throw an error, camp.