[general] Random number generation in tar_make vs. tar_make_future()

[svraka]

Help

• I understand and agree to https://books.ropensci.org/targets/help.html.

Description

Apologies, if this is obvious but I kept my head banging against the desk before figuring it out, so I thought it might be useful for others.

I have a pipeline which started its life as a fairly simple one, with no need for parallel execution. As the project evolved, I added some targets which could benefit from parallelism. While setting things up for local workers for tar_make_future(), I noticed targets using RNGs changing their output. See this reprex:

library(targets)

tar_dir({
  tar_script({
    library(targets)
    list(tar_target(rand, runif(1)))
  })

  tar_make()
  meta_tar_make <- tar_meta()
  tar_destroy(ask = FALSE)
  tar_make_future()
  meta_tar_make_future <- tar_meta()
})
#> ▶ start target rand
#> ● built target rand [0 seconds]
#> ▶ end pipeline [0.08 seconds]
#> ▶ start target rand
#> ● built target rand [0 seconds]
#> ▶ end pipeline [0.31 seconds]

meta_tar_make[, c("data", "seed")]
#> # A tibble: 1 × 2
#>   data                   seed
#>   <chr>                 <int>
#> 1 82500bbe87ba979b 1053913074
meta_tar_make_future[, c("data", "seed")]
#> # A tibble: 1 × 2
#>   data                   seed
#>   <chr>                 <int>
#> 1 54500d29355b0f94 1053913074

Note that both runs use the same seeds!

If the problematic target is set to run in the main pipeline, we get back the old results. I eventually ended up with tar_option_set(deployment = "main") for the whole project as for most of my targets are it doesn’t make much sense to run in a separate worker.

library(targets)

tar_dir({
  tar_script({
    library(targets)
    tar_option_set(deployment = "main")
    list(tar_target(rand, runif(1)))
  })

  tar_make()
  meta_tar_make <- tar_meta()
  tar_destroy(ask = FALSE)
  tar_make_future()
  meta_tar_make_future <- tar_meta()
})
#> ▶ start target rand
#> ● built target rand [0 seconds]
#> ▶ end pipeline [0.06 seconds]
#> ▶ start target rand
#> ● built target rand [0 seconds]
#> ▶ end pipeline [0.29 seconds]

meta_tar_make[, c("data", "seed")]
#> # A tibble: 1 × 2
#>   data                   seed
#>   <chr>                 <int>
#> 1 82500bbe87ba979b 1053913074
meta_tar_make_future[, c("data", "seed")]
#> # A tibble: 1 × 2
#>   data                   seed
#>   <chr>                 <int>
#> 1 82500bbe87ba979b 1053913074

What is the explanation though? Seed are the same but the state is different between runs. Does setting up a future worker use RNGs?

library(targets)

tar_dir({
  tar_script({
    library(targets)
    list(tar_target(rand, .Random.seed))
  })

  tar_make()
  meta_tar_make <- tar_meta()
  tar_destroy(ask = FALSE)
  tar_make_future()
  meta_tar_make_future <- tar_meta()
})
#> ▶ start target rand
#> ● built target rand [0 seconds]
#> ▶ end pipeline [0.08 seconds]
#> ▶ start target rand
#> ● built target rand [0 seconds]
#> ▶ end pipeline [0.31 seconds]

meta_tar_make[, c("data", "seed")]
#> # A tibble: 1 × 2
#>   data                   seed
#>   <chr>                 <int>
#> 1 9e751f017a8e9f85 1053913074
meta_tar_make_future[, c("data", "seed")]
#> # A tibble: 1 × 2
#>   data                   seed
#>   <chr>                 <int>
#> 1 a5e373229aab9234 1053913074

Anyways, hopefully this little example can help someone who is just as confused as I was.

^{Created on 2023-10-04 with reprex v2.0.2}

Session info

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.3.1 (2023-06-16 ucrt)
#>  os       Windows 10 x64 (build 19045)
#>  system   x86_64, mingw32
#>  ui       RTerm
#>  language (EN)
#>  collate  Hungarian_Hungary.utf8
#>  ctype    Hungarian_Hungary.utf8
#>  tz       Europe/Budapest
#>  date     2023-10-04
#>  pandoc   3.1.8 @ C:/scoop/shims/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version    date (UTC) lib source
#>  backports     1.4.1      2021-12-13 [1] CRAN (R 4.3.0)
#>  base64url     1.4        2018-05-14 [1] CRAN (R 4.3.1)
#>  callr         3.7.3      2022-11-02 [1] CRAN (R 4.3.1)
#>  cli           3.6.1      2023-03-23 [1] CRAN (R 4.3.1)
#>  codetools     0.2-19     2023-02-01 [2] CRAN (R 4.3.1)
#>  data.table    1.14.9     2023-07-27 [1] Github (Rdatatable/data.table@8803918)
#>  digest        0.6.33     2023-07-07 [1] CRAN (R 4.3.1)
#>  evaluate      0.21       2023-05-05 [1] CRAN (R 4.3.1)
#>  fansi         1.0.4      2023-01-22 [1] CRAN (R 4.3.1)
#>  fastmap       1.1.1      2023-02-24 [1] CRAN (R 4.3.1)
#>  fs            1.6.3      2023-07-20 [1] CRAN (R 4.3.1)
#>  future        1.33.0     2023-07-01 [1] CRAN (R 4.3.1)
#>  globals       0.16.2     2022-11-21 [1] CRAN (R 4.3.0)
#>  glue          1.6.2      2022-02-24 [1] CRAN (R 4.3.1)
#>  htmltools     0.5.6      2023-08-10 [1] CRAN (R 4.3.1)
#>  igraph        1.5.1      2023-08-10 [1] CRAN (R 4.3.1)
#>  knitr         1.43       2023-05-25 [1] CRAN (R 4.3.1)
#>  lifecycle     1.0.3      2022-10-07 [1] CRAN (R 4.3.1)
#>  listenv       0.9.0      2022-12-16 [1] CRAN (R 4.3.1)
#>  magrittr      2.0.3      2022-03-30 [1] CRAN (R 4.3.1)
#>  parallelly    1.36.0     2023-05-26 [1] CRAN (R 4.3.0)
#>  pillar        1.9.0      2023-03-22 [1] CRAN (R 4.3.1)
#>  pkgconfig     2.0.3      2019-09-22 [1] CRAN (R 4.3.1)
#>  processx      3.8.2      2023-06-30 [1] CRAN (R 4.3.1)
#>  ps            1.7.5      2023-04-18 [1] CRAN (R 4.3.1)
#>  R6            2.5.1      2021-08-19 [1] CRAN (R 4.3.1)
#>  reprex        2.0.2      2022-08-17 [1] CRAN (R 4.3.1)
#>  rlang         1.1.1      2023-04-28 [1] CRAN (R 4.3.1)
#>  rmarkdown     2.24       2023-08-14 [1] CRAN (R 4.3.1)
#>  rstudioapi    0.15.0     2023-07-07 [1] CRAN (R 4.3.1)
#>  sessioninfo   1.2.2      2021-12-06 [1] CRAN (R 4.3.1)
#>  targets     * 1.2.2.9001 2023-08-23 [1] Github (ropensci/targets@efa5da8)
#>  tibble        3.2.1      2023-03-20 [1] CRAN (R 4.3.1)
#>  tidyselect    1.2.0      2022-10-10 [1] CRAN (R 4.3.1)
#>  utf8          1.2.3      2023-01-31 [1] CRAN (R 4.3.1)
#>  vctrs         0.6.3      2023-06-14 [1] CRAN (R 4.3.1)
#>  withr         2.5.0      2022-03-03 [1] CRAN (R 4.3.1)
#>  xfun          0.40       2023-08-09 [1] CRAN (R 4.3.1)
#>  yaml          2.3.7      2023-01-23 [1] CRAN (R 4.3.0)
#> 
#>  [1] C:/Users/SvrakaA/AppData/Local/R/win-library/4.3
#>  [2] c:/R/R-4.3.1/library
#> 
#> ──────────────────────────────────────────────────────────────────────────────

[wlandau]

I am almost positive this is due to futureverse/furrr#265. For a furrr task, I got different results in different R sessions despite setting the same seed. I think it's something going on in future which prevents users from controlling seeds manually.

Anyway, I recommend tar_make() with crew rather than tar_make_future(): https://books.ropensci.org/targets/crew.html

[svraka]

Thanks, I thought this was intended like this.

In the current project I'm gonna stick to future with the workaround, as I don't want to mess with the renv setup (and annoy my co-authors) but moving forward I'll give crew a try!

[wlandau]

Turns out I can fix this on targets' end with seed = FALSE in future::future(): b6c8ad8