[general] how to pass parameter to a target workflow

[weizhu365]

Help

• I understand and agree to https://books.ropensci.org/targets/help.html.

Description

I tried to pass some customized configurations/parameters to the target workflow, for example, loc="./user_files". I cannot figure a solution out.

Thank you in advance.

[joelnitta]

Can you please provide a reproducible example? It's pretty hard to figure out what you are trying to do from that description.

Anyways, you should be able to define objects outside of the targets list. You could also use an environmental variable. I've shown both below (note that use of tar_dir() and tar_script() are not needed when you write an actual _targets.R file; those are so that the code can be run as a reproducible example).

library(targets)

tar_dir({ # tar_dir() runs code from a temp dir
  # Write and run a target script:
  tar_script({
    library(targets)
    # Method 1: define object outside of targets list
    x <- 1
    # Method 2: use environmental variable
    Sys.setenv(y = "A")
    list(
      tar_target(
        my_target,
        x
      ),
      tar_target(
        my_other_target,
        Sys.getenv("y")
      )
    )
  })
  tar_make()
  print(tar_read(my_target))
  print(tar_read(my_other_target))
})
#> ▶ dispatched target my_other_target
#> ● completed target my_other_target [0 seconds, 53 bytes]
#> ▶ dispatched target my_target
#> ● completed target my_target [0 seconds, 51 bytes]
#> ▶ ended pipeline [0.057 seconds]
#> [1] 1
#> [1] "A"

^{Created on 2024-12-17 with reprex v2.1.0}

Session info

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.4.1 (2024-06-14)
#>  os       macOS Sonoma 14.6.1
#>  system   aarch64, darwin20
#>  ui       X11
#>  language (EN)
#>  collate  en_US.UTF-8
#>  ctype    UTF-8
#>  tz       Asia/Tokyo
#>  date     2024-12-17
#>  pandoc   3.1.2 @ /usr/local/bin/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version date (UTC) lib source
#>  backports     1.5.0   2024-05-23 [1] CRAN (R 4.4.0)
#>  base64url     1.4     2018-05-14 [1] CRAN (R 4.4.0)
#>  callr         3.7.6   2024-03-25 [1] CRAN (R 4.4.0)
#>  cli           3.6.3   2024-06-21 [1] CRAN (R 4.4.0)
#>  codetools     0.2-20  2024-03-31 [1] CRAN (R 4.4.1)
#>  data.table    1.15.4  2024-03-30 [1] CRAN (R 4.4.0)
#>  digest        0.6.36  2024-06-23 [1] CRAN (R 4.4.0)
#>  evaluate      0.24.0  2024-06-10 [1] CRAN (R 4.4.0)
#>  fansi         1.0.6   2023-12-08 [1] CRAN (R 4.4.0)
#>  fastmap       1.2.0   2024-05-15 [1] CRAN (R 4.4.0)
#>  fs            1.6.4   2024-04-25 [1] CRAN (R 4.4.0)
#>  glue          1.7.0   2024-01-09 [1] CRAN (R 4.4.0)
#>  htmltools     0.5.8.1 2024-04-04 [1] CRAN (R 4.4.0)
#>  igraph        2.0.3   2024-03-13 [1] CRAN (R 4.4.0)
#>  knitr         1.48    2024-07-07 [1] CRAN (R 4.4.0)
#>  lifecycle     1.0.4   2023-11-07 [1] CRAN (R 4.4.0)
#>  magrittr      2.0.3   2022-03-30 [1] CRAN (R 4.4.0)
#>  pillar        1.9.0   2023-03-22 [1] CRAN (R 4.4.0)
#>  pkgconfig     2.0.3   2019-09-22 [1] CRAN (R 4.4.0)
#>  processx      3.8.4   2024-03-16 [1] CRAN (R 4.4.0)
#>  ps            1.8.0   2024-09-12 [1] CRAN (R 4.4.1)
#>  purrr         1.0.2   2023-08-10 [1] CRAN (R 4.4.0)
#>  R.cache       0.16.0  2022-07-21 [1] CRAN (R 4.4.0)
#>  R.methodsS3   1.8.2   2022-06-13 [1] CRAN (R 4.4.0)
#>  R.oo          1.26.0  2024-01-24 [1] CRAN (R 4.4.0)
#>  R.utils       2.12.3  2023-11-18 [1] CRAN (R 4.4.0)
#>  R6            2.5.1   2021-08-19 [1] CRAN (R 4.4.0)
#>  reprex        2.1.0   2024-01-11 [1] CRAN (R 4.4.0)
#>  rlang         1.1.4   2024-06-04 [1] CRAN (R 4.4.0)
#>  rmarkdown     2.27    2024-05-17 [1] CRAN (R 4.4.0)
#>  secretbase    1.0.0   2024-06-16 [1] CRAN (R 4.4.0)
#>  sessioninfo   1.2.2   2021-12-06 [1] CRAN (R 4.4.0)
#>  styler        1.10.3  2024-04-07 [1] CRAN (R 4.4.0)
#>  targets     * 1.8.0   2024-10-02 [1] CRAN (R 4.4.1)
#>  tibble        3.2.1   2023-03-20 [1] CRAN (R 4.4.0)
#>  tidyselect    1.2.1   2024-03-11 [1] CRAN (R 4.4.0)
#>  utf8          1.2.4   2023-10-22 [1] CRAN (R 4.4.0)
#>  vctrs         0.6.5   2023-12-01 [1] CRAN (R 4.4.0)
#>  withr         3.0.1   2024-07-31 [1] CRAN (R 4.4.0)
#>  xfun          0.45    2024-06-16 [1] CRAN (R 4.4.0)
#>  yaml          2.3.9   2024-07-05 [1] CRAN (R 4.4.0)
#> 
#>  [1] /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library
#> 
#> ──────────────────────────────────────────────────────────────────────────────

[weizhu365]

@joelnitta Thank you for your prompt reply. And the method you provide can partially solve my problem. I am looking for a more elegant solution if possible, something like tar_option_set or tar_config_set. Unfortunately, neither tar_option_set nor tar_config_set can pass an R object (if I understood correctly). For example, I want to pass an R list my_config=list(x=1, y=2) to the specific targets workflow, and my_config could be generated from the command-line arguments. Sys.setenv is more like global variable: it can pass a character (but not any R object) and it might be some side effect due to its scope.

[joelnitta]

tar_option_set() and tar_config_set() are specifically for setting the default arguments to tar_target().

Are you trying to wrap tar_make() in a command-line call (i.e., put it in a shell script)?

[weizhu365]

Yes. More specifically, I am trying to run tar_make in a non-interactive way, which could be more reproducible. I think it is good to have options to provide those non-default arguments to the workflow. If there is no good way, I am going to use R save/load to pass a general R object.

[joelnitta]

The code executed bytar_make() runs in a non-interactive session. From that standpoint, it is already highly reproducible.

I think it is good to have options to provide those non-default arguments to the workflow

It is still not clear to me what you are trying to do.

• If you want to modify the default arguments to tar_target(), use tar_option_set() and/or tar_config_set().

• If you want to set values to arguments of functions, just pass those values in your code.

• If you want to change the default arguments of functions, there are various ways to do that; this blog post may be helpful.

[weizhu365]

Sorry that I have not made it clear. Here is one example.

Rscript mycode.R --x 1 --y "str"

mycode.R will run tar_make and start a workflow _targets.R where the variables of "x" and "y" need to be assigned and used by tar_targets.

More generally, I would like to pass an R object: var=list(x=1, y="str") to a tar_script _targets.R.

[joelnitta]

Sounds like you want commandArgs(). See this example on SO.

FWIW I think this question is getting beyond the scope of {targets}. It's more a matter of how to pass things from the terminal to R.

[weizhu365]

I am developing a R program with many command-line options and tar_targets. Personally, I prefer to separate CLI from the code dealing with the actual work. :-)

[joelnitta]

How about Rscript -e "targets::tar_make()"?

[joelnitta]

... OK, I think I finally understand what's going on here.

It sounds like you have a bigger workflow running things outside of R, and you want to use {targets} for some part of it in R? If that is the case, I would recommend considering inverting it: make {targets} your primary workflow manager, and call other programs from R with functions like processx::run().

IMHO, {targets} works best as a primary workflow manager for projects in R. If you go outside of that paradigm, you are stretching what it was designed to do.

[weizhu365]

No, I am developing a program for data analyses using R. I am using Snakemake for other things in general. Nevertheless, this is a nice discussion. Thank you very much.

[joelnitta]

No problem. Good luck!