devtools::check() is slow

[vandepitte]

The preparing step in devtools::check() is very slow. It takes about 5 minutes.

─ preparing '[my-package-name]': (5m 42.6s)

I noticed that removing directories with lots of files like .git helps, but this is not very practical. My package manager is also installing files in the project directory, so I guess this is also slowing down.

Is there a way to speed this up? I looked into the documentation and I tried using the .Rbuildignore but it does not ignore the files during this step.

[jimhester]

Thanks for opening this issue!

We are tracking this in r-lib/rcmdcheck#90 and r-lib/pkgbuild#59. However it is really due to the behavior of R CMD build. It is largely out of our hands and would require a patch for R to be written and someone on R core to accept it.

I am not sure it is possible to write without breaking backwards compatibility, and I have doubts on R core accepting it, which is why no one has undertaken it to this date.

I would suggest you move directories with lots of files outside the package directory.

[vandepitte]

Thanks for pointing me to the existing open issues and sorry having created a new one for that. Wasn't aware of them.

Unfortunately, it is not possible for me to move the .git directory nor to change the lib path. Will have to wait until a solution is available, and meanwhile run less checks.

Maybe I will start writing my own check that first copies everything except the files matching in .Rbuildignore to a temp directory, and start the check from there

[vandepitte]

FYI, did write such function (okay, maybe not perfect) to check the project much faster, by first copying the files to a build directory (ignoring files specified .Rbuildignore), then performing devtools::check.

check <- function(pkg = ".", tmp_pkg = "build", ...) {
  rules <- readLines(file.path(pkg, ".Rbuildignore"))
  list.files(path = pkg, recursive = T) %>% 
    purrr::discard(function(f) purrr::map(rules, function(r) grepl(r, f)) %>% purrr::reduce(sum) > 0) %>%
    purrr::walk(function(f) {
      new_file <-file.path(tmp_pkg, f)
      dir.create(dirname(new_file),  showWarnings = F, recursive = T)
      file.copy(f, new_file, overwrite = T)
    })
  devtools::check(tmp_pkg, ...)
}

The good thing about this approach is:

1. it's entirely written in R,
2. it is much faster now
3. and it consumes the .Rbuildignore files to do the filtering (so the normal check will have no files to filter anymore).

Maybe a good idea to include this in devtools ? E.g. call it devtools::check_filtered

The bad thing: it is fast, but can even be faster

I then wrote a shell script to perform the same, but using rsync

#!/usr/bin/env bash
rsync -a --exclude build --exclude .git --exclude packrat --exclude .Rprofile . build
R --no-save -q -e "devtools::check('build')"

The good thing: it's really fast

The bad thing: rsync does not work with regular expressions, so I can't use the .Rbuildignore. This means you have to update the script for your own needs (I excluded .git, packrat and .Rprofile (because it initializes packrat, which itself is excluded)

Enjoy