Refactor file bundling functions #684
Conversation
| @@ -166,6 +166,28 @@ isShinyRmd <- function(filename) { | |||
| } | |||
| is_shiny_prerendered(yaml$runtime, yaml$server) | |||
| } | |||
|
|
|||
| yamlFromRmd <- function(filename) { | |||
There was a problem hiding this comment.
Discovered this was in the wrong place while extracting the bundling functions.
|
|
||
| # remove any files lines listed .rscignore | ||
| if (".rscignore" %in% contents) { | ||
| ignoreContents <- readLines(file.path(dir, ".rscignore")) |
There was a problem hiding this comment.
Two changes here:
readLines(file.path(dir, ".rscignore"))instead ofreadLines(".rscignore"); if that ever worked before it was by coincidence- Also remove
.rscignoreitself from the bundle
There was a problem hiding this comment.
Likely worked because:
- ignore files usually at top level
- working directory almost always the content directory
|
To do:
|
R/bundleFiles.R
Outdated
| children <- character() | ||
| for (name in contents) { | ||
| if (isTRUE(is_dir[[name]])) { | ||
| dirList <- recursiveBundleFiles(file.path(dir, name), depth + 1) |
There was a problem hiding this comment.
This change removes of getOption("rsconnect.max.bundle.size") and getOption("rsconnect.max.bundle.files") as the walk progresses. This is an important check because folks occasionally use an appDir which is massive (a consequence of deploying $HOME/document.Rmd, for example).
Some historical issues with this code:
- maxDirectoryList tracks the running number of files and cumulative size #463 - incorrect size calculations
- Getting an error from rsconnect while publishing on bookdown.org #514 - permissions problems when enumerating
- enforce bundle limits when exploding file enumerations #546 - enforcement on
appFiles
I'm not finding the original issue, but folks observed "hangs" (very long runtime) if there's no feedback until after the entire directory is processed.
There was a problem hiding this comment.
Oh bummer 😬 I'll think about how to restore this.
#Conflicts: # NEWS.md # R/bundle.R # tests/testthat/_snaps/bundle.md # tests/testthat/test-bundle.R
And add tests
|
@aronatkins restored eager evaluation and added a test. |
| children <- append(children, name) | ||
| totalFiles <- totalFiles + 1 | ||
| totalSize <- totalSize + file_size(file.path(dir, name)) | ||
| } |
There was a problem hiding this comment.
Enforcement checks should happen after each name is processed rather than after the fill set of list.files. Consider a directory with 20k files, a directory containing 10 2GB files, or a deep, broad directory trees.
You do not need checks both after each file and prior to exit; checking in the loop body is sufficient.
There was a problem hiding this comment.
Yeah, I misunderstand the recursion here.
|
|
||
| # remove any files lines listed .rscignore | ||
| if (".rscignore" %in% contents) { | ||
| ignoreContents <- readLines(file.path(dir, ".rscignore")) |
There was a problem hiding this comment.
Likely worked because:
- ignore files usually at top level
- working directory almost always the content directory
bundleFiles.RlistBundleFiles()andexplodeFiles()to performenforceBundleLimits()at end. IMO this makes the recursion used in these functions much easier to understand.listBundleFiles()no errors if too big/too many files, rather than silently truncating, thus matching the behavior ofstandardizeAppFiles().listBundleFiles(), revealing a bug in the support for.rscignore. Fixes .rcsignore file not ignoring files #568.