-
-
Notifications
You must be signed in to change notification settings - Fork 967
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Force a render when user pkgs don't match the pkgs used in a rendered, shiny-prerendered document #1420
Conversation
behavior: * return false if there is a prerender_option = 0 or yell if not 0 or 1 * if the original file is newer than the compiled file, return true * if any external resource is newer, return true * NEW: if any htmldep's package version is different than the installed, return true * return false
This LGTM and will be a huge improvement on what we have currently (which is obviously much too fragile). |
Based on a code-pairing between me and Barret, there are some changes that incorporate R package dependencies. One thing that Barret noted was that the package dependencies (in the current form below) will incorporate packages like Here are the changes per file: util.R add: get_loaded_packages <- function() {
packages <- setdiff(loadedNamespaces(), "base")
loadedversion <- vapply(packages, getNamespaceVersion, "")
attached <- paste0("package:", packages) %in% search()
data.frame(packages = packages[attached], loadedversion = loadedversion[attached],
row.names = NULL, stringsAsFactors = FALSE)
} render.R change (ln 362): output_options$dependency_resolver <- function(deps) {
shiny_prerendered_dependencies <<- list(deps = deps, packages = get_loaded_packages())
list()
} shiny_prerendered.R change (ln 184): dependencies <- lapply(shiny_prerendered_dependencies$deps, function(dependency) { add (after ln 228): dependencies_pkgs_json <- jsonlite::serializeJSON(shiny_prerendered_dependencies$packages, pretty = FALSE)
shiny_prerendered_append_context(con, "package_dependencies", dependencies_pkgs_json) |
I think it is fine to ignore base R packages. The original problem was caused by packages that ship HTML dependencies. Base R packages don't have such things. |
Thanks, Yihui. To ignore the base R packages, the function get_loaded_packages <- function() {
base_r_packages <-
c("base", "compiler", "datasets", "graphics", "grDevices",
"grid", "methods", "parallel", "splines",
"stats", "stats4", "tcltk", "tools", "utils")
packages <- sort(setdiff(loadedNamespaces(), base_r_packages))
loadedversion <- vapply(packages, getNamespaceVersion, "")
attached <- paste0("package:", packages) %in% search()
data.frame(packages = packages[attached], loadedversion = loadedversion[attached],
row.names = NULL, stringsAsFactors = FALSE)
} |
Documenting decision towards using loaded packages (after execution) vs detecting library and There were two main approaches:
We decided to go with the loaded packages, as it can not be easily fooled and most renderings of a document are done within RStudio's clean environment and on hosted sites, which will compile in a clean environment. The only caveat are with console users, where rendering is known to not be in a clean environment by default. The good part about being aggressive with the execution package version matching is that users will get a cohesive experience. The worst interaction of not matching the execution packages is having to re-render the shiny prerendered document. This will be the first interaction of most any pre-render documents, but after re-rendering, it wont happen until the user (or service) updates their local packages. |
That sounds reasonable to me. |
…in prerender calculation Thanks @rich-iannone for the help!
R/util.R
Outdated
"base", "compiler", "datasets", "graphics", "grDevices", | ||
"grid", "methods", "parallel", "splines", | ||
"stats", "stats4", "tcltk", "tools", "utils" | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can use rownames(installed.packages(priority = 'base'))
instead of hardcoding the package names.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Aha! I thought there was a better way (but I couldn't find it). Thanks!
R/shiny_prerendered.R
Outdated
# check that the major R versions match | ||
if (!identical(R.version$major, execution_info$rMajorVersion)) { | ||
return(TRUE) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove -- we can use exact version matching for base packages.
R/util.R
Outdated
|
||
packages <- sort(setdiff(loadedNamespaces(), base_r_packages)) | ||
version <- vapply(packages, get_package_version_string, character(1)) | ||
attached <- paste0("package:", packages) %in% search() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to look at all loaded packages, not just the attached ones. For example, if someone does library(shiny)
, then this will only record the version of shiny, but not dependencies like htmltools.
Another case is if they do htmltools::div()
, it will not attach htmltools, and this code will also not record the version.
R/util.R
Outdated
# find all loaded packages. | ||
# May contain extra packages, but contains all packages used while knitting | ||
get_loaded_packages <- function() { | ||
base_r_packages <- rownames(installed.packages(priority = 'base')) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This can be removed -- we might as well look at the base R packages as well, since it's (marginally) safer to use them, and the code will be slightly simpler without excluding them. It also lets us not record the R major version because that comes along with these packages.
R/shiny_prerendered.R
Outdated
f = function(package, version) { | ||
!identical(get_package_version_string(package), version) | ||
} | ||
)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Change to for
loop with a short-circuit return.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See comments.
R/shiny_prerendered.R
Outdated
@@ -298,7 +293,8 @@ shiny_prerendered_append_dependencies <- function(input, # always UTF-8 | |||
|
|||
# write r major version and execution package dependencies | |||
execution_json <- jsonlite::serializeJSON( | |||
shiny_prerendered_dependencies[c("rMajorVersion", "packages")], | |||
# visibly display what is being stored | |||
shiny_prerendered_dependencies[c("packages")], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't need c()
here
We just need to make sure that Shiny Server and RSC run these in a way that ensures that no render occurs (on the assumption that they are in read-only directories). They can I believe do this via an environment variable (which they may or may not already be setting, worth checking on). |
@jjallaire The original envvar check of rstudio/learnr@942df45 now checks if the If |
@yihui removed [WIP], ready for merge when you're ready. Thank you! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is excellent. Thanks!
…, shiny-prerendered document (rstudio#1420) Fixes rstudio/learnr#169
Address: rstudio/learnr#169
By keeping track of the package version, we can force a new render if the user does not have the same package versions as the pre-rendered document.
This solves the error where a pre-rendered resource can not be mounted (because it doesn't exist!).