Restoring older versions of r-universe packages doesn't work anymore?

[andrewheiss]

(I'm not sure if this is an r-universe issue or an {renv} issue…)

Based on r-universe's documentation (and #789 and #1359), {renv} seems like it's supposed to be able to do some magic behind-the-scenes work to install older versions of packages released on r-universe by looking up GitHub SHAs.

However, as of {renv} 1.0.7, it doesn't seem to be working anymore. For instance, as of right now, the most current version of {tinytable} at https://vincentarelbundock.r-universe.dev is 0.3.0.33, and this lockfile works fine with renv::restore():

{
  "R": {
    "Version": "4.4.0",
    "Repositories": [
      {
        "Name": "CRAN",
        "URL": "https://cloud.r-project.org"
      }
    ]
  },
  "Packages": {
    "renv": {
      "Package": "renv",
      "Version": "1.0.7",
      "Source": "Repository",
      "Repository": "CRAN",
      "Requirements": [
        "utils"
      ],
      "Hash": "397b7b2a265bc5a7a06852524dabae20"
    },
    "tinytable": {
      "Package": "tinytable",
      "Version": "0.3.0.33",
      "Source": "Repository",
      "Repository": "https://vincentarelbundock.r-universe.dev",
      "Requirements": [
        "R",
        "methods"
      ],
      "Hash": "d65e3b0b02d1cfe4714cfe4b00063141"
    }
  }
}

However, if the current version at r-universe changes (or if I change the lockfile manually to 0.3.0.32):

...
"tinytable": {
      "Package": "tinytable",
      "Version": "0.3.0.32",
...

…running renv::restore() results in this error:

> renv::restore()
The following package(s) will be updated:

# https://vincentarelbundock.r-universe.dev ----------------------------------
- tinytable   [0.3.0.33 -> 0.3.0.32]

Do you want to proceed? [Y/n]: 
y

# Downloading packages -------------------------------------------------------
- Querying repositories for available source packages ... Done!
Traceback (most recent calls last):
16: renv::restore()
15: renv_restore_run_actions(project, diff, current, lockfile, rebuild)
14: retrieve(packages)
13: handler(package, renv_retrieve_impl(package))
12: renv_retrieve_impl(package)
11: renv_retrieve_repos(record)
10: (function() {
        renv_scope_options(warn = 1L)
        for (error in errors$data()) warning(error)
    })()
 9: warning(error)
 8: withRestarts({
        .Internal(.signalCondition(cond, message, call))
        .Internal(.dfltWarn(message, call))
    }, muffleWarning = function() NULL)
 7: withOneRestart(expr, restarts[[1L]])
 6: doWithOneRestart(return(expr), restart)
 5: <condition-handler>(...)
 4: <condition-handler>(...)
 3: invoke_option_error_handler() at <text>#41
 2: eval(hnd, globalenv()) at <text>#194
 1: eval(hnd, globalenv())
Error:
failed to find binary for 'tinytable 0.3.0.32' in package repositories

For now, the only workaround I've found is to install older versions directly from GitHub SHAs (renv::install("package/name@some-SHA")) instead of using r-universe.

---

I ran into this issue when building a Docker container that worked great when I first created it, but after a couple days, when one of the r-universe packages it relied on incremented its version, things broke. (The solution there was to manually edit the version in the lockfile to the most recent version at r-universe: andrewheiss/mountainous-mackerel-docker#1 (comment))

[kevinushey]

Thanks for the bug report! This seems like an issue with renv; normally, the lockfile should include entries like RemoteSha and such so that we can identify the original git source for the package, but those are being excluded from the lockfile in this scenario.

Note that just manually updating the package version in the lockfile wouldn't normally be sufficient, since you'd need to be able to map the package version to the associated commit hash, but renv isn't currently able to do that. (r-universe probably has an API for this?)

[kevinushey]

I think I might know what's going on. renv::install() now writes out package metadata in a way that's compatible with pak; for example:

renv/R/package.R

Lines 155 to 189 in 6d80457

	renv_package_augment_standard <- function(record) {
	# only done for repository remotes
	if (!identical(record$Source, "Repository"))
	return(record)
	# check whether we tagged a url + type for this package
	url <- attr(record, "url", exact = TRUE)
	type <- attr(record, "type", exact = TRUE)
	name <- attr(record, "name", exact = TRUE)
	if (is.null(url) || is.null(type))
	return(record)
	# figure out base of repository URL
	pattern <- "/(?:bin|src)/"
	index <- regexpr(pattern, url, perl = TRUE)
	repos <- substring(url, 1L, index - 1L)
	# figure out the platform
	platform <- if (identical(type, "binary")) R.version$platform else "source"
	# build pak-compatible standard remote reference
	remotes <- list(
	RemoteType = "standard",
	RemotePkgRef = record$Package,
	RemoteRef = record$Package,
	RemoteRepos = repos,
	RemoteReposName = name,
	RemotePkgPlatform = platform,
	RemoteSha = record$Version
	)
	overlay(record, remotes)
	}

This change happened with:

0a4cc3b

However, to avoid lockfile noise from spurious diffs, renv tries to omit Remote fields for packages installed from a CRAN / standard remote. (The logic here being that users might choose to install CRAN packages using either pak or another tool, and having those diffs could be frustrating.)

We need to make sure renv does not omit this metadata for packages installed from r-universe.

[kevinushey]

I think this should now be resolved with the development builds of renv -- you should hopefully see the relevant RemoteSha fields populated when calling renv::snapshot() with a package installed from r-universe. Please let me know if you're still seeing any issues!

[andrewheiss]

Ooh it works and writes out a more complete entry now:

"tinytable": {
      "Package": "tinytable",
      "Version": "0.3.0.33",
      "Source": "Repository",
      "Repository": "https://vincentarelbundock.r-universe.dev",
      "RemoteUrl": "https://github.com/vincentarelbundock/tinytable",
      "RemoteSha": "3bc16a4c668e7a5321994c512cbd636f04f4c7a8",
      "Requirements": [
        "R",
        "methods"
      ],
      "Hash": "d65e3b0b02d1cfe4714cfe4b00063141"
    }