Handling of local packages (compiled from local source)

[florisvdh]

This topic originated in #789 but will be continued here. 'Local package' means a package compiled and installed from a local source directory (not by renv) - regardless of whether that package is also present in a repository.

Even though the local package may be in a different dev state than the versions available in one of the active Repositories, @kevinushey explained (#789 (comment)) it is intended behaviour of renv to link the local package with an available Repository in renv.lock, if applicable.

In that regard three related aspects may profit from further improvement (we could split this in more issues if needed):

1. In the case of a r-universe repository there is some inconsistency in renv.lock (and related output of renv::snapshot()):

• local packages get the Repository name as the value for their Repository key. This is consistent with the case for packages installed from CRAN and their local counterparts.
• packages effectively installed from a r-universe repository, get the Repository URL as value for their Repository key

Examples (partly from #789)

    "rlang": {
      "Package": "rlang",
      "Version": "0.99.0.9009",
      "Source": "Repository",
      "Repository": "rlib",
      "Hash": "29e7067079a29a891ac42418806585b5"
    },

{
  "R": {
    "Version": "4.1.2",
    "Repositories": [
      {
        "Name": "rlib",
        "URL": "https://r-lib.r-universe.dev"
      }
    ]
  },
  "Packages": {
    "rlang": {
      "Package": "rlang",
      "Version": "0.4.11.9001",
      "Source": "Repository",
      "Repository": "https://r-lib.r-universe.dev",
      "RemoteUrl": "https://github.com/r-lib/rlang",
      "RemoteRef": "HEAD",
      "RemoteSha": "26c4a878473578cf7b4c630f457a3bdfc46c8c31"
    }
  }
}

    "rlang": {
      "Package": "rlang",
      "Version": "0.99.0.9009",
      "Source": "Repository",
      "Repository": "CRAN",
      "Hash": "29e7067079a29a891ac42418806585b5"
    },

From #789 (comment):

Regarding how the 'inbo' r-universe is depicted in two different ways, I was referring to the difference between packages INBOtheme and n2khab in the output of renv::init():

# https://inbo.r-universe.dev ========  ## THIS LINE
- INBOtheme       [* -> 0.5.9]

versus

# inbo ===============================  ## THIS LINE
- n2khab          [* -> 0.5.0.900]

Same in the above example renv.lock:

      "Source": "Repository",
      "Repository": "https://inbo.r-universe.dev",

versus

      "Source": "Repository",
      "Repository": "inbo",

Sure it is not wrong, it's just not 100% consistent; I'd expect either the name or the URL in both cases.

2. In contrast with renv 0.14.0, renv 0.14.0-50 does no longer complain when registering a package that cannot be linked to a known Repository - instead the Repository gets value null and no further messaging occurs.

Details (copied from #789)

Both below cases only have the CRAN repo enabled (options(repos = c(CRAN = "https://cloud.r-project.org/"))).

with renv 0.14.0-50 (github):

This locally built package (which is not on CRAN, only in the inbo r-universe repo = disabled in this session) is now stored like:

    "n2khab": {
      "Package": "n2khab",
      "Version": "0.5.0.900",
      "Source": "Repository",
      "Repository": null,
      "Hash": "9aafc0465394ce0bd93a7220c20db697"
    },

So a local package is now always set as coming from a repository, here being the 'null' repository (before, it would have been source 'unknown', not considered as coming from a repo).

In this case no warning came anymore from snapshot() (which now reports this package under # Repository ========) or status().

==> Suggestion: should a warning not have been risen, as it was in 0.14.0? - see below

with renv 0.14.0 (CRAN):

Local package registered as:

    "n2khab": {
      "Package": "n2khab",
      "Version": "0.5.0.900",
      "Source": "unknown",
      "Hash": "9aafc0465394ce0bd93a7220c20db697"
    },

Where warnings occurred:

> renv::snapshot()
The following package(s) were installed from an unknown source:
         _
  n2khab   [0.5.0.900]

renv may be unable to restore these packages in the future.
Consider reinstalling these packages from a known source (e.g. CRAN).

Do you want to proceed? [y/N]: y
The following package(s) will be updated in the lockfile:
# CRAN ===============================
...
# (Unknown Source) ===================
- n2khab       [* -> 0.5.0.900]
Do you want to proceed? [y/N]: y
* Lockfile written to '/.../renv.lock'.

> renv::status()
* The project is already synchronized with the lockfile.
The following package(s) were installed from an unknown source:
         _
  n2khab   [0.5.0.900]

renv may be unable to restore these packages in the future.
Consider reinstalling these packages from a known source (e.g. CRAN).

3. It could be argued that renv should always warn the user when registering a local package, since it can be in a different state than the version available in a Repository.

[hadley]

IMO there's relatively little more we can do for locally installed packages because we have to guess where they come from. I think there are real dangers to making the existing heuristics more powerful instead of encouraging people to install packages in a way that gives a full specification of their source.