GitHub Actions CI Issue: dependent package "is not a valid binary, it does not contain 'Hmisc/Meta/package.rds'" #467
Comments
|
It is a bug in pak. A workaround is to use the |
|
Probably the same in another repo: https://github.com/tidymodels/recipes/actions/runs/4348497699/jobs/7614837809 This can happen if pak expects a binary package, but RSPM sends a source package. So this is definitely a bug. OTOH, for the packages that fail RSPM sends binaries, so something else must be going on as well. pkgdepends/pkgcache will have to set the |
|
In case, also happening here, if of any help: |
|
This should be fixed now, I reverted a commit in pkgcache: I still don't know what the bug is, though, but at least this Dockerfile, using the previous devel version of pak reproduces it: # -*- mode: Dockerfile -*-
FROM ubuntu:22.04
RUN apt-get update && \
apt-get install -y curl && \
curl -LO https://cdn.posit.co/r/ubuntu-2204/pkgs/r-devel_1_amd64.deb && \
apt install -y ./r-devel_1_amd64.deb && \
rm r-devel*.deb
RUN ln -s /opt/R/devel/bin/R /usr/local/bin/R && \
ln -s /opt/R/devel/bin/Rscript /usr/local/bin/Rscript
RUN echo 'options( \
repos = c(RSPM = "https://packagemanager.posit.co/cran/__linux__/jammy/latest", CRAN = "https://cloud.r-project.org"),\
HTTPUserAgent="R/4.2.2 R (4.2.2 x86_64-pc-linux-gnu x86_64 linux-gnu) on GitHub Actions"\
)' \
>> $HOME/.Rprofile
RUN apt-get update && \
apt-get install -y git && \
git clone --depth 1 https://github.com/r-lib/pkgcache
RUN curl -L -H 'Authorization: Bearer QQ==' -o pak.tar.gz \
https://ghcr.io/v2/r-lib/pak/blobs/sha256:ecafed8beab831d350856a06a0e68b4101014ea96783f8f5d1eb9a90183bb31a && \
R CMD INSTALL pak.tar.gz
RUN cd pkgcache && \
Rscript -e 'pak::lockfile_create(c("deps::.", "any::rcmdcheck", "any::sessioninfo"), dependencies = "all")'
RUN cd pkgcache && \
cat pkg.lock
RUN cd pkgcache && \
Rscript -e 'pak::lockfile_install()' |
|
@gaborcsardi can confirm it fixed the issue for me! 😊 |
|
It turns out that pak does not actually use binaries from RSPM on R-devel, because the pak subprocess does not read On the one hand, it is weird that we never actually noticed this. OTOH it would explain why we never saw errors when trying to use the release binaries for devel builds. So now this causes an error because now the install plan is to download binaries, but then we get source packages, and this is not handled. |
|
I'm running into this issue too. Right now PPM for Jammy/latest seems to be serving MASS as source package, which causes my CI jobs to fail. The suggestion above, to use |
|
@nbenn can you link to the workflow and the failing build? |
|
Unfortunately it's a private repo and it's not a Gh actions workflow (I'm using drone). I might be able to put together the necessary pieces to reproduce though if you want to try and have a look. I'll go ahead and describe the setup: It's a c(CRAN = "https://packagemanager.rstudio.com/cran/__linux__/jammy/latest")I install pak as install.packages("pak", repos = "https://r-lib.github.io/p/pak/stable/")And in my lockfile I then get (among other entries) {
"ref": "MASS",
"package": "MASS",
"version": "7.3-59",
"type": "standard",
"direct": false,
"binary": true,
"dependencies": [],
"vignettes": false,
"needscompilation": false,
"metadata": {
"RemoteType": "standard",
"RemotePkgRef": "MASS",
"RemoteRef": "MASS",
"RemoteRepos": "https://packagemanager.rstudio.com/cran/__linux__/jammy/latest",
"RemotePkgPlatform": "x86_64-pc-linux-gnu-ubuntu-22.04",
"RemoteSha": "7.3-59"
},
"sources": ["https://packagemanager.rstudio.com/cran/__linux__/jammy/latest/src/contrib/MASS_7.3-59.tar.gz", "https://packagemanager.rstudio.com/cran/__linux__/jammy/latest/src/contrib/Archive/MASS/MASS_7.3-59.tar.gz"],
"target": "src/contrib/x86_64-pc-linux-gnu-ubuntu-22.04/4.2/MASS_7.3-59.tar.gz",
"platform": "x86_64-pc-linux-gnu-ubuntu-22.04",
"rversion": "4.2",
"directpkg": false,
"license": "GPL-2 | GPL-3",
"dep_types": ["Depends", "Imports", "LinkingTo"],
"params": [],
"install_args": "",
"repotype": "cran"
}The PPM web UI currently says
Happy to share more if you're interested. |
|
Seems like RSPM is sending a binary package to me: > pak::pkg_install("MASS?reinstall")
> Will install 1 package.
> Will download 1 package with unknown size.
+ MASS 7.3-59 [dl]
i Getting 1 pkg with unknown size
v Got MASS 7.3-59 (x86_64-pc-linux-gnu-ubuntu-22.04) (1.12 MB)
v Downloaded 1 package (1.12 MB)in 4.9s
v Installed MASS 7.3-59 (270ms)
v 1 pkg: added 1, dld 1 (1.12 MB) [8.2s]
> .Last.value[c("sources", "platform")]
sources
1 https://packagemanager.posit.co/cran/__linux__/jammy/latest/src/contrib/MASS_7.3-59.tar.gz
platform
1 x86_64-pc-linux-gnu-ubuntu-22.04
> getRversion()
[1] '4.2.0'How are your repos set up? What is the output of |
|
Are you sure, you're being served a binary release? If I download the file at As requested: |
|
Yeah, it is a binary package: > dl <- pak::pkg_download("MASS")
i Getting 2 pkgs (1.03 MB) and 2 pkgs with unknown sizes
v Got MASS 7.3-59 (source) (515.84 kB)
v Got MASS 7.3-59 (source) (515.84 kB)
v Got MASS 7.3-59 (x86_64-pc-linux-gnu-ubuntu-22.04) (1.12 MB)
v Got MASS 7.3-59 (x86_64-pc-linux-gnu-ubuntu-22.04) (1.12 MB)
v Downloaded 4 packages (3.27 MB)in 7.1s> dl$sources
[[1]]
[1] "https://packagemanager.posit.co/cran/__linux__/jammy/latest/src/contrib/4.4.0/Recommended/MASS_7.3-59.tar.gz"
[[2]]
[1] "https://packagemanager.posit.co/cran/__linux__/jammy/latest/src/contrib/MASS_7.3-59.tar.gz"
[[3]]
[1] "https://cloud.r-project.org/src/contrib/MASS_7.3-59.tar.gz"
[2] "https://cloud.r-project.org/src/contrib/Archive/MASS/MASS_7.3-59.tar.gz"
[[4]]
[1] "https://cloud.r-project.org/src/contrib/4.4.0/Recommended/MASS_7.3-59.tar.gz"
[2] "https://cloud.r-project.org/src/contrib/Archive/MASS/MASS_7.3-59.tar.gz"> dl$fulltarget[2]
[1] "./src/contrib/x86_64-pc-linux-gnu-ubuntu-22.04/4.2/MASS_7.3-59.tar.gz"
> untar(dl$fulltarget[2], list = TRUE)[1:10]
[1] "MASS/CITATION" "MASS/DESCRIPTION" "MASS/INDEX"
[4] "MASS/Meta/" "MASS/Meta/Rd.rds" "MASS/Meta/data.rds"
[7] "MASS/Meta/features.rds" "MASS/Meta/hsearch.rds" "MASS/Meta/links.rds"
[10] "MASS/Meta/nsInfo.rds" |
|
I'm sorry if all of this comes down to me somehow misunderstanding things, but for the URL you have in url <- "https://packagemanager.posit.co/cran/__linux__/jammy/latest/src/contrib/MASS_7.3-59.tar.gz"
tmp <- tempfile()
download.file(url, tmp)
untar(tmp, list = TRUE)[1:10]
#> [1] "MASS/" "MASS/NAMESPACE" "MASS/LICENCE.note"
#> [4] "MASS/ChangeLog" "MASS/data/" "MASS/data/Boston.rda"
#> [7] "MASS/data/immer.rda" "MASS/data/geyser.rda" "MASS/data/npr1.rda"
#> [10] "MASS/data/phones.rda"
unlink(tmp)Which I find unsurprising given the What am I missing here? Are the two of us getting different files? |
|
Yes, RSPM sends different files depending on your User Agent header: https://packagemanager.posit.co/__docs__/admin/serving-binaries/#binary-user-agents > getOption("HTTPUserAgent")
[1] "R/4.2.0 R (4.2.0 x86_64-pc-linux-gnu x86_64 linux-gnu)" |
|
Many thanks for clarifying this bit & sorry for the detour with my past ~3 msgs. I was running some of that under macOS. I can still reproduce the problem though:
|
|
This is probably an edge case in pak. I don't see it if I set up my repos like this: Hopefully that works as a workaround for you. As for the bug itself, I think it happens because RSPM has two versions of MASS, and only has a binary for one version. pak sees that RSPM has binaries for MASS, and assumes that both builds are binary. |
|
Yes, I can confirm that the problem goes away when setting repos as options(
repos = c(
RSPM = "https://packagemanager.rstudio.com/cran/__linux__/jammy/latest",
CRAN = "https://cloud.r-project.org"
)
)I then get the same results as you do. In terms of workarounds, I'm currently using a PPM snapshot from before the last update to MASS (sometime last week). Which works fine for me for now. In terms of general advice, if you consider this an edge case, would you suggest something like "always set up a proper CRAN repo alongside PPM for pak to function properly"? At any rate, thanks for taking the time to discuss (and for filling me in on the importance of user agent headers for interacting with PPM). |
Oh, an edge case is still a bug, and I'll fix it asap. |
|
Ah, sorry, misunderstood you there. Thanks! |
AFAIU the issue is not due to the lack of a "proper" CRAN repo - PPM also provides the same sources as the CRAN repo besides the binaries - but that {pak} errors after having received the binary, i.e. in its post-processing step. The only scenario and reason in which you would not see this issue with https://cloud.r-project.org/ as a single repo would be that {pak} would just directly get the source instead of the binary (because CRAN does not provide Linux binaries) and hence can't fail at the "is not a valid binary" step. |
|
There are about four interacting issues here.
|
|
To fix this, first I need to fix 4. and handle the case when RSPM replies with an unexpected source package. Then we can think about avoiding the duplicate downloads. I also reported the issue of sending a source package instead of a binary to the RSPM team. |
|
This will be eventually fixed in PPM: https://github.com/rstudio/package-manager/issues/10471 (private repo, sorry). I'll also add a workaround in a minute. |
To work around r-lib/pak#467 Until rstudio/package-manager#10471 is fixed.
|
OK, there is a workaround now, in the pkgcache package that pak uses internally. It will be in tomorrow's nightly pak |
|
@gaborcsardi Any ETA when this will make it into "stable"? I have a bunch of WFs running on "stable" and would like to avoid having to point all of them to "devel" and then back again. Here's one example for which {KernSmooth} is the issue instead of {MASS}: https://github.com/mlr-org/mlr3spatiotempcv/actions/runs/5046852603/jobs/9052924843?pr=223#step:11:602 |
|
I'm running into the same issue with this GHA, specifically related to the {withr} package: https://github.com/ketchbrookanalytics/fcall/actions/runs/7560235802/job/20585804754 Is this perhaps due to the fact that {withr} v3.0.0 was released just a few hours ago? Apologies if this is an issue with my understanding when/how often binaries are made available. |
|
@mthomas-ketchbrook I think that's a different issue that manifests in a similar way, and it happens because of a PPM (formerly known as RSPM) bug. A workaround is to turn off PPM until it is fixed. |
I also ran into this PPM-{withr} problem; 3 hours ago PPM still provided {withr} 2.5.2 = no problem. I can confirm that neither the |
|
The PPM-{withr} problem reported above in #467 (comment) has been gone (example log). |
When using GitHub Actions, I get the following error that appears to originate from
pak. It may be related to R version 4.3 since it consistently happens for the Ubuntu-devel version.The error is occurring here:
https://github.com/nlmixr2/rxode2/actions/runs/4355986650/jobs/7613357209#step:6:6313
The text was updated successfully, but these errors were encountered: