"remotes" downloads incorrect SOILWAT2 submodule commit

[dylanbeaudette]

Hi, trying to compile on Mac OS, R 4.0.2.

Any suggestions?

remotes::install_github('DrylandEcology/rSOILWAT2')

OILWAT2-a7f4185/DESCRIPTION’ ...
─  preparing ‘rSOILWAT2’: (534ms)
✓  checking DESCRIPTION meta-information ...
─  cleaning src
─  checking for LF line-endings in source and make files and shell scripts (652ms)
─  checking for empty or unneeded directories
   Removed empty directory ‘rSOILWAT2/inst/extdata/example1/Output’
─  building ‘rSOILWAT2_4.0.4.tar.gz’
   
Installing package into ‘/Users/sgtpepp/Library/R/4.0/library’
(as ‘lib’ is unspecified)
* installing *source* package ‘rSOILWAT2’ ...
** using staged installation
** libs
clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -DRSOILWAT  -I/usr/local/include   -fPIC  -Wall -g -O2  -c SW_R_init.c -o SW_R_init.o
clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -DRSOILWAT  -I/usr/local/include   -fPIC  -Wall -g -O2  -c SW_R_lib.c -o SW_R_lib.o
clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -DRSOILWAT  -I/usr/local/include   -fPIC  -Wall -g -O2  -c rSW_Carbon.c -o rSW_Carbon.o
clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -DRSOILWAT  -I/usr/local/include   -fPIC  -Wall -g -O2  -c rSW_Control.c -o rSW_Control.o
clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -DRSOILWAT  -I/usr/local/include   -fPIC  -Wall -g -O2  -c rSW_Files.c -o rSW_Files.o
clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -DRSOILWAT  -I/usr/local/include   -fPIC  -Wall -g -O2  -c rSW_Markov.c -o rSW_Markov.o
clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -DRSOILWAT  -I/usr/local/include   -fPIC  -Wall -g -O2  -c rSW_Model.c -o rSW_Model.o
clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -DRSOILWAT  -I/usr/local/include   -fPIC  -Wall -g -O2  -c rSW_Output.c -o rSW_Output.o
clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -DRSOILWAT  -I/usr/local/include   -fPIC  -Wall -g -O2  -c rSW_Site.c -o rSW_Site.o
rSW_Site.c:218:48: error: too many arguments to function call, expected 2, have 3
                calculate_soilBulkDensity(matricd, f_gravel, lyrno);
                ~~~~~~~~~~~~~~~~~~~~~~~~~                    ^~~~~
./SOILWAT2/SW_Site.h:149:1: note: 'calculate_soilBulkDensity' declared here
RealD calculate_soilBulkDensity(RealD matricDensity, RealD fractionGravel);
^
1 error generated.
make: *** [rSW_Site.o] Error 1
ERROR: compilation failed for package ‘rSOILWAT2’
* removing ‘/Users/sgtpepp/Library/R/4.0/library/rSOILWAT2’
Error: Failed to install 'rSOILWAT2' from GitHub:
  (converted from warning) installation of package ‘/var/folders/cf/bv1wmc896dq6vjy55w9jw7j80000gn/T//RtmpJ3SlZn/file8ee54960fa5b/rSOILWAT2_4.0.4.tar.gz’ had non-zero exit status

[dylanbeaudette]

Attempting with PR #172

remotes::install_github('DrylandEcology/rSOILWAT2#172')

... and it works! So maybe I just need to wait for this to be merged?

[dschlaep]

Sorry for your trouble and the late response, @dylanbeaudette! Great to see that you figured out how to get it working.

I can reproduce the error on my machine. The current rSOILWAT2 main branch works only with the second to latest version of SOILWAT2 main branch: the error message you see is due to changes in the latest version of SOILWAT2 that have not been reflected in rSOILWAT2. This is done on the development branch that you used with in the pull request #172.

I had assumed that the specific commit that is registered for the SOILWAT2 submodule (in this case the second to last version) would be pulled when installing rSOILWAT2, but apparently it doesn't work and pulls the latest commit instead.

Thanks for posting the error!

[dylanbeaudette]

Thanks @dschlaep. Any suggestions on when these changes will be merged with the main branch? I don't mind using the PR, but would like to eventually use rSOILWAT2 in production code.

[dschlaep]

It seems that the issue we are having here is due to r-lib/remotes#260, i.e., "remotes" downloads the zip archive (instead of as a git repository) from github. Because of this, "remotes" cannot do what would be the correct way, i.e., git submodule update --init --recursive, and "remotes" cannot know what the specified commit for each submodule should be (because that is stored in ".git" that isn't part of the zip archive). Instead "remotes" clones the submodules (which grabs the latest commits on the specified branch) and fails in situations as here where a specific submodule commit is required.

Unfortunately, it appears that this is not something that we can fix on our side -- except avoiding the problem by attempting to not let SOILWAT2 and rSOILWAT2 get out of sync ...

The only way to get an installation of rSOILWAT2 working correctly in this situation, as far as I know, that is until "remotes" correctly handles submodules, is to avoid using "remotes" (please tell me if you know of a way to make this work with the current "remotes"!), e.g.,

    system2(
      command = "git",
      args = paste(
        "clone -b main --single-branch --recursive",
        "https://github.com/DrylandEcology/rSOILWAT2.git",
        "rSOILWAT2"
      )
    )
    tools::Rcmd(args = "INSTALL rSOILWAT2")

I am working on finalizing #172 and hopefully it will be done in the near future. Thanks for your interest in rSOILWAT2, @dylanbeaudette !

[dschlaep]

@dylanbeaudette For your information: I finalized the pull request, released as v5.0.1, and remotes::install_github('DrylandEcology/rSOILWAT2') works again out of the box -- at least on my end, i.e., HEAD of main rSOILWAT2 branch is compatible with HEAD of main SOILWAT2 branch.

[dylanbeaudette]

Excellent thank you! Issue closed then?

[dylanbeaudette]

Hi @dschlaep, thanks for all of the assistance. I'm finally returning to SOILWAT2 and rSOILWAT2.

This time attempting to compile on Windows, using R CMD INSTALL rSOILWAT2 and remotes::install_github(). Towards the end of compilation getting this error:

*** No rule to make target 'Files/rtools42/x86_64-w64-mingw32.static.posix/include'.  Stop.

R version 4.2.1 (2022-06-23 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19044)

Any suggestions?

[dschlaep]

Sorry, that this is not working for you. I'm not sure what is going wrong, however, I don't believe it is an issue with rSOILWAT2:

• Installing rSOILWAT2 works for us on "windows-latest" via Github Actions (https://github.com/DrylandEcology/rSOILWAT2/blob/main/.github/workflows/check-standard.yml).
• I also just tested it on a colleague's work Windows computer (using Rtools42) and it worked without problem.

The error message may suggest that Rtools42 doesn't fully work, e.g., that "make" is not working properly?

[dylanbeaudette]

Thanks for testing. I'll keep trying. I wasn't able to get a usable build from rhub because R CMD build won't complete locally.

Could this have anything to do with our local installation of RTools?

[dylanbeaudette]

It seems that our non-standard installation directory for RTools was the problem. Thanks to @brownag, the solution is:

Sys.setenv("RTOOLS42_HOME" = "C:/PROGRA~1/rtools42")
remotes::install_github("DrylandEcology/rSOILWAT2", build_vignettes = FALSE)