Updating sf from 1.0.3 to 1.0.4 fails under PROJ 8.1.1-GEOS 3.9.0-GDAL 3.2.2-R 4.1.2-Debian 11

[iagogv3]

All seems fine until

** testing if installed package can be loaded from temporary location

 *** caught segfault ***
address 0x558a19fc5aae, cause 'memory not mapped'
An irrecoverable exception occurred. R is aborting now ...
Segmentation fault
ERROR: loading failed
* removing ‘/home/iago/R/x86_64-pc-linux-gnu-library/4.1/sf’
* restoring previous ‘/home/iago/R/x86_64-pc-linux-gnu-library/4.1/sf’
Warning in install.packages :
  installation of package ‘sf’ had non-zero exit status

The downloaded source packages are in
	‘/tmp/Rtmp68H5qh/downloaded_packages’

Under

R version 4.1.2 (2021-11-01) -- "Bird Hippie"
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

Let me know if I can give more information.

I should add that I am working on Debian 11, but I have installed proj-bin 8.1.1-1 from testing distribution. May it be related to the issue?

[edzer]

See https://github.com/r-spatial/sf#multiple-gdal-geos-andor-proj-versions-on-your-system

You most likely have multiple versions installed from PROJ, GEOS and/or GDAL.

[iagogv3]

@edzer I believe I have a unique version from each library:

$ pkg-config --modversion proj                        
8.1.1
$ pkg-config --modversion gdal                     
3.2.2
$ pkg-config --modversion geos                        
3.9.0

Let me know if I should do another check.

[rsbivand]

It looks as though you use the OS packaging system for installing external software. Is this related to r-spatial/spatialreg#22 and specifically r-spatial/spatialreg#22 (comment), so maybe back out of testing? I only install PROJ/GEOS/GDAL in sequence from source, and on Fedora, so whether transient package builds are fragile isn't something I know about, but I'd revert to the same level as the other software, not least because GDAL uses PROJ, so might expect the PROJ from its level.

[iagogv3]

@rsbivand Thanks,

Is this related to r-spatial/spatialreg#22 and specifically r-spatial/spatialreg#22 (comment), so maybe back out of testing?

It seems a related issue indeed. Thanks for pointing me to it.

It looks as though you use the OS packaging system for installing external software.

Yes, I try to use apt always than possible, in particular for R, PROJ, GEOS and GDAL (did you mean this? isn't it?).

I'd revert to the same level as the other software, not least because GDAL uses PROJ, so might expect the PROJ from its level.

I will check this, so maybe having all these packages installed from testing should work.

[rsbivand]

Please try and report back.My own current experience in upgrading Fedora 34 to 35 has been that although compilers and basic libraries were the same versions (F34 was fully updated), I had to rebuild my source installed PROJ/GEOS/GDAL because libnetcdf on which GDAL depends had changed version between F34 and F35. So second or third degree dependencies somewhere in the dependency tree can be a problem. Of cource I re-install R anyway from source, but that is just normal caution.

[iagogv3]

I have checked the 3 libraries for PROJ/GEOS/GDAL are installed from testing distribution. I have also reinstalled r-base-dev from the beginning and, FWIW, reinstalled Rcpp. Now trying to install sf, I get

* installing *source* package ‘sf’ ...
** package ‘sf’ successfully unpacked and MD5 sums checked
** using staged installation
configure: CC: gcc -std=gnu99 -std=gnu11
configure: CXX: g++ -std=gnu++11
checking for gdal-config... /usr/bin/gdal-config
checking gdal-config usability... yes
configure: GDAL: 3.2.2
checking GDAL version >= 2.0.1... yes
checking for gcc... gcc -std=gnu99 -std=gnu11
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc -std=gnu99 -std=gnu11 accepts -g... yes
checking for gcc -std=gnu99 -std=gnu11 option to accept ISO C89... none needed
checking how to run the C preprocessor... gcc -std=gnu99 -std=gnu11 -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking gdal.h usability... yes
checking gdal.h presence... yes
checking for gdal.h... yes
checking GDAL: linking with --libs only... yes
checking GDAL: /usr/share/gdal/pcs.csv readable... no
checking GDAL: checking whether PROJ is available for linking:... yes
checking GDAL: checking whether PROJ is available fur running:... yes
configure: GDAL: 3.2.2
configure: pkg-config proj exists, will use it
configure: using proj.h.
configure: PROJ: 8.1.1
checking PROJ: checking whether PROJ and sqlite3 are available for linking:... yes
checking for geos-config... /usr/bin/geos-config
checking geos-config usability... yes
configure: GEOS: 3.9.0
checking GEOS version >= 3.4.0... yes
checking geos_c.h usability... yes
checking geos_c.h presence... yes
checking for geos_c.h... yes
checking geos: linking with -L/usr/lib/x86_64-linux-gnu -lgeos_c... yes
configure: Package CPP flags:   -DHAVE_PROJ_H -I/usr/include/gdal -I/usr/include
configure: Package LIBS: -lproj   -L/usr/lib -lgdal -L/usr/lib/x86_64-linux-gnu -lgeos_c
configure: creating ./config.status
config.status: creating src/Makevars
** libs
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DHAVE_PROJ_H -I/usr/include/gdal -I/usr/include -I'/home/iago/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include'    -fpic  -g -O2 -ffile-prefix-map=/home/jranke/git/r-backports/bullseye/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c RcppExports.cpp -o RcppExports.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DHAVE_PROJ_H -I/usr/include/gdal -I/usr/include -I'/home/iago/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include'    -fpic  -g -O2 -ffile-prefix-map=/home/jranke/git/r-backports/bullseye/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c bbox.cpp -o bbox.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DHAVE_PROJ_H -I/usr/include/gdal -I/usr/include -I'/home/iago/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include'    -fpic  -g -O2 -ffile-prefix-map=/home/jranke/git/r-backports/bullseye/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c gdal.cpp -o gdal.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DHAVE_PROJ_H -I/usr/include/gdal -I/usr/include -I'/home/iago/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include'    -fpic  -g -O2 -ffile-prefix-map=/home/jranke/git/r-backports/bullseye/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c gdal_geom.cpp -o gdal_geom.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DHAVE_PROJ_H -I/usr/include/gdal -I/usr/include -I'/home/iago/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include'    -fpic  -g -O2 -ffile-prefix-map=/home/jranke/git/r-backports/bullseye/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c gdal_read.cpp -o gdal_read.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DHAVE_PROJ_H -I/usr/include/gdal -I/usr/include -I'/home/iago/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include'    -fpic  -g -O2 -ffile-prefix-map=/home/jranke/git/r-backports/bullseye/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c gdal_utils.cpp -o gdal_utils.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DHAVE_PROJ_H -I/usr/include/gdal -I/usr/include -I'/home/iago/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include'    -fpic  -g -O2 -ffile-prefix-map=/home/jranke/git/r-backports/bullseye/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c gdal_write.cpp -o gdal_write.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DHAVE_PROJ_H -I/usr/include/gdal -I/usr/include -I'/home/iago/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include'    -fpic  -g -O2 -ffile-prefix-map=/home/jranke/git/r-backports/bullseye/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c geos.cpp -o geos.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DHAVE_PROJ_H -I/usr/include/gdal -I/usr/include -I'/home/iago/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include'    -fpic  -g -O2 -ffile-prefix-map=/home/jranke/git/r-backports/bullseye/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c hex.cpp -o hex.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DHAVE_PROJ_H -I/usr/include/gdal -I/usr/include -I'/home/iago/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include'    -fpic  -g -O2 -ffile-prefix-map=/home/jranke/git/r-backports/bullseye/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c mdim.cpp -o mdim.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DHAVE_PROJ_H -I/usr/include/gdal -I/usr/include -I'/home/iago/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include'    -fpic  -g -O2 -ffile-prefix-map=/home/jranke/git/r-backports/bullseye/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c ops.cpp -o ops.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DHAVE_PROJ_H -I/usr/include/gdal -I/usr/include -I'/home/iago/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include'    -fpic  -g -O2 -ffile-prefix-map=/home/jranke/git/r-backports/bullseye/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c polygonize.cpp -o polygonize.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DHAVE_PROJ_H -I/usr/include/gdal -I/usr/include -I'/home/iago/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include'    -fpic  -g -O2 -ffile-prefix-map=/home/jranke/git/r-backports/bullseye/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c proj.cpp -o proj.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DHAVE_PROJ_H -I/usr/include/gdal -I/usr/include -I'/home/iago/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include'    -fpic  -g -O2 -ffile-prefix-map=/home/jranke/git/r-backports/bullseye/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c proj_info.cpp -o proj_info.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DHAVE_PROJ_H -I/usr/include/gdal -I/usr/include -I'/home/iago/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include'    -fpic  -g -O2 -ffile-prefix-map=/home/jranke/git/r-backports/bullseye/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c raster2sf.cpp -o raster2sf.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DHAVE_PROJ_H -I/usr/include/gdal -I/usr/include -I'/home/iago/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include'    -fpic  -g -O2 -ffile-prefix-map=/home/jranke/git/r-backports/bullseye/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c sfg.cpp -o sfg.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DHAVE_PROJ_H -I/usr/include/gdal -I/usr/include -I'/home/iago/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include'    -fpic  -g -O2 -ffile-prefix-map=/home/jranke/git/r-backports/bullseye/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c signed_area.cpp -o signed_area.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DHAVE_PROJ_H -I/usr/include/gdal -I/usr/include -I'/home/iago/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include'    -fpic  -g -O2 -ffile-prefix-map=/home/jranke/git/r-backports/bullseye/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c stars.cpp -o stars.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DHAVE_PROJ_H -I/usr/include/gdal -I/usr/include -I'/home/iago/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include'    -fpic  -g -O2 -ffile-prefix-map=/home/jranke/git/r-backports/bullseye/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c wkb.cpp -o wkb.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -DHAVE_PROJ_H -I/usr/include/gdal -I/usr/include -I'/home/iago/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include'    -fpic  -g -O2 -ffile-prefix-map=/home/jranke/git/r-backports/bullseye/r-base-4.1.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c zm_range.cpp -o zm_range.o
g++ -std=gnu++11 -shared -L/usr/lib/R/lib -Wl,-z,relro -o sf.so RcppExports.o bbox.o gdal.o gdal_geom.o gdal_read.o gdal_utils.o gdal_write.o geos.o hex.o mdim.o ops.o polygonize.o proj.o proj_info.o raster2sf.o sfg.o signed_area.o stars.o wkb.o zm_range.o -lproj -L/usr/lib -lgdal -L/usr/lib/x86_64-linux-gnu -lgeos_c -L/usr/lib/R/lib -lR
installing to /home/iago/R/x86_64-pc-linux-gnu-library/4.1/00LOCK-sf/00new/sf/libs
** R
** demo
** inst
** byte-compile and prepare package for lazy loading
in method for ‘dbWriteTable’ with signature ‘"PostgreSQLConnection","character","sf"’: no definition for class “PostgreSQLConnection”
in method for ‘dbDataType’ with signature ‘"PostgreSQLConnection","sf"’: no definition for class “PostgreSQLConnection”
in method for ‘coerce’ with signature ‘"Spatial","sf"’: no definition for class “Spatial”
in method for ‘coerce’ with signature ‘"Spatial","sfc"’: no definition for class “Spatial”
in method for ‘coerce’ with signature ‘"sf","Spatial"’: no definition for class “Spatial”
in method for ‘coerce’ with signature ‘"sfc","Spatial"’: no definition for class “Spatial”
in method for ‘coerce’ with signature ‘"XY","Spatial"’: no definition for class “Spatial”
in method for ‘coerce’ with signature ‘"crs","CRS"’: no definition for class “CRS”
in method for ‘coerce’ with signature ‘"sgbp","sparseMatrix"’: no definition for class “sparseMatrix”
** help
*** installing help indices
*** copying figures
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location

 *** caught segfault ***
address 0x55bca2b713bf, cause 'memory not mapped'
An irrecoverable exception occurred. R is aborting now ...
Segmentation fault
ERROR: loading failed
* removing ‘/home/iago/R/x86_64-pc-linux-gnu-library/4.1/sf’
* restoring previous ‘/home/iago/R/x86_64-pc-linux-gnu-library/4.1/sf’
Warning in install.packages :
  installation of package ‘sf’ had non-zero exit status

The downloaded source packages are in
	‘/tmp/RtmpWLmv3H/downloaded_packages’

[rsbivand]

With the same PROJ/GEOS/GDAL, try to install terra and rgdal. If they (and say vapour) all fail at the same point, @edzer 's suspicion that you have multiple installs lurking on your system would be strengthened. Crucially, some other package may draw in a different version of GDAL with just the dynamic libraries, but these on loading come ahead of the GDAL used for building. If the installs of the other R packages do not fail, there may be someting in the packaging of the specific versions for your target platform, and I'd bring in people who know about your platform.

[iagogv3]

Similar happens with terra:

** byte-compile and prepare package for lazy loading
in method for ‘coerce’ with signature ‘"SpatRaster","Raster"’: no definition for class “Raster”
in method for ‘coerce’ with signature ‘"Raster","SpatRaster"’: no definition for class “Raster”
in method for ‘coerce’ with signature ‘"sf","SpatVector"’: no definition for class “sf”
in method for ‘coerce’ with signature ‘"sfc","SpatVector"’: no definition for class “sfc”
in method for ‘coerce’ with signature ‘"sfg","SpatVector"’: no definition for class “sfg”
in method for ‘coerce’ with signature ‘"XY","SpatVector"’: no definition for class “XY”
in method for ‘coerce’ with signature ‘"im","SpatRaster"’: no definition for class “im”
in method for ‘coerce’ with signature ‘"SpatVector","Spatial"’: no definition for class “Spatial”
in method for ‘coerce’ with signature ‘"Spatial","SpatVector"’: no definition for class “Spatial”
Creating a generic function for ‘ncol’ from package ‘base’ in package ‘terra’
in method for ‘ext’ with signature ‘x="sf"’: no definition for class “sf”
in method for ‘ext’ with signature ‘x="Extent"’: no definition for class “Extent”
in method for ‘ext’ with signature ‘x="Raster"’: no definition for class “Raster”
in method for ‘ext’ with signature ‘x="Spatial"’: no definition for class “Spatial”
in method for ‘wrap’ with signature ‘x="Spatial"’: no definition for class “Spatial”
in method for ‘show’ with signature ‘"Rcpp_SpatDataFrame"’: no definition for class “Rcpp_SpatDataFrame”
in method for ‘show’ with signature ‘"Rcpp_SpatCategories"’: no definition for class “Rcpp_SpatCategories”
in method for ‘geomtype’ with signature ‘x="Spatial"’: no definition for class “Spatial”
in method for ‘vect’ with signature ‘x="Spatial"’: no definition for class “Spatial”
in method for ‘vect’ with signature ‘x="sf"’: no definition for class “sf”
in method for ‘vect’ with signature ‘x="sfc"’: no definition for class “sfc”
in method for ‘vect’ with signature ‘x="XY"’: no definition for class “XY”
** help
*** installing help indices
*** copying figures
** building package indices
** testing if installed package can be loaded from temporary location

 *** caught segfault ***
address 0x56434a6d65ec, cause 'memory not mapped'
An irrecoverable exception occurred. R is aborting now ...
Segmentation fault
ERROR: loading failed
* removing ‘/home/iago/R/x86_64-pc-linux-gnu-library/4.1/terra’
* restoring previous ‘/home/iago/R/x86_64-pc-linux-gnu-library/4.1/terra’
Warning in install.packages :
  installation of package ‘terra’ had non-zero exit status

and with vapour:

** byte-compile and prepare package for lazy loading
** help
*** installing help indices
*** copying figures
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
free(): invalid pointer
Aborted
ERROR: loading failed
* removing ‘/home/iago/R/x86_64-pc-linux-gnu-library/4.1/vapour’
Warning in install.packages :
  installation of package ‘vapour’ had non-zero exit status

I did not try with rgdal (which I have currently installed), but I do not expect success.

I only installed these packages using apt, and then I upgraded to testing distribution, but I did not any other install of them. What I have installed are Python modules which use them (like these R libraries), could this be the reason? I assume that installations inside a docker image (where there I have indeed other GDAL... versions) should not have any impact here.

[edzer]

I think we diagnosed your problem, but have to leave solving it up to you.

[rsbivand]

Installs for Python could very well be the reason. Maybe try on a completely separate bare machine, installing each component separately in sequence. Trying to get Python to use my PROJ/GEOS/GDAL is painful and difficult, as the level of predictive package coordination is simply not comparable to R. So maybe either Python stuff in its container, and R in the system, the reverse, or both in their own containers?

[iagogv3]

Thanks both for the help. I will check the issue #844 carefully and against the above outputs trying to install these packages .

[iagogv3]

Finally I reinstalled stable versions of PROJ, GEOS and GDAL and I could update both sf and terra so I close the issue.

Considering what @rsbivand commented, my hypothesis for all this is that when installing QGis through apt-get, it implied the installation also from Debian repositories of python3-pyproj and python3-gdal among others, so when later I upgraded both PROJ and GDAL python libraries through pip, it possibly produce "multiple versions installed" for PROJ, GEOS, GDAL.