[request] gdal/3.0.4

[sebhmg]

Package Details

• Package Name/Version: gdal/3.0.4
• Website: https://gdal.org/
• Source code: https://github.com/OSGeo/gdal

Description Of The Library / Tool

GDAL is a translator library for raster and vector geospatial data formats that is released under an X/MIT style Open Source License by the Open Source Geospatial Foundation. As a library, it presents a single raster abstract data model and single vector abstract data model to the calling application for all supported formats. It also comes with a variety of useful command line utilities for data translation and processing. The NEWS page describes the January 2020 GDAL/OGR 3.0.4 release.

Notes

Building for Visual Studio 2017

The AppVeyor file might be a good starting point for a a recipe for Visual Studio 2017. See https://github.com/OSGeo/gdal/blob/v3.0.4/appveyor.yml

Build requirements

GDAL 3.x depends on PROJ version 6 (source code of PROJ: https://github.com/OSGeo/PROJ)

Besides, it seems there are quite a bit of dependencies tied to the support of various GDAL drivers (a driver roughly corresponds to a file format, though some drivers may be gateways to many formats):

• build requirements for Raster Drivers: https://gdal.org/drivers/raster/index.html
• build requirements for Vector Drivers: https://gdal.org/drivers/vector/index.html

The recipe could possibly have options to decide which drivers to support.

[Croydon]

There is also this where we can have a look at: https://github.com/osechet/conan-gdal

[SpaceIm]

Gdal has a lot of optional dependencies (and transitive dependencies).

For the moment, here is what is missing in CCI to be able to create a proper recipe:

library	url	PR
proj	https://github.com/OSGeo/PROJ	#1272
geos	https://trac.osgeo.org/geos	#829
libgeotiff	https://github.com/OSGeo/libgeotiff	#1317
libkml	https://github.com/libkml/libkml	#792
cryptopp	https://cryptopp.com	#1724
lerclib	https://github.com/Esri/lerc	#1159
freexl	https://www.gaia-gis.it/fossil/freexl/index	#1846
raslib	https://rasdaman.org
libspatialite	https://www.gaia-gis.it/fossil/libspatialite/index
rasterlite2	https://www.gaia-gis.it/fossil/librasterlite2/index
ogdi	https://github.com/libogdi/ogdi
fyba	https://github.com/kartverket/fyba
crunch	https://github.com/BinomialLLC/crunch
netcdf-c	https://github.com/Unidata/netcdf-c
kealib	http://kealib.org/
libbpg	https://bellard.org/bpg
armadillo	http://arma.sourceforge.net
tiledb	https://github.com/TileDB-Inc/TileDB
cfitsio	https://heasarc.gsfc.nasa.gov/fitsio	#1121
poppler	https://github.com/freedesktop/poppler
podofo	https://sourceforge.net/projects/podofo
pdfium	https://pdfium.googlesource.com/pdfium
sfcgal	https://github.com/Oslandia/SFCGAL
libopencad	https://github.com/sandyre/libopencad
epsilon	https://sourceforge.net/projects/epsilon-project/
charls	https://github.com/team-charls/charls	#1039
libdap	https://www.opendap.org/pub/source
file-geodatabase-api	https://github.com/Esri/file-geodatabase-api
opencl-headers	https://github.com/KhronosGroup/OpenCL-Headers/
qhull	https://github.com/qhull/qhull	#1728
mongo-cxx-driver	https://github.com/mongodb/mongo-cxx-driver
pcraster-rasterformat	https://github.com/pcraster/rasterformat
hdf4	https://portal.hdfgroup.org/display/support/Download+HDF4	#1070
mariadb-connector-c	https://github.com/mariadb-corporation/mariadb-connector-c

(+ their transitive -eventually optionals- dependencies: pnetcdf, lapack, libidn, nss, fontconfig, cairo...)

and maybe others... (I don't take into account close commercial libraries)

[sebhmg]

Thanks @SpaceIm,
From this post on the OSGeo wiki (5 years old though), it seems the GDAL can be configured to build with only a minimal set of drivers. It should help reducing the number of dependencies.

./configure \
    --prefix=${PREFIX} \
    --with-geos \
    --with-geotiff=internal \
    --with-hide-internal-symbols \
    --with-libtiff=internal \
    --with-libz=internal \
    --with-threads \
    --without-bsb \
    --without-cfitsio \
    --without-cryptopp \
    --without-curl \
    --without-dwgdirect \
    --without-ecw \
    --without-expat \
    --without-fme \
    --without-freexl \
    --without-gif \
    --without-gif \
    --without-gnm \
    --without-grass \
    --without-grib \
    --without-hdf4 \
    --without-hdf5 \
    --without-idb \
    --without-ingres \
    --without-jasper \
    --without-jp2mrsid \
    --without-jpeg \
    --without-kakadu \
    --without-libgrass \
    --without-libkml \
    --without-libtool \
    --without-mrf \
    --without-mrsid \
    --without-mysql \
    --without-netcdf \
    --without-odbc \
    --without-ogdi \
    --without-openjpeg \
    --without-pcidsk \
    --without-pcraster \
    --without-pcre \
    --without-perl \
    --without-pg \
    --without-php \
    --without-png \
    --without-python \
    --without-qhull \
    --without-sde \
    --without-sqlite3 \
    --without-webp \
    --without-xerces \
    --without-xml2

There could be enough value in supporting only the drivers which dependencies are available in CCI or bincrafters, and increase the support later from there.

[SpaceIm]

Now that PROJ is in CCI, GDAL can be packaged with available drivers. There is a nice configure.ac of ~6000 lines to check 😅

[sebhmg]

@SpaceIm, nice! It does not seem so daunting anymore 😃

[SpaceIm]

Well, after a first attempt of packaging, gdal configure.ac seems to be very fragile regarding dependencies (one of the first AC_CHECK_LIB fails on PROJ, probably because it doesn't rely on pkg_config to list PROJ dependencies).
It will take time.

[SpaceIm]

I've made some progress.

• I have a recipe which build and link on Linux with all gcc version >= 4.9, with a minimal set of dependencies (the required ones: json-c, libgeotiff, libtiff, proj, and also libpng + zlib).

• It fails at some point with clang and libc++ (in linking phase of a gdal executable).

• It might work on MacOS, but outputs are so verbose that Travis kills my build even before the end of the first one.

• Didn't try to make it work on Windows yet (other build files to check and patch).

• gdal has also 2 required dependencies more or less hidden in source code: flatbuffers (this one could have been optional in gdal build files, because only one driver requires it) and shapelib. So it requires some patches in order to use conan recipes for these two dependencies instead of embedded ones.

[SpaceIm]

I'm not too far from a working recipe for 3.1.0

• On Linux:
  • gcc: pretty robust with all combinations of optional dependencies which can be provided by CCI or by internal code (zlib, zstd, pg, cfitsio, pcraster, png, gta, pcidsk, jpeg, charls, gif, hdf4, hdf5, jasper, openjpeg, gnm, mysql, xerces, expat, libkml, odbc, curl, xml2, sqlite3, pcre, webp, geos, qhull, crypto, lerc, openexr)
  • clang: works with libstdc++ but not with libc++
• On Windows:
  • mingw: takes a lot of time to compile, archive and install (gdal with minimum set of dependencies: ~7 min on my computer with gcc on Linux Mint against ~35 min on Windows 10 with MinGW), but it works.
  • visual studio: not yet implemented in the recipe
• On Macos: maybe it works, I'm not sure

Some conflicts in CCI need to be fixed because some recipes depends on different versions of the same dependency, that gdal can't override (because they are not direct dependencies).