Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add gdal/3.1.0 #1722

Merged
merged 6 commits into from
Jun 11, 2020
Merged

add gdal/3.1.0 #1722

merged 6 commits into from
Jun 11, 2020

Conversation

SpaceIm
Copy link
Contributor

@SpaceIm SpaceIm commented May 24, 2020

Specify library name and version: gdal/3.1.0

  • I've read the guidelines for contributing.
  • I've followed the PEP8 style guides for Python code in the recipes.
  • I've used the latest Conan client version.
  • I've tried at least one configuration locally with the
    conan-center hook activated.

closes #1023

@SpaceIm SpaceIm marked this pull request as draft May 24, 2020 22:25
@conan-center-bot
Copy link
Collaborator

Failure in build 1 (7956dd1f8831272dcb2df81d6f154e9da0445e8b):
gdal/3.1.0:

  • Error processing recipe: Windows x86_64, Debug, Visual Studio 14, MTd. Options: gdal:shared-False
    Unexpected error happened:
ERROR: gdal/3.1.0: Error in configure() method, line 177
	if self.settings.os == "Windows" and self.options.with_obdc:
	ConanException: 'options.with_obdc' doesn't exist
Possible options are ['shared', 'simd_intrinsics', 'with_zstd', 'with_pg', 'with_cfitsio', 'with_pcraster', 'with_gta', 'with_pcidsk', 'with_jpeg', 'with_charls', 'with_gif', 'with_hdf4', 'with_hdf5', 'with_jasper', 'with_openjpeg', 'with_gnm', 'with_mysql', 'with_xerces', 'with_expat', 'with_libkml', 'with_odbc', 'with_curl', 'with_xml2', 'with_sqlite3', 'with_webp', 'with_geos', 'with_qhull', 'without_pam', 'with_crypto', 'without_lerc', 'with_exr']

@SpaceIm
Copy link
Contributor Author

SpaceIm commented May 24, 2020

Few comments:

  • Linux:
    • build with gcc is quite robust
    • build with clang and libc++ fails at link time, I don't know why and I need help. It's like libc++ was not linked. Is it related to libtool?
      2020-05-25T09:54:15.7677428Z /bin/bash /home/conan/.conan/data/gdal/3.1.0/bincrafters/testing/build/4c51049431b2ca7ca5e455a2d25db4c01f5bdc93/source_subfolder/gdal/libtool --mode=compile --silent --tag=CXX clang++ -I/home/conan/.conan/data/gdal/3.1.0/bincrafters/testing/build/4c51049431b2ca7ca5e455a2d25db4c01f5bdc93/source_subfolder/gdal/port -I/home/conan/.conan/data/gdal/3.1.0/bincrafters/testing/build/4c51049431b2ca7ca5e455a2d25db4c01f5bdc93/source_subfolder/gdal/gcore -I/home/conan/.conan/data/gdal/3.1.0/bincrafters/testing/build/4c51049431b2ca7ca5e455a2d25db4c01f5bdc93/source_subfolder/gdal/alg -I/home/conan/.conan/data/gdal/3.1.0/bincrafters/testing/build/4c51049431b2ca7ca5e455a2d25db4c01f5bdc93/source_subfolder/gdal/ogr -I/home/conan/.conan/data/gdal/3.1.0/bincrafters/testing/build/4c51049431b2ca7ca5e455a2d25db4c01f5bdc93/source_subfolder/gdal/ogr/ogrsf_frmts -I/home/conan/.conan/data/gdal/3.1.0/bincrafters/testing/build/4c51049431b2ca7ca5e455a2d25db4c01f5bdc93/source_subfolder/gdal/gnm -I/home/conan/.conan/data/gdal/3.1.0/bincrafters/testing/build/4c51049431b2ca7ca5e455a2d25db4c01f5bdc93/source_subfolder/gdal/apps -DHAVE_SSE_AT_COMPILE_TIME -m64 -O3 -fPIC -stdlib=libc++ -fvisibility=hidden  -Wall -Wextra -Winit-self -Wunused-parameter -Wformat -Werror=format-security -Wno-format-nonliteral -Wshorten-64-to-32 -Wshadow -Werror=vla -Wdate-time -Wnull-dereference -Wextra-semi -Wcomma -Wfloat-conversion -Wdocumentation -Wno-documentation-deprecated-sync -Wunused-private-field -Wmissing-declarations -Wnon-virtual-dtor -Woverloaded-virtual -fno-operator-names -Wzero-as-null-pointer-constant -Wimplicit-fallthrough  -I/home/conan/.conan/data/gdal/3.1.0/bincrafters/testing/build/4c51049431b2ca7ca5e455a2d25db4c01f5bdc93/source_subfolder/gdal/frmts/vrt -DGNM_ENABLED -I/home/conan/.conan/data/json-c/0.13.1/_/_/package/961755854760682044d37dabfc2d998658edf961/include -I/home/conan/.conan/data/libgeotiff/1.6.0/_/_/package/a8010b9b5b442ae49d438aa47df569210b11ae65/include -I/home/conan/.conan/data/flatbuffers/1.12.0/_/_/package/7a6b4153449d93fcb618404a0cb0dc0b69dc87fa/include -I/home/conan/.conan/data/libpng/1.6.37/_/_/package/9de5bc6401cdef5fe3faeb686f8f0b96f5878e0b/include -I/home/conan/.conan/data/proj/7.0.0/_/_/package/662ea4ce1322083e130b4b86d136eb9806ac931a/include -I/home/conan/.conan/data/libtiff/4.1.0/_/_/package/88993a48f0fa66db4e2e8a60db8185d9ea62e8c3/include -I/home/conan/.conan/data/sqlite3/3.31.1/_/_/package/8024ee1d28473555b0d438e4cee5b9e2ae03b794/include -I/home/conan/.conan/data/libcurl/7.69.1/_/_/package/17d38f99c1a7aa10a96bfbd885785d51639b5b77/include -I/home/conan/.conan/data/zlib/1.2.11/_/_/package/961755854760682044d37dabfc2d998658edf961/include -I/home/conan/.conan/data/xz_utils/5.2.4/_/_/package/961755854760682044d37dabfc2d998658edf961/include -I/home/conan/.conan/data/libjpeg/9d/_/_/package/961755854760682044d37dabfc2d998658edf961/include -I/home/conan/.conan/data/jbig/20160605/_/_/package/961755854760682044d37dabfc2d998658edf961/include -I/home/conan/.conan/data/zstd/1.4.3/_/_/package/961755854760682044d37dabfc2d998658edf961/include -I/home/conan/.conan/data/libwebp/1.0.3/_/_/package/397baf88a0282b06dbebb69e534eed679c964aa1/include -I/home/conan/.conan/data/openssl/1.1.1g/_/_/package/961755854760682044d37dabfc2d998658edf961/include -DLIBJPEG_STATIC -DLZMA_API_STATIC -DCURL_STATICLIB=1 -DNDEBUG -I/home/conan/.conan/data/gdal/3.1.0/bincrafters/testing/build/4c51049431b2ca7ca5e455a2d25db4c01f5bdc93/source_subfolder/gdal/port -I/home/conan/.conan/data/libjpeg/9d/_/_/package/961755854760682044d37dabfc2d998658edf961 -I/home/conan/.conan/data/libjpeg/9d/_/_/package/961755854760682044d37dabfc2d998658edf961/include -I/home/conan/.conan/data/libgeotiff/1.6.0/_/_/package/a8010b9b5b442ae49d438aa47df569210b11ae65/include -I/home/conan/.conan/data/libtiff/4.1.0/_/_/package/88993a48f0fa66db4e2e8a60db8185d9ea62e8c3/include -I/home/conan/.conan/data/libpng/1.6.37/_/_/package/9de5bc6401cdef5fe3faeb686f8f0b96f5878e0b -I/home/conan/.conan/data/libpng/1.6.37/_/_/package/9de5bc6401cdef5fe3faeb686f8f0b96f5878e0b/include  -DGDAL_COMPILATION -I/home/conan/.conan/data/json-c/0.13.1/_/_/package/961755854760682044d37dabfc2d998658edf961/include/json-c -I/home/conan/.conan/data/proj/7.0.0/_/_/package/662ea4ce1322083e130b4b86d136eb9806ac931a/include -I/home/conan/.conan/data/gdal/3.1.0/bincrafters/testing/build/4c51049431b2ca7ca5e455a2d25db4c01f5bdc93/source_subfolder/gdal/ogr/ogrsf_frmts/geojson -I/home/conan/.conan/data/gdal/3.1.0/bincrafters/testing/build/4c51049431b2ca7ca5e455a2d25db4c01f5bdc93/source_subfolder/gdal/ogr/ogrsf_frmts/generic -g -I/home/conan/.conan/data/gdal/3.1.0/bincrafters/testing/build/4c51049431b2ca7ca5e455a2d25db4c01f5bdc93/source_subfolder/gdal/gnm -c -o gnmanalyse.lo gnmanalyse.cpp
      2020-05-25T09:54:17.0368605Z /bin/bash /home/conan/.conan/data/gdal/3.1.0/bincrafters/testing/build/4c51049431b2ca7ca5e455a2d25db4c01f5bdc93/source_subfolder/gdal/libtool --mode=link --silent clang++ -m64 -L/home/conan/.conan/data/json-c/0.13.1/_/_/package/961755854760682044d37dabfc2d998658edf961/lib -L/home/conan/.conan/data/libgeotiff/1.6.0/_/_/package/a8010b9b5b442ae49d438aa47df569210b11ae65/lib -L/home/conan/.conan/data/flatbuffers/1.12.0/_/_/package/7a6b4153449d93fcb618404a0cb0dc0b69dc87fa/lib -L/home/conan/.conan/data/libpng/1.6.37/_/_/package/9de5bc6401cdef5fe3faeb686f8f0b96f5878e0b/lib -L/home/conan/.conan/data/proj/7.0.0/_/_/package/662ea4ce1322083e130b4b86d136eb9806ac931a/lib -L/home/conan/.conan/data/libtiff/4.1.0/_/_/package/88993a48f0fa66db4e2e8a60db8185d9ea62e8c3/lib -L/home/conan/.conan/data/sqlite3/3.31.1/_/_/package/8024ee1d28473555b0d438e4cee5b9e2ae03b794/lib -L/home/conan/.conan/data/libcurl/7.69.1/_/_/package/17d38f99c1a7aa10a96bfbd885785d51639b5b77/lib -L/home/conan/.conan/data/zlib/1.2.11/_/_/package/961755854760682044d37dabfc2d998658edf961/lib -L/home/conan/.conan/data/xz_utils/5.2.4/_/_/package/961755854760682044d37dabfc2d998658edf961/lib -L/home/conan/.conan/data/libjpeg/9d/_/_/package/961755854760682044d37dabfc2d998658edf961/lib -L/home/conan/.conan/data/jbig/20160605/_/_/package/961755854760682044d37dabfc2d998658edf961/lib -L/home/conan/.conan/data/zstd/1.4.3/_/_/package/961755854760682044d37dabfc2d998658edf961/lib -L/home/conan/.conan/data/libwebp/1.0.3/_/_/package/397baf88a0282b06dbebb69e534eed679c964aa1/lib -L/home/conan/.conan/data/openssl/1.1.1g/_/_/package/961755854760682044d37dabfc2d998658edf961/lib gdalinfo_bin.lo  /home/conan/.conan/data/gdal/3.1.0/bincrafters/testing/build/4c51049431b2ca7ca5e455a2d25db4c01f5bdc93/source_subfolder/gdal/libgdal.la  -o gdalinfo
      2020-05-25T09:54:17.2231170Z /bin/bash /home/conan/.conan/data/gdal/3.1.0/bincrafters/testing/build/4c51049431b2ca7ca5e455a2d25db4c01f5bdc93/source_subfolder/gdal/libtool --mode=link --silent clang++ -m64 -L/home/conan/.conan/data/json-c/0.13.1/_/_/package/961755854760682044d37dabfc2d998658edf961/lib -L/home/conan/.conan/data/libgeotiff/1.6.0/_/_/package/a8010b9b5b442ae49d438aa47df569210b11ae65/lib -L/home/conan/.conan/data/flatbuffers/1.12.0/_/_/package/7a6b4153449d93fcb618404a0cb0dc0b69dc87fa/lib -L/home/conan/.conan/data/libpng/1.6.37/_/_/package/9de5bc6401cdef5fe3faeb686f8f0b96f5878e0b/lib -L/home/conan/.conan/data/proj/7.0.0/_/_/package/662ea4ce1322083e130b4b86d136eb9806ac931a/lib -L/home/conan/.conan/data/libtiff/4.1.0/_/_/package/88993a48f0fa66db4e2e8a60db8185d9ea62e8c3/lib -L/home/conan/.conan/data/sqlite3/3.31.1/_/_/package/8024ee1d28473555b0d438e4cee5b9e2ae03b794/lib -L/home/conan/.conan/data/libcurl/7.69.1/_/_/package/17d38f99c1a7aa10a96bfbd885785d51639b5b77/lib -L/home/conan/.conan/data/zlib/1.2.11/_/_/package/961755854760682044d37dabfc2d998658edf961/lib -L/home/conan/.conan/data/xz_utils/5.2.4/_/_/package/961755854760682044d37dabfc2d998658edf961/lib -L/home/conan/.conan/data/libjpeg/9d/_/_/package/961755854760682044d37dabfc2d998658edf961/lib -L/home/conan/.conan/data/jbig/20160605/_/_/package/961755854760682044d37dabfc2d998658edf961/lib -L/home/conan/.conan/data/zstd/1.4.3/_/_/package/961755854760682044d37dabfc2d998658edf961/lib -L/home/conan/.conan/data/libwebp/1.0.3/_/_/package/397baf88a0282b06dbebb69e534eed679c964aa1/lib -L/home/conan/.conan/data/openssl/1.1.1g/_/_/package/961755854760682044d37dabfc2d998658edf961/lib gdalserver.lo  /home/conan/.conan/data/gdal/3.1.0/bincrafters/testing/build/4c51049431b2ca7ca5e455a2d25db4c01f5bdc93/source_subfolder/gdal/libgdal.la -o gdalserver
      2020-05-25T09:54:17.4373625Z /bin/bash /home/conan/.conan/data/gdal/3.1.0/bincrafters/testing/build/4c51049431b2ca7ca5e455a2d25db4c01f5bdc93/source_subfolder/gdal/libtool --mode=link --silent clang++ -m64 -L/home/conan/.conan/data/json-c/0.13.1/_/_/package/961755854760682044d37dabfc2d998658edf961/lib -L/home/conan/.conan/data/libgeotiff/1.6.0/_/_/package/a8010b9b5b442ae49d438aa47df569210b11ae65/lib -L/home/conan/.conan/data/flatbuffers/1.12.0/_/_/package/7a6b4153449d93fcb618404a0cb0dc0b69dc87fa/lib -L/home/conan/.conan/data/libpng/1.6.37/_/_/package/9de5bc6401cdef5fe3faeb686f8f0b96f5878e0b/lib -L/home/conan/.conan/data/proj/7.0.0/_/_/package/662ea4ce1322083e130b4b86d136eb9806ac931a/lib -L/home/conan/.conan/data/libtiff/4.1.0/_/_/package/88993a48f0fa66db4e2e8a60db8185d9ea62e8c3/lib -L/home/conan/.conan/data/sqlite3/3.31.1/_/_/package/8024ee1d28473555b0d438e4cee5b9e2ae03b794/lib -L/home/conan/.conan/data/libcurl/7.69.1/_/_/package/17d38f99c1a7aa10a96bfbd885785d51639b5b77/lib -L/home/conan/.conan/data/zlib/1.2.11/_/_/package/961755854760682044d37dabfc2d998658edf961/lib -L/home/conan/.conan/data/xz_utils/5.2.4/_/_/package/961755854760682044d37dabfc2d998658edf961/lib -L/home/conan/.conan/data/libjpeg/9d/_/_/package/961755854760682044d37dabfc2d998658edf961/lib -L/home/conan/.conan/data/jbig/20160605/_/_/package/961755854760682044d37dabfc2d998658edf961/lib -L/home/conan/.conan/data/zstd/1.4.3/_/_/package/961755854760682044d37dabfc2d998658edf961/lib -L/home/conan/.conan/data/libwebp/1.0.3/_/_/package/397baf88a0282b06dbebb69e534eed679c964aa1/lib -L/home/conan/.conan/data/openssl/1.1.1g/_/_/package/961755854760682044d37dabfc2d998658edf961/lib gdal_translate_bin.lo  /home/conan/.conan/data/gdal/3.1.0/bincrafters/testing/build/4c51049431b2ca7ca5e455a2d25db4c01f5bdc93/source_subfolder/gdal/libgdal.la -o gdal_translate
      2020-05-25T09:54:17.6085638Z /bin/bash /home/conan/.conan/data/gdal/3.1.0/bincrafters/testing/build/4c51049431b2ca7ca5e455a2d25db4c01f5bdc93/source_subfolder/gdal/libtool --mode=link --silent clang++ -m64 -L/home/conan/.conan/data/json-c/0.13.1/_/_/package/961755854760682044d37dabfc2d998658edf961/lib -L/home/conan/.conan/data/libgeotiff/1.6.0/_/_/package/a8010b9b5b442ae49d438aa47df569210b11ae65/lib -L/home/conan/.conan/data/flatbuffers/1.12.0/_/_/package/7a6b4153449d93fcb618404a0cb0dc0b69dc87fa/lib -L/home/conan/.conan/data/libpng/1.6.37/_/_/package/9de5bc6401cdef5fe3faeb686f8f0b96f5878e0b/lib -L/home/conan/.conan/data/proj/7.0.0/_/_/package/662ea4ce1322083e130b4b86d136eb9806ac931a/lib -L/home/conan/.conan/data/libtiff/4.1.0/_/_/package/88993a48f0fa66db4e2e8a60db8185d9ea62e8c3/lib -L/home/conan/.conan/data/sqlite3/3.31.1/_/_/package/8024ee1d28473555b0d438e4cee5b9e2ae03b794/lib -L/home/conan/.conan/data/libcurl/7.69.1/_/_/package/17d38f99c1a7aa10a96bfbd885785d51639b5b77/lib -L/home/conan/.conan/data/zlib/1.2.11/_/_/package/961755854760682044d37dabfc2d998658edf961/lib -L/home/conan/.conan/data/xz_utils/5.2.4/_/_/package/961755854760682044d37dabfc2d998658edf961/lib -L/home/conan/.conan/data/libjpeg/9d/_/_/package/961755854760682044d37dabfc2d998658edf961/lib -L/home/conan/.conan/data/jbig/20160605/_/_/package/961755854760682044d37dabfc2d998658edf961/lib -L/home/conan/.conan/data/zstd/1.4.3/_/_/package/961755854760682044d37dabfc2d998658edf961/lib -L/home/conan/.conan/data/libwebp/1.0.3/_/_/package/397baf88a0282b06dbebb69e534eed679c964aa1/lib -L/home/conan/.conan/data/openssl/1.1.1g/_/_/package/961755854760682044d37dabfc2d998658edf961/lib gdaladdo.lo  /home/conan/.conan/data/gdal/3.1.0/bincrafters/testing/build/4c51049431b2ca7ca5e455a2d25db4c01f5bdc93/source_subfolder/gdal/libgdal.la -o gdaladdo
      2020-05-25T09:54:17.7886859Z /usr/bin/ld: .libs/gdal_translate_bin.o: undefined reference to symbol '_ZNKSt3__121__basic_string_commonILb1EE20__throw_length_errorEv'
      2020-05-25T09:54:17.7889374Z /usr/bin/ld: /lib/x86_64-linux-gnu/libc++.so.1: error adding symbols: DSO missing from command line
      
      => understood, AutoToolsBuildEnvironment doesn't append -stdlib=libc++ in LDFLAGS [question] AutoToolsBuildEnvironment: add -stdlib in LDFLAGS if clang? conan#7109. As a workaround, I manually inject -stdlib=libc++ in LDFLAGS if clang and libc++ on Linux.
  • Windows:
    • MinGW works most of the time, but configure fails to even test compiler if there are too many dependencies enabled (maybe something related to command line length, I don't know) => ok, issue was coming from xerces-c recipe xerces-c: fix cpp_info.libs for MinGW + cache cmake + pthread in system_libs #1787
      I need to check if it works in Debug mode, because configure.ac does a lot of AC_CHECK_LIB (I already had to remove a lot of them for robust Linux and Macos builds) which can break if lib name has a debug suffix, which is not uncommon on Windows.
    • Visual Studio (nmake):
  • MacOS: don't know (autotools outputs are so verbose that Travis CI kills my build :( )

I've tried to use as much as possible external dependencies instead of internal ones, except:

  • those which don't have a recipe obviously (pcidsk, pcraster-rasterformat, qhull, libopencad)
  • shapelib: because in gdal it's compiled with a definition which lead to include a gdal header. External lib can't do that.
  • lerc: gdal uses a private header (maybe a header in progress in main repo of lerc, but it doesn't properly exports its symbols, so I decided to use internal lerc in gdal).

I've also tried to fix a little bit nmake files because there are discrepancies between autotools and nmake files (some options can't be disabled in nmake files).

Conflicts: there are conflicts in the dependency tree of CCI. For example, libxml2 requires libiconv/1.15 while odbc requires libiconv/1.16 (these 2 dependencies are not enabled by default). If it's not overridden by gdal (with_libiconv=False), installation fails:

ERROR: Conflict in libxml2/2.9.10:
    'libxml2/2.9.10' requires 'libiconv/1.15' while 'odbc/2.3.7' requires 'libiconv/1.16'.
    To fix this conflict you need to override the package 'libiconv' in your root package.

Choice for default options is not straightforward.
I've decided to set to:

  • False:
    • dependencies which are disabled by default
    • dependencies which are tested in system by default, but gdal disable them if not found (this is the most common case)
  • True:
    • dependencies which are internal by default
    • maybe enable geos by default

(this recipe was hard to create, a lot of things are very fragile in gdal build system)

@conan-center-bot
Copy link
Collaborator

Some configurations of 'gdal/3.1.0' failed in build 2 (30b839150b87c8525c2b09562df8f1989a801138):

@conan-center-bot
Copy link
Collaborator

Some configurations of 'gdal/3.1.0' failed in build 3 (e88247568faee204307717ce453e8b029202310e):

@SpaceIm
Copy link
Contributor Author

SpaceIm commented May 25, 2020

How to deal with that 😓 :

gdal/3.1.0: WARN: Build folder is dirty, removing it: C:\J\c3ipr\pr_1722_3_0_2\.conan\data\gdal\3.1.0\_\_\build\883f8a0dda83088f9bc4d671549ec9101b7d7632
gdal/3.1.0: WARN: Trying to remove corrupted source folder
gdal/3.1.0: WARN: This can take a while for big packages
ERROR: gdal/3.1.0: Error in source() method, line 303
	tools.get(**self.conan_data["sources"][self.version])
	FileNotFoundError: [Errno 2] No such file or directory: '.\\gdal-3.1.0\\autotest\\gdrivers\\data\\fake_sentinel2_l2a_MSIL2A\\S2A_MSIL2A_20180818T094031_N0208_R036_T34VFJ_20180818T120345.SAFE\\GRANULE\\L2A_T34VFJ_A016478_20180818T094030\\IMG_DATA\\R10m\\T34VFJ_20180818T094031_AOT_10m.jp2'

@conan-center-bot
Copy link
Collaborator

Some configurations of 'gdal/3.1.0' failed in build 4 (6c334c4b535fc9ddbc82579557de741f43167539):

@SpaceIm
Copy link
Contributor Author

SpaceIm commented May 26, 2020

It probably makes more sense to not build GDAL executables in this recipe (rather in a dedicated recipe):

  • GDAL has 27 executables.
  • If debug build type, static and default options, it's ~143 MB per executable (~3.8 GB) on Linux with gcc-9. Moreover, there is no reason to use debug executables.

@SpaceIm
Copy link
Contributor Author

SpaceIm commented May 26, 2020

@danimtb @uilianries @SSE4

Could you clean up manually this condiguration please? CI fails to remove dirty source folder:

gdal/3.1.0: WARN: Build folder is dirty, removing it: C:\J\c3ipr\pr_1722_3_0_2\.conan\data\gdal\3.1.0\_\_\build\883f8a0dda83088f9bc4d671549ec9101b7d7632
gdal/3.1.0: WARN: Trying to remove corrupted source folder
gdal/3.1.0: WARN: This can take a while for big packages
ERROR: gdal/3.1.0: Error in source() method, line 303
	tools.get(**self.conan_data["sources"][self.version])
	FileNotFoundError: [Errno 2] No such file or directory: '.\\gdal-3.1.0\\autotest\\gdrivers\\data\\fake_sentinel2_l2a_MSIL2A\\S2A_MSIL2A_20180818T094031_N0208_R036_T34VFJ_20180818T120345.SAFE\\GRANULE\\L2A_T34VFJ_A016478_20180818T094030\\IMG_DATA\\R10m\\T34VFJ_20180818T094031_AOT_10m.jp2'

Removing autotest/gdrivers/data folder in source method may be a workaround to prevent this issue.

@uilianries
Copy link
Member

@danimtb seems like we have an internal problem here.

@SpaceIm
Copy link
Contributor Author

SpaceIm commented May 27, 2020

@SpaceIm SpaceIm marked this pull request as ready for review May 27, 2020 08:29
if self.options.threadsafe:
self.cpp_info.system_libs.append("pthread")
elif self.settings.os == "Windows":
self.cpp_info.system_libs.extend(["psapi", "ws2_32"])
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GDAL directly call functions defined in kernel32 if using win32 threads. Should I add this lib also? It's not required by MinGW Posix Threads I think.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, kernel32 is not required. I used to write code with WinAPI in past, including windows threads, kernel32 was not required.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

functions like CreateThread reside in kernel32.lib. it's usually included by default in Visual Studio, but for other compilers (MinGW, Clang, Intel, Borland, etc.) it might not be the case.

@conan-center-bot
Copy link
Collaborator

Some configurations of 'gdal/3.1.0' failed in build 5 (f9d11c3054fe7481b0804917e5c8669d0f888ebc):

@conan-center-bot
Copy link
Collaborator

All green in build 6 (7d1e41f531ed07b53b777b29e9c481e0a0e61638)! 😊

@SpaceIm
Copy link
Contributor Author

SpaceIm commented May 27, 2020

Great, it builds.
I've also made a lot of builds with several non default options enabled or disabled, on Linux, Visual Studio, MinGW, and it seems to be very neat now (it honors options). There is still an issue when too many options are enabled with MinGW (ok, it was an issue in xerces-c recipe: #1787).

Now, important question: #1722 (comment)

I think that GDAL executables should have a dedicated recipe (too big in debug mode, and I don't see any reason to use debug executables).

@uilianries
Copy link
Member

uilianries commented May 28, 2020

I think that GDAL executables should have a dedicated recipe (too big in debug mode, and I don't see any reason to use debug executables).

Only if they are distributed separated. We are trying to avoid it, because we had many problems with installer package, and now we have build context, which allows get them at same package. Otherwise, you can add an option to build or not the executables, with build context, you can different profiles if it's used as build requirement.

@SpaceIm
Copy link
Contributor Author

SpaceIm commented May 28, 2020

Disk space is not an issue? I didn't check but it's probably something like 150 GB of executables in CCI for each (version, revision)
(against something like 1.7GB for Windows + Linux + Macos if executables have their own recipe, no issue with shared lib on Linux/Macos, no slow debug executable, no need to build from source just because you don't want executables).

@uilianries
Copy link
Member

Disk space is not a problem at all, also both VCPKG and Conda provide all together.

@SpaceIm
Copy link
Contributor Author

SpaceIm commented May 28, 2020

Well ok, therefore this PR is ready for review.

@SpaceIm SpaceIm requested review from SSE4 and uilianries May 28, 2020 14:06
@SpaceIm SpaceIm requested a review from danimtb June 4, 2020 16:10
@conan-center-bot
Copy link
Collaborator

All green in build 9 (bd215a37a06c07810e1ff7f2a46baded9c4a1ebb)! 😊

uilianries
uilianries previously approved these changes Jun 5, 2020
Copy link
Member

@uilianries uilianries left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@conan-center-bot
Copy link
Collaborator

All green in build 10 (b48b4b16204bedac56161be847f168a630a8450a)! 😊

danimtb
danimtb previously approved these changes Jun 10, 2020
@SpaceIm
Copy link
Contributor Author

SpaceIm commented Jun 10, 2020

I can add a small patch to prevent adding CPL_DISABLE_DLL in cpp_info.defines if Visual Studio and shared: OSGeo/gdal#2664

@SpaceIm SpaceIm requested a review from danimtb June 10, 2020 14:59
options = {
"shared": [True, False],
"fPIC": [True, False],
"simd_intrinsics": [None, "sse", "ssse3", "avx"],
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I should use False instead of None for consistency like

"with_libevent": [False, "libevent", "libev"],

@conan-center-bot
Copy link
Collaborator

All green in build 11 (f5eecdf5fef6b06ca9693e7b6acf1a8bb4b114c2)! 😊

Copy link
Contributor

@jgsogo jgsogo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would love to know projects people are working on using all these libraries from Conan Center 😃

@SSE4 SSE4 merged commit 112dec0 into conan-io:master Jun 11, 2020
@SpaceIm SpaceIm deleted the gdal/3.1.0 branch June 11, 2020 10:57
@sebhmg
Copy link

sebhmg commented Jun 11, 2020

I would love to know projects people are working on using all these libraries from Conan Center 😃

@jgsogo, we are using parts of GDAL at Mira Geoscience in two of our products:

There was no easy way to get the latest SDK of GDAL. Now there is Conan! Thank you @SpaceIm and all the contributors for your amazing work here!

I have just started to explore Conan a couple of months ago as a better way to handle our third parties.

@b1x
Copy link

b1x commented Jun 11, 2020

This is a great work! Than you @SpaceIm ))
May I ask you about small change, please don't remove "share" directory from package, this folder (share\gdal) contains resources for a proper gdal drivers work. For example s57objectclasses.csv contains attributes schema for s-57 charts. So please don't remove this folder"

@SpaceIm
Copy link
Contributor Author

SpaceIm commented Jun 11, 2020

Good catch @b1x

We need to:

  • rename share to res (share is not allowed in package folder).
  • add res path to S57_CSV env var, and maybe other env var into which to add this path ? (is GDAL_DATA enough?)

Could you open an issue?

@b1x
Copy link

b1x commented Jun 11, 2020

Good catch @b1x

We need to:

* rename `share` to `res` (share is not allowed in package folder).

* add `res` path to `S57_CSV` env var, and maybe other env var into which to add this path ? (is `GDAL_DATA` enough?)

Could you open an issue?

I will try it)

@SpaceIm
Copy link
Contributor Author

SpaceIm commented Jun 11, 2020

For the PR, I already have something ready.

@b1x
Copy link

b1x commented Jun 11, 2020

For linux this patch, works for me, but unfortunately I don't have Windows

diff --git a/recipes/gdal/all/conanfile.py b/recipes/gdal/all/conanfile.py
index 8ec7b013..d53759f1 100644
--- a/recipes/gdal/all/conanfile.py
+++ b/recipes/gdal/all/conanfile.py
@@ -461,6 +461,7 @@ class GdalConan(ConanFile):
         self._autotools = AutoToolsBuildEnvironment(self, win_bash=tools.os_info.is_windows)
 
         args = []
+        args.extend(["--datarootdir=${prefix}/res"])
         # Shared/Static
         if self.options.shared:
             args.extend(["--disable-static", "--enable-shared"])

@SpaceIm
Copy link
Contributor Author

SpaceIm commented Jun 11, 2020

Well there are several things:

  • remove some definitions during gdal compilation which can hardcode path (defeats binaries built in another machine).
  • install data in res/gdal subfolder of package folder.
  • append this path into GDAL_DATA env var (in package_info method).
  • maybe add an env variable for plugins dir, even though this recipe doesn't currently build plugins.

@SpaceIm
Copy link
Contributor Author

SpaceIm commented Jun 11, 2020

should be fixed in #1884

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[request] gdal/3.0.4
9 participants