PROJ 6.1+ (pyproj 2.2+ wheels) incompatible with previous versions

[snowman2]

Related to:

• 'init' keyword on Proj() causing segfault #367
• Segmentation fault: pyproj chrashes python when changing crs #351
• pyproj >= 2 segmentation fault geopandas/geopandas#1040

Problem:

When proj<6 is installed in the environment and a pyproj wheel with PROJ 6.1+ is installed, a segfault occurs.

Steps to reproduce:

conda create -n test python=3.6 proj4=5.2.0
conda activate test
(test) pip install pyproj==2.2.1
(test)python -c "import pyproj; print(pyproj.proj_version_str)"
6.1.1
(test) python -c "from pyproj import CRS; CRS(init='epsg:4326')"
Segmentation fault (core dumped)

However, if you install a version with PROJ 6.0.0, it works okay:

conda create -n test python=3.6 proj4=5.2.0
conda activate test
(test) pip install pyproj==2.1.3
(test) python -c "import pyproj; print(pyproj.proj_version_str)"
6.0.0
(test) python -c "from pyproj import CRS; CRS(init='epsg:4326')"

Still fails with pyproj==2.1.3 with PROJ 6.1+

conda create -n test python=3.6 proj4=5.2.0
conda activate test
(test) pip install pip install http://a365fff413fe338398b6-1c8a9b3114517dc5fe17b7c3f8c63a43.r19.cf2.rackcdn.com/pyproj-2.1.3.post0-cp36-cp36m-manylinux1_x86_64.whl
(test) python -c "import pyproj; print(pyproj.proj_version_str)"
6.1.1
(test) python -c "from pyproj import CRS; CRS(init='epsg:4326')"
Segmentation fault (core dumped)

PROJ error location

$ gdb python
GNU gdb (Ubuntu 8.1-0ubuntu3) 8.1.0.20180409-git
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from python...done.
(gdb) run dump.py
Starting program: /home/snowal/miniconda3/envs/projmix/bin/python dump.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
/home/snowal/miniconda3/envs/projmix/lib/python3.6/site-packages/pyproj/crs.py:113: FutureWarning: '+init=<authority>:<code>' syntax is deprecated. '<authority>:<code>' is the preferred initialization method.
  FutureWarning,

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff6279c88 in osgeo::proj::io::PROJStringParser::createFromPROJString(std::string const&) ()
   from /home/snowal/miniconda3/envs/projmix/lib/python3.6/site-packages/pyproj/.libs/libproj-e5ef822e.so.15.1.1

Seems to be:

osgeo::proj::io::PROJStringParser::createFromPROJString(std::string const&) ()

[jorisvandenbossche]

It would be nice to at least be able to detect it to give a more informative error message to the user. But, I would think it should actually be solvable (although in the end, it is a quite fundamental difference between pip and conda how they handle dependencies ..).

[jorisvandenbossche]

But, I would think it should actually be solvable

Since the so name of the included proj library gets mangled, shouldn't the pyproj wheel be able to get the correct proj lib? (the one included) Even when there are other libraries with the same name available on the system
I thought that was the point of mangling names of the external libraries included in wheels?

[snowman2]

The part that has me confused is that it works with PROJ 6.0.0 but not with 6.1.1. I am curious as to what changed between the versions that might impact that.

[snowman2]

I wonder if it has to do with the version of cython used? They have had some releases recently https://github.com/cython/cython/releases

[aspedisca]

On my manjaro system I have
usr/lib/libproj.so.15 is owned by proj 6.1.0-1

and in virtual env I have
pyproj2.2.1

but I get the following error on import of cartopy

libproj.so.13: cannot open shared object file: No such file or directory

I have these versions of libproj on my system
****/venv/lib/python3.7/site-packages/fiona/.libs/libproj-bd876d1a.so.12.0.0
****/venv/lib/python3.7/site-packages/pyproj/.libs/libproj-1b1cfd8e.so.15.1.0
/usr/lib/libproj.so
/usr/lib/libproj.so.15
/usr/lib/libproj.so.15.1.0

I assume this problem is related to this bug.
I can not downgrade proj in a sane manner and I was wondering if there is an easy way to fix the issue?

[snowman2]

@aspedisca cartopy does not use pyproj. I would recommend opening an issue on the cartopy repo.

[snowman2]

This may help you: SciTools/cartopy#1289 (comment)

[snowman2]

I am hoping this will finally be addressed in 2.3.1 🤞

[snowman2]

Fix confirmed:

$ python
Python 3.6.7 | packaged by conda-forge | (default, Jul  2 2019, 02:18:42) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyproj
>>> pyproj.show_versions()

System:
    python: 3.6.7 | packaged by conda-forge | (default, Jul  2 2019, 02:18:42)  [GCC 7.3.0]
executable: /home/snowal/miniconda3/envs/geocube/bin/python
   machine: Linux-4.15.0-58-generic-x86_64-with-debian-buster-sid

PROJ:
      PROJ: 6.1.1
  data dir: /home/snowal/miniconda3/envs/geocube/lib/python3.6/site-packages/pyproj/proj_dir/share/proj

Python deps:
    pyproj: 2.3.1
       pip: 19.1.1
setuptools: 40.8.0
    Cython: None
>>> pyproj.CRS(init="epsg:4326")
<Geographic 2D CRS: +init=epsg:4326 +type=crs>
Name: WGS 84
Axis Info [ellipsoidal]:
- lon[east]: Longitude (degree)
- lat[north]: Latitude (degree)
Area of Use:
- name: World
- bounds: (-180.0, -90.0, 180.0, 90.0)
Datum: World Geodetic System 1984
- Ellipsoid: WGS 84
- Prime Meridian: Greenwich

>>> exit()
$ conda list proj
# packages in environment at /home/snowal/miniconda3/envs/geocube:
#
# Name                    Version                   Build  Channel
proj4                     5.2.0             he1b5a44_1003    conda-forge
pyproj                    2.3.1                    pypi_0    pypi