building against system libs and external pythons

[dirteat]

Hi there,
I have built cloudmap out of pypi and various issues showed up. Here some summary and patches proposal:

1. missing deps to various python packages which are actually built-in modules. Datetime is already built-in in python2 and python3, configparser only in python3:

--- cloudmap-0.9.2/setup.py.orig        2017-10-27 17:18:52.515799486 +0200
+++ cloudmap-0.9.2/setup.py     2017-10-27 17:20:57.620080091 +0200
@@ -70,12 +70,11 @@
 cmdclass['install'] = Install

 if sys.version_info >= (3, 2):
-    install_requires = ['pyresample', 'numpy', 'scipy', 'requests', 'datetime',
-                        'pillow>=3.0.0', 'setuptools>=0.7.2',
-                        'configparser>=3.5.0b1']
+    install_requires = ['pyresample', 'numpy', 'scipy', 'requests',
+                        'pillow>=3.0.0', 'setuptools>=0.7.2']
 else:
-    install_requires = ['pyresample', 'numpy', 'scipy', 'requests', 'datetime',
-                        'pillow>=3.0.0', 'setuptools>=0.7.2', 'configparser']
+    install_requires = ['pyresample', 'numpy', 'scipy', 'requests',
+                        'pillow>=3.0.0', 'setuptools>=0.7.2', 'configparser>=3.5.0b1']

 setup(
     name='CreateCloudMap',

2. eggs with incorrect version numbers

--- cloudmap-0.9.2/cloudmap/_version.py.orig    2017-10-27 16:50:06.144781102 +0200
+++ cloudmap-0.9.2/cloudmap/_version.py 2017-10-27 16:50:34.823468553 +0200
@@ -9,7 +9,7 @@
 # versioneer-0.11 (https://github.com/warner/python-versioneer)

 # these strings will be replaced by git during git-archive
-git_refnames = " (HEAD -> master, tag: 0.9.2, tag: 0.9.1)"
+git_refnames = " (HEAD -> master, tag: 0.9.2)"
 git_full = "ef3df6ebcf543854c66dec14acd77a9c966b7f23"

 # these strings are filled in when 'setup.py versioneer' creates _version.py

3. the "pc" function in satellites.py seems to miss some informations for the proj_dict. The following fix works, but that's just a wild guess from what works in geo_dundee!

--- cloudmap-0.9.2/cloudmap/satellites.py.orig  2017-10-27 17:49:33.401520400 +0200
+++ cloudmap-0.9.2/cloudmap/satellites.py       2017-10-27 17:50:27.639033323 +0200
@@ -126,7 +126,8 @@
 def pc(x_size, y_size):
     from pyresample import geometry
     """Create area defintion for world map in Plate Carree projection"""
-    proj_dict = {'proj': 'eqc'}
+    proj_dict = {'a': '6378137.0', 'b': '6356752.3',
+                     'h': '35785831.0', 'proj': 'eqc'}
     area_extent = (-20037508.34, -10018754.17, 20037508.34, 10018754.17)
     pc = geometry.AreaDefinition('pc', 'Plate Carree world map', 'pc',
                                  proj_dict, x_size, y_size, area_extent)

Hope it helps!
Cheers,
Chris.

[jmozmoz]

I updated the package to fix the problems with the dependencies and the version number.

But what is the problem with the projection?

[dirteat]

Great, thanks a lot!
Without the patch on satellite.py, I am getting the following error:

Download image date/time:  2017-10-27 15:00 UTC
Satellite file: images/2017_10_27_1500_MSG1_7_S2.jpeg
Satellite file: images/2017_10_27_1500_MSG3_9_S2.jpeg
Satellite file: images/2017_10_27_1500_GOES13_4_S2.jpeg
Satellite file: images/2017_10_27_1500_GOES15_4_S2.jpeg
Satellite file: images/2017_10_27_1500_MTSAT3_7_S2.jpeg
Traceback (most recent call last):
  File "/usr/bin/create_map", line 11, in <module>
    load_entry_point('CreateCloudMap==0.9.2', 'console_scripts', 'create_map')()
  File "/usr/lib/python3.6/site-packages/cloudmap/create_map.py", line 95, in main
    satellite_list.overlay(args.debug)
  File "/usr/lib/python3.6/site-packages/cloudmap/satellites.py", line 268, in overlay
    img = satellite.project()
  File "/usr/lib/python3.6/site-packages/cloudmap/geo_dundee.py", line 177, in project
    return self.project_pyresample()
  File "/usr/lib/python3.6/site-packages/cloudmap/geo_dundee.py", line 257, in project_pyresample
    self.outheight))
  File "/usr/lib/python3.6/site-packages/cloudmap/satellites.py", line 132, in pc
    proj_dict, x_size, y_size, area_extent)
  File "/usr/lib64/python3.6/site-packages/pyresample/geometry.py", line 738, in __init__
    proj = _spatial_mp.Proj(**proj_dict)
  File "/usr/lib64/python3.6/site-packages/pyproj/__init__.py", line 358, in __new__
    return _proj.Proj.__new__(self, projstring)
  File "_proj.pyx", line 84, in _proj.Proj.__cinit__ (_proj.c:1170)
RuntimeError: b'major axis or radius = 0 or not given'

and I have been using a system-wide install of pyproj-1.9.5.1 against libproj4 version 4.9.2. It seems that the proj_dict dictionary is required to have a and b with these versions.

Cheers,
Chris.

[jmozmoz]

I try to reproduce your error, but with the same libproj4 version (4.9.2) and pyproj-1.9.5.1 I am not able to get the error. What is your pyresample version? I am using 1.7.0 (the current one).

[eatdust]

Yes, same here, this is pyresample-1.7.0. That's weird, I'll try to figure out what is wrong with my install then!
Thanks for the feedback!

[jmozmoz]

Can you start with a fresh virtual environment and install CreateCloudMap plus all dependencies there by pip?

By the way, these looks somehow similar:
pytroll/pyresample#26
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=792068
http://lists.maptools.org/pipermail/proj/2011-October/005905.html
(the last link includes something about an initialization file. What could that be?)

[dirteat]

Yes, within pip only, in a clean environment, it works fine. It seems that I indeed miss some proj4 or pyproj default files. I'll try to figure it out and post it here. Thanks for the help!

[dirteat]

Ok, this is a well-known libproj issue:
OSGeo/PROJ#274

I am closing this issue, thanks a lot for your help!
Best,
Chris.

[dirteat]

To be specific, this file should be present for pyproj and proj to use default values:

/usr/share/proj/proj_def.dat

[jmozmoz]

Thank you for traking this down.