Best practice around PROJ version / included datum ensemble members

[jorisvandenbossche]

In the geopandas test suite, I am running into the following issue: if you save a geoparquet file with a recent PROJ version (and use it to create the PROJJSON representation of the crs), and then read it with software that uses an older PROJ version, this fails in you run into unknown IDs.

I specifically ran into this with a crs of EPSG:4326 (but OGC:CRS84 has the same issue), where PROJ uses an ensemble datum for this crs. Depending on the database version included in PROJ, the datum ensemble might get new datum additions. But those new datums are not known by older versions of the database.
For example, with a recent version of PROJ, the datum ensemble now includes "WGS 1984 G2139" (EPSG:1309), but that's not part of the EPSG database included in an older version of PROJ.

Of course this is a general issue. You can also run into this with just an actual crs that didn't exist yet in an older version of the database), but in that case I think the user can be held responsible to choose a CRS that already exists for some time to have it compatible with older PROJ versions. While for a crs with a datum ensemble (like EPSG:4326 and OGC:CRS84), 1) the actual crs (the top-level ID) actually already existed before as well, and 2) it is also normal that the datum ensemble gets updated (that's the point of the ensemble).

---

Do we have any recommendation about how to deal with this situation?
We could recommend geoparquet writers to not include the actual ensemble members for those cases? (like the example JSON object for OGC:CRS84 at https://github.com/opengeospatial/geoparquet/blob/main/format-specs/geoparquet.md#ogccrs84-details) Or does that defeat the purpose of using a datum ensemble? (although for PROJ itself for doing a transformation, it doesn't really matter in practice if you don't use an epoch?)
Or should a PROJJSON consumer try to be more flexible for this case? (and ignore unknown datum ensemble members / just check the top-level EPSG:4326 ID)

cc @rouault

Example error of parsing a PROJJSON object for EPSG:4326 (produced with PROJ 8.2) using PROJ 7.0:

PROJJSON object (dict version):

{'$schema': 'https://proj.org/schemas/v0.4/projjson.schema.json',
 'type': 'GeographicCRS',
 'name': 'WGS 84',
 'datum_ensemble': {'name': 'World Geodetic System 1984 ensemble',
  'members': [{'name': 'World Geodetic System 1984 (Transit)',
    'id': {'authority': 'EPSG', 'code': 1166}},
   {'name': 'World Geodetic System 1984 (G730)',
    'id': {'authority': 'EPSG', 'code': 1152}},
   {'name': 'World Geodetic System 1984 (G873)',
    'id': {'authority': 'EPSG', 'code': 1153}},
   {'name': 'World Geodetic System 1984 (G1150)',
    'id': {'authority': 'EPSG', 'code': 1154}},
   {'name': 'World Geodetic System 1984 (G1674)',
    'id': {'authority': 'EPSG', 'code': 1155}},
   {'name': 'World Geodetic System 1984 (G1762)',
    'id': {'authority': 'EPSG', 'code': 1156}},
   {'name': 'World Geodetic System 1984 (G2139)',
    'id': {'authority': 'EPSG', 'code': 1309}}],
  'ellipsoid': {'name': 'WGS 84',
   'semi_major_axis': 6378137,
   'inverse_flattening': 298.257223563},
  'accuracy': '2.0',
  'id': {'authority': 'EPSG', 'code': 6326}},
 'coordinate_system': {'subtype': 'ellipsoidal',
  'axis': [{'name': 'Geodetic latitude',
    'abbreviation': 'Lat',
    'direction': 'north',
    'unit': 'degree'},
   {'name': 'Geodetic longitude',
    'abbreviation': 'Lon',
    'direction': 'east',
    'unit': 'degree'}]},
 'scope': 'Horizontal component of 3D system.',
 'area': 'World.',
 'bbox': {'south_latitude': -90,
  'west_longitude': -180,
  'north_latitude': 90,
  'east_longitude': 180},
 'id': {'authority': 'EPSG', 'code': 4326}}

Reading using pyproj:

In [26]: pyproj.__proj_version__                                                                                                                                                                                   
Out[26]: '7.0.0'

In [27]: pyproj.CRS({"$schema":"https://proj.org/schemas/v0.4/projjson.schema.json","type":"GeographicCRS","name":"WGS ...
...
CRSError: Invalid projection: ... (Internal Proj Error: proj_create: No Datum of code 1309)

[rouault]

This is a unfortunate bug in PROJ that I've just fixed in OSGeo/PROJ#3221 and whose fix will go in PROJ 9.0.1. Of course, I can't fix the past...
To avoid running into the issue, producers can strip out the "id" on the members of a datum ensemble. They are not super essential. I'll fix the GDAL implementation to do this (this actually mostly affects projected CRS based on WGS 84, since GDAL by default uses the "no explicit CRS = WGS 84 long-lat" in reading & writing).

[rouault]

GDAL related workarounds in OSGeo/gdal#5872

[jorisvandenbossche]

Thanks for the quick follow-up!