Added transformation from JGD2011 Geographic 3D to JGD2011 height using GSIGEO2011

[hernando]

Completes OSGeo/PROJ-data#49 to enable the Japanese geoid GSIGEO2011 in PROJ

[rouault]

you also need to add a record in grid_alternatives.sql

[hernando]

you also need to add a record in grid_alternatives.sql

This grid is not an alternative to any other mentioned in the EPSG registry that I know of, thus I have no idea about what should be the original name to use for original_grid_name.

[rouault]

This grid is not an alternative to any other mentioned in the EPSG registry that I know of, thus I have no idea about what should be the original name to use for original_grid_name.

Yeah the name 'alternative' isn't particularly good here. But you need to put an entry there so that PROJ knows the grid is known by it and available on the CDN. original_grid_name and proj_grid_name will be the same here : jp_gsi_gsigeo2011_mainland.tif

As the file is now on PROJ CDN, make sure to test with PROJ_NETWORK=ON and cs2cs that things work well.

[hernando]

I've added the grid alternative and tested as requested and it works:

echo 34.290  135.630 0 | PROJ_NETWORK=ON cs2cs EPSG:6667 EPSG:6668+6695
34.29   135.63 -39.86

Before adding the grid I've sorted the grid_alternatives.sql by agency as it was a bit messy in my opinion. In the process I've spotted a few grids that are listed in PROJ CDN that don't appear there:

• au_ga_AUSGeoid98.tif
• ca_nrc_NAD83v6VG.tif
• ca_nrc_NAD83v70VG.tif
• fr_ign_ggker08v2.tif
• fr_ign_ggm04v1.tif
• fr_ign_ggpf*.tif
• is_lmi_ISN_vel_beta.tif
• nz_linz_nzgd2000*.tif
• us_noaa all horizontal grids and many vertical ones

I also found a couple of grids where the original grid name contains NOT-YET-IN-GRID-TRANSFORMATION, they are: nl_nsgi_rdcorr2018.tif, nl_nsgi_naptrans2018.tif.

[rouault]

Before adding the grid I've sorted the grid_alternatives.sql by agency as it was a bit messy in my opinion.

Thanks

a few grids that are listed in PROJ CDN that don't appear there:
* au_ga_AUSGeoid98.tif

Mapping done through SQL request in data/sql/customizations.sql

* ca_nrc_NAD83v6VG.tif
* ca_nrc_NAD83v70VG.tif

Those are in EPSG but point motion operations aren't handled properly by PROJ yet, hence no automatic mapping for now

* fr_ign_ggker08v2.tif
* fr_ign_ggm04v1.tif
* fr_ign_ggpf*.tif
* is_lmi_ISN_vel_beta.tif

Yep, extra stuff not in EPSG

nz_linz_nzgd2000*.tif

The TIFF files are indirectly referenced by the deformation model .json files

us_noaa all horizontal grids and many vertical ones

Those are in data/sql/grid_alternatives_generated_noaa.sql

I also found a couple of grids where the original grid name contains NOT-YET-IN-GRID-TRANSFORMATION, they are: nl_nsgi_rdcorr2018.tif, nl_nsgi_naptrans2018.tif.

Yes available for manual transformations, but not (yet ?) in EPSG

There is a scripts/grid_checks.py script that can be done to check the content of proj.db vs PROJ-data

[rouault]

@hernando One point I don't remember if I've asked: does GSI has a plan to submit a record for this transformation / geoid model to EPSG ? If not, could you approach them to encourage to do so ? This would simplify a lot maintenance on our side, since we could remove the custom entry in our database and just rely on the one of a future EPSG dataset release.

[hernando]

@hernando One point I don't remember if I've asked: does GSI has a plan to submit a record for this transformation / geoid model to EPSG ? If not, could you approach them to encourage to do so ? This would simplify a lot maintenance on our side, since we could remove the custom entry in our database and just rely on the one of a future EPSG dataset release.

No, we didn't discuss this before. Can I assume they are familiar with the process for submitting a new transformation to the EPSG registry? Because I'm not.

[rouault]

This is explained at https://epsg.org/dataset-change-requests.html . Mostly a matter of them sending an email to feedback@epsg.org saying "we have a transformation from JGD2011 Geographic 3D to JGD2011 height using GSIGEO2011 geoid model whose file name is XXXXXX, whose area of use is YYYYY and whose accuracy is ZZZZZ metres"

[hernando]

This brings to my head the question of the area of use for the geoid. I cropped the original data to match the bounds of EPSG:3263 Japan - onshore mainland because I was not sure if keeping the whole area would case issues. In any case for taking advantage of the increased area additional transformations would be required, because from the myriad of areas of use related to Japan, I didn't find any matching exactly the geoid bounds and I can't confidently decide which ones to use to cover all the data-containing regions of the geoid (https://epsg.io/?q=Japan%20kind:AREA). I guess this is something that GSI has to figure out, but it also means that we may need to change the geoid file on PROJ-data.

[kbevers]

I cropped the original data

I would say that it is better to leave it uncropped. People frequently use transformations outside their formal boundaries and get surprised when they don't work. A slightly bigger geoid grid makes this less of a problem. The grid can of course also be used by other transformations with different areas of use.

we may need to change the geoid file on PROJ-data

I would suggest doing that right away. I have just released PROJ-data 1.3RC1 so there's still time to get this sorted before the final release on sunday.

[rouault]

I would say that it is better to leave it uncropped. People frequently use transformations outside their formal boundaries and get surprised when they don't work. A slightly bigger geoid grid makes this less of a problem

I concur with that. For proj.db purposes, if you don't find an existing extent/area_of_use that matches the grid extent, you can keep "Japan -on shore mainland" for now. With the EPSG v10 data model, you could actually create several "usages" that would use different extents, but PROJ logic is probably not fully ready to make use of that, so I'd recommend staying with one single usage.

[hernando]

Good that I brought this up, because I thought you had noticed the cropping in the original PR.
Now that I went back to the README there's another reason for which I didn't use the whole extent. As it I wrote: " The data for the Tokara, Amami, Okinawa, and Sakishima Islands in the Ryukyu Archipelago, the Daitö Islands and the Ogasawara Islands has been discarded due to the lack of a vertical coordinate reference system in the EPSG catalog." (I remember searching the names of the places that were getting discarded, but I didn't remember they were so many).
This brings some questions to my mind. Can the geoid declare a vertical CRS with a smaller area of use than the grid bounds?
Is the metadata attached to the GeoTIFF using vertoffset_grid_to_gtiff.py just for information purposes and it's not really used by PROJ to infer anything about how to use it?

[rouault]

Can the geoid declare a vertical CRS with a smaller area of use than the grid bounds?

Not sure to understand this. The GeoTIFF file only refers to the vertical CRS through its EPSG code. The extent of the GeoTIFF file may just partially overlaps the one of the Geograpphic3D and VerticalCRS though

Is the metadata attached to the GeoTIFF using vertoffset_grid_to_gtiff.py just for information purposes and it's not really used by PROJ to infer anything about how to use it?

Yes this is informational only. PROJ only makes use of proj.db to determine which grid to use, and then once it has selected a grid, it uses the GeoTIFF keys to determine how to find nodes in the grid.

[hernando]

Can the geoid declare a vertical CRS with a smaller area of use than the grid bounds?

Not sure to understand this. The GeoTIFF file only refers to the vertical CRS through its EPSG code. The extent of the GeoTIFF file may just partially overlaps the one of the Geograpphic3D and VerticalCRS though

The issue is that there's no vertical CRS with an area of use as big as the geoid (at least I don't find it and nether I find other vertical CRSs for the smaller areas like Okinawa). This means that the extent of the grid will be a lot larger than the area of use of the most appropriate vertical CRS indicated in the metadata. But if this data is only for informational purposes, then it doesn't matter very much. That said, I find limited practical usage for this grid applied to areas for which there's no vertical coordinate system available.

[kbevers]

This means that the extent of the grid will be a lot larger than the area of use of the most appropriate vertical CRS indicated in the metadata

That is not a problem. The other way round would be though :-)

That said, I find limited practical usage for this grid applied to areas for which there's no vertical coordinate system available.

As I said previously, people tend to abuse CRS and their transformations. Transformations are frequently used outside their intended area. Providing a larger grid makes that possible in more cases. Also, there are uses for a geoid grid outside of official transformations defined by the local geodetic office. Scientists will find ways to use this in all sorts of non-standard ways. That's a good thing!

[hernando]

Should I rename the grid to jp_gsi_gsigeo2011.tif? (Which implies having to do another PR for PROJ a well).

[kbevers]

Should I rename the grid to jp_gsi_gsigeo2011.tif? (Which implies having to do another PR for PROJ a well).

Yes, I think that would be fitting.