Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new GCI_ constants in particular for InfraRed and SAR, and 'standardize' … #10721

Merged
merged 5 commits into from
Sep 19, 2024

Conversation

rouault
Copy link
Member

@rouault rouault commented Sep 3, 2024

…a band-level IMAGERY metadata domain

Implements discussion thread started at https://lists.osgeo.org/pipermail/gdal-dev/2024-August/thread.html#59403

Added values:
* GCI_PanBand: Panchromatic band [0.40 - 1.00 um]
* GCI_CoastalBand: Coastal band [0.40 - 0.45 um]
* GCI_RedEdgeBand: Red-edge band [0.69 - 0.79 um]
* GCI_NIRBand: Near-InfraRed (NIR) band [0.75 - 1.40 um]
* GCI_SWIRBand: Short-Wavelength InfraRed (SWIR) band [1.40 - 3.00 um]
* GCI_MWIRBand: Mid-Wavelength InfraRed (MWIR) band [3.00 - 8.00 um]
* GCI_LWIRBand: Long-Wavelength InfraRed (LWIR) band [8.00 - 15 um]
* GCI_TIRBand: Thermal InfraRed (TIR) band (MWIR or LWIR) [3 - 15 um]
* GCI_OtherIRBand: Other infrared band [0.75 - 1000 um]

They can be in particular used for the GeoTIFF driver, being serialized in the GDAL_METADATA tag.

For spectral bands, the wavelength ranges are indicative only, and may vary depending on sensors.

STACIT driver enhanced to map STAC common_name to GCI_ constants

gdalinfo -json output enhanced to output STAC common_name from GCI_ constants

Items in the IMAGERY band-level metadata domain:

  • CENTRAL_WAVELENGTH_UM: Central Wavelength in micrometers.
  • FWHM_UM: Full-width half-maximum (FWHM) in micrometers.

Filled by the SENTINEL2 and ENVI drivers (if corresponding metadata items are found in the ENVI header)
May also be found in other drivers handling arbitrary GDAL metadata, such as the ones using the GDAL Persistent Auxiliary Mechanism (PAM / .aux.xml side car files) or VRT drivers. The GTiff driver also supports serializing and deserializing the band IMAGERY metadata domain in the GDAL_METADATA TIFF tag.

@rouault rouault added this to the 3.10.0 milestone Sep 3, 2024
@coveralls
Copy link
Collaborator

coveralls commented Sep 3, 2024

Coverage Status

coverage: 69.382% (+0.01%) from 69.371%
when pulling d80f555 on rouault:new_gci_values
into 2742d2b on OSGeo:master.

@rouault rouault force-pushed the new_gci_values branch 3 times, most recently from 58451a9 to 9c0ba10 Compare September 3, 2024 18:57
@sgillies
Copy link
Contributor

sgillies commented Sep 4, 2024

@rouault does support in 2 drivers justify adding this to a core module? Adding more constants has a cost downstream.

None of this scales to the future world of ubiguitous hyperspectral data, right? I'm not sure where we draw the line. Maybe I'm missing something, in which case I apologize.

@rouault
Copy link
Member Author

rouault commented Sep 4, 2024

does support in 2 drivers justify adding this to a core module?

The new GCI constants are also supported automatically by GTiff, VRT and GTI.

None of this scales to the future world of ubiguitous hyperspectral data, right?

I'm not very well versed in that domain, but I guess for that use case advertizing the central wavelength and FWHM is the way to go.

I'm not sure where we draw the line. Maybe I'm missing something, in which case I apologize.

Well, I thought this would be useful. I've always been frustrating opening GeoTIFF with B,G,R,Undefined bands where this was meant to be B,G,R,NIR

Copy link
Contributor

@jjimenezshaw jjimenezshaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @rouault

If I had an ultraviolet sensor (that I do not plan to have), how should I tag it? gray, with central wavelength and fwhm?

@@ -1396,6 +1396,7 @@ InfoFormat
InfoOptions
Informatisé
Informix
InfraRed
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any reason for the upper-case R? infrared is a word in English https://www.collinsdictionary.com/dictionary/english/infrared

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any reason for the upper-case R? infrared is a word in English

I'm just fixing sphinx-spelling warnings

- CENTRAL_WAVELENGTH_UM: Central Wavelength in micrometers.
- FWHM_UM: Full-width half-maximum (FWHM) in micrometers.

At time of writing, they are set by the SENTINEL2 and ENVI drivers (if corresponding metadata items are found in the ENVI header)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe you can say that they are available in other formats, but not populated by default. Otherwise it is easy to think that they are only for these two drivers mentioned here.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also mentioning which function can be used to set them in other drivers could be useful.

@klaussnd
Copy link

klaussnd commented Sep 4, 2024

Thanks, this is a really useful addition.

Some details:

GCI_TIRBand: Thermal InfraRed (TIR) band (MWIR or LWIR) [3 - 15 um]
GCI_OtherIRBand: Other infrared band [0.75 - 1000 um]

Why do we have those enum values that are a superset of other ones? Wouldn't it be better if the band types are not overlapping and the user would have to choose either MWIR or LWIR, instead of using TIR, for example?

@rouault
Copy link
Member Author

rouault commented Sep 4, 2024

@jjimenezshaw

If I had an ultraviolet sensor (that I do not plan to have), how should I tag it? gray, with central wavelength and fwhm?

or undefined. + central wavelength + fwhm

@klaussnd

Also mentioning which function can be used to set them in other drivers could be useful.

Standard GDALRasterBand::GetMetadataItem/SetMetadataItem API

GCI_TIRBand: Thermal InfraRed (TIR) band (MWIR or LWIR) [3 - 15 um]
GCI_OtherIRBand: Other infrared band [0.75 - 1000 um]
Why do we have those enum values that are a superset of other ones?

TIR: based on feedback of https://lists.osgeo.org/pipermail/gdal-dev/2024-August/059404.html
OtherIR: provision for other band types. For example Sentinel2 "cirrus" B10 band does not fit well in other enumeration terms, and is mapped to that one

@jjimenezshaw + @klaussnd I've enhanced the doc to be:

Such metadata items can be get (resp. set) with :cpp:func:`GDALRasterBand::GetMetadataItem()`
(resp. :cpp:func:`GDALRasterBand::SetMetadataItem()`).`

They are specifically set by the :ref:`raster.sentinel2` and
:ref:`raster.envi` drivers (if corresponding metadata items are found in the ENVI header),
but may also be found in other drivers handling arbitrary GDAL metadata, such as
the one using the GDAL Persistent Auxiliary Mechanism (PAM / .aux.xml side car files)
or :ref:`raster.vrt` drivers. The :ref:`raster.gtiff` driver also supports serializing
and deserializing the band IMAGERY metadata domain in the ``GDAL_METADATA`` TIFF tag.

@tbonfort
Copy link
Member

tbonfort commented Sep 4, 2024

@rouault does support in 2 drivers justify adding this to a core module? Adding more constants has a cost downstream.

None of this scales to the future world of ubiguitous hyperspectral data, right? I'm not sure where we draw the line. Maybe I'm missing something, in which case I apologize.

It is my understanding that this metadata will be serialized to .aux.xml PAM files, which opens up the list to more than 2 drivers (@rouault please correct me if I'm wrong).
I agree on your comment on where to draw the line, but given that GDAL's colorinterpretation tag is in itself rather ill-defined (e.g. what is "red" exactly for a given sensor) it is to be expected that the list of available entries can and will grow over time. As long as this metadata remains purely informative as it is now, I see it as a nice user-friendly addition for folks opening a dataset in qgis/etc and directly having the information as to what to expect from each band. The advanced user or the purist will already know that information, or get precise information from an authoritative source.

@rouault
Copy link
Member Author

rouault commented Sep 4, 2024

I've asked for feedback to a SAR data producer and he pointed me at https://www.earthdata.nasa.gov/learn/backgrounders/what-is-sar. So perhaps add GCI_SAR_[Ka|K|Ku|X|C|S|L|P]_Band ?

@rouault
Copy link
Member Author

rouault commented Sep 4, 2024

So perhaps add GCI_SAR_[Ka|K|Ku|X|C|S|L|P]_Band ?

Done in the last commit. I've also added #define's GCI_IR_Start and GCI_IR_End so that all IR band values are in that range (can make it easier for code wanting very broad categorization), as well as GCI_SAR_Start, GCI_SAR_End. I've also added a few reserved values so we can extend those ranges a bit in the future. The design constraint is that all values in [GCI_Undefined, GCI_Max] must be valid enumeration items

@pedros007
Copy link
Contributor

This is a nice feature. It is certainly good to reduce ambiguity around B,G,R,Undefined bands (or worse: B,G,R,Alpha) when data is actually B,G,R,NIR. Every constellation will likely have slightly different center wavelengths, so users should consult data providers for specific center wavelengths.

Does it make sense to add GCI_YellowBand? Maxar's 8-band satellites collect a yellow band 585 - 625 nm. Landsat-Next is planned to have a yellow band of 585-615nm. However, I don't know where to draw the line as Landsat Next has additional bands like Violet, Orange, etc.

@rouault
Copy link
Member Author

rouault commented Sep 4, 2024

Does it make sense to add GCI_YellowBand?

it is actually already existing, but was limited to the yellow of CMYK color space. In this proposal, I also extend it to "Yellow band of CMYK image, or yellow spectral band [0.58 - 0.62 um]"

@rouault rouault changed the title Add new GCI_ constants in particular for InfraRed, and 'standardize' … Add new GCI_ constants in particular for InfraRed and SAR, and 'standardize' … Sep 4, 2024
@rouault rouault mentioned this pull request Sep 11, 2024
11 tasks
bradh

This comment was marked as resolved.

…he GDAL_METADATA, expose it in a COLOR_INTERPRETATION metadata item
…a band-level IMAGERY metadata domain

Added values:
    * GCI_PanBand: Panchromatic band [0.40 - 1.00 um]
    * GCI_CoastalBand: Coastal band [0.40 - 0.45 um]
    * GCI_RedEdgeBand: Red-edge band [0.69 - 0.79 um]
    * GCI_NIRBand: Near-InfraRed (NIR) band [0.75 - 1.40 um]
    * GCI_SWIRBand: Short-Wavelength InfraRed (SWIR) band [1.40 - 3.00 um]
    * GCI_MWIRBand: Mid-Wavelength InfraRed (MWIR) band [3.00 - 8.00 um]
    * GCI_LWIRBand: Long-Wavelength InfraRed (LWIR) band [8.00 - 15 um]
    * GCI_TIRBand: Thermal InfraRed (TIR) band (MWIR or LWIR) [3 - 15 um]
    * GCI_OtherIRBand: Other infrared band [0.75 - 1000 um]

For spectral bands, the wavelength ranges are indicative only, and may vary
depending on sensors.

STACIT driver enhanced to map STAC common_name to GCI_ constants

gdalinfo -json output enhanced to output STAC common_name from GCI_
constants

Items in the IMAGERY band-level metadata domain:
- CENTRAL_WAVELENGTH_UM: Central Wavelength in micrometers.
- FWHM_UM: Full-width half-maximum (FWHM) in micrometers.

Filled by the SENTINEL2 and ENVI drivers (if corresponding metadata items are found in
the ENVI header)
… by default (unless COPY_SRC_MDD or SRC_MDD creation options are set)
…_[IR|SAR]_[Start|End]

as well as GCI_IR_Reserved[1|2|3|4] and GCI_SAR_Reserved_[1|2]
@rouault rouault merged commit 9ada0c8 into OSGeo:master Sep 19, 2024
36 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants