`describeCoverage` & `getCoverage` not returning all rangeType info for multiband coverages #88

annakrystalli · 2022-08-22T13:58:59Z

For the following RGB multiband coverage example, the rangeType object returned by DescribeCoverage only gives the details of the last band (BLUE).

library(ows4R)
#> Loading required package: geometa
#> Loading ISO 19139 XML schemas...
#> Loading ISO 19115 codelists...
#> Loading IANA mime types...
#> No encoding supplied: defaulting to UTF-8.
#> Loading required package: keyring
wcs <- ows4R::WCSClient$new(url = "https://geoserver.emodnet-physics.eu/geoserver/wcs",
                            serviceVersion = "2.0.1", logger = "DEBUG")
#> [ows4R][INFO] OWSGetCapabilities - Fetching https://geoserver.emodnet-physics.eu/geoserver/wcs?service=WCS&version=2.0.1&request=GetCapabilities
desc <- wcs$describeCoverage("emodnet__EP_GEO_NER_OTHR_NN_NN_RAS")
#> [ows4R][INFO] WCSClient - Fetching coverageSummary description for 'emodnet__EP_GEO_NER_OTHR_NN_NN_RAS' ... 
#> [ows4R][INFO] WCSDescribeCoverage - Fetching https://geoserver.emodnet-physics.eu/geoserver/wcs?service=WCS&version=2.0.1&coverageId=emodnet__EP_GEO_NER_OTHR_NN_NN_RAS&request=DescribeCoverage
#> Loading required package: sf
#> Linking to GEOS 3.9.1, GDAL 3.4.0, PROJ 8.1.1; sf_use_s2() is TRUE
# Only last band details are returned
desc$rangeType
#> <GMLElement>
#> ....|-- DataRecord <GMLElement>
#> ........|-- field <GMLElement>
#> ............|-- Quantity <GMLElement>
#> ................|-- description <GMLElement>
#> ....................|-- value: BLUE_BAND
#> ................|-- uom <GMLElement>
#> ................|-- constraint <GMLElement>
#> ....................|-- AllowedValues <GMLElement>
#> ........................|-- interval <GMLElement>
#> ............................|-- value: 0 255
# getRange() returns list of length 0
desc$getRange()
#> list()

^{Created on 2022-08-22 by the reprex package (v2.0.1)}

Session info

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.1.3 (2022-03-10)
#>  os       macOS Big Sur/Monterey 10.16
#>  system   x86_64, darwin17.0
#>  ui       X11
#>  language (EN)
#>  collate  en_GB.UTF-8
#>  ctype    en_GB.UTF-8
#>  tz       Europe/Athens
#>  date     2022-08-22
#>  pandoc   2.18 @ /Applications/RStudio.app/Contents/MacOS/quarto/bin/tools/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version   date (UTC) lib source
#>  askpass       1.1       2019-01-13 [1] CRAN (R 4.1.0)
#>  assertthat    0.2.1     2019-03-21 [1] CRAN (R 4.1.0)
#>  class         7.3-20    2022-01-16 [1] CRAN (R 4.1.3)
#>  classInt      0.4-7     2022-06-10 [1] CRAN (R 4.1.2)
#>  cli           3.3.0     2022-04-25 [1] CRAN (R 4.1.2)
#>  codetools     0.2-18    2020-11-04 [1] CRAN (R 4.1.3)
#>  curl          4.3.2     2021-06-23 [1] CRAN (R 4.1.0)
#>  DBI           1.1.3     2022-06-18 [1] CRAN (R 4.1.2)
#>  digest        0.6.29    2021-12-01 [1] CRAN (R 4.1.0)
#>  dplyr         1.0.9     2022-04-28 [1] CRAN (R 4.1.2)
#>  e1071         1.7-11    2022-06-07 [1] CRAN (R 4.1.2)
#>  evaluate      0.15      2022-02-18 [1] CRAN (R 4.1.2)
#>  fansi         1.0.3     2022-03-24 [1] CRAN (R 4.1.2)
#>  fastmap       1.1.0     2021-01-25 [1] CRAN (R 4.1.0)
#>  fs            1.5.2     2021-12-08 [1] CRAN (R 4.1.0)
#>  generics      0.1.2     2022-01-31 [1] CRAN (R 4.1.2)
#>  geometa     * 0.6-6     2022-01-26 [1] CRAN (R 4.1.2)
#>  glue          1.6.2     2022-02-24 [1] CRAN (R 4.1.2)
#>  highr         0.9       2021-04-16 [1] CRAN (R 4.1.0)
#>  htmltools     0.5.3     2022-07-18 [1] CRAN (R 4.1.2)
#>  httr          1.4.3     2022-05-04 [1] CRAN (R 4.1.2)
#>  jsonlite      1.8.0     2022-02-22 [1] CRAN (R 4.1.2)
#>  KernSmooth    2.23-20   2021-05-03 [1] CRAN (R 4.1.3)
#>  keyring     * 1.3.0     2021-11-29 [1] CRAN (R 4.1.0)
#>  knitr         1.39      2022-04-26 [1] CRAN (R 4.1.2)
#>  lifecycle     1.0.1     2021-09-24 [1] CRAN (R 4.1.0)
#>  magrittr      2.0.3     2022-03-30 [1] CRAN (R 4.1.2)
#>  openssl       2.0.2     2022-05-24 [1] CRAN (R 4.1.2)
#>  ows4R       * 0.3       2022-08-17 [1] Github (eblondel/ows4R@1cd9dbc)
#>  pillar        1.8.0     2022-07-18 [1] CRAN (R 4.1.2)
#>  pkgconfig     2.0.3     2019-09-22 [1] CRAN (R 4.1.0)
#>  proxy         0.4-27    2022-06-09 [1] CRAN (R 4.1.2)
#>  purrr         0.3.4     2020-04-17 [1] CRAN (R 4.1.0)
#>  R.cache       0.16.0    2022-07-21 [1] CRAN (R 4.1.2)
#>  R.methodsS3   1.8.2     2022-06-13 [1] CRAN (R 4.1.2)
#>  R.oo          1.25.0    2022-06-12 [1] CRAN (R 4.1.2)
#>  R.utils       2.12.0    2022-06-28 [1] CRAN (R 4.1.2)
#>  R6            2.5.1     2021-08-19 [1] CRAN (R 4.1.0)
#>  Rcpp          1.0.9     2022-07-08 [1] CRAN (R 4.1.2)
#>  reprex        2.0.1     2021-08-05 [1] CRAN (R 4.1.0)
#>  rlang         1.0.4     2022-07-12 [1] CRAN (R 4.1.2)
#>  rmarkdown     2.14      2022-04-25 [1] CRAN (R 4.1.2)
#>  rstudioapi    0.13      2020-11-12 [1] CRAN (R 4.1.0)
#>  sessioninfo   1.2.2     2021-12-06 [1] CRAN (R 4.1.0)
#>  sf          * 1.0-7     2022-03-07 [1] CRAN (R 4.1.2)
#>  stringi       1.7.6     2021-11-29 [1] CRAN (R 4.1.0)
#>  stringr       1.4.0     2019-02-10 [1] CRAN (R 4.1.2)
#>  styler        1.7.0     2022-03-13 [1] CRAN (R 4.1.2)
#>  terra         1.5-21    2022-02-17 [1] CRAN (R 4.1.2)
#>  tibble        3.1.8     2022-07-22 [1] CRAN (R 4.1.2)
#>  tidyselect    1.1.2     2022-02-21 [1] CRAN (R 4.1.2)
#>  units         0.8-0     2022-02-05 [1] CRAN (R 4.1.2)
#>  utf8          1.2.2     2021-07-24 [1] CRAN (R 4.1.0)
#>  vctrs         0.4.1     2022-04-13 [1] CRAN (R 4.1.2)
#>  withr         2.5.0     2022-03-03 [1] CRAN (R 4.1.2)
#>  xfun          0.31      2022-05-10 [1] CRAN (R 4.1.2)
#>  XML           3.99-0.10 2022-06-09 [1] CRAN (R 4.1.2)
#>  yaml          2.3.5     2022-02-21 [1] CRAN (R 4.1.2)
#> 
#>  [1] /Library/Frameworks/R.framework/Versions/4.1/Resources/library
#> 
#> ──────────────────────────────────────────────────────────────────────────────

Using https://geoserver.emodnet-physics.eu/geoserver/wcs?service=WCS&version=2.0.1&coverageId=emodnet__EP_GEO_NER_OTHR_NN_NN_RAS&request=DescribeCoverage in the browser returns the full details

<wcs:CoverageDescriptions xmlns:wcs="http://www.opengis.net/wcs/2.0" xmlns:ows="http://www.opengis.net/ows/2.0" xmlns:gml="http://www.opengis.net/gml/3.2" xmlns:gmlcov="http://www.opengis.net/gmlcov/1.0" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:swe="http://www.opengis.net/swe/2.0" xmlns:wcsgs="http://www.geoserver.org/wcsgs/2.0" xsi:schemaLocation=" http://www.opengis.net/wcs/2.0 http://schemas.opengis.net/wcs/2.0/wcsDescribeCoverage.xsd http://www.geoserver.org/wcsgs/2.0 https://geoserver.emodnet-physics.eu/geoserver/schemas/wcs/2.0/wcsgs.xsd">
<wcs:CoverageDescription gml:id="emodnet__EP_GEO_NER_OTHR_NN_NN_RAS">
<gml:description>Generated from WorldImage</gml:description>
<gml:name>Natural Earth features 5 types of raster files at 1:10 million-scale to suit your bandwidth and content focus. Two versions of the 10 million-scale raster data are offered: high resolution files at 21,600 x 10,800 pixels and low resolution at 16,200 x 8,100. The raster files register precisely with the 10m vector data. Embedded raster content includes: land cover, shaded relief, ocean water, and drainages with lakes. All files include a TFW world file.</gml:name>
<gml:boundedBy>
<gml:Envelope srsName="http://www.opengis.net/def/crs/EPSG/0/4326" axisLabels="Lat Long" uomLabels="Deg Deg" srsDimension="2">
<gml:lowerCorner>-90.000000000036 -180.0</gml:lowerCorner>
<gml:upperCorner>90.00000000000001 180.00000000007202</gml:upperCorner>
</gml:Envelope>
</gml:boundedBy>
<wcs:CoverageId>emodnet__EP_GEO_NER_OTHR_NN_NN_RAS</wcs:CoverageId>
<gml:coverageFunction>
<gml:GridFunction>
<gml:sequenceRule axisOrder="+2 +1">Linear</gml:sequenceRule>
<gml:startPoint>0 0</gml:startPoint>
</gml:GridFunction>
</gml:coverageFunction>
<gmlcov:metadata>
<gmlcov:Extension>
<ows:Keywords>
<ows:Keyword>WCS</ows:Keyword>
<ows:Keyword>WorldImage</ows:Keyword>
<ows:Keyword>NE1_HR_LC_SR_W_DR</ows:Keyword>
</ows:Keywords>
<ows:Metadata xlink:type="simple" xlink:href="https://catalogue.emodnet-physics.eu/geonetwork/srv/eng/xml_iso19139?uuid=bbeb010ad1d51578cd4a946f45eb7aea940a7e0d"/>
</gmlcov:Extension>
</gmlcov:metadata>
<gml:domainSet>
<gml:RectifiedGrid gml:id="grid00__emodnet__EP_GEO_NER_OTHR_NN_NN_RAS" dimension="2">
<gml:limits>
<gml:GridEnvelope>
<gml:low>0 0</gml:low>
<gml:high>21599 10799</gml:high>
</gml:GridEnvelope>
</gml:limits>
<gml:axisLabels>i j</gml:axisLabels>
<gml:origin>
<gml:Point gml:id="p00_emodnet__EP_GEO_NER_OTHR_NN_NN_RAS" srsName="http://www.opengis.net/def/crs/EPSG/0/4326">
<gml:pos>89.99166666666667 -179.99166666666667</gml:pos>
</gml:Point>
</gml:origin>
<gml:offsetVector srsName="http://www.opengis.net/def/crs/EPSG/0/4326">0.0 0.01666666666667</gml:offsetVector>
<gml:offsetVector srsName="http://www.opengis.net/def/crs/EPSG/0/4326">-0.01666666666667 0.0</gml:offsetVector>
</gml:RectifiedGrid>
</gml:domainSet>
<gmlcov:rangeType>
<swe:DataRecord>
<swe:field name="RED_BAND">
<swe:Quantity>
<swe:description>RED_BAND</swe:description>
<swe:uom code="W.m-2.Sr-1"/>
<swe:constraint>
<swe:AllowedValues>
<swe:interval>0 255</swe:interval>
</swe:AllowedValues>
</swe:constraint>
</swe:Quantity>
</swe:field>
<swe:field name="GREEN_BAND">
<swe:Quantity>
<swe:description>GREEN_BAND</swe:description>
<swe:uom code="W.m-2.Sr-1"/>
<swe:constraint>
<swe:AllowedValues>
<swe:interval>0 255</swe:interval>
</swe:AllowedValues>
</swe:constraint>
</swe:Quantity>
</swe:field>
<swe:field name="BLUE_BAND">
<swe:Quantity>
<swe:description>BLUE_BAND</swe:description>
<swe:uom code="W.m-2.Sr-1"/>
<swe:constraint>
<swe:AllowedValues>
<swe:interval>0 255</swe:interval>
</swe:AllowedValues>
</swe:constraint>
</swe:Quantity>
</swe:field>
</swe:DataRecord>
</gmlcov:rangeType>
<wcs:ServiceParameters>
<wcs:CoverageSubtype>RectifiedGridCoverage</wcs:CoverageSubtype>
<wcs:nativeFormat>image/tiff</wcs:nativeFormat>
</wcs:ServiceParameters>
</wcs:CoverageDescription>
</wcs:CoverageDescriptions>

The text was updated successfully, but these errors were encountered:

annakrystalli · 2022-08-30T10:43:29Z

Hey @eblondel , any updates on this? Sorry to pester but at this point, this issue is holding up EMODnetWCS development.

eblondel · 2022-09-26T15:07:56Z

Hi @annakrystalli back from annual leave, i 'll look asap why multiple fields are not kept although not tackled as proper SWE bindings, and we get only one. I will create a ticket in geometa. It's a tricky issue but i'll do my best to solve it asap so you can keep going on EMODnetWCS devs.

Behind that, this deals with generic types defined geometa that depend on a standard (SWE - used essentially in SOS standard) that is not yet integrated in geometa (see eblondel/geometa#187). Some recent tests were done, but it's not complete yet.

annakrystalli · 2022-09-26T16:27:43Z

Great! And thanks for the background info.

eblondel · 2022-09-26T21:50:53Z

@annakrystalli Please update geometa from Github. I've added the SWE DataRecord and fixed issue dealing with decoding. With this, you will get the rangeType as SWEDataRecord (from geometa binding), including the lit of fields.

Code:

require(ows4R)
wcs <- ows4R::WCSClient$new(url = "https://geoserver.emodnet-physics.eu/geoserver/wcs", serviceVersion = "2.0.1", logger = "DEBUG")
desc <- wcs$describeCoverage("emodnet__EP_GEO_NER_OTHR_NN_NN_RAS")
desc$rangeType

Output:

<SWEDataRecord>
....|-- field <SWEQuantity>
........|-- description: RED_BAND
........|-- uom[code=W.m-2.Sr-1]
........|-- constraint: 0 255
........|-- value: NA
....|-- field <SWEQuantity>
........|-- description: GREEN_BAND
........|-- uom[code=W.m-2.Sr-1]
........|-- constraint: 0 255
........|-- value: NA
....|-- field <SWEQuantity>
........|-- description: BLUE_BAND
........|-- uom[code=W.m-2.Sr-1]
........|-- constraint: 0 255
........|-- value: NA

annakrystalli · 2022-09-27T09:11:31Z

Awesome! Thanks @eblondel !

annakrystalli · 2022-09-29T12:15:48Z

Just looking into this today and although I can now get info on all bands, I cannot get nilValues anymore:

library(ows4R)
#> Loading required package: geometa
#> Loading ISO 19139 XML schemas...
#> Loading ISO 19115 codelists...
#> Loading IANA mime types...
#> No encoding supplied: defaulting to UTF-8.
#> Loading required package: keyring
wcs <- WCSClient$new(url = "https://geo.vliz.be/geoserver/Emodnetbio/wcs",
                            serviceVersion = "2.0.1")
summary <- wcs$getCapabilities()$findCoverageSummaryById("Emodnetbio__aca_spp_19582016_L1", exact = TRUE)
summary$getDescription()$rangeType
#> Loading required package: sf
#> Linking to GEOS 3.10.2, GDAL 3.4.2, PROJ 8.2.1; sf_use_s2() is TRUE
#> <SWEDataRecord>
#> ....|-- field <SWEQuantity>
#> ........|-- description: relative_abundance
#> ........|-- uom[code=W.m-2.Sr-1]
#> ........|-- constraint: -3.4028235E38 3.4028235E38
#> ........|-- value: NA

^{Created on 2022-09-29 with reprex v2.0.2}

nilValue is returned by the server though:

https://github.com/EMODnet/EMODnetWCS/blob/ca4ffe9603381112654e249a33c97bf9872e0259/tests/fixtures/biology-description/geo.vliz.be/geoserver/Emodnetbio/wcs-206fed.xml#L756-L775

annakrystalli · 2022-09-29T13:07:26Z

BTW do you know when the dependency of ows4R on the corrected version of geometa will be formalised, either here or on CRAN?

Currently if someone installs EMODnetWCS (or latest dev version of ows4R from GitHub which EMODnetWCS currently requires) they do not get the right version of geometa installed by default. I tried adding the dev version of geometa as an explicit dependency of EMODnetWCS but R CMD CHECK complains with a note because geometa is not actually used in code in EMODnetWCS.

Any chance you could formalise the dependency on the dev version of geometa in the dev version of ows4R until they are both updated on CRAN?

eblondel · 2022-09-29T13:10:31Z

@annakrystalli Sorry for that you are right. Doing it right now. I will also look into the nilValues problem.

eblondel · 2022-09-29T13:16:18Z

@annakrystalli For the nilValues can you send me an example of coverage description concerned by that? thanks

annakrystalli · 2022-09-29T13:24:01Z

Sorry, how do you mean? The example I gave should be retuning the nilValue (it did with the previous version and I had written a function to extract it: https://emodnet.github.io/EMODnetWCS/reference/emdn_get_bbox.html) but isn't at all anymore so no way to adjust the function as I did for band descriptions, uoms etc. Here's the relevant section of the server response to the getDescription() request: https://github.com/EMODnet/EMODnetWCS/blob/ca4ffe9603381112654e249a33c97bf9872e0259/tests/fixtures/biology-description/geo.vliz.be/geoserver/Emodnetbio/wcs-206fed.xml#L756-L775

eblondel · 2022-09-29T13:29:29Z

Yes thanks for the xml example. nilValues here are yet another binding not yet added in geometa... I will have to support it there

eblondel · 2022-09-29T14:28:49Z

@annakrystalli ok you can reinstall geometa from github.
I've enabled the decoding of nilvalues. SWE is still at experimental by the way...
Normally you should get something like

<SWEDataRecord>
....|-- field <SWEQuantity>
........|-- description: relative_abundance
........|-- nilValues <SWENilValues>
............|-- nilValue: 9.969209968386869E36
........|-- uom[code=W.m-2.Sr-1]
........|-- constraint: -3.4028235E38 3.4028235E38
........|-- value: NA

annakrystalli · 2022-09-29T15:03:44Z

excellent thank you! I'll test it out now

eblondel · 2022-09-30T21:48:39Z

@annakrystalli SWE support in geometa is still experimental, so there may be changes in the way SWE objects are modelled. SWE introduces some more challenges in geometa to support generic encoding/decoding of XML. for WCS support we are more focusing on decoding, but for next needs in ows4R (in particular for SOS support) the SWE encoding is key.

In particular for the above, the nilValue will include a value field and not give the direct decimal value. For this I need to rely on simple SWE Element that is required to be able to decode/encode properly some fields such as nilValues. Similar logic had to be implemented to support GML elements in the past.

As print mode, it will look like this:

<SWEDataRecord>
....|-- field <SWEQuantity>
........|-- description  
............|-- value: relative_abundance
........|-- nilValues  <SWENilValues>
............|-- nilValue [reason=http://www.opengis.net/def/nil/OGC/0/unknown]
................|-- value: 9.969209968386869E36
........|-- uom [code=W.m-2.Sr-1] 
........|-- constraint  
............|-- AllowedValues  
................|-- interval  
....................|-- value: -3.4028235E38 3.4028235E38

This was referenced Aug 22, 2022

describeCoverage & getCoverage not returning all rangeType info for multiband coverages EMODnet/EMODnetWCS#20

Closed

Create info.R functions EMODnet/EMODnetWCS#3

Closed

eblondel added OGC - WCS user support labels Sep 22, 2022

This was referenced Sep 26, 2022

N cardinality of XML children not caught when element is an abstract GML element eblondel/geometa#197

Closed

Support for SWE data model eblondel/geometa#187

Open

eblondel closed this as completed Sep 27, 2022

eblondel added a commit that referenced this issue Sep 29, 2022

add geometa as remotes #88

68e86ca

eblondel self-assigned this Sep 29, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`describeCoverage` & `getCoverage` not returning all rangeType info for multiband coverages #88

`describeCoverage` & `getCoverage` not returning all rangeType info for multiband coverages #88

annakrystalli commented Aug 22, 2022

annakrystalli commented Aug 30, 2022

eblondel commented Sep 26, 2022 •

edited

Loading

annakrystalli commented Sep 26, 2022

eblondel commented Sep 26, 2022

annakrystalli commented Sep 27, 2022

annakrystalli commented Sep 29, 2022

annakrystalli commented Sep 29, 2022

eblondel commented Sep 29, 2022 •

edited

Loading

eblondel commented Sep 29, 2022

annakrystalli commented Sep 29, 2022

eblondel commented Sep 29, 2022

eblondel commented Sep 29, 2022

annakrystalli commented Sep 29, 2022

eblondel commented Sep 30, 2022

describeCoverage & getCoverage not returning all rangeType info for multiband coverages #88

describeCoverage & getCoverage not returning all rangeType info for multiband coverages #88

Comments

annakrystalli commented Aug 22, 2022

annakrystalli commented Aug 30, 2022

eblondel commented Sep 26, 2022 • edited Loading

annakrystalli commented Sep 26, 2022

eblondel commented Sep 26, 2022

annakrystalli commented Sep 27, 2022

annakrystalli commented Sep 29, 2022

annakrystalli commented Sep 29, 2022

eblondel commented Sep 29, 2022 • edited Loading

eblondel commented Sep 29, 2022

annakrystalli commented Sep 29, 2022

eblondel commented Sep 29, 2022

eblondel commented Sep 29, 2022

annakrystalli commented Sep 29, 2022

eblondel commented Sep 30, 2022

`describeCoverage` & `getCoverage` not returning all rangeType info for multiband coverages #88

`describeCoverage` & `getCoverage` not returning all rangeType info for multiband coverages #88

eblondel commented Sep 26, 2022 •

edited

Loading

eblondel commented Sep 29, 2022 •

edited

Loading