Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

"Filter error: unimplemented filter encountered" when installing from conda-forge #1182

Closed
bertcoerver opened this issue Aug 22, 2022 · 10 comments
Closed

"Filter error: unimplemented filter encountered" when installing from conda-forge #1182

bertcoerver opened this issue Aug 22, 2022 · 10 comments

Comments

@bertcoerver
Copy link

According to the installation instructions, it should be possible to install using either pip or conda. However:

When I create a new conda environment using:

conda create -n test1_hdf --yes -c conda-forge python netcdf4
conda activate test1_hdf

and then run the following code in Python:

import netCDF4
ds = netCDF4.Dataset(r"/Users/hmcoerver/Local/collect_test/PROBAV/PROBAV/PROBAV_S5_TOC_X21Y04_20210701_100M_V101.HDF5")
ds["LEVEL3/NDVI"]["NDVI"][...]

I get the following error (I can see the metadata of the file, it's just the opening of the actual data with [...] that causes the error):

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "src/netCDF4/_netCDF4.pyx", line 4739, in netCDF4._netCDF4.Variable.__getitem__
  File "src/netCDF4/_netCDF4.pyx", line 5688, in netCDF4._netCDF4.Variable._get
  File "src/netCDF4/_netCDF4.pyx", line 1965, in netCDF4._netCDF4._ensure_nc_success
RuntimeError: NetCDF: Filter error: unimplemented filter encountered

However, when I create a conda environment like this:

conda create -n test2_hdf --yes -c conda-forge python
conda activate test2_hdf
pip install netcdf4

The code works fine.

Python 3.10.5 | packaged by conda-forge | (main, Jun 14 2022, 07:05:37) [Clang 13.0.1 ] on darwin 

>>> netCDF4.__version__
'1.6.0'
@jswhit
Copy link
Collaborator

jswhit commented Aug 22, 2022 •
edited
Loading

Does that file use one of the new compression plugins in netcdf-c 4.9.0? If so, it looks like the conda installed package does not have support for that compression plugin. Try 'ncdump -sh'

@bertcoerver
Copy link
Author

I don't know about the compression, but here is its ncdump -sh:

netcdf PROBAV_S5_TOC_X21Y04_20210701_100M_V101 {
dimensions:
	lat = 10080 ;
	lon = 10080 ;
	phony_dim_2 = 1 ;
variables:
	char crs(phony_dim_2) ;
		string crs:grid_mapping_name = "latitude_longitude" ;
		crs:longitude_of_prime_meridian = 0. ;
		crs:semi_major_axis = 6378137. ;
		crs:inverse_flattening = 298.257223563 ;
		string crs:_CoordinateTransformType = "Projection" ;
		string crs:_CoordinateAxisTypes = "GeoX GeoY" ;
		string crs:spatial_ref = "GEOGCS[\"WGS 84\",DATUM[\"WGS_1984\",SPHEROID[\"WGS 84\",6378137,298.257223563,AUTHORITY[\"EPSG\",\"7030\"]],AUTHORITY[\"EPSG\",\"6326\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4326\"]]" ;
		string crs:GeoTransform = "30.0000000000 0.0009920635 0 35.0000000000 0 -0.0009920635 " ;
		crs:_Storage = "contiguous" ;
		crs:_NoFill = "true" ;
	double lat(lat) ;
		string lat:DIMENSION_LABELS = "lat" ;
		string lat:units = "degrees_north" ;
		string lat:axis = "Y" ;
		string lat:long_name = "latitude" ;
		string lat:standard_name = "latitude" ;
		string lat:_CoordinateAxisType = "Lat" ;
		lat:_Storage = "contiguous" ;
		lat:_Endianness = "little" ;
		lat:_NoFill = "true" ;
	double lon(lon) ;
		string lon:DIMENSION_LABELS = "lon" ;
		string lon:units = "degrees_east" ;
		string lon:axis = "X" ;
		string lon:long_name = "longitude" ;
		string lon:standard_name = "longitude" ;
		string lon:_CoordinateAxisType = "Lon" ;
		lon:_Storage = "contiguous" ;
		lon:_Endianness = "little" ;
		lon:_NoFill = "true" ;

// global attributes:
		string :PRODUCT_REFERENCE = "Synthesis_PROBAV_20210701_S5_TOC_100M_V101" ;
		:PRODUCT_VERSION = 101 ;
		string :DESCRIPTION = "PROBA-V Level3 Top Of Canopy Five-Daily Synthesis at 100M resolution" ;
		string :PLATFORM = "PROBA-1" ;
		string :INSTRUMENT = "VEGETATION" ;
		string :PROCESSING_DATE = "2021-07-06" ;
		string :PROCESSING_TIME = "22:53:50" ;
		string :MAP_PROJECTION_NAME = "GEOGRAPHIC_LAT_LON_WGS84" ;
		string :MAP_PROJECTION_FAMILY = "GEOGRAPHIC" ;
		string :MAP_PROJECTION_UNITS = "DEGREES" ;
		string :MAP_PROJECTION_REFERENCE = "EPSG:4326" ;
		string :MAP_PROJECTION_WKT = "GEOGCS[\"WGS 84\",DATUM[\"WGS_1984\",SPHEROID[\"WGS 84\",6378137,298.257223563,AUTHORITY[\"EPSG\",\"7030\"]],AUTHORITY[\"EPSG\",\"6326\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4326\"]]" ;
		:SYNTHESIS_PERIOD = 5 ;
		string :_CoordSysBuilder = "ucar.nc2.dataset.conv.CF1Convention" ;
		string :Conventions = "CF-1.6" ;
		string :institution = "Vito NV" ;
		string :source = "PROBA-V" ;
		:_SuperblockVersion = 0 ;
		:_IsNetcdf4 = 1 ;
		:_Format = "netCDF-4" ;

group: LEVEL3 {

  // group attributes:
  		string :PROCESSINGINFO_GEOMODELLING = "PROBAV_GEOMODELLING_V1.0" ;
  		string :PROCESSINGINFO_RADIOMODELLING = "PROBAV_RADIOMODELLING_V1.0" ;
  		string :PROCESSINGINFO_MAPPING = "PROBAV_MAPPING_V1.0" ;
  		string :PROCESSINGINFO_MOSAIC = "PROBAV_MOSAIC_V1.0" ;
  		string :PROCESSINGINFO_CLOUDICESNOWDETECTION = "PROBAV_CLOUDICESNOWDETECTION_V2.0" ;
  		string :PROCESSINGINFO_SHADOWDETECTION = "PROBAV_SHADOWDETECTION_V1.0" ;
  		string :PROCESSINGINFO_ATMOSPHERIC_CORRECTION = "PROBAV_ATMCORR_SMAC_V1.0" ;
  		string :PROCESSINGINFO_COMPOSITING = "PROBAV_COMPOSITING_MVC_V2.0" ;

  group: GEOMETRY {
    variables:
    	ubyte SAA(lat, lon) ;
    		SAA:SCALE = 0.66667f ;
    		SAA:OFFSET = 0.f ;
    		string SAA:UNITS = "Degrees" ;
    		SAA:NO_DATA = 255.f ;
    		string SAA:MAPPING = "Geographic Lat/Lon", "0.5", "0.5", "30.000000000000000", "35.000000000000000", "0.000992063492063", "0.000992063492063", "WGS84", "Degrees" ;
    		string SAA:DESCRIPTION = "Solar Azimuth Angles" ;
    		string SAA:DIMENSION_LABELS = "lat", "lon" ;
    		SAA:scale_factor = 1.5 ;
    		SAA:add_offset = -0. ;
    		SAA:_FillValue = 255.f ;
    		string SAA:units = "degree" ;
    		string SAA:long_name = "Solar Azimuth Angles" ;
    		string SAA:grid_mapping = "crs" ;
    		string SAA:standard_name = "solar_azimuth_angle" ;
    		SAA:_Storage = "chunked" ;
    		SAA:_ChunkSizes = 1, 10080 ;
    		SAA:_Filter = "4,32,8" ;
    		SAA:_NoFill = "true" ;
    	ubyte SZA(lat, lon) ;
    		SZA:SCALE = 2.f ;
    		SZA:OFFSET = 0.f ;
    		string SZA:UNITS = "Degrees" ;
    		SZA:NO_DATA = 255.f ;
    		string SZA:MAPPING = "Geographic Lat/Lon", "0.5", "0.5", "30.000000000000000", "35.000000000000000", "0.000992063492063", "0.000992063492063", "WGS84", "Degrees" ;
    		string SZA:DESCRIPTION = "Solar Zenith Angles" ;
    		string SZA:DIMENSION_LABELS = "lat", "lon" ;
    		SZA:scale_factor = 0.5 ;
    		SZA:add_offset = -0. ;
    		SZA:_FillValue = 255.f ;
    		string SZA:units = "degree" ;
    		string SZA:long_name = "Solar Zenith Angles" ;
    		string SZA:grid_mapping = "crs" ;
    		string SZA:standard_name = "solar_zenith_angle" ;
    		SZA:_Storage = "chunked" ;
    		SZA:_ChunkSizes = 1, 10080 ;
    		SZA:_Filter = "4,32,8" ;
    		SZA:_NoFill = "true" ;

    // group attributes:
    		:TOP_LEFT_LONGITUDE = 30.f ;
    		:TOP_LEFT_LATITUDE = 35.f ;
    		:TOP_RIGHT_LONGITUDE = 39.99901f ;
    		:TOP_RIGHT_LATITUDE = 35.f ;
    		:BOTTOM_LEFT_LONGITUDE = 30.f ;
    		:BOTTOM_LEFT_LATITUDE = 25.00099f ;
    		:BOTTOM_RIGHT_LONGITUDE = 39.99901f ;
    		:BOTTOM_RIGHT_LATITUDE = 25.00099f ;
    		:CENTER_LONGITUDE = 34.9995f ;
    		:CENTER_LATITUDE = 30.0005f ;
    		:TOP_LEFT_X = 30240.f ;
    		:TOP_LEFT_Y = 35280.f ;
    		:TOP_RIGHT_X = 40319.f ;
    		:TOP_RIGHT_Y = 35280.f ;
    		:BOTTOM_LEFT_X = 30240.f ;
    		:BOTTOM_LEFT_Y = 25201.f ;
    		:BOTTOM_RIGHT_X = 40319.f ;
    		:BOTTOM_RIGHT_Y = 25201.f ;
    		:CENTER_X = 35279.5f ;
    		:CENTER_Y = 30240.5f ;

    group: SWIR {
      variables:
      	ubyte VAA(lat, lon) ;
      		VAA:SCALE = 0.66667f ;
      		VAA:OFFSET = 0.f ;
      		string VAA:UNITS = "Degrees" ;
      		VAA:NO_DATA = 255.f ;
      		string VAA:MAPPING = "Geographic Lat/Lon", "0.5", "0.5", "30.000000000000000", "35.000000000000000", "0.000992063492063", "0.000992063492063", "WGS84", "Degrees" ;
      		string VAA:DESCRIPTION = "Viewing Azimuth Angles SWIR detector" ;
      		VAA:scale_factor = 1.5 ;
      		string VAA:DIMENSION_LABELS = "lat", "lon" ;
      		VAA:add_offset = -0. ;
      		VAA:_FillValue = 255.f ;
      		string VAA:units = "degree" ;
      		string VAA:long_name = "Viewing Azimuth Angles SWIR detector" ;
      		string VAA:grid_mapping = "crs" ;
      		string VAA:standard_name = "sensor_azimuth_angle" ;
      		VAA:_Storage = "chunked" ;
      		VAA:_ChunkSizes = 1, 10080 ;
      		VAA:_Filter = "4,32,8" ;
      		VAA:_NoFill = "true" ;
      	ubyte VZA(lat, lon) ;
      		VZA:SCALE = 2.f ;
      		VZA:OFFSET = 0.f ;
      		string VZA:UNITS = "Degrees" ;
      		VZA:NO_DATA = 255.f ;
      		string VZA:MAPPING = "Geographic Lat/Lon", "0.5", "0.5", "30.000000000000000", "35.000000000000000", "0.000992063492063", "0.000992063492063", "WGS84", "Degrees" ;
      		string VZA:DESCRIPTION = "Viewing Azimuth Angles SWIR detector" ;
      		string VZA:DIMENSION_LABELS = "lat", "lon" ;
      		VZA:scale_factor = 0.5 ;
      		VZA:add_offset = -0. ;
      		VZA:_FillValue = 255.f ;
      		string VZA:units = "degree" ;
      		string VZA:long_name = "Viewing Azimuth Angles SWIR detector" ;
      		string VZA:grid_mapping = "crs" ;
      		string VZA:standard_name = "sensor_zenith_angle" ;
      		VZA:_Storage = "chunked" ;
      		VZA:_ChunkSizes = 1, 10080 ;
      		VZA:_Filter = "4,32,8" ;
      		VZA:_NoFill = "true" ;

      // group attributes:
      } // group SWIR

    group: VNIR {
      variables:
      	ubyte VAA(lat, lon) ;
      		VAA:SCALE = 0.66667f ;
      		VAA:OFFSET = 0.f ;
      		string VAA:UNITS = "Degrees" ;
      		VAA:NO_DATA = 255.f ;
      		string VAA:MAPPING = "Geographic Lat/Lon", "0.5", "0.5", "30.000000000000000", "35.000000000000000", "0.000992063492063", "0.000992063492063", "WGS84", "Degrees" ;
      		string VAA:DESCRIPTION = "Viewing Azimuth Angles VNIR detector" ;
      		string VAA:DIMENSION_LABELS = "lat", "lon" ;
      		VAA:scale_factor = 1.5 ;
      		VAA:add_offset = -0. ;
      		VAA:_FillValue = 255.f ;
      		string VAA:units = "degree" ;
      		string VAA:long_name = "Viewing Azimuth Angles VNIR detector" ;
      		string VAA:grid_mapping = "crs" ;
      		string VAA:standard_name = "sensor_azimuth_angle" ;
      		VAA:_Storage = "chunked" ;
      		VAA:_ChunkSizes = 1, 10080 ;
      		VAA:_Filter = "4,32,8" ;
      		VAA:_NoFill = "true" ;
      	ubyte VZA(lat, lon) ;
      		VZA:SCALE = 2.f ;
      		VZA:OFFSET = 0.f ;
      		string VZA:UNITS = "Degrees" ;
      		VZA:NO_DATA = 255.f ;
      		string VZA:MAPPING = "Geographic Lat/Lon", "0.5", "0.5", "30.000000000000000", "35.000000000000000", "0.000992063492063", "0.000992063492063", "WGS84", "Degrees" ;
      		string VZA:DESCRIPTION = "Viewing Zenith Angles VNIR detector" ;
      		string VZA:DIMENSION_LABELS = "lat", "lon" ;
      		VZA:scale_factor = 0.5 ;
      		VZA:add_offset = -0. ;
      		VZA:_FillValue = 255.f ;
      		string VZA:units = "degree" ;
      		string VZA:long_name = "Viewing Zenith Angles VNIR detector" ;
      		string VZA:grid_mapping = "crs" ;
      		string VZA:standard_name = "sensor_zenith_angle" ;
      		VZA:_Storage = "chunked" ;
      		VZA:_ChunkSizes = 1, 10080 ;
      		VZA:_Filter = "4,32,8" ;
      		VZA:_NoFill = "true" ;

      // group attributes:
      } // group VNIR
    } // group GEOMETRY

  group: NDVI {
    variables:
    	ubyte NDVI(lat, lon) ;
    		NDVI:SCALE = 250.f ;
    		NDVI:OFFSET = 20.f ;
    		string NDVI:UNITS = "-" ;
    		NDVI:NO_DATA = 255.f ;
    		string NDVI:MAPPING = "Geographic Lat/Lon", "0.5", "0.5", "30.000000000000000", "35.000000000000000", "0.000992063492063", "0.000992063492063", "WGS84", "Degrees" ;
    		string NDVI:DESCRIPTION = "Normalized Difference Vegetation Index" ;
    		string NDVI:DIMENSION_LABELS = "lat", "lon" ;
    		NDVI:scale_factor = 0.004 ;
    		NDVI:add_offset = -0.08 ;
    		NDVI:_FillValue = 255.f ;
    		string NDVI:units = "1" ;
    		string NDVI:long_name = "Normalized Difference Vegetation Index" ;
    		string NDVI:grid_mapping = "crs" ;
    		string NDVI:standard_name = "normalized_difference_vegetation_index" ;
    		NDVI:_Storage = "chunked" ;
    		NDVI:_ChunkSizes = 1, 10080 ;
    		NDVI:_Filter = "4,32,8" ;
    		NDVI:_NoFill = "true" ;

    // group attributes:
    } // group NDVI

  group: QUALITY {
    variables:
    	ubyte SM(lat, lon) ;
    		SM:SCALE = 1.f ;
    		SM:OFFSET = 0.f ;
    		string SM:UNITS = "-" ;
    		SM:NO_DATA = 2.f ;
    		string SM:MAPPING = "Geographic Lat/Lon", "0.5", "0.5", "30.000000000000000", "35.000000000000000", "0.000992063492063", "0.000992063492063", "WGS84", "Degrees" ;
    		string SM:DESCRIPTION = "LeastSignificantBit 0-2 = { 000 = clear, 010 = undefined, 011 = cloud, 100 = ice/snow, 001 = shadow };LeastSignificantBit 3 = { 0 = sea, 1 = land };LeastSignificantBit 4 = { 0 = bad SWIR, 1 = good SWIR };LeastSignificantBit 5 = { 0 = bad NIR, 1 = good NIR };LeastSignificantBit 6 = { 0 = bad RED, 1 = good RED };LeastSignificantBit 7 = { 0 = bad BLUE, 1 = good BLUE }" ;
    		string SM:DIMENSION_LABELS = "lat", "lon" ;
    		SM:scale_factor = 1. ;
    		SM:add_offset = -0. ;
    		SM:_FillValue = 2.f ;
    		string SM:units = "-" ;
    		string SM:long_name = "LeastSignificantBit 0-2 = { 000 = clear, 010 = undefined, 011 = cloud, 100 = ice/snow, 001 = shadow };LeastSignificantBit 3 = { 0 = sea, 1 = land };LeastSignificantBit 4 = { 0 = bad SWIR, 1 = good SWIR };LeastSignificantBit 5 = { 0 = bad NIR, 1 = good NIR };LeastSignificantBit 6 = { 0 = bad RED, 1 = good RED };LeastSignificantBit 7 = { 0 = bad BLUE, 1 = good BLUE }" ;
    		string SM:grid_mapping = "crs" ;
    		string SM:standard_name = "status_mask" ;
    		string SM:flag_masks = "0b, 3b, 3b, 3b, 4b, 8b, 16b, 32b, 64b, 128b" ;
    		string SM:flag_values = "0b, 1b, 2b, 3b, 4b, 8b, 16b, 32b, 64b, 128b" ;
    		string SM:flag_meanings = "Clear Shadow Undefined Cloud IceSnow Land Good_Swir Good_Nir Good_Red Good_Blue" ;
    		SM:_Storage = "chunked" ;
    		SM:_ChunkSizes = 1, 10080 ;
    		SM:_Filter = "4,32,8" ;
    		SM:_NoFill = "true" ;

    // group attributes:
    		:PERCENTAGE_CLOUD = 7.938f ;
    		:PERCENTAGE_SNOW = 0.f ;
    		:PERCENTAGE_LAND = 73.069f ;
    		:PERCENTAGE_MISSING_DATA = 0.f ;
    } // group QUALITY

  group: RADIOMETRY {

    // group attributes:

    group: BLUE {
      variables:
      	short TOC(lat, lon) ;
      		TOC:SCALE = 2000.f ;
      		TOC:OFFSET = 0.f ;
      		string TOC:UNITS = "-" ;
      		TOC:NO_DATA = -1.f ;
      		string TOC:MAPPING = "Geographic Lat/Lon", "0.5", "0.5", "30.000000000000000", "35.000000000000000", "0.000992063492063", "0.000992063492063", "WGS84", "Degrees" ;
      		string TOC:DESCRIPTION = "Top Of Canopy Reflectance BLUE - channel" ;
      		string TOC:DIMENSION_LABELS = "lat", "lon" ;
      		TOC:scale_factor = 0.0005 ;
      		TOC:add_offset = -0. ;
      		TOC:_FillValue = -1.f ;
      		string TOC:units = "1" ;
      		string TOC:long_name = "Top Of Canopy Reflectance BLUE - channel" ;
      		string TOC:grid_mapping = "crs" ;
      		string TOC:standard_name = "toc_reflectance_blue" ;
      		TOC:_Storage = "chunked" ;
      		TOC:_ChunkSizes = 1, 10080 ;
      		TOC:_Filter = "4,32,8" ;
      		TOC:_Endianness = "little" ;
      		TOC:_NoFill = "true" ;

      // group attributes:
      		string :DETECTOR = "VNIR" ;
      		string :OBSERVATION_START_DATE = "2021-07-01" ;
      		string :OBSERVATION_START_TIME = "00:00:00" ;
      		string :OBSERVATION_END_DATE = "2021-07-05" ;
      		string :OBSERVATION_END_TIME = "23:59:59" ;
      		:GAIN_FACTOR = 5.8f ;
      		:SOLAR_IRRADIANCE = 2025.57f ;
      } // group BLUE

    group: NIR {
      variables:
      	short TOC(lat, lon) ;
      		TOC:SCALE = 2000.f ;
      		TOC:OFFSET = 0.f ;
      		string TOC:UNITS = "-" ;
      		TOC:NO_DATA = -1.f ;
      		string TOC:MAPPING = "Geographic Lat/Lon", "0.5", "0.5", "30.000000000000000", "35.000000000000000", "0.000992063492063", "0.000992063492063", "WGS84", "Degrees" ;
      		string TOC:DESCRIPTION = "Top Of Canopy Reflectance NIR - channel" ;
      		string TOC:DIMENSION_LABELS = "lat", "lon" ;
      		TOC:scale_factor = 0.0005 ;
      		TOC:add_offset = -0. ;
      		TOC:_FillValue = -1.f ;
      		string TOC:units = "1" ;
      		string TOC:long_name = "Top Of Canopy Reflectance NIR - channel" ;
      		string TOC:grid_mapping = "crs" ;
      		string TOC:standard_name = "toc_reflectance_nir" ;
      		TOC:_Storage = "chunked" ;
      		TOC:_ChunkSizes = 1, 10080 ;
      		TOC:_Filter = "4,32,8" ;
      		TOC:_Endianness = "little" ;
      		TOC:_NoFill = "true" ;

      // group attributes:
      		string :DETECTOR = "VNIR" ;
      		string :OBSERVATION_START_DATE = "2021-07-01" ;
      		string :OBSERVATION_START_TIME = "00:00:00" ;
      		string :OBSERVATION_END_DATE = "2021-07-05" ;
      		string :OBSERVATION_END_TIME = "23:59:59" ;
      		:GAIN_FACTOR = 1.997f ;
      		:SOLAR_IRRADIANCE = 1036.18f ;
      } // group NIR

    group: RED {
      variables:
      	short TOC(lat, lon) ;
      		TOC:SCALE = 2000.f ;
      		TOC:OFFSET = 0.f ;
      		string TOC:UNITS = "-" ;
      		TOC:NO_DATA = -1.f ;
      		string TOC:MAPPING = "Geographic Lat/Lon", "0.5", "0.5", "30.000000000000000", "35.000000000000000", "0.000992063492063", "0.000992063492063", "WGS84", "Degrees" ;
      		string TOC:DESCRIPTION = "Top Of Canopy Reflectance RED - channel" ;
      		string TOC:DIMENSION_LABELS = "lat", "lon" ;
      		TOC:scale_factor = 0.0005 ;
      		TOC:add_offset = -0. ;
      		TOC:_FillValue = -1.f ;
      		string TOC:units = "1" ;
      		string TOC:long_name = "Top Of Canopy Reflectance RED - channel" ;
      		string TOC:grid_mapping = "crs" ;
      		string TOC:standard_name = "toc_reflectance_red" ;
      		TOC:_Storage = "chunked" ;
      		TOC:_ChunkSizes = 1, 10080 ;
      		TOC:_Filter = "4,32,8" ;
      		TOC:_Endianness = "little" ;
      		TOC:_NoFill = "true" ;

      // group attributes:
      		string :DETECTOR = "VNIR" ;
      		string :OBSERVATION_START_DATE = "2021-07-01" ;
      		string :OBSERVATION_START_TIME = "00:00:00" ;
      		string :OBSERVATION_END_DATE = "2021-07-05" ;
      		string :OBSERVATION_END_TIME = "23:59:59" ;
      		:GAIN_FACTOR = 2.298f ;
      		:SOLAR_IRRADIANCE = 1553.579f ;
      } // group RED

    group: SWIR {
      variables:
      	short TOC(lat, lon) ;
      		TOC:SCALE = 2000.f ;
      		TOC:OFFSET = 0.f ;
      		string TOC:UNITS = "-" ;
      		TOC:NO_DATA = -1.f ;
      		string TOC:MAPPING = "Geographic Lat/Lon", "0.5", "0.5", "30.000000000000000", "35.000000000000000", "0.000992063492063", "0.000992063492063", "WGS84", "Degrees" ;
      		string TOC:DESCRIPTION = "Top Of Canopy Reflectance SWIR - channel" ;
      		string TOC:DIMENSION_LABELS = "lat", "lon" ;
      		TOC:scale_factor = 0.0005 ;
      		TOC:add_offset = -0. ;
      		TOC:_FillValue = -1.f ;
      		string TOC:units = "1" ;
      		string TOC:long_name = "Top Of Canopy Reflectance SWIR - channel" ;
      		string TOC:grid_mapping = "crs" ;
      		string TOC:standard_name = "toc_reflectance_swir" ;
      		TOC:_Storage = "chunked" ;
      		TOC:_ChunkSizes = 1, 10080 ;
      		TOC:_Filter = "4,32,8" ;
      		TOC:_Endianness = "little" ;
      		TOC:_NoFill = "true" ;

      // group attributes:
      		string :DETECTOR = "SWIR" ;
      		string :OBSERVATION_START_DATE = "2021-07-01" ;
      		string :OBSERVATION_START_TIME = "00:00:00" ;
      		string :OBSERVATION_END_DATE = "2021-07-05" ;
      		string :OBSERVATION_END_TIME = "23:59:59" ;
      		:GAIN_FACTOR = 0.777f ;
      		:SOLAR_IRRADIANCE = 250.734f ;
      } // group SWIR
    } // group RADIOMETRY

  group: TIME {
    variables:
    	ushort TIME(lat, lon) ;
    		TIME:SCALE = 1.f ;
    		TIME:OFFSET = 0.f ;
    		string TIME:UNITS = "Minutes" ;
    		TIME:NO_DATA = 0.f ;
    		string TIME:MAPPING = "Geographic Lat/Lon", "0.5", "0.5", "30.000000000000000", "35.000000000000000", "0.000992063492063", "0.000992063492063", "WGS84", "Degrees" ;
    		string TIME:DESCRIPTION = "Time grid" ;
    		string TIME:DIMENSION_LABELS = "lat", "lon" ;
    		TIME:scale_factor = 1. ;
    		TIME:add_offset = -0. ;
    		TIME:_FillValue = 0.f ;
    		string TIME:units = "Minutes since 2021-07-01T00:00:00Z" ;
    		string TIME:long_name = "Time grid" ;
    		string TIME:grid_mapping = "crs" ;
    		string TIME:calendar = "standard" ;
    		TIME:_Storage = "chunked" ;
    		TIME:_ChunkSizes = 1, 10080 ;
    		TIME:_Filter = "4,32,8" ;
    		TIME:_Endianness = "little" ;
    		TIME:_NoFill = "true" ;

    // group attributes:
    		string :OBSERVATION_START_DATE = "2021-07-01" ;
    		string :OBSERVATION_START_TIME = "00:00:00" ;
    		string :OBSERVATION_END_DATE = "2021-07-05" ;
    		string :OBSERVATION_END_TIME = "23:59:59" ;
    } // group TIME
  } // group LEVEL3
}

@jswhit
Copy link
Collaborator

jswhit commented Aug 23, 2022

Looks like that dataset is using the optional szip compression filter. The binary wheels (installed via pip) have support for szip, but apparently the conda packages do not. I suggest creating an issue at https://github.com/conda-forge/netcdf4-feedstock.

@ocefpaf
Copy link
Collaborator

ocefpaf commented Aug 23, 2022

We cannot support szip BTW due to its more restrictive licenses on re-distribution.

@jswhit
Copy link
Collaborator

jswhit commented Aug 23, 2022

The binary wheels use libaec, which I understand is a drop-in replacement for szip with better licensing. It apparently already exists in conda.

@ocefpaf
Copy link
Collaborator

ocefpaf commented Aug 23, 2022

Indeed libaec would do. I'll look into it.

@bertcoerver
Copy link
Author

Looks like that dataset is using the optional szip compression filter. The binary wheels (installed via pip) have support for szip, but apparently the conda packages do not. I suggest creating an issue at https://github.com/conda-forge/netcdf4-feedstock.

@ocefpaf should I still open an issue at netcdf4-feedstock?

@ocefpaf
Copy link
Collaborator

ocefpaf commented Aug 24, 2022

@ocefpaf should I still open an issue at netcdf4-feedstock?

Please do! Be sure to mention libaec so folks don't close it when they see the issue. Thanks!

@jswhit
Copy link
Collaborator

jswhit commented Aug 24, 2022

@ocefpaf have you considered what it would take to add the new netcdf filter plugins in the conda packages?

@bertcoerver bertcoerver mentioned this issue Aug 25, 2022
Closed
1 task
@ocefpaf
Copy link
Collaborator

ocefpaf commented Aug 25, 2022

@ocefpaf have you considered what it would take to add the new netcdf filter plugins in the conda packages?

Not yet but it should possible. I'm returning from a "week off" so there is a lot on my plate at the moment but I'll get to it ASAP.

@jswhit jswhit closed this as completed Aug 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jswhit @ocefpaf @bertcoerver