Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

resampling fails with ValueError if not padding data #497

Closed
gerritholl opened this issue Feb 6, 2023 · 2 comments
Closed

resampling fails with ValueError if not padding data #497

gerritholl opened this issue Feb 6, 2023 · 2 comments
Labels
bug

Comments

@gerritholl
Copy link
Collaborator

When we're using satpy to load geostationary data with pad_data=False and then pass it to pyresample for resampling, pyresample is no longer able to handle it correctly. In v1.25.1 it worked. In v1.26.0 it fails with a ValueError.

Code Sample, a minimal, complete, and verifiable piece of code

import hdf5plugin
import pathlib
import os
from satpy import Scene
from glob import glob
from satpy.utils import debug_on; debug_on()
bd = pathlib.Path("/media/nas/x21308/MTG_test_data/2022_05_MTG_Testdata/RC0040/")
sc = Scene(filenames=[os.fspath(f) for f in bd.glob("*BODY*003[789].nc")],reader="fci_l1c_nc")
sc.load(["vis_04"], pad_data=False)
ls = sc.resample("nqeuro1km")
ls.save_datasets()

Problem description

This fails with a ValueError. It worked prior to fd27e14 (found with git bisect).

Expected Output

With pyresample 1.15.1 
/data/gholl/checkouts/satpy/satpy/_config.py:129: DeprecationWarning: SelectableGroups dict interface is deprecated. Use select.
  for entry_point in cached_entry_points().get(name, []):
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/fci_l1c_nc.yaml',)
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.yaml_reader] Assigning to fci_l1c_nc: ['/media/nas/x21308/MTG_test_data/2022_05_MTG_Testdata/RC0040/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20170920064421_GTT_DEV_20170920063900_20170920063921_N_JLS_T_0040_0039.nc', '/media/nas/x21
308/MTG_test_data/2022_05_MTG_Testdata/RC0040/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20170920064407_GTT_DEV_20170920063835_20170920063907_N_JLS_T_0040_0037.nc', '/media/nas/x21308/MTG_test_data/2022_05_MTG_Testdata/RC0040/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDH
SI-FD--CHK-BODY---NC4E_C_EUMT_20170920064411_GTT_DEV_20170920063849_20170920063911_N_JLS_T_0040_0038.nc']
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.fci_l1c_nc] Reading: /media/nas/x21308/MTG_test_data/2022_05_MTG_Testdata/RC0040/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20170920064421_GTT_DEV_20170920063900_20170920063921_N_JLS_T_0040_0039.nc
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.fci_l1c_nc] Start: 2017-09-20 06:39:00
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.fci_l1c_nc] End: 2017-09-20 06:39:21
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.fci_l1c_nc] Reading: /media/nas/x21308/MTG_test_data/2022_05_MTG_Testdata/RC0040/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20170920064407_GTT_DEV_20170920063835_20170920063907_N_JLS_T_0040_0037.nc
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.fci_l1c_nc] Start: 2017-09-20 06:38:35
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.fci_l1c_nc] End: 2017-09-20 06:39:07
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.fci_l1c_nc] Reading: /media/nas/x21308/MTG_test_data/2022_05_MTG_Testdata/RC0040/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20170920064411_GTT_DEV_20170920063849_20170920063911_N_JLS_T_0040_0038.nc
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.fci_l1c_nc] Start: 2017-09-20 06:38:49
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.fci_l1c_nc] End: 2017-09-20 06:39:11
[DEBUG: 2023-02-06 18:19:25 : satpy.composites.config_loader] Looking for composites config file fci.yaml
[DEBUG: 2023-02-06 18:19:25 : satpy.composites.config_loader] Looking for composites config file visir.yaml
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.fci_l1c_nc] Reading vis_04 from /media/nas/x21308/MTG_test_data/2022_05_MTG_Testdata/RC0040/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20170920064407_GTT_DEV_20170920063835_20170920063907_N_JLS_T_0040_0037.nc
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.fci_l1c_nc] Reading vis_04 from /media/nas/x21308/MTG_test_data/2022_05_MTG_Testdata/RC0040/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20170920064411_GTT_DEV_20170920063849_20170920063911_N_JLS_T_0040_0038.nc
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.fci_l1c_nc] Reading vis_04 from /media/nas/x21308/MTG_test_data/2022_05_MTG_Testdata/RC0040/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20170920064421_GTT_DEV_20170920063900_20170920063921_N_JLS_T_0040_0039.nc
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.fci_l1c_nc] Channel vis_04 resolution: 11136
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.fci_l1c_nc] Row/Cols: 278 / 11136
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.fci_l1c_nc] Calculated area extent: (-5567999.998550739, 4731999.998768358, 5567999.998550762, 4453999.998840717)
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.fci_l1c_nc] Channel vis_04 resolution: 11136
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.fci_l1c_nc] Row/Cols: 279 / 11136
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.fci_l1c_nc] Calculated area extent: (-5567999.998550739, 5010999.99869574, 5567999.998550762, 4731999.998768358)
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.fci_l1c_nc] Channel vis_04 resolution: 11136
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.fci_l1c_nc] Row/Cols: 278 / 11136
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.fci_l1c_nc] Calculated area extent: (-5567999.998550739, 5288999.99862338, 5567999.998550762, 5010999.99869574)
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.yaml_reader] Requested orientation for Dataset vis_04 is 'native' (default). No flipping is applied.
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.fci_l1c_nc] Reading vis_04_pixel_quality from /media/nas/x21308/MTG_test_data/2022_05_MTG_Testdata/RC0040/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20170920064407_GTT_DEV_20170920063835_20170920063907_N_JLS_T_0040_0037.nc
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.fci_l1c_nc] Reading vis_04_pixel_quality from /media/nas/x21308/MTG_test_data/2022_05_MTG_Testdata/RC0040/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20170920064411_GTT_DEV_20170920063849_20170920063911_N_JLS_T_0040_0038.nc
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.fci_l1c_nc] Reading vis_04_pixel_quality from /media/nas/x21308/MTG_test_data/2022_05_MTG_Testdata/RC0040/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20170920064421_GTT_DEV_20170920063900_20170920063921_N_JLS_T_0040_0039.nc
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
[DEBUG: 2023-02-06 18:19:25 : satpy.readers.yaml_reader] Requested orientation for Dataset None is 'native' (default). No flipping is applied.
[DEBUG: 2023-02-06 18:19:25 : satpy.scene] Resampling DataID(name='vis_04', wavelength=WavelengthRange(min=0.384, central=0.444, max=0.504, unit='µm'), resolution=1000, calibration=<1>, modifiers=())
/data/gholl/checkouts/pyresample/pyresample/spherical.py:34: RuntimeWarning: invalid value encountered in remainder
  return (val + mod) % (2 * mod) - mod
/data/gholl/checkouts/pyresample/pyresample/spherical.py:370: RuntimeWarning: invalid value encountered in cos
  self.cvertices = np.array([np.cos(self.lat) * np.cos(self.lon),
/data/gholl/checkouts/pyresample/pyresample/spherical.py:371: RuntimeWarning: invalid value encountered in cos
  np.cos(self.lat) * np.sin(self.lon),
/data/gholl/checkouts/pyresample/pyresample/spherical.py:372: RuntimeWarning: invalid value encountered in sin
  np.sin(self.lat)]).T * radius
/data/gholl/checkouts/pyresample/pyresample/spherical.py:58: RuntimeWarning: invalid value encountered in scalar subtract
  ad = np.sin(lat1 - lat2) * np.cos((lon1 - lon2) / 2.0)
/data/gholl/checkouts/pyresample/pyresample/spherical.py:59: RuntimeWarning: invalid value encountered in sin
  be = np.sin(lat1 + lat2) * np.sin((lon1 - lon2) / 2.0)
/data/gholl/checkouts/pyresample/pyresample/spherical.py:62: RuntimeWarning: invalid value encountered in cos
  g = np.cos(lat1)
/data/gholl/checkouts/pyresample/pyresample/spherical.py:63: RuntimeWarning: invalid value encountered in cos
  h = np.cos(lat2)
/data/gholl/checkouts/pyresample/pyresample/spherical.py:91: RuntimeWarning: invalid value encountered in scalar subtract
  return 2 * np.arcsin((np.sin((point.lat - self.lat) / 2.0) ** 2.0 +
/data/gholl/checkouts/pyresample/pyresample/spherical.py:92: RuntimeWarning: invalid value encountered in cos
  np.cos(point.lat) * np.cos(self.lat) *
/data/gholl/checkouts/pyresample/pyresample/spherical.py:58: RuntimeWarning: invalid value encountered in sin
  ad = np.sin(lat1 - lat2) * np.cos((lon1 - lon2) / 2.0)
/data/gholl/checkouts/pyresample/pyresample/spherical.py:91: RuntimeWarning: invalid value encountered in sin
  return 2 * np.arcsin((np.sin((point.lat - self.lat) / 2.0) ** 2.0 +
[INFO: 2023-02-06 18:19:31 : satpy.resample] Using default KDTree resampler
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/numpy/lib/function_base.py:1448: RuntimeWarning: invalid value encountered in subtract
  a = op(a[slice1], a[slice2])
[DEBUG: 2023-02-06 18:19:31 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2023-02-06 18:19:31 : satpy.resample] Resampling None
[DEBUG: 2023-02-06 18:19:31 : satpy.scene] Resampling DataID(name='vis_04_pixel_quality', resolution=1000, modifiers=())
[DEBUG: 2023-02-06 18:19:31 : satpy.resample] Computing kd-tree parameters
[DEBUG: 2023-02-06 18:19:31 : satpy.resample] Resampling pixel_quality
[DEBUG: 2023-02-06 18:19:31 : satpy.writers] Reading ['/data/gholl/checkouts/satpy/satpy/etc/writers/geotiff.yaml']
[DEBUG: 2023-02-06 18:19:31 : rasterio.session] Could not import boto3, continuing with reduced functionality.
/data/gholl/checkouts/satpy/satpy/_config.py:129: DeprecationWarning: SelectableGroups dict interface is deprecated. Use select.
  for entry_point in cached_entry_points().get(name, []):
[DEBUG: 2023-02-06 18:19:31 : satpy.writers] Adding enhancement configuration from file: /data/gholl/checkouts/satpy/satpy/etc/enhancements/generic.yaml
[DEBUG: 2023-02-06 18:19:31 : satpy.writers] Adding enhancement configuration from file: /home/gholl/checkouts/pytroll-dwd-config/enhancements/generic.yaml
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
[DEBUG: 2023-02-06 18:19:31 : satpy.writers] Data for DataID(name='vis_04', wavelength=WavelengthRange(min=0.384, central=0.444, max=0.504, unit='µm'), resolution=1000, calibration=<1>, modifiers=()) will be enhanced with options:
        [{'name': 'linear_stretch', 'method': <function stretch at 0x7f41cad78180>, 'kwargs': {'stretch': 'crude', 'min_stretch': 0.0, 'max_stretch': 100.0}}, {'name': 'gamma', 'method': <function gamma at 0x7f41cad78220>, 'kwargs': {'gamma': 1.5}}]
[DEBUG: 2023-02-06 18:19:31 : trollimage.xrimage] Applying stretch crude with parameters {'min_stretch': 0.0, 'max_stretch': 100.0}
[DEBUG: 2023-02-06 18:19:31 : trollimage.xrimage] Applying gamma 1.5
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
[DEBUG: 2023-02-06 18:19:31 : rasterio.env] Entering env context: <rasterio.env.Env object at 0x7f41cbe115d0>
[DEBUG: 2023-02-06 18:19:31 : rasterio.env] Starting outermost env
[DEBUG: 2023-02-06 18:19:31 : rasterio.env] No GDAL environment exists
[DEBUG: 2023-02-06 18:19:31 : rasterio.env] New GDAL environment <rasterio._env.GDALEnv object at 0x7f41cad58550> created
[DEBUG: 2023-02-06 18:19:31 : rasterio._filepath] Installing FilePath filesystem handler plugin...
[DEBUG: 2023-02-06 18:19:31 : rasterio._env] GDAL_DATA found in environment.
[DEBUG: 2023-02-06 18:19:31 : rasterio._env] PROJ_DATA found in environment.
[DEBUG: 2023-02-06 18:19:31 : rasterio._env] Started GDALEnv: self=<rasterio._env.GDALEnv object at 0x7f41cad58550>.
[DEBUG: 2023-02-06 18:19:31 : rasterio.env] Entered env context: <rasterio.env.Env object at 0x7f41cbe115d0>
[DEBUG: 2023-02-06 18:19:31 : rasterio._io] Path: _UnparsedPath(path='vis_04_20170920_063835.tif'), mode: w, driver: GTiff
[DEBUG: 2023-02-06 18:19:31 : rasterio._base] Nodata success: 0, Nodata value: 0.000000
[DEBUG: 2023-02-06 18:19:31 : rasterio._base] Nodata success: 0, Nodata value: 0.000000
[DEBUG: 2023-02-06 18:19:31 : rasterio.env] Exiting env context: <rasterio.env.Env object at 0x7f41cbe115d0>
[DEBUG: 2023-02-06 18:19:31 : rasterio.env] Cleared existing <rasterio._env.GDALEnv object at 0x7f41cad58550> options
[DEBUG: 2023-02-06 18:19:31 : rasterio._env] Stopped GDALEnv <rasterio._env.GDALEnv object at 0x7f41cad58550>.
[DEBUG: 2023-02-06 18:19:31 : rasterio.env] Exiting outermost env
[DEBUG: 2023-02-06 18:19:31 : rasterio.env] Exited env context: <rasterio.env.Env object at 0x7f41cbe115d0>
[INFO: 2023-02-06 18:19:31 : satpy.writers] Computing and writing results...
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/dask/core.py:119: RuntimeWarning: invalid value encountered in cos
  return func(*(_execute_task(a, cache) for a in args))
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/dask/core.py:119: RuntimeWarning: invalid value encountered in sin
  return func(*(_execute_task(a, cache) for a in args))
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/dask/core.py:119: RuntimeWarning: invalid value encountered in cos
  return func(*(_execute_task(a, cache) for a in args))
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/dask/core.py:119: RuntimeWarning: invalid value encountered in sin
  return func(*(_execute_task(a, cache) for a in args))
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/dask/array/chunk.py:281: RuntimeWarning: invalid value encountered in cast
  return x.astype(astype_dtype, **kwargs)

And the image:

vis

Actual Result, Traceback if applicable

With v1.26.0 
/data/gholl/checkouts/satpy/satpy/_config.py:129: DeprecationWarning: SelectableGroups dict interface is deprecated. Use select.
  for entry_point in cached_entry_points().get(name, []):
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/fci_l1c_nc.yaml',)
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.yaml_reader] Assigning to fci_l1c_nc: ['/media/nas/x21308/MTG_test_data/2022_05_MTG_Testdata/RC0040/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20170920064407_GTT_DEV_20170920063835_20170920063907_N_JLS_T_0040_0037.nc', '/media/nas/x21
308/MTG_test_data/2022_05_MTG_Testdata/RC0040/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20170920064421_GTT_DEV_20170920063900_20170920063921_N_JLS_T_0040_0039.nc', '/media/nas/x21308/MTG_test_data/2022_05_MTG_Testdata/RC0040/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDH
SI-FD--CHK-BODY---NC4E_C_EUMT_20170920064411_GTT_DEV_20170920063849_20170920063911_N_JLS_T_0040_0038.nc']
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.fci_l1c_nc] Reading: /media/nas/x21308/MTG_test_data/2022_05_MTG_Testdata/RC0040/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20170920064407_GTT_DEV_20170920063835_20170920063907_N_JLS_T_0040_0037.nc
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.fci_l1c_nc] Start: 2017-09-20 06:38:35
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.fci_l1c_nc] End: 2017-09-20 06:39:07
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.fci_l1c_nc] Reading: /media/nas/x21308/MTG_test_data/2022_05_MTG_Testdata/RC0040/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20170920064421_GTT_DEV_20170920063900_20170920063921_N_JLS_T_0040_0039.nc
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.fci_l1c_nc] Start: 2017-09-20 06:39:00
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.fci_l1c_nc] End: 2017-09-20 06:39:21
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.fci_l1c_nc] Reading: /media/nas/x21308/MTG_test_data/2022_05_MTG_Testdata/RC0040/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20170920064411_GTT_DEV_20170920063849_20170920063911_N_JLS_T_0040_0038.nc
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.fci_l1c_nc] Start: 2017-09-20 06:38:49
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.fci_l1c_nc] End: 2017-09-20 06:39:11
[DEBUG: 2023-02-06 18:21:32 : satpy.composites.config_loader] Looking for composites config file fci.yaml
[DEBUG: 2023-02-06 18:21:32 : satpy.composites.config_loader] Looking for composites config file visir.yaml
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.fci_l1c_nc] Reading vis_04 from /media/nas/x21308/MTG_test_data/2022_05_MTG_Testdata/RC0040/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20170920064407_GTT_DEV_20170920063835_20170920063907_N_JLS_T_0040_0037.nc
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.fci_l1c_nc] Reading vis_04 from /media/nas/x21308/MTG_test_data/2022_05_MTG_Testdata/RC0040/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20170920064411_GTT_DEV_20170920063849_20170920063911_N_JLS_T_0040_0038.nc
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.fci_l1c_nc] Reading vis_04 from /media/nas/x21308/MTG_test_data/2022_05_MTG_Testdata/RC0040/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20170920064421_GTT_DEV_20170920063900_20170920063921_N_JLS_T_0040_0039.nc
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.fci_l1c_nc] Channel vis_04 resolution: 11136
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.fci_l1c_nc] Row/Cols: 278 / 11136
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.fci_l1c_nc] Calculated area extent: (-5567999.998550739, 4731999.998768358, 5567999.998550762, 4453999.998840717)
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.fci_l1c_nc] Channel vis_04 resolution: 11136
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.fci_l1c_nc] Row/Cols: 279 / 11136
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.fci_l1c_nc] Calculated area extent: (-5567999.998550739, 5010999.99869574, 5567999.998550762, 4731999.998768358)
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.fci_l1c_nc] Channel vis_04 resolution: 11136
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.fci_l1c_nc] Row/Cols: 278 / 11136
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.fci_l1c_nc] Calculated area extent: (-5567999.998550739, 5288999.99862338, 5567999.998550762, 5010999.99869574)
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.yaml_reader] Requested orientation for Dataset vis_04 is 'native' (default). No flipping is applied.
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.fci_l1c_nc] Reading vis_04_pixel_quality from /media/nas/x21308/MTG_test_data/2022_05_MTG_Testdata/RC0040/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20170920064407_GTT_DEV_20170920063835_20170920063907_N_JLS_T_0040_0037.nc
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.fci_l1c_nc] Reading vis_04_pixel_quality from /media/nas/x21308/MTG_test_data/2022_05_MTG_Testdata/RC0040/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20170920064411_GTT_DEV_20170920063849_20170920063911_N_JLS_T_0040_0038.nc
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.fci_l1c_nc] Reading vis_04_pixel_quality from /media/nas/x21308/MTG_test_data/2022_05_MTG_Testdata/RC0040/W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20170920064421_GTT_DEV_20170920063900_20170920063921_N_JLS_T_0040_0039.nc
/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/xarray/core/pycompat.py:22: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  duck_array_version = LooseVersion(duck_array_module.__version__)
[DEBUG: 2023-02-06 18:21:32 : satpy.readers.yaml_reader] Requested orientation for Dataset None is 'native' (default). No flipping is applied.
[DEBUG: 2023-02-06 18:21:33 : satpy.scene] Resampling DataID(name='vis_04', wavelength=WavelengthRange(min=0.384, central=0.444, max=0.504, unit='µm'), resolution=1000, calibration=<1>, modifiers=())
/data/gholl/checkouts/pyresample/pyresample/spherical.py:34: RuntimeWarning: invalid value encountered in remainder
  return (val + mod) % (2 * mod) - mod
/data/gholl/checkouts/pyresample/pyresample/spherical.py:547: RuntimeWarning: invalid value encountered in cos
  self.cvertices = np.array([np.cos(self.lat) * np.cos(self.lon),
/data/gholl/checkouts/pyresample/pyresample/spherical.py:548: RuntimeWarning: invalid value encountered in cos
  np.cos(self.lat) * np.sin(self.lon),
/data/gholl/checkouts/pyresample/pyresample/spherical.py:549: RuntimeWarning: invalid value encountered in sin
  np.sin(self.lat)]).T * radius
Traceback (most recent call last):
  File "/data/gholl/checkouts/satpy/satpy/scene.py", line 901, in _reduce_data
    (slice_x, slice_y), source_area = reductions[key]
                                      ~~~~~~~~~~^^^^^
KeyError/data/gholl/mambaforge/envs/py311/lib/python3.11/site-packages/pyproj/crs/crs.py:1296: UserWarning: You will likely lose important projection information when converting to a PROJ string from another format. See: https://proj.org/faq.html#what-is-the-best-format-for-describing-coordinate-reference-systems
  proj = self._crs.to_proj4(version=version)
: Area ID: mtg_fci_fdss_1km
Description: MTG FCI Full Disk Scanning Service area definition with 1 km resolution
Projection: {'ellps': 'WGS84', 'h': '35786400', 'lon_0': '0', 'no_defs': 'None', 'proj': 'geos', 'type': 'crs', 'units': 'm', 'x_0': '0', 'y_0': '0'}
Number of columns: 11136
Number of rows: 835
Area extent: (-5567999.9986, 5288999.9986, 5567999.9986, 4453999.9988)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/gholl/checkouts/protocode/fci-resample-problem.py", line 10, in <module>
    ls = sc.resample("nqeuro1km")
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/gholl/checkouts/satpy/satpy/scene.py", line 954, in resample
    self._resampled_scene(new_scn, destination, resampler=resampler,
  File "/data/gholl/checkouts/satpy/satpy/scene.py", line 865, in _resampled_scene
    dataset, source_area = self._reduce_data(dataset, source_area, destination_area,
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/gholl/checkouts/satpy/satpy/scene.py", line 908, in _reduce_data
    slice_x, slice_y = source_area.get_area_slices(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/gholl/checkouts/pyresample/pyresample/geometry.py", line 2616, in get_area_slices
    intersection = data_boundary.contour_poly.intersection(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/gholl/checkouts/pyresample/pyresample/spherical.py", line 713, in intersection
    return self._bool_oper(other, -1)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/gholl/checkouts/pyresample/pyresample/spherical.py", line 649, in _bool_oper
    arcs1 = [edge for edge in self.aedges()]
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/gholl/checkouts/pyresample/pyresample/spherical.py", line 649, in <listcomp>
    arcs1 = [edge for edge in self.aedges()]
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/gholl/checkouts/pyresample/pyresample/spherical.py", line 571, in aedges
    yield Arc(SCoordinate(lon_start, lat_start),
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/gholl/checkouts/pyresample/pyresample/spherical.py", line 131, in __init__
    lon, lat = _check_lon_lat(lon, lat)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/gholl/checkouts/pyresample/pyresample/spherical.py", line 119, in _check_lon_lat
    _check_lat_validity(lat)
  File "/data/gholl/checkouts/pyresample/pyresample/spherical.py", line 109, in _check_lat_validity
    raise ValueError("Latitude values can not contain inf values.")
ValueError: Latitude values can not contain inf values.

Versions of Python, package at hand and relevant dependencies

As indicated, this works with pyresample 1.25.1 but fails with 1.26.0. According to git bisect, the problem was introduced in commit
fd27e14.
@gerritholl gerritholl added the bug label Feb 6, 2023
@djhoese
Copy link
Member

djhoese commented Feb 6, 2023

Duplicate of #492

@djhoese djhoese marked this as a duplicate of #492 Feb 6, 2023
@djhoese djhoese closed this as completed Feb 6, 2023
@gerritholl
Copy link
Collaborator Author

Oops, sorry. I thought it was a different problem, but you're probably right.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@gerritholl @djhoese