Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

combine_by_coords should allow for missing panels in hypercube #3648

Closed
bolliger32 opened this issue Dec 19, 2019 · 0 comments · Fixed by #3649
Closed

combine_by_coords should allow for missing panels in hypercube #3648

bolliger32 opened this issue Dec 19, 2019 · 0 comments · Fixed by #3649

Comments

@bolliger32
Copy link
Contributor

MCVE Code Sample

import numpy as np
import xarray as xr
x1 = xr.Dataset(
     {
         "temperature": (("y", "x"), 20 * np.random.rand(6).reshape(2, 3))
     },
     coords={"y": [0, 1], "x": [10, 20, 30]},
)
x2 = xr.Dataset(
     {
         "temperature": (("y", "x"), 20 * np.random.rand(6).reshape(2, 3))
     },
     coords={"y": [2, 3], "x": [10, 20, 30]},
)
x3 = xr.Dataset(
     {
         "temperature": (("y", "x"), 20 * np.random.rand(6).reshape(2, 3))
     },
     coords={"y": [2, 3], "x": [40, 50, 60]},
)
xr.combine_by_coords([x1,x2,x3])

Expected Output

<xarray.Dataset>
Dimensions:      (x: 6, y: 4)
Coordinates:
  * x            (x) int64 10 20 30 40 50 60
  * y            (y) int64 0 1 2 3
Data variables:
    temperature  (y, x) float64 14.11 19.19 10.77 nan ... 4.86 10.57 4.38 15.09

Problem Description

Currently, it throws the following error:

ValueError: The supplied objects do not form a hypercube because sub-lists do not have consistent lengths along dimension0

This is b/c combine_by_coords calls xr.core.combine._check_shape_tile_ids, which mandates that the passed datasets form a complete hypercube. This check functiono also serves the purpose of mandating that the dimension depths are the same. Could we pull that part out as a separate function and, for combine_by_coords, only call this first part but NOT mandate that the hypercube is complete? The expected behavior, in my mind, should be to simply replace the missing tiles of the hypercube with fill_value. I'll file a PR to this effect and welcome comments.

Output of xr.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.7.3 | packaged by conda-forge | (default, Dec 6 2019, 08:54:18) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 4.14.150+ machine: x86_64 processor: x86_64 byteorder: little LC_ALL: en_US.UTF-8 LANG: en_US.UTF-8 LOCALE: en_US.UTF-8 libhdf5: 1.10.5 libnetcdf: 4.7.1

xarray: 0.14.1
pandas: 0.25.3
numpy: 1.17.3
scipy: 1.3.2
netCDF4: 1.5.3
pydap: None
h5netcdf: 0.7.4
h5py: 2.10.0
Nio: None
zarr: 2.3.2
cftime: 1.0.4.2
nc_time_axis: 1.2.0
PseudoNetCDF: None
rasterio: 1.1.1
cfgrib: None
iris: 2.2.0
bottleneck: None
dask: 2.8.1
distributed: 2.8.1
matplotlib: 3.1.2
cartopy: 0.17.0
seaborn: 0.9.0
numbagg: None
setuptools: 42.0.2.post20191201
pip: 19.3.1
conda: 4.8.0
pytest: 5.3.1
IPython: 7.10.1
sphinx: 2.2.2

TomNicholas pushed a commit that referenced this issue Dec 24, 2019
* allow incomplete hypercubes in combine_by_coords

* If fill_value=None then still requires complete hypercube 

* Closes #3648
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant