You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Code Sample, a minimal, complete, and verifiable piece of code
Calling SwathDefinition.update_hash() after having sliced the objects raises an error.
importnumpyasnpfrompyresampleimportSwathDefinitiondtype="float32"# "float64" # "int32"lons=np.ones((2,100), dtype=dtype)
lats=np.ones((2,100), dtype=dtype)
# Create dummy SwathDef swath_def=SwathDefinition(lons=lons, lats=lats)
print(swath_def.lats.data.contiguous) # Trueswath_def.update_hash() # Value Error # Slice SwathDefswath_def_subset=swath_def[:, slice(0,5)]
print(swath_def_subset.lats.data.contiguous) # False# Try to update the hashswath_def_subset.update_hash() # Value Error
Problem description
This error does not allow parallelized computations (using dask delayed) over blocks of SwathDefinition objects.
The following approach fails because update_hash() is called by dask:
import dask
@dask.delayed
def delayed_fun(swath_def):
# whatever
pass
return None
list_slices = [slice(0,3), slice(2,5)]
results = []
for slc in list_slices:
swath_def_block = swath_def[:, slc]
result = delayed_fun(swath_def_block)
results.append(result)
out = dask.compute(*results) # FAILS
results[0].compute() # FAILS
Expected Output
SwathDefinition.update_hash() should not raise an error when the lats and lons arrays are not contiguous (i.e. after having sliced the objects).
The problem does not occur with AreaDefinition objects.
Traceback
Traceback (most recent call last):
File ~/anaconda3/envs/satpy39/lib/python3.9/site-packages/pyresample/geometry.py:668 in get_array_hashable
return arr.name.encode('utf-8') # dask array
AttributeError: 'numpy.ndarray' object has no attribute 'name'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
Input In [145] in <cell line: 13>
swath_def_subset.update_hash() # Value Error
File ~/anaconda3/envs/satpy39/lib/python3.9/site-packages/pyresample/geometry.py:122 in update_hash
existing_hash.update(get_array_hashable(self.lons))
File ~/anaconda3/envs/satpy39/lib/python3.9/site-packages/pyresample/geometry.py:670 in get_array_hashable
return np.asarray(arr).view(np.uint8) # np array
ValueError: To change to a dtype of a different size, the array must be C-contiguous
The text was updated successfully, but these errors were encountered:
So the main problem seems to be the conversion to uint8. This makes sense since the sliced array would not have a nice even striding when converted from individual 4-byte elements (32-bit floats) to groups of 4 uint8 elements. @mraspaud do you remember why we needed to convert to np.uint8? Does the numpy buffer protocol implementation not allow for buffer access to floats?
Code Sample, a minimal, complete, and verifiable piece of code
Calling
SwathDefinition.update_hash()
after having sliced the objects raises an error.Problem description
This error does not allow parallelized computations (using dask delayed) over blocks of
SwathDefinition
objects.The following approach fails because
update_hash()
is called by dask:Expected Output
SwathDefinition.update_hash()
should not raise an error when thelats
andlons
arrays are not contiguous (i.e. after having sliced the objects).The problem does not occur with AreaDefinition objects.
Traceback
The text was updated successfully, but these errors were encountered: