-
-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
⬆️ Bump cryptography from 41.0.3 to 41.0.4 #123
Conversation
Bumps [cryptography](https://github.com/pyca/cryptography) from 41.0.3 to 41.0.4. - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](pyca/cryptography@41.0.3...41.0.4) --- updated-dependencies: - dependency-name: cryptography dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>
Make non-required dependencies optional again! Workaround by running `poetry lock --no-update` until dependabot/dependabot-core#3104 is resolved.
The latest stackstac 0.5.0 version installed via readthedocs does sets the dtype on read-in using rasterio, instead of after reading with `np.astype`, see gjoseph92/stackstac#208. Ideally, this should be fixed upstream in rasterio and GDAL by supporting reading of GeoTIFFs into float16 dtype. Alternative is to cast to float32 instead, though it would be a waste of memory if doing 16-bit precision training.
post_install: | ||
# Install stackstac=0.4.4 instead of 0.5.0 to prevent | ||
# TypeError: Unsupported data type float16 | ||
# because stackstac casts to float16 at read-in instead of post-read | ||
# see https://github.com/gjoseph92/stackstac/pull/208 | ||
# Need to wait for rasterio/GDAL to support float16 | ||
# see https://gdal.org/api/raster_c_api.html#_CPPv412GDALDataType | ||
- "pip install stackstac==0.4.4" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Was getting this error with stackstac==0.5.0
at https://readthedocs.org/projects/zen3geo/builds/21996237/:
/home/docs/checkouts/readthedocs.org/user_builds/zen3geo/checkouts/123/docs/stacking.md: WARNING: Executing notebook failed: CellExecutionError
Traceback (most recent call last):
File "/home/docs/checkouts/readthedocs.org/user_builds/zen3geo/envs/123/lib/python3.11/site-packages/jupyter_cache/executors/utils.py", line 58, in single_nb_execution
executenb(
File "/home/docs/checkouts/readthedocs.org/user_builds/zen3geo/envs/123/lib/python3.11/site-packages/nbclient/client.py", line 1305, in execute
return NotebookClient(nb=nb, resources=resources, km=km, **kwargs).execute()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/docs/checkouts/readthedocs.org/user_builds/zen3geo/envs/123/lib/python3.11/site-packages/jupyter_core/utils/__init__.py", line 166, in wrapped
return loop.run_until_complete(inner)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/docs/.asdf/installs/python/3.11.4/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/home/docs/checkouts/readthedocs.org/user_builds/zen3geo/envs/123/lib/python3.11/site-packages/nbclient/client.py", line 705, in async_execute
await self.async_execute_cell(
File "/home/docs/checkouts/readthedocs.org/user_builds/zen3geo/envs/123/lib/python3.11/site-packages/nbclient/client.py", line 1058, in async_execute_cell
await self._check_raise_for_error(cell, cell_index, exec_reply)
File "/home/docs/checkouts/readthedocs.org/user_builds/zen3geo/envs/123/lib/python3.11/site-packages/nbclient/client.py", line 914, in _check_raise_for_error
raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
------------------
dataslice = datacube.sel(time="2022-02-23T11:41:54.329096000").compute()
fig, ax = plt.subplots(nrows=2, ncols=2, figsize=(11, 12), sharex=True, sharey=True)
dataslice.vh.plot.imshow(ax=ax[0][0], cmap="bone", robust=True)
ax[0][0].set_title("Sentinel-1 RTC 20220223 VH")
dataslice.vv.plot.imshow(ax=ax[0][1], cmap="bone", robust=True)
ax[0][1].set_title("Sentinel-1 RTC 20220223 VV")
dataslice.dem.plot.imshow(ax=ax[1][0], cmap="gist_earth")
ax[1][0].set_title("Copernicus DEM")
dataslice.mask.plot.imshow(ax=ax[1][1], cmap="binary_r")
ax[1][1].set_title("Landslide mask")
plt.show()
------------------
�[0;31m---------------------------------------------------------------------------�[0m
�[0;31mKeyError�[0m Traceback (most recent call last)
File �[0;32m~/checkouts/readthedocs.org/user_builds/zen3geo/envs/123/lib/python3.11/site-packages/rasterio/dtypes.py:66�[0m, in �[0;36m_get_gdal_dtype�[0;34m(type_name)�[0m
�[1;32m 65�[0m �[38;5;28;01mtry�[39;00m:
�[0;32m---> 66�[0m �[38;5;28;01mreturn�[39;00m �[43mdtype_rev�[49m�[43m[�[49m�[43mtype_name�[49m�[43m]�[49m
�[1;32m 67�[0m �[38;5;28;01mexcept�[39;00m �[38;5;167;01mKeyError�[39;00m:
�[0;31mKeyError�[0m: 'float16'
During handling of the above exception, another exception occurred:
�[0;31mTypeError�[0m Traceback (most recent call last)
File �[0;32m~/checkouts/readthedocs.org/user_builds/zen3geo/envs/123/lib/python3.11/site-packages/stackstac/rio_reader.py:387�[0m, in �[0;36mAutoParallelRioReader.read�[0;34m(self, window, **kwargs)�[0m
�[1;32m 386�[0m �[38;5;28;01mtry�[39;00m:
�[0;32m--> 387�[0m result �[38;5;241m=�[39m �[43mreader�[49m�[38;5;241;43m.�[39;49m�[43mread�[49m�[43m(�[49m
�[1;32m 388�[0m �[43m �[49m�[43mwindow�[49m�[38;5;241;43m=�[39;49m�[43mwindow�[49m�[43m,�[49m
�[1;32m 389�[0m �[43m �[49m�[43mout_dtype�[49m�[38;5;241;43m=�[39;49m�[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43mdtype�[49m�[43m,�[49m
�[1;32m 390�[0m �[43m �[49m�[43mmasked�[49m�[38;5;241;43m=�[39;49m�[38;5;28;43;01mTrue�[39;49;00m�[43m,�[49m
�[1;32m 391�[0m �[43m �[49m�[38;5;66;43;03m# ^ NOTE: we always do a masked array, so we can safely apply scales and offsets�[39;49;00m
�[1;32m 392�[0m �[43m �[49m�[38;5;66;43;03m# without potentially altering pixels that should have been the ``fill_value``�[39;49;00m
�[1;32m 393�[0m �[43m �[49m�[38;5;241;43m*�[39;49m�[38;5;241;43m*�[39;49m�[43mkwargs�[49m�[43m,�[49m
�[1;32m 394�[0m �[43m �[49m�[43m)�[49m
�[1;32m 395�[0m �[38;5;28;01mexcept�[39;00m �[38;5;167;01mException�[39;00m �[38;5;28;01mas�[39;00m e:
File �[0;32m~/checkouts/readthedocs.org/user_builds/zen3geo/envs/123/lib/python3.11/site-packages/stackstac/rio_reader.py:228�[0m, in �[0;36mThreadLocalRioDataset.read�[0;34m(self, window, **kwargs)�[0m
�[1;32m 227�[0m �[38;5;28;01mwith�[39;00m �[38;5;28mself�[39m�[38;5;241m.�[39m_env�[38;5;241m.�[39mread:
�[0;32m--> 228�[0m �[38;5;28;01mreturn�[39;00m �[38;5;28;43mself�[39;49m�[38;5;241;43m.�[39;49m�[43mdataset�[49m�[38;5;241;43m.�[39;49m�[43mread�[49m�[43m(�[49m�[43mwindow�[49m�[38;5;241;43m=�[39;49m�[43mwindow�[49m�[43m,�[49m�[43m �[49m�[38;5;241;43m*�[39;49m�[38;5;241;43m*�[39;49m�[43mkwargs�[49m�[43m)�[49m
File �[0;32mrasterio/_warp.pyx:1240�[0m, in �[0;36mrasterio._warp.WarpedVRTReaderBase.read�[0;34m()�[0m
File �[0;32mrasterio/_io.pyx:651�[0m, in �[0;36mrasterio._io.DatasetReaderBase.read�[0;34m()�[0m
File �[0;32mrasterio/_io.pyx:975�[0m, in �[0;36mrasterio._io.DatasetReaderBase._read�[0;34m()�[0m
File �[0;32mrasterio/_io.pyx:140�[0m, in �[0;36mrasterio._io.io_multi_band�[0;34m()�[0m
File �[0;32m~/checkouts/readthedocs.org/user_builds/zen3geo/envs/123/lib/python3.11/site-packages/rasterio/dtypes.py:68�[0m, in �[0;36m_get_gdal_dtype�[0;34m(type_name)�[0m
�[1;32m 67�[0m �[38;5;28;01mexcept�[39;00m �[38;5;167;01mKeyError�[39;00m:
�[0;32m---> 68�[0m �[38;5;28;01mraise�[39;00m �[38;5;167;01mTypeError�[39;00m(
�[1;32m 69�[0m �[38;5;124mf�[39m�[38;5;124m"�[39m�[38;5;124mUnsupported data type �[39m�[38;5;132;01m{�[39;00mtype_name�[38;5;132;01m}�[39;00m�[38;5;124m. �[39m�[38;5;124m"�[39m
�[1;32m 70�[0m �[38;5;124mf�[39m�[38;5;124m"�[39m�[38;5;124mAllowed data types: �[39m�[38;5;132;01m{�[39;00m�[38;5;28mlist�[39m(dtype_rev)�[38;5;132;01m}�[39;00m�[38;5;124m.�[39m�[38;5;124m"�[39m
�[1;32m 71�[0m )
�[0;31mTypeError�[0m: Unsupported data type float16. Allowed data types: [None, 'uint8', 'uint16', 'int16', 'uint32', 'int32', 'float32', 'float64', 'complex_int16', 'complex64', 'complex128', 'int64', 'uint64', 'complex', 'int8'].
The above exception was the direct cause of the following exception:
�[0;31mRuntimeError�[0m Traceback (most recent call last)
Cell �[0;32mIn[26], line 1�[0m
�[0;32m----> 1�[0m dataslice �[38;5;241m=�[39m �[43mdatacube�[49m�[38;5;241;43m.�[39;49m�[43msel�[49m�[43m(�[49m�[43mtime�[49m�[38;5;241;43m=�[39;49m�[38;5;124;43m"�[39;49m�[38;5;124;43m2022-02-23T11:41:54.329096000�[39;49m�[38;5;124;43m"�[39;49m�[43m)�[49m�[38;5;241;43m.�[39;49m�[43mcompute�[49m�[43m(�[49m�[43m)�[49m
�[1;32m 3�[0m fig, ax �[38;5;241m=�[39m plt�[38;5;241m.�[39msubplots(nrows�[38;5;241m=�[39m�[38;5;241m2�[39m, ncols�[38;5;241m=�[39m�[38;5;241m2�[39m, figsize�[38;5;241m=�[39m(�[38;5;241m11�[39m, �[38;5;241m12�[39m), sharex�[38;5;241m=�[39m�[38;5;28;01mTrue�[39;00m, sharey�[38;5;241m=�[39m�[38;5;28;01mTrue�[39;00m)
�[1;32m 5�[0m dataslice�[38;5;241m.�[39mvh�[38;5;241m.�[39mplot�[38;5;241m.�[39mimshow(ax�[38;5;241m=�[39max[�[38;5;241m0�[39m][�[38;5;241m0�[39m], cmap�[38;5;241m=�[39m�[38;5;124m"�[39m�[38;5;124mbone�[39m�[38;5;124m"�[39m, robust�[38;5;241m=�[39m�[38;5;28;01mTrue�[39;00m)
File �[0;32m~/checkouts/readthedocs.org/user_builds/zen3geo/envs/123/lib/python3.11/site-packages/xarray/core/dataset.py:991�[0m, in �[0;36mDataset.compute�[0;34m(self, **kwargs)�[0m
�[1;32m 972�[0m �[38;5;250m�[39m�[38;5;124;03m"""Manually trigger loading and/or computation of this dataset's data�[39;00m
�[1;32m 973�[0m �[38;5;124;03mfrom disk or a remote source into memory and return a new dataset.�[39;00m
�[1;32m 974�[0m �[38;5;124;03mUnlike load, the original dataset is left unaltered.�[39;00m
�[0;32m (...)�[0m
�[1;32m 988�[0m �[38;5;124;03mdask.compute�[39;00m
�[1;32m 989�[0m �[38;5;124;03m"""�[39;00m
�[1;32m 990�[0m new �[38;5;241m=�[39m �[38;5;28mself�[39m�[38;5;241m.�[39mcopy(deep�[38;5;241m=�[39m�[38;5;28;01mFalse�[39;00m)
�[0;32m--> 991�[0m �[38;5;28;01mreturn�[39;00m �[43mnew�[49m�[38;5;241;43m.�[39;49m�[43mload�[49m�[43m(�[49m�[38;5;241;43m*�[39;49m�[38;5;241;43m*�[39;49m�[43mkwargs�[49m�[43m)�[49m
File �[0;32m~/checkouts/readthedocs.org/user_builds/zen3geo/envs/123/lib/python3.11/site-packages/xarray/core/dataset.py:825�[0m, in �[0;36mDataset.load�[0;34m(self, **kwargs)�[0m
�[1;32m 822�[0m chunkmanager �[38;5;241m=�[39m get_chunked_array_type(�[38;5;241m*�[39mlazy_data�[38;5;241m.�[39mvalues())
�[1;32m 824�[0m �[38;5;66;03m# evaluate all the chunked arrays simultaneously�[39;00m
�[0;32m--> 825�[0m evaluated_data �[38;5;241m=�[39m �[43mchunkmanager�[49m�[38;5;241;43m.�[39;49m�[43mcompute�[49m�[43m(�[49m�[38;5;241;43m*�[39;49m�[43mlazy_data�[49m�[38;5;241;43m.�[39;49m�[43mvalues�[49m�[43m(�[49m�[43m)�[49m�[43m,�[49m�[43m �[49m�[38;5;241;43m*�[39;49m�[38;5;241;43m*�[39;49m�[43mkwargs�[49m�[43m)�[49m
�[1;32m 827�[0m �[38;5;28;01mfor�[39;00m k, data �[38;5;129;01min�[39;00m �[38;5;28mzip�[39m(lazy_data, evaluated_data):
�[1;32m 828�[0m �[38;5;28mself�[39m�[38;5;241m.�[39mvariables[k]�[38;5;241m.�[39mdata �[38;5;241m=�[39m data
File �[0;32m~/checkouts/readthedocs.org/user_builds/zen3geo/envs/123/lib/python3.11/site-packages/xarray/core/daskmanager.py:70�[0m, in �[0;36mDaskManager.compute�[0;34m(self, *data, **kwargs)�[0m
�[1;32m 67�[0m �[38;5;28;01mdef�[39;00m �[38;5;21mcompute�[39m(�[38;5;28mself�[39m, �[38;5;241m*�[39mdata: DaskArray, �[38;5;241m*�[39m�[38;5;241m*�[39mkwargs) �[38;5;241m-�[39m�[38;5;241m>�[39m �[38;5;28mtuple�[39m[np�[38;5;241m.�[39mndarray, �[38;5;241m.�[39m�[38;5;241m.�[39m�[38;5;241m.�[39m]:
�[1;32m 68�[0m �[38;5;28;01mfrom�[39;00m �[38;5;21;01mdask�[39;00m�[38;5;21;01m.�[39;00m�[38;5;21;01marray�[39;00m �[38;5;28;01mimport�[39;00m compute
�[0;32m---> 70�[0m �[38;5;28;01mreturn�[39;00m �[43mcompute�[49m�[43m(�[49m�[38;5;241;43m*�[39;49m�[43mdata�[49m�[43m,�[49m�[43m �[49m�[38;5;241;43m*�[39;49m�[38;5;241;43m*�[39;49m�[43mkwargs�[49m�[43m)�[49m
File �[0;32m~/checkouts/readthedocs.org/user_builds/zen3geo/envs/123/lib/python3.11/site-packages/dask/base.py:628�[0m, in �[0;36mcompute�[0;34m(traverse, optimize_graph, scheduler, get, *args, **kwargs)�[0m
�[1;32m 625�[0m postcomputes�[38;5;241m.�[39mappend(x�[38;5;241m.�[39m__dask_postcompute__())
�[1;32m 627�[0m �[38;5;28;01mwith�[39;00m shorten_traceback():
�[0;32m--> 628�[0m results �[38;5;241m=�[39m �[43mschedule�[49m�[43m(�[49m�[43mdsk�[49m�[43m,�[49m�[43m �[49m�[43mkeys�[49m�[43m,�[49m�[43m �[49m�[38;5;241;43m*�[39;49m�[38;5;241;43m*�[39;49m�[43mkwargs�[49m�[43m)�[49m
�[1;32m 630�[0m �[38;5;28;01mreturn�[39;00m repack([f(r, �[38;5;241m*�[39ma) �[38;5;28;01mfor�[39;00m r, (f, a) �[38;5;129;01min�[39;00m �[38;5;28mzip�[39m(results, postcomputes)])
File �[0;32m~/checkouts/readthedocs.org/user_builds/zen3geo/envs/123/lib/python3.11/site-packages/stackstac/to_dask.py:189�[0m, in �[0;36mfetch_raster_window�[0;34m(reader_table, slices, dtype, fill_value)�[0m
�[1;32m 182�[0m �[38;5;66;03m# Only read if the window we're fetching actually overlaps with the asset�[39;00m
�[1;32m 183�[0m �[38;5;28;01mif�[39;00m windows�[38;5;241m.�[39mintersect(current_window, asset_window):
�[1;32m 184�[0m �[38;5;66;03m# NOTE: when there are multiple assets, we _could_ parallelize these reads with our own threadpool.�[39;00m
�[1;32m 185�[0m �[38;5;66;03m# However, that would probably increase memory usage, since the internal, thread-local GDAL datasets�[39;00m
�[1;32m 186�[0m �[38;5;66;03m# would end up copied to even more threads.�[39;00m
�[1;32m 187�[0m
�[1;32m 188�[0m �[38;5;66;03m# TODO when the Reader won't be rescaling, support passing `output` to avoid the copy?�[39;00m
�[0;32m--> 189�[0m data �[38;5;241m=�[39m �[43mreader�[49m�[38;5;241;43m.�[39;49m�[43mread�[49m�[43m(�[49m�[43mcurrent_window�[49m�[43m)�[49m
�[1;32m 191�[0m �[38;5;28;01mif�[39;00m all_empty:
�[1;32m 192�[0m �[38;5;66;03m# Turn `output` from a broadcast-trick array to a real array, so it's writeable�[39;00m
�[1;32m 193�[0m �[38;5;28;01mif�[39;00m (
�[1;32m 194�[0m np�[38;5;241m.�[39misnan(data)
�[1;32m 195�[0m �[38;5;28;01mif�[39;00m np�[38;5;241m.�[39misnan(fill_value)
�[1;32m 196�[0m �[38;5;28;01melse�[39;00m np�[38;5;241m.�[39mequal(data, fill_value)
�[1;32m 197�[0m )�[38;5;241m.�[39mall():
�[1;32m 198�[0m �[38;5;66;03m# Unless the data we just read is all empty anyway�[39;00m
File �[0;32m~/checkouts/readthedocs.org/user_builds/zen3geo/envs/123/lib/python3.11/site-packages/stackstac/rio_reader.py:401�[0m, in �[0;36mAutoParallelRioReader.read�[0;34m(self, window, **kwargs)�[0m
�[1;32m 398�[0m warnings�[38;5;241m.�[39mwarn(msg)
�[1;32m 399�[0m �[38;5;28;01mreturn�[39;00m nodata_for_window(window, �[38;5;28mself�[39m�[38;5;241m.�[39mfill_value, �[38;5;28mself�[39m�[38;5;241m.�[39mdtype)
�[0;32m--> 401�[0m �[38;5;28;01mraise�[39;00m �[38;5;167;01mRuntimeError�[39;00m(msg) �[38;5;28;01mfrom�[39;00m �[38;5;21;01me�[39;00m
�[1;32m 403�[0m �[38;5;66;03m# When the GeoTIFF doesn't have a nodata value, and we're using a VRT, pixels�[39;00m
�[1;32m 404�[0m �[38;5;66;03m# outside the dataset don't get properly masked (they're just 0). Using `add_alpha`�[39;00m
�[1;32m 405�[0m �[38;5;66;03m# in the `WarpedVRT`, we get an explicit alpha channel, which we use as a mask instead.�[39;00m
�[1;32m 406�[0m �[38;5;66;03m# See https://github.com/gjoseph92/stackstac/issues/217.�[39;00m
�[1;32m 407�[0m �[38;5;28;01mif�[39;00m result�[38;5;241m.�[39mshape[�[38;5;241m0�[39m] �[38;5;241m==�[39m �[38;5;241m2�[39m:
�[0;31mRuntimeError�[0m: Error reading Window(col_off=0, row_off=0, width=491, height=579) from 'https://elevationeuwest.blob.core.windows.net/copernicus-dem/COP30_hh/Copernicus_DSM_COG_10_S01_00_E100_00_DEM.tif?st=2023-09-20T21%3A28%3A02Z&se=2023-09-22T21%3A28%3A02Z&sp=rl&sv=2021-06-08&sr=c&skoid=c85c15d6-d1ae-42d4-af60-e2ca0f81359b&sktid=72f988bf-86f1-41af-91ab-2d7cd011db47&skt=2023-09-21T05%3A22%3A55Z&ske=2023-09-28T05%3A22%3A55Z&sks=b&skv=2021-06-08&sig=WelnMcHXxjdEuDHJJ1i9%2BV83uQXToDYmOkbL3WVGWpc%3D': TypeError("Unsupported data type float16. Allowed data types: [None, 'uint8', 'uint16', 'int16', 'uint32', 'int32', 'float32', 'float64', 'complex_int16', 'complex64', 'complex128', 'int64', 'uint64', 'complex', 'int8'].")
Not exactly stackstac's fault as rasterio/GDAL doesn't support reading GeoTIFFs into float16 dtype directly (and probably won't soon given the effort).
Will have a follow-up PR to make things compatible with stackstac=0.5.0
. Key is on how to deal with NaNs, since only float types support having NaNs. Either cast things to float32 instead of float16, or use native uint16 and cast back to float16? If only there were nullable uint16 dtype 🤔
Bumps cryptography from 41.0.3 to 41.0.4.
Changelog
Sourced from cryptography's changelog.
Commits
fc11bce
bump for 41.0.4 (#9629)Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase
.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
@dependabot rebase
will rebase this PR@dependabot recreate
will recreate this PR, overwriting any edits that have been made to it@dependabot merge
will merge this PR after your CI passes on it@dependabot squash and merge
will squash and merge this PR after your CI passes on it@dependabot cancel merge
will cancel a previously requested merge and block automerging@dependabot reopen
will reopen this PR if it is closed@dependabot close
will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually@dependabot show <dependency name> ignore conditions
will show all of the ignore conditions of the specified dependency@dependabot ignore this major version
will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this minor version
will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this dependency
will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)You can disable automated security fix PRs for this repo from the Security Alerts page.