'save_cog_with_dask' fails with numpy > 2

[remi-braun]

Hello,

With numpy <2, save_cog_with_dask fails because illegal casting.

• With numpy < 2, everything works fine.
• With numpy 2.0.1, I have the following error: OverflowError: Python integer 999999 out of bounds for uint8
• With numpy 2.1.0, this one: TypeError: Cannot cast scalar from dtype('int64') to dtype('uint8') according to the rule 'same_kind'

This issue is based on this discussion: https://discourse.pangeo.io/t/save-cog-with-dask-cannot-convert-fill-value-999999-to-dtype-uint8/4713/9

As @maxrjones said there:

This is most likely coming from NumPy’s defaults for masked arrays - numpy.ma.default_fill_value — NumPy v2.1 Manual.

Minimal working example:

import numpy as np
import rioxarray as rxr
from odc.geo import cog, xr  # noqa
xds = rxr.open_rasterio(r"not_cog.tif", chunks=True)
nodata = 0
cog.save_cog_with_dask(
    xds.copy(data=xds.fillna(nodata).astype(np.uint8)).rio.set_nodata(
        nodata
    ),
    r"cog.tif",
).compute()

Data : not_cog.zip

I'm on Windows, maybe this doesn't help...

[remi-braun]

Maybe linked to numpy/numpy#25677 and more specifically this comment

[Kirill888]

Looks like the error is in the part of the logic that deals with "stats" computation, min/max/avg pixel statistics. Error itself seems to be coming from numpy implementation of masked arrays, or possibly the interaction with Dask DataArray and numpy2

You can disable stats computation with stats=False parameter, that way COG generation succeeds, at least on my system, however without those stats large images might take long time to load up in QGIS for the first time.