Pickle error and never ending code with 1.43.1

[TabeaW]

With the new version 1.43.1 we receive a pickle error while opening Sentinel-3 OLCI data. The error isn't really helpful, as everything comes from the distributed/pickle/python side, the error:

distributed.protocol.pickle - ERROR - Failed to serialize <pykdtree.dktree.KDTree object 
...
TypeError: no default __reduce__ due to non-trivial __cinit__

After the error, the code hangs and does not end.
There was no error with the version 1.43.0.

[remi-braun]

Hi there,

We indeed added a deeper use of Dask in this 1.43.1 release, so maybe it has broken some weird things like that 🤔
I think maybe it's more of an eoreader issue.

I'll investiguate.

[bastiencyr]

Hello @TabeaW ,
Do you use threading or cython or multithreading somwhere in your code ?
Notice that sertit-utils will not use Dask if you dont have it in your environement

[bastiencyr]

The issue is linked to storpipfugl/pykdtree#81

[bastiencyr]

rasters now creates a local threaded cluster which should solve your issue. You can test your code against the latest version of sertit-utils (on main branch)

[remi-braun]

Done, released in 1.43.2