Release GIL in gradient search resampling #445
Labels
enhancement
performance
improves speed or decreases memory consumption, but does not otherwise change functionality
Problem description
It was noted in recent EWA profiling that the Python Global Interpreter Lock (GIL) was not being released during processing which caused CPU usage to hover around 100% even though multiple dask workers were being used. Adding
with nogil:
to the proper spots in the Cython code improved execution time and increased CPU usage to ~700% usage for 8 workers. This same lack of releasing the GIL was noticed in the gradient search code. Specifically here:pyresample/pyresample/gradient/_gradient_search.pyx
Lines 124 to 133 in c930465
and here:
pyresample/pyresample/gradient/_gradient_search.pyx
Lines 240 to 248 in c930465
My initial tests at naively adding
with nogil:
around these calls resulted in a segmentation fault. My hope is that someone like @mraspaud who is familiar with the algorithm can figure out why this might be happening.The text was updated successfully, but these errors were encountered: