You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Almost all of them come from using ICP in a pipeline, or on its own.
We'd save 6min (half of test durations), by finding a way to speed up those. I'll look into it.
The text was updated successfully, but these errors were encountered:
OK, so it looks like the slowdown comes largely from the apply() part of ICP, which we already optimized as much as we could in #531. It's unavoidable when applying those transformations step-by-step on such large rasters.
So the best way to address this slow issue for tests is simply to reduce the size of the input rasters, which works well by simply adding some cropping in load_examples():
# Crop to smaller extents for test speedres=reference_dem.rescrop_geom= (reference_dem.bounds.left, reference_dem.bounds.bottom,
reference_dem.bounds.left+res[0] *300, reference_dem.bounds.bottom+res[1] *300)
reference_dem=reference_dem.crop(crop_geom)
to_be_aligned_dem=to_be_aligned_dem.crop(crop_geom)
With this quick fix (which I'm adding in #530), we bring the total test duration of xDEM down from 12min to 5min, and in particular running just the test_coreg folder is now about 4x times faster @adebardo This should save you time in #563 😉
Further optimizations?
The only main optimization we could add now that we covered re-gridding in #531 would be specific to CoregPipeline: an automated way to detect when the next method in the pipeline accepts point directly, we don't need to apply a transformation to the full raster, but only the point subsample which is much faster. See this old issue on this: #79.
Another little optimization would be to avoid running apply to the last step of a pipeline fit (which is unusued), as pointed in this other old issue that is still relevant: #234. I'll address this in #530.
Related to #455.
Here's an overview of our test lengths that are longer than 10s:
Almost all of them come from using ICP in a pipeline, or on its own.
We'd save 6min (half of test durations), by finding a way to speed up those. I'll look into it.
The text was updated successfully, but these errors were encountered: