PERF ND reader flexibility #227
Conversation
A subclass of FramesSequenceND now should only define 1 (or more) reader
method that can return any axes (even single pixels), given by the
`@reads_frames('yx')` decorator. The reader then figures out how to
iterate over the reader methods to return the desired axes.
Register a get_frame method in the initialization using `self._register_get_frame(method, axes)`
Mainly for backwards compatibility
|
Does anybody have time to look into this PR? The performance increase is great for my purposes and I would like to see it in master. |
That's very exciting and in line with some things @tacaswell and I are interested in here as well. @vivekthampy might also be excited about this. I gave this a quick read -- looks fine to me. Merge? |
|
Merge is fine by me! I am happy to collaborate on extensions or generalizations on this, let me know. I leave the merge to you or @tacaswell |
| def _register_get_frame(self, method, axes): | ||
| axes = tuple([a for a in axes]) | ||
| if not hasattr(self, '_get_frame_dict'): | ||
| self._get_frame_dict = dict() |
There was a problem hiding this comment.
I would suggest putting this initialization in FrameSequenceND.__init__ and requrining sub-classes to call super() before doing anything else. Likely will have to do this test + warn for awhile to maintain back-compat.
|
Just one picky comment |
|
@tacaswell I pushed the changes that you suggested, ready to go |
See soft-matter/pims#227 . Wait with merge for release of this PR (probably pims v0.4)
This enables FramesSequenceND readers to use the frame dimensions of the underlying reader in a smart way. For instance: currently when an RGB frame is loaded from a movie, the frame is actually read 3 times, each time only using 1 channel.
Subclassed FramesSequenceND readers have to register their
get_framemethods, using the axes names:When
bundle_axesis set, the reader wraps theget_frame_2Dmethod with atranspose,iterateordropoperation so that the correct axes are returned. It always tries to minimize IO traffic: several calls toget_frame_2Dis referred over reading too much data.Any combination between
get_frameaxes andbundle_axesis possible (and tested).This enhancement paves the way to make the readers actually lazy in all dimensions. Cropping could be pushed down to reader level, when the underlying reader supports reading pixel lines or single pixels.