-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Streamed Volumes #52
Streamed Volumes #52
Conversation
- split NiftiVolume trait to have a separate random access API - `Dim` construct provides nice and efficient volume shape access - volume::streamed module containing volumes read slice by slice - deprecate InMemNiftiVolume::new_from_stream in favor of from_reader - make Nifti object impl more generic, use type alias to retain API - provide nifti object for streamed volumes - add more info to NiftiError::IncompatibleLength
- Improve volume::streamed module documentation - rename methods in NiftiObject away from "stream", "reader" is better
- deprecate old functions - "reader" is more appropriate here, and prevents confusion with async streams
- move Dim to new module volume::shape - add Idx to volume::shape (ndimensional index) - add `indexed` method to streamed volume - document some methods in util
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting work. Thank you. Did you do some benchmark? Is it a lot slower but necessary on huge file, or as fast and we should use it whenever we can?
staged by mistake
- split FromSource trait to have options invariant to source - add "_rank" variants of streamed object construction - move common implementations to generic impl scope - rustfmt
Thanks for the initial review, @nilgoyette. A few more changes were pushed just now, some of which reduce redundancy. The subtle differences in each kind of volume make it pretty hard to merge common routines. I haven't benchmarked this yet, but it would be nice to see some measurements on large volumes. The real benefit comes from using much less memory, at the cost of potentially a bit of extra maintenance overhead (plus more allocations if the idiomatic iterator is used instead of the streaming iterator). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Only minor details, good job.
- remove redundant commented code - reduce redundancy in GenericNiftiObject::from_reader - rustfmt tests/volume
This gathers a bunch of additions and changes to the crate so as to provide slice-by-slice reading of volumes. (#44)
A summary of all changes below:
NiftiVolume
trait to have a separate random access API:RandomAccessNiftiVolume
Dim
andIdx
type definitions for containing a NIfTI-compliant volume shape and index to volume, provides a nicer abstraction than working with the 8-element array directly.volume::streamed
module contains a volume implementation which reads the voxel data slice by slice.InMemNiftiVolume::new_from_stream
and all otherfrom_stream
associated functions in favor offrom_reader
, which is more idiomatic and does not conflict with the futures concept of streamGenericNiftiObject
.InMemoryNiftiObject
andStreamedNiftiObject
are now aliases to this type.NiftiError::IncompatibleLength