Support NIFTI-2, resolves #89 #90

benkay86 · 2022-01-13T22:04:10Z

Adds support for the NIFTI-2 header format. The PR is just enough to compile and run tests. If approved, documentation and additional test cases will be needed in a subsequent PR. These changes are not backwards compatible.

Types and methods outside the header have been changed to default to the larger NIFTI-2 types. For example, nifti::volume::shape::Dim is now based off u64 instead of u16.
NiftiHeader is changed to an enum that contains either a Nifti1Header or Nifti2Header structure. NiftiHeader implements getter/setter methods to modify the underlying header without having to worry about which version it is. The underlying structure can also be accessed directly.

let mut hdr = NiftiHeader::default();
hdr.slice_duration = 1.0; // no longer works
hdr.set_slice_duration(1.0); // OK
let slice_duration: f32 = hdr.get_slice_duration(); // no longer works
let slice_duration: f64 = hdr.get_slice_duration(); // OK
// Conversion to larger NIFTI-2 types always succeeds.
let mut hdr2: Nifti2Header = hdr.into_nifti2();
hdr2.slice_duration = 1.0; // OK
let slice_duration: f64 = hdr2.slice_duration; // OK
// Conversion to smaller NIFTI-1 types is fallible.
let mut hdr1: Nifti1Header = hdr2.into_nifti().into_nifti1().unwrap();
hdr1.slice_duration = 1.0; // OK
let slice_duration: f32 = hdr1.slice_duration; // OK

Migration should otherwise be relatively painless.

NiftiHeader::from_file() and from_reader() automatically detect the header version in the given file or input stream.
WriterOptions::write_nifti() automatically writes out whichever header version is referenced by the WriterOptions.
NiftiHeader::into_nifti1() and into_nifti2() conveniently convert between header versions.

nilgoyette

I can't review anything thoroughly now, but here are some questions and comments.

Btw, good job. This is a lot of work. Thank you.

nilgoyette · 2022-01-16T18:51:51Z

src/header.rs

+/// Magic code for full NIFTI-1 files (extention ".nii[.gz]").
+pub const MAGIC_CODE_NIP2: &[u8; 8] = b"n+2\0\r\n\x1A\n";
+
+/// Abstraction over NIFTI version 1 and 2 headers.  This is the high-level type


May I ask you why you use double space after all sentences? I never seen that before.

I heard rumors that typewriters had issues which were resolved with the double spacing, and with that some people grew into the aesthetics of separating sentences like this. 🤔

Nowadays though, I practice a form of semantic linefeeds or semantic line breaks, which recommends introducing a new line at the end of each sentence.

In any case, this isn't a critical enough concern to be a blocker.

nilgoyette · 2022-01-16T18:53:39Z

src/header.rs

+    /// Header size, must be 348 for a NIFTI-1 header and 540 for NIFTI-2.
+    /// There is no corresponding `set_sizeof_hdr()` because you should never
+    /// need to set this field manually.
+    pub fn get_sizeof_hdr(&self) -> u32 {


I think @Enet4 once told me to not use get_ in Rust. I don't have a clear opinion on this matter, but it seems it's not idiomatic. Maybe he can confirm.

Yup, it's the C-GETTER guideline. Getters in Rust usually usually aren't prefixed with get_ by convention.

Suggested change

pub fn get_sizeof_hdr(&self) -> u32 {

pub fn sizeof_hdr(&self) -> u32 {

nilgoyette · 2022-01-16T19:02:00Z

src/header.rs

+
+    // All done, return header with populated fields.
+    Ok(h)
+}


Do you use an automatic formatter? (Don't search what's not formatted here ^^)

nilgoyette · 2022-01-16T19:03:05Z

src/object.rs

-        let len = header.vox_offset as usize;
-        let len = if len < 352 { 0 } else { len - 352 };
+        let len: usize = header.get_vox_offset()?.try_into()?;
+        let len = if len == 0 { 0 } else { len - TryInto::<usize>::try_into(header.get_sizeof_hdr())? }; // TODO!


Can you explain more what's left TODO? We need to know if we ever need to do it.

nilgoyette · 2022-01-16T19:09:12Z

tests/writer.rs

@@ -256,23 +258,10 @@ mod tests {
        assert_eq!(new_data, data);
    }

-    #[test]
-    fn write_descrip_panic() {


Is this not tested anymore?

It is no longer possible because descrip was turned back into an 80-byte array. It was kind of what I originally had in mind, but large arrays were not as nice to work with in Rust. That changed in recent versions, so unless we witness a significant performance regression, we can save the description in a [u8; 80] here instead of Vec<u8>.

smallvec? https://docs.rs/smallvec/latest/smallvec/

Enet4

Thank you for this major contribution, @benkay86! I went through the code and made a quick review. What I feel are the two major concerns are:

getter naming convention (as already mentioned, they should not use get_)
linear transformations for volume conversion will use f64 instead of f32 in many cases, which I suspect could affect cases where the original data is all in f32. If all voxel values have to go through f64, this could translate to a bigger memory footprint and maybe less performance. It might help to try a stress test with f32 volumes and see if performance and memory usage changed significantly.

Enet4 · 2022-01-19T18:54:43Z

src/error.rs

+        /// Invalid header size.  Header size must be 540 for NIfTI-2 or 348 for
+        /// NIfTI-1.
+        InvalidHeaderSize(sizeof_hdr: i32) {
+            display("Invalid header size {} must eb 540 for NIfTI-2 or 348 for NIfTI-1.", sizeof_hdr)


Suggested change

display("Invalid header size {} must eb 540 for NIfTI-2 or 348 for NIfTI-1.", sizeof_hdr)

display("Invalid header size {}: must be 540 for NIfTI-2 or 348 for NIfTI-1.", sizeof_hdr)

Enet4 · 2022-01-19T18:57:39Z

src/header.rs

+    /// Header size, must be 348 for a NIFTI-1 header and 540 for NIFTI-2.
+    /// There is no corresponding `set_sizeof_hdr()` because you should never
+    /// need to set this field manually.
+    pub fn get_sizeof_hdr(&self) -> u32 {


Yup, it's the C-GETTER guideline. Getters in Rust usually usually aren't prefixed with get_ by convention.

Suggested change

pub fn get_sizeof_hdr(&self) -> u32 {

pub fn sizeof_hdr(&self) -> u32 {

Enet4 · 2022-01-19T19:07:55Z

src/header.rs

-        (self.pixdim[0].abs() - 1.).abs() < 1e-11
+impl Nifti2Header {
+    /// Place this `Nifti2Header` into a version-agnostic [`NiftiHeader`] enum.
+    pub fn into_nifti(self) -> NiftiHeader {


It would be idiomatic to also have implementations of From<Nifti1Header> and From<Nifti2Header> for NiftiHeader. This way, a conversion would be as simple as header.into().

Enet4 · 2022-01-19T19:13:25Z

src/header.rs

@@ -379,11 +1477,11 @@ impl NiftiHeader {
    pub fn affine<T>(&self) -> Matrix4<T>
    where
        T: RealField,
-        f32: SubsetOf<T>,
+        f64: SubsetOf<T>,


This seems to suggest that all affine transformations will from now on use f64 for calculations, regardless of the nature of the original data. Volume data value transformations also seem to have changed accordingly. Not sure whether this breaks any expectations to existing consumers (@nilgoyette?).

Enet4 · 2022-01-19T19:25:33Z

src/volume/element.rs

@@ -42,17 +42,17 @@ pub trait LinearTransform<T: 'static + Copy> {
 /// affine transformation, then converted back to the original type. Ideal for
 /// small, low precision types such as `u8` and `i16`.
 #[derive(Debug)]
-pub struct LinearTransformViaF32;
+pub struct LinearTransformViaF32; // TODO remove this for NIFTI-2?


Were these changes merely a consequence of the wider nifti header? Perhaps by letting the API consumer tap into nifti-1, they could keep linear transformations in f32.

Enet4 · 2022-01-19T19:31:33Z

tests/writer.rs

@@ -256,23 +258,10 @@ mod tests {
        assert_eq!(new_data, data);
    }

-    #[test]
-    fn write_descrip_panic() {


It is no longer possible because descrip was turned back into an 80-byte array. It was kind of what I originally had in mind, but large arrays were not as nice to work with in Rust. That changed in recent versions, so unless we witness a significant performance regression, we can save the description in a [u8; 80] here instead of Vec<u8>.

liningpan · 2022-05-18T20:52:37Z

Hi, NIFTI2 support would be very helpful!

Can we add additional intent codes from the HCP pipeline?

https://github.com/Washington-University/workbench/blob/master/src/FilesBase/nifti2.h#L33-L49

const int32_t NIFTI_INTENT_CONNECTIVITY_UNKNOWN=3000;
const int32_t NIFTI_INTENT_CONNECTIVITY_DENSE=3001;
const int32_t NIFTI_INTENT_CONNECTIVITY_DENSE_TIME=3002;//CIFTI-1 name
const int32_t NIFTI_INTENT_CONNECTIVITY_DENSE_SERIES=3002;//CIFTI-2 name
const int32_t NIFTI_INTENT_CONNECTIVITY_PARCELLATED=3003;
const int32_t NIFTI_INTENT_CONNECTIVITY_PARCELLATED_TIME=3004;//ditto
const int32_t NIFTI_INTENT_CONNECTIVITY_PARCELLATED_SERIES=3004;
const int32_t NIFTI_INTENT_CONNECTIVITY_CONNECTIVITY_TRAJECTORY=3005;//ditto
const int32_t NIFTI_INTENT_CONNECTIVITY_DENSE_TRAJECTORY=3005;
const int32_t NIFTI_INTENT_CONNECTIVITY_DENSE_SCALARS=3006;
const int32_t NIFTI_INTENT_CONNECTIVITY_DENSE_LABELS=3007;
const int32_t NIFTI_INTENT_CONNECTIVITY_PARCELLATED_SCALAR=3008;
const int32_t NIFTI_INTENT_CONNECTIVITY_PARCELLATED_DENSE=3009;
const int32_t NIFTI_INTENT_CONNECTIVITY_DENSE_PARCELLATED=3010;
const int32_t NIFTI_INTENT_CONNECTIVITY_PARCELLATED_PARCELLATED_SERIES=3011;
const int32_t NIFTI_INTENT_CONNECTIVITY_PARCELLATED_PARCELLATED_SCALAR=3012;

However, several variables map to the same number which is not possible for rust enum.

benkay86 · 2022-05-18T21:10:56Z

Apologies, I had a baby and kind of lost track of this! It looks like there is still broad interest in NIFTI-2 support, and aside from some formatting issues and the need for more test cases, the biggest obstacle is what to name getter methods.

@nilgoyette is right that Rust eschews getter methods in general, preferring the simplicity and explicitness of accessing a struct's field directly. The current PR allows this by resolving the NiftiHeader enum to a Nifti1Header or Nifti2Header struct that can be manipulated directly. In addition to this, convenient getter and setter methods which are agnostic to the underlying header version are provided for automatic conversion between field types.

In such cases where getter/setter methods are needed, the Rust convention is to name the setter for the field foo set_foo() and simply name the getter foo(). However, there is a problem for nifti-rs because a few fields already have getter-like functions with the same name. For example, NiftiHeader::dim is a [u16;8] and NiftiHeader::dim() returns a validated slice of this field. What should we name a getter that returns the raw field?

Possibilities:

Name raw getters get_foo(). While this is verbose and goes against Rust convention, it is easily understood and avoids name conflicts with existing methods.
Rename the existing methods something else like foo_validated() and name the raw getter foo(). This is more idiomatic but breaks the API. (Maybe not such a big deal since the PR contains other breaking changes in corner cases.)
Don't provide a raw getter for fields that require some kind of validation or parsing. Make the user cast the enum down to a Nifti1Header or Nifti2Header if raw access to the field is desired.

Thoughts?

liningpan · 2022-05-18T21:34:50Z

I also tried to implement nifti-2 support myself. I took a more aggressive approach and cast nifti-1 and nifti-2 header to a common type derived from nifti-2. The common type also exposes Enum types directly instead of through a getter. I believe connectome workbench took a similar approach. However, this will complete nuke the current interface.

I also took a few test cases from nibabel, for nifti-2 unit tests.

You can take a look my experimental branch here if you are interested. https://github.com/liningpan/nifti-rs/tree/nifti2

Enet4 · 2022-05-19T09:57:32Z

However, there is a problem for nifti-rs because a few fields already have getter-like functions with the same name. For example, NiftiHeader::dim is a [u16;8] and NiftiHeader::dim() returns a validated slice of this field. What should we name a getter that returns the raw field?

I don't take issue in introducing a breaking change to the API, especially for introducing such a big feature to the crate. raw_dim would make a decent method name for the slice as saved in the header (thus only available behind specific version headers).

Aside from this, I would also like to ensure that 1) it stays possible to work with a specific version explicitly, and that 2) calculations on NIfTI-1 can still use f32 as common ground, as mentioned in this comment.

benkay86 added 14 commits January 13, 2022 11:03

expand NiftiError

c883612

tweak types in NIFTI-1 header to prevent misuse

0fd81bb

store header in same file as data by default

36e957b

add Nifti2Header and rename Nifti1Header

fd54853

add NiftiHeader and conversion methods

95866c9

add getter/setters to NiftiHeader

4c20845

move high-level methods to NiftiHeader

374ea78

use getters/setters instead of raw fields

dcd1134

use larger NIFTI-2 date types

d9d33e6

change how headers are read

17a6d4a

read extensions from NIFTI-2

37e5e19

change how headers are written

13b83a3

make existing tests work with abstract NiftiHeader

9a94e89

make nalgebra work with abstract NiftiHeader

98bccee

benkay86 mentioned this pull request Jan 13, 2022

support nifti-2 files #89

Open

nilgoyette reviewed Jan 16, 2022

View reviewed changes

Enet4 self-requested a review January 18, 2022 18:05

Enet4 requested changes Jan 19, 2022

View reviewed changes

liningpan mentioned this pull request Jul 10, 2022

NIfTI-2 Support #96

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support NIFTI-2, resolves #89 #90

Support NIFTI-2, resolves #89 #90

benkay86 commented Jan 13, 2022 •

edited

Loading

nilgoyette left a comment

nilgoyette Jan 16, 2022

Enet4 May 19, 2022

nilgoyette Jan 16, 2022

Enet4 Jan 19, 2022

nilgoyette Jan 16, 2022 •

edited

Loading

nilgoyette Jan 16, 2022 •

edited

Loading

nilgoyette Jan 16, 2022

Enet4 Jan 19, 2022

liningpan Jul 10, 2022

Enet4 left a comment

Enet4 Jan 19, 2022

Enet4 Jan 19, 2022

Enet4 Jan 19, 2022

Enet4 Jan 19, 2022

Enet4 Jan 19, 2022

Enet4 Jan 19, 2022

liningpan commented May 18, 2022

benkay86 commented May 18, 2022

liningpan commented May 18, 2022

Enet4 commented May 19, 2022

	pub fn get_sizeof_hdr(&self) -> u32 {
	pub fn sizeof_hdr(&self) -> u32 {

	display("Invalid header size {} must eb 540 for NIfTI-2 or 348 for NIfTI-1.", sizeof_hdr)
	display("Invalid header size {}: must be 540 for NIfTI-2 or 348 for NIfTI-1.", sizeof_hdr)

Support NIFTI-2, resolves #89 #90

Are you sure you want to change the base?

Support NIFTI-2, resolves #89 #90

Conversation

benkay86 commented Jan 13, 2022 • edited Loading

nilgoyette left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nilgoyette Jan 16, 2022 • edited Loading

Choose a reason for hiding this comment

nilgoyette Jan 16, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Enet4 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liningpan commented May 18, 2022

benkay86 commented May 18, 2022

liningpan commented May 18, 2022

Enet4 commented May 19, 2022

benkay86 commented Jan 13, 2022 •

edited

Loading

nilgoyette Jan 16, 2022 •

edited

Loading

nilgoyette Jan 16, 2022 •

edited

Loading