MergeSeries has no affine tolerance

[smazurchuk]

What happened?

I recently upgraded my preprocessing pipeline from v20.1.0 to v22.0.0. We use a singularity container and there are a few participants who ran with no problems on the older version, who now throw errors. It seems perhaps an old bug was re-introduced in the most recent version.

Seemingly equivalent issue in previous version: #2706 (new comments at the bottom of this post since the issue has been closed)

Small differences in the affine matrix lead to failure

What command did you use?

fMRIPrep command: /opt/conda/bin/fmriprep /scratch/u/smazurchuk/cat/bids /scratch/g/lconant/derivatives/fmriprep-v22.0.0 participant --skip_bids_validation --participant_label soe123 --mem 90000 --nthreads 12 --omp-nthreads 4 --output-spaces MNI152NLin2009cAsym fsLR:den-32k --medial-surface-nan --random-seed 1234567890 --fs-license-file /work_dir/license.txt --no-submm-recon --cifti-output 91k --work-dir /scratch/g/lconant/tmp/fmriprep_soe123 --stop-on-first-crash

What version of fMRIPrep are you running?

22.0.0

How are you running fMRIPrep?

Singularity

Is your data BIDS valid?

Yes

Are you reusing any previously computed results?

FreeSurfer

Please copy and paste any relevant log output.

Node: fmriprep_22_0_wf.single_subject_soe123_wf.fmap_preproc_wf.wf_auto_00000.concat_blips
Working directory: /scratch/g/lconant/tmp/fmriprep_soe123/fmriprep_22_0_wf/single_subject_soe123_wf/fmap_preproc_wf/wf_auto_00000/concat_blips

Node inputs:

affine_tolerance = <undefined>
allow_4D = True
in_files = <undefined>

Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 524, in run
    result = self._run_interface(execute=True)
  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 642, in _run_interface
    return self._run_command(execute)
  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 750, in _run_command
    raise NodeExecutionError(
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node concat_blips.

Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/nipype/interfaces/base/core.py", line 398, in run
    runtime = self._run_interface(runtime)
  File "/opt/conda/lib/python3.9/site-packages/niworkflows/interfaces/nibabel.py", line 227, in _run_interface
    img_4d = nb.concat_images(
  File "/opt/conda/lib/python3.9/site-packages/nibabel/funcs.py", line 141, in concat_images
    raise ValueError(f'Affine for image {i} does not match affine for first image')
ValueError: Affine for image 5 does not match affine for first image

Additional information / screenshots

Here is some sample code demonstrating the magnitude of the difference

import nibabel as nib

loadedF = [nib.load(k) for k in allFiles]

loadedF[0].affine
Out[5]: 
array([[ -1.625     ,   0.        ,  -0.        , 101.31230164],
       [ -0.        ,   1.625     ,  -0.        , -99.4499054 ],
       [  0.        ,   0.        ,   2.        , -38.88479996],
       [  0.        ,   0.        ,   0.        ,   1.        ]])

loadedF[0].affine == loadedF[5].affine
Out[6]: 
array([[ True,  True,  True,  True],
       [ True,  True,  True,  True],
       [ True,  True,  True, False],
       [ True,  True,  True,  True]])

loadedF[0].affine - loadedF[5].affine
Out[7]: 
array([[0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00],
       [0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00],
       [0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 3.81469727e-06],
       [0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00]])

[cmpetty]

This fix from here this issue is not present in : niworkflows/interfaces/nibabel.py

        #img_4d = nb.concat_images(
        #    nii_list,
        #    check_affines=not bool(self.inputs.affine_tolerance)
        #)
        img_4d = nb.concat_images(nii_list, check_affines=False)

i added it locally and v22.0.0 runs without the concat_blips error

[effigies]

Thanks for identifying this. It turns out that while the fix in niworkflows was in both versions, the patch in sdcflows did not get merged into the development branch. This should be fixed in the next release of fMRIPrep.

[smazurchuk]

thanks @effigies . Without seeing a new pull request, is it safe to assume that this is fixed and I should close this, or are you saying that you will link a commit and I should close it after?

[effigies]

It's fixed in sdcflows, but now we're having trouble releasing: nipreps/sdcflows#275

When there is a new release, this will be resolved.

[effigies]

Pushed new sdcflows to PyPI manually. cc @mgxd

[jsmentch]

we've also been encountering this error - any idea roughly when the next release will be?

[effigies]

Tonight/tomorrow.

[mgxd]

22.0.1 is now available