Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The Dockerfile is missing installation of dwidenoise from Mrtrix3 to run the dwi workflow #1175

Closed
celprov opened this issue Jan 25, 2024 · 2 comments · Fixed by #1196
Closed
Labels

Comments

@celprov
Copy link
Collaborator

celprov commented Jan 25, 2024

What happened?

I built the singularity image from the Dockerfile as it currently is in nipreps/mriqc:master and tried to run it on our dense-sampling dataset containing DWI images.

But that image is missing the installation of dwidenoise to run the dwi workflow of MRIQC.
dwidenoise is a package from mrtrix3.

The Dockerfile thus needs to be adapted to include this package.

What command did you use?

docker build \
-t local/mriqc:latest \
--build-arg BUILD_DATE=`date -u +"%Y-%m-%dT%H:%M:%SZ"` \
--build-arg VCS_REF=`git rev-parse --short HEAD` \
--build-arg VERSION="23.2.0" /data/code/mriqc

docker run --privileged -t --rm \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v /absolute/path/to/output/folder:/output \
    singularityware/docker2singularity \
    local/mriqc:latest

singularity run -e -B $DATADIR:/data:ro \
-B ${WORKDIR}:/work \
-B ${OUTDIR}:/out \ 
/oak/stanford/groups/russpold/users/cprovins/singularity_images/local-mriqc-23.2.0.simg \
/data /out participant \
-w /work --session-id ${SES#ses-} \
--verbose-reports --nprocs 4 --mem 40G --omp-nthreads 8 -vv

What version of the software are you running?

nipreps/mriqc:master

How are you running this software?

Singularity

Is your data BIDS valid?

Yes

Are you reusing any previously computed results?

No

Please copy and paste any relevant log output.

File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node dwidenoise.

Cmdline:
	dwidenoise -mask /work/mriqc_wf/dwiMRIQC/synthstrip_wf/_in_file_..data..sub-001..ses-001..dwi..sub-001_ses-001_acq-highres_dir-LR_dwi.nii.gz/synthstrip/clipped_corrected_desc-brain_mask.nii.gz -noise noisemap.nii.gz -nthreads 8 /work/mriqc_wf/dwiMRIQC/_in_file_..data..sub-001..ses-001..dwi..sub-001_ses-001_acq-highres_dir-LR_dwi.nii.gz/dti_filter/sub-001_ses-001_acq-highres_dir-LR_dwi_valid_nodriftfull_dti.nii.gz sub-001_ses-001_acq-highres_dir-LR_dwi_valid_nodriftfull_dti_denoised.nii.gz
Stdout:

Stderr:

Traceback:
	Traceback (most recent call last):
	  File "/opt/conda/lib/python3.9/site-packages/nipype/interfaces/base/core.py", line 397, in run
	    runtime = self._run_interface(runtime)
	  File "/opt/conda/lib/python3.9/site-packages/nipype/interfaces/base/core.py", line 752, in _run_interface
	    raise IOError(
	OSError: No command "dwidenoise" found on host sh02-06n01.int. Please check that the corresponding package is installed.

Additional information / screenshots

No response

@celprov celprov added the bug label Jan 25, 2024
@effigies
Copy link
Member

https://www.mrtrix.org/download/linux-anaconda/

Install mrtrix3 from the mrtrix3 channel.

conda install -c mrtrix3 mrtrix3

@effigies
Copy link
Member

Installing on its own has a lot of dependencies
❯ mamba create -n mrtrix3 -c mrtrix3 mrtrix3

Looking for: ['mrtrix3']

mrtrix3/linux-64 (check zst)                        Checked  0.2s
mrtrix3/noarch (check zst)                          Checked  0.1s
conda-forge/linux-64                                        Using cache
conda-forge/noarch                                          Using cache
pkgs/main/linux-64                                          Using cache
pkgs/main/noarch                                            Using cache
pkgs/r/linux-64                                             Using cache
pkgs/r/noarch                                               Using cache
mrtrix3/noarch                                     116.0 B @ 455.0 B/s  0.3s
mrtrix3/linux-64                                   797.0 B @   1.8kB/s  0.4s
Transaction

  Prefix: /home/chris/mambaforge/envs/mrtrix3

  Updating specs:

   - mrtrix3


  Package                           Version  Build               Channel           Size
─────────────────────────────────────────────────────────────────────────────────────────
  Install:
─────────────────────────────────────────────────────────────────────────────────────────

  + _libgcc_mutex                       0.1  conda_forge         conda-forge     Cached
  + libstdcxx-ng                     13.2.0  h7e041cc_4          conda-forge     Cached
  + ca-certificates              2023.11.17  hbcca054_0          conda-forge     Cached
  + ld_impl_linux-64                   2.40  h41732ed_0          conda-forge     Cached
  + libgomp                          13.2.0  h807b86a_4          conda-forge      422kB
  + _openmp_mutex                       4.5  2_gnu               conda-forge     Cached
  + libgcc-ng                        13.2.0  h807b86a_4          conda-forge     Cached
  + xorg-xf86vidmodeproto             2.3.1  h7f98852_1002       conda-forge     Cached
  + xorg-kbproto                      1.0.7  h7f98852_1002       conda-forge     Cached
  + xorg-xproto                      7.0.31  h7f98852_1007       conda-forge     Cached
  + xorg-damageproto                  1.2.1  h7f98852_1002       conda-forge       25kB
  + xorg-renderproto                 0.11.1  h7f98852_1002       conda-forge     Cached
  + xorg-randrproto                   1.5.0  h7f98852_1001       conda-forge       33kB
  + xorg-recordproto                 1.14.2  h7f98852_1002       conda-forge        8kB
  + xorg-inputproto                   2.3.2  h7f98852_1002       conda-forge       20kB
  + graphite2                        1.3.13  h58526e2_1001       conda-forge     Cached
  + xorg-libxdmcp                     1.1.3  h7f98852_0          conda-forge     Cached
  + pthread-stubs                       0.4  h36c2ea0_1001       conda-forge     Cached
  + libxcrypt                        4.4.36  hd590300_1          conda-forge     Cached
  + ncurses                             6.4  h59595ed_2          conda-forge     Cached
  + xorg-compositeproto               0.4.2  h7f98852_1001       conda-forge       12kB
  + libnsl                            2.0.1  hd590300_0          conda-forge     Cached
  + attr                              2.5.1  h166bdaf_1          conda-forge     Cached
  + mpg123                           1.32.4  h59595ed_0          conda-forge     Cached
  + libogg                            1.3.4  h7f98852_1          conda-forge     Cached
  + lame                              3.100  h166bdaf_1003       conda-forge     Cached
  + lz4-c                             1.9.4  hcb278e6_0          conda-forge     Cached
  + libopus                           1.3.1  h7f98852_1          conda-forge     Cached
  + xorg-util-macros                 1.19.3  h7f98852_0          conda-forge       52kB
  + xorg-xextproto                    7.3.0  h0b41bf4_1003       conda-forge     Cached
  + pixman                           0.43.2  h59595ed_0          conda-forge      387kB
  + openssl                           3.2.0  hd590300_1          conda-forge     Cached
  + libiconv                           1.17  hd590300_2          conda-forge     Cached
  + libffi                            3.4.2  h7f98852_5          conda-forge     Cached
  + giflib                            5.2.1  h0b41bf4_3          conda-forge     Cached
  + xorg-libxau                      1.0.11  hd590300_0          conda-forge     Cached
  + bzip2                             1.0.8  hd590300_5          conda-forge     Cached
  + keyutils                          1.6.1  h166bdaf_0          conda-forge     Cached
  + libuuid                          2.38.1  h0b41bf4_0          conda-forge     Cached
  + xorg-libice                       1.1.1  hd590300_0          conda-forge     Cached
  + gettext                          0.21.1  h27087fc_0          conda-forge     Cached
  + icu                                73.2  h59595ed_0          conda-forge     Cached
  + nspr                               4.35  h27087fc_0          conda-forge     Cached
  + libexpat                          2.5.0  hcb278e6_1          conda-forge     Cached
  + alsa-lib                         1.2.10  hd590300_0          conda-forge     Cached
  + libgfortran5                     13.2.0  ha4646dd_4          conda-forge     Cached
  + libjpeg-turbo                     3.0.0  hd590300_1          conda-forge     Cached
  + libzlib                          1.2.13  hd590300_5          conda-forge     Cached
  + libwebp-base                      1.3.2  hd590300_0          conda-forge     Cached
  + libdeflate                         1.19  hd590300_0          conda-forge     Cached
  + xz                                5.2.6  h166bdaf_0          conda-forge     Cached
  + lerc                              4.0.0  h27087fc_0          conda-forge     Cached
  + libedit                    3.1.20191231  he28a2e2_2          conda-forge     Cached
  + readline                            8.2  h8228510_1          conda-forge     Cached
  + libcap                             2.69  h0f662aa_0          conda-forge     Cached
  + libvorbis                         1.3.7  h9c3ff4c_0          conda-forge     Cached
  + xorg-fixesproto                     5.0  h7f98852_1002       conda-forge     Cached
  + mysql-common                     8.0.33  hf1915f5_6          conda-forge     Cached
  + libevent                         2.1.12  hf998b51_1          conda-forge     Cached
  + libxcb                             1.15  h0b41bf4_0          conda-forge     Cached
  + xorg-libsm                        1.2.4  h7391055_0          conda-forge     Cached
  + libgpg-error                       1.47  h71f35ed_0          conda-forge     Cached
  + libflac                           1.4.3  h59595ed_0          conda-forge     Cached
  + expat                             2.5.0  hcb278e6_1          conda-forge     Cached
  + libgfortran-ng                   13.2.0  h69a702a_4          conda-forge     Cached
  + tk                               8.6.13  noxft_h4845f30_101  conda-forge     Cached
  + pcre2                             10.42  hcad00b1_0          conda-forge     Cached
  + libsqlite                        3.44.2  h2797004_0          conda-forge     Cached
  + libpng                           1.6.39  h753d276_0          conda-forge     Cached
  + zlib                             1.2.13  hd590300_5          conda-forge     Cached
  + zstd                              1.5.5  hfc55251_0          conda-forge     Cached
  + libxml2                          2.12.4  h232c23b_1          conda-forge     Cached
  + krb5                             1.21.2  h659d440_0          conda-forge     Cached
  + xcb-util-wm                       0.4.1  h8ee46fc_1          conda-forge     Cached
  + xcb-util-keysyms                  0.4.0  h8ee46fc_1          conda-forge     Cached
  + xcb-util                          0.4.0  hd590300_1          conda-forge     Cached
  + xcb-util-renderutil               0.3.9  hd590300_1          conda-forge     Cached
  + xorg-libx11                       1.8.7  h8ee46fc_0          conda-forge     Cached
  + libgcrypt                        1.10.3  hd590300_0          conda-forge     Cached
  + libsndfile                        1.2.2  hc60ed4a_1          conda-forge     Cached
  + fftw                             3.3.10  nompi_hc118613_108  conda-forge     Cached
  + libglib                          2.78.3  h783c2da_0          conda-forge     Cached
  + nss                                3.97  h1d7d5a4_0          conda-forge     Cached
  + freetype                         2.12.1  h267a509_2          conda-forge     Cached
  + mysql-libs                       8.0.33  hca2cd23_6          conda-forge     Cached
  + libtiff                           4.6.0  ha9c0a0a_2          conda-forge     Cached
  + libllvm15                        15.0.7  hb3ce162_4          conda-forge     Cached
  + libpq                              16.1  h33b98f1_7          conda-forge     Cached
  + libcups                           2.3.3  h4637d8d_4          conda-forge     Cached
  + xcb-util-image                    0.4.0  h8ee46fc_1          conda-forge     Cached
  + xkeyboard-config                   2.40  hd590300_0          conda-forge     Cached
  + xorg-libxext                      1.3.4  h0b41bf4_2          conda-forge     Cached
  + xorg-libxfixes                    5.0.3  h7f98852_1004       conda-forge     Cached
  + xorg-libxrender                  0.9.11  hd590300_0          conda-forge     Cached
  + libsystemd0                         255  h3516f8a_0          conda-forge     Cached
  + glib-tools                       2.78.3  hfc55251_0          conda-forge     Cached
  + dbus                             1.13.6  h5008d03_3          conda-forge     Cached
  + fontconfig                       2.14.2  h14ed4e7_0          conda-forge     Cached
  + libwebp                           1.3.2  h658648e_1          conda-forge     Cached
  + libclang13                       15.0.7  default_ha2b6cf4_4  conda-forge     Cached
  + libxkbcommon                      1.6.0  hd429924_1          conda-forge     Cached
  + xorg-libxcomposite                0.4.6  h0b41bf4_1          conda-forge       13kB
  + xorg-libxi                       1.7.10  h7f98852_0          conda-forge       47kB
  + xorg-libxdamage                   1.1.5  h7f98852_1          conda-forge       12kB
  + xorg-libxrandr                    1.5.2  h7f98852_1          conda-forge       30kB
  + pulseaudio-client                  16.1  hb77b528_5          conda-forge     Cached
  + libclang                         15.0.7  default_hb11cfb5_4  conda-forge     Cached
  + xorg-libxtst                      1.2.3  h7f98852_1002       conda-forge       32kB
  + font-ttf-dejavu-sans-mono          2.37  hab24e00_0          conda-forge     Cached
  + font-ttf-inconsolata              3.000  h77eed37_0          conda-forge     Cached
  + font-ttf-source-code-pro          2.038  h77eed37_0          conda-forge     Cached
  + font-ttf-ubuntu                    0.83  h77eed37_1          conda-forge     Cached
  + tzdata                            2023d  h0c530f3_0          conda-forge     Cached
  + fonts-conda-forge                     1  0                   conda-forge     Cached
  + fonts-conda-ecosystem                 1  0                   conda-forge     Cached
  + python                           3.12.1  hab00c5b_1_cpython  conda-forge     Cached
  + cairo                            1.18.0  h3faef2a_0          conda-forge     Cached
  + harfbuzz                          8.3.0  h3d44ed6_0          conda-forge     Cached
  + wheel                            0.42.0  pyhd8ed1ab_0        conda-forge     Cached
  + setuptools                       69.0.3  pyhd8ed1ab_0        conda-forge     Cached
  + pip                              23.3.2  pyhd8ed1ab_0        conda-forge     Cached
  + glib                             2.78.3  hfc55251_0          conda-forge     Cached
  + gstreamer                        1.22.9  h98fc4e7_0          conda-forge     Cached
  + gst-plugins-base                 1.22.9  h8e1006c_0          conda-forge     Cached
  + qt-main                          5.15.8  h450f30e_18         conda-forge     Cached
  + qt-webengine                     5.15.8  h7517aa4_5          conda-forge       58MB
  + qt                               5.15.8  hf11cfaa_0          conda-forge       17kB
  + mrtrix3                           3.0.4  h2bc3f7f_0          mrtrix3           21MB

  Summary:

  Install: 128 packages

  Total download: 81MB

─────────────────────────────────────────────────────────────────────────────────────────


Confirm changes: [Y/n] y
xorg-randrproto                                     33.0kB @ 193.2kB/s  0.2s
xorg-recordproto                                     8.0kB @  45.6kB/s  0.2s
xorg-inputproto                                     19.6kB @ 111.6kB/s  0.2s
xorg-damageproto                                    25.4kB @ 119.2kB/s  0.2s
xorg-libxdamage                                     12.0kB @  50.2kB/s  0.1s
qt                                                  17.1kB @  66.9kB/s  0.1s
xorg-libxcomposite                                  13.4kB @  44.0kB/s  0.1s
xorg-libxtst                                        32.1kB @ 103.7kB/s  0.1s
xorg-util-macros                                    52.5kB @ 157.0kB/s  0.1s
libgomp                                            422.1kB @   1.3MB/s  0.3s
xorg-libxrandr                                      29.7kB @  77.1kB/s  0.1s
xorg-libxi                                          47.3kB @ 113.5kB/s  0.1s
xorg-compositeproto                                 12.4kB @  24.9kB/s  0.1s
pixman                                             386.8kB @ 737.1kB/s  0.2s
mrtrix3                                             21.1MB @   7.6MB/s  2.5s
qt-webengine                                        58.4MB @  17.8MB/s  3.1s

Downloading and Extracting Packages:

Preparing transaction: done
Verifying transaction: done
Executing transaction: done

To activate this environment, use

     $ mamba activate mrtrix3

To deactivate an active environment, use

     $ mamba deactivate

That's a lot of potential conflicts. We could do a build stage where we install and then copy, but maybe instead it makes sense to copy from their docker image: https://hub.docker.com/r/mrtrix3/mrtrix3/tags

# Tested, and we need at least these libraries to run dwidenoise --help
RUN apt update && apt install --no-install-recommends -y libtiff5 libpng16-16

copy --from=mrtrix3/mrtrix3:3.0.4 /opt/mrtrix3/bin/dwidenoise /usr/local/bin
copy --from=mrtrix3/mrtrix3:3.0.4 /opt/mrtrix3/lib/libmrtrix.so /usr/local/lib    

oesteban added a commit to oesteban/mriqc that referenced this issue Mar 18, 2024
Update the docker build procedure to leverage multi-stage builds and lay
the ground for nipreps#1175.
oesteban added a commit that referenced this issue Mar 19, 2024
Implements [@effigies'
solution](#1175 (comment))
leveraging Docker multi-staged builds.

Checked linked libraries within the Docker container:

```
$ ldd /usr/local/bin/dwidenoise
	linux-vdso.so.1 (0x00007fff6db65000)
	libmrtrix.so => /usr/local/bin/../lib/libmrtrix.so (0x00007f4e2b757000)
	libstdc++.so.6 => /opt/conda/lib/libstdc++.so.6 (0x00007f4e2b574000)
	libm.so.6 => /usr/lib/x86_64-linux-gnu/libm.so.6 (0x00007f4e2b48d000)
	libgcc_s.so.1 => /opt/conda/lib/libgcc_s.so.1 (0x00007f4e2b472000)
	libpthread.so.0 => /usr/lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f4e2b46d000)
	libc.so.6 => /usr/lib/x86_64-linux-gnu/libc.so.6 (0x00007f4e2b242000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f4e2bac2000)
	libz.so.1 => /opt/conda/lib/libz.so.1 (0x00007f4e2b227000)
	libtiff.so.5 => /usr/lib/x86_64-linux-gnu/libtiff.so.5 (0x00007f4e2b19f000)
	libpng16.so.16 => /opt/conda/lib/libpng16.so.16 (0x00007f4e2b162000)
	librt.so.1 => /usr/lib/x86_64-linux-gnu/librt.so.1 (0x00007f4e2b15d000)
	libwebp.so.7 => /opt/conda/lib/libwebp.so.7 (0x00007f4e2b0bb000)
	libzstd.so.1 => /opt/conda/lib/libzstd.so.1 (0x00007f4e2afa7000)
	liblzma.so.5 => /opt/conda/lib/liblzma.so.5 (0x00007f4e2af7e000)
	libjbig.so.0 => /usr/lib/x86_64-linux-gnu/libjbig.so.0 (0x00007f4e2af6d000)
	libjpeg.so.8 => /opt/conda/lib/libjpeg.so.8 (0x00007f4e2ae87000)
	libdeflate.so.0 => /opt/conda/lib/libdeflate.so.0 (0x00007f4e2ae71000)
	libsharpyuv.so.0 => /opt/conda/lib/./libsharpyuv.so.0 (0x00007f4e2ae65000)

ldd /usr/local/lib/libmrtrix.so
	linux-vdso.so.1 (0x00007ffc0a0b6000)
	libz.so.1 => /opt/conda/lib/libz.so.1 (0x00007f431b203000)
	libtiff.so.5 => /usr/lib/x86_64-linux-gnu/libtiff.so.5 (0x00007f431b17b000)
	libpng16.so.16 => /opt/conda/lib/libpng16.so.16 (0x00007f431b13e000)
	libstdc++.so.6 => /opt/conda/lib/libstdc++.so.6 (0x00007f431af5b000)
	libm.so.6 => /usr/lib/x86_64-linux-gnu/libm.so.6 (0x00007f431ae74000)
	libgcc_s.so.1 => /opt/conda/lib/libgcc_s.so.1 (0x00007f431ae57000)
	libpthread.so.0 => /usr/lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f431ae52000)
	libc.so.6 => /usr/lib/x86_64-linux-gnu/libc.so.6 (0x00007f431ac29000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f431b4d5000)
	libwebp.so.7 => /opt/conda/lib/libwebp.so.7 (0x00007f431ab89000)
	libzstd.so.1 => /opt/conda/lib/libzstd.so.1 (0x00007f431aa75000)
	liblzma.so.5 => /opt/conda/lib/liblzma.so.5 (0x00007f431aa4c000)
	libjbig.so.0 => /usr/lib/x86_64-linux-gnu/libjbig.so.0 (0x00007f431aa39000)
	libjpeg.so.8 => /opt/conda/lib/libjpeg.so.8 (0x00007f431a953000)
	libdeflate.so.0 => /opt/conda/lib/libdeflate.so.0 (0x00007f431a93d000)
	librt.so.1 => /usr/lib/x86_64-linux-gnu/librt.so.1 (0x00007f431a938000)
	libsharpyuv.so.0 => /opt/conda/lib/./libsharpyuv.so.0 (0x00007f431a92e000)
```

And also tested *MRIQC* locally on DWI data.

Resolves: #1175.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants