Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Docker attestation and provenance #445

Merged
merged 3 commits into from
Feb 23, 2024
Merged

Add Docker attestation and provenance #445

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4e45d83
Dockerfile
dkuegler Jan 31, 2024
9123515
Docker/Dockerfile
dkuegler Feb 19, 2024
0f08ecd
build.py
dkuegler Feb 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 39 additions & 10 deletions Docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,12 @@ ARG FREESURFER_BUILD_IMAGE=build_freesurfer
ARG CONDA_BUILD_IMAGE=build_conda
ARG RUNTIME_BASE_IMAGE=ubuntu:22.04
ARG BUILD_BASE_IMAGE=ubuntu:22.04
# BUILDKIT_SBOM:SCAN_CONTEXT enables buildkit to provide and scan build images
# this is active by default to provide proper SBOM manifests, however, it may also
# include parts that are not part of the distributed image (specifically build image
# parts installed in the build image, but not transfered to the runtime image such as
# git, wget, the miniconda installer, etc.)
ARG BUILDKIT_SBOM_SCAN_CONTEXT=true

## Start with ubuntu base to build the conda base stage
FROM $BUILD_BASE_IMAGE AS build_base
Expand Down Expand Up @@ -87,7 +93,8 @@ COPY ./env/fastsurfer.yml ./Docker/conda_pack.sh ./Docker/install_env.py /instal
# Install conda for gpu

ARG DEBUG=false
RUN python /install/install_env.py -m base -i /install/fastsurfer.yml -o /install/base-env.yml && \
RUN python /install/install_env.py -m base -i /install/fastsurfer.yml \
-o /install/base-env.yml && \
mamba env create -f "/install/base-env.yml" | tee /install/env-create.log ; \
if [ "${DEBUG}" != "true" ]; then \
rm /install/base-env.yml ; \
Expand All @@ -98,8 +105,10 @@ FROM build_common AS build_conda
ARG DEBUG=false
ARG DEVICE=cu118
# install additional packages for cuda/rocm/cpu
RUN python /install/install_env.py -m ${DEVICE} -i /install/fastsurfer.yml -o /install/${DEVICE}-env.yml && \
mamba env update -n "fastsurfer" -f "/install/${DEVICE}-env.yml" | tee /install/env-update.log && \
RUN python /install/install_env.py -m ${DEVICE} -i /install/fastsurfer.yml \
-o /install/${DEVICE}-env.yml && \
mamba env update -n "fastsurfer" -f "/install/${DEVICE}-env.yml" \
| tee /install/env-update.log && \
/install/conda_pack.sh "fastsurfer" && \
echo "DEBUG=$DEBUG\nDEVICE=$DEVICE\n" > /install/build_conda.args ; \
if [ "${DEBUG}" != "true" ]; then \
Expand All @@ -114,8 +123,10 @@ FROM build_base AS build_freesurfer
COPY ./Docker/install_fs_pruned.sh /install/
SHELL ["/bin/bash", "--login", "-c"]

ARG FREESURFER_URL=default

# install freesurfer and point to new python location
RUN /install/install_fs_pruned.sh /opt --upx && \
RUN /install/install_fs_pruned.sh /opt --upx --url $FREESURFER_URL && \
rm /opt/freesurfer/bin/fspython && \
rm -R /install && \
ln -s /venv/bin/python3 /opt/freesurfer/bin/fspython
Expand All @@ -124,7 +135,8 @@ ln -s /venv/bin/python3 /opt/freesurfer/bin/fspython
# =======================================================
# Here, we create references to the requested build image
# =======================================================
# This is needed because COPY --from=<image/stage> does not accept variables as part of the image/stage name
# This is needed because COPY --from=<image/stage> does not accept variables as part of
# the image/stage name
# selected_freesurfer_build_image -> $FREESURFER_BUILD_IMAGE
FROM $FREESURFER_BUILD_IMAGE AS selected_freesurfer_build_image
# selected_conda_build_image -> $CONDA_BUILD_IMAGE
Expand Down Expand Up @@ -169,7 +181,8 @@ SHELL ["/bin/bash", "--login", "-c"]

# Copy fastsurfer venv and pruned freesurfer from build images

# Note, since COPY does not support variables in the --from parameter, so we point to a reference here, and the
# Note, since COPY does not support variables in the --from parameter, so we point to a
# reference here, and the
# seletced_<name>_build_image is a only a reference to $<NAME>_BUILD_IMAGE
COPY --from=selected_freesurfer_build_image /opt/freesurfer /opt/freesurfer
COPY --from=selected_conda_build_image /venv /venv
Expand All @@ -180,15 +193,31 @@ COPY . /fastsurfer/
ENV PYTHONPATH=/fastsurfer:/opt/freesurfer/python/packages:$PYTHONPATH \
FASTSURFER_HOME=/fastsurfer

# Download all remote network checkpoints already, compile all FastSurfer scripts into bytecode and update
# the build file with checkpoints md5sums and pip packages.
# Download all remote network checkpoints already, compile all FastSurfer scripts into
# bytecode and update the build file with checkpoints md5sums and pip packages.
RUN cd /fastsurfer ; python3 FastSurferCNN/download_checkpoints.py --all && \
python3 -m compileall * && \
python3 FastSurferCNN/version.py --sections +git+checkpoints+pip --build_cache BUILD.info -o fullBUILD.info && \
python3 FastSurferCNN/version.py --sections +git+checkpoints+pip \
--build_cache BUILD.info -o fullBUILD.info && \
mv fullBUILD.info BUILD.info

# TODO: SBOM info of FastSurfer and FreeSurfer are missing, it is unclear how to add
# those at the moment, as the buildscanner syft does not find simple package.json
# or pyproject.toml files right now. The easiest option seems to be to "setup"
# fastsurfer and freesurfer via pip instll.
#ENV BUILDKIT_SCAN_SOURCE_EXTRAS="/fastsurfer"
#ARG BUILDKIT_SCAN_SOURCE_EXTRAS="/fastsurfer"
#RUN <<EOF > /fastsurfer/package.json
#{
# "name": "fastsurfer",
# "version": "$(python3 FastSurferCNN/version.py)",
# "author": "David Kügler <david.kuegler@dzne.de>"
#}
#EOF

# Set FastSurfer workdir and entrypoint
# the script entrypoint ensures that our conda env is active
USER nonroot
WORKDIR "/fastsurfer"
ENTRYPOINT ["/fastsurfer/Docker/entrypoint.sh","/fastsurfer/run_fastsurfer.sh"]
CMD ["--help"]
Expand All @@ -199,4 +228,4 @@ FROM runtime as runtime_cuda

ENV NVIDIA_VISIBLE_DEVICES=all \
NVIDIA_DRIVER_CAPABILITIES=compute,utility \
NVIDIA_REQUIRE_CUDA="cuda>=8.0"
NVIDIA_REQUIRE_CUDA="cuda>=8.0"
51 changes: 48 additions & 3 deletions Docker/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,13 @@ docker run --gpus all -v /home/user/my_mri_data:/data \
* `-v`: This commands mount your data, output and directory with the FreeSurfer license file into the docker container. Inside the container these are visible under the name following the colon (in this case /data, /output, and /fs_license).
* `--rm`: The flag takes care of removing the container once the analysis finished.
* `-d`: This is optional. You can add this flag to run in detached mode (no screen output and you return to shell)
* `--user $(id -u):$(id -g)`: This part automatically runs the container with your group- (id -g) and user-id (id -u). All generated files will then belong to the specified user. Without the flag, the docker container will be run as root which is strongly discouraged.
* `--user $(id -u):$(id -g)`: Run the container with your account (your user-id and group-id), which are determined by `$(id -u)` and `$(id -g)`, respectively. Running the docker container as root `-u 0:0` is strongly discouraged.

##### Advanced Docker Flags
* `--group-add <list of groups>`: If additional user groups are required to access files, additional groups may be added via `--group-add <group id>[,...]` or `--group-add $(id -G <group name>)`.

##### FastSurfer Flags
* The `--fs_license` points to your FreeSurfer license which needs to be available on your computer in the my_fs_license_dir that was mapped above.
* The `--fs_license` points to your FreeSurfer license which needs to be available on your computer in the `my_fs_license_dir` that was mapped above.
* The `--t1` points to the t1-weighted MRI image to analyse (full path, with mounted name inside docker: /home/user/my_mri_data => /data)
* The `--sid` is the subject ID name (output folder name)
* The `--sd` points to the output directory (its mounted name inside docker: /home/user/my_fastsurfer_analysis => /output)
Expand Down Expand Up @@ -69,7 +72,10 @@ The build script `build.py` supports additional args, targets and options, see `
Note, that the build script's main function is to select parameters for build args, but also create the FastSurfer-root/BUILD.info file, which will be used by FastSurfer to document the version (including git hash of the docker container). This BUILD.info file must exist for the docker build to be successful.
In general, if you specify `--dry_run` the command will not be executed but sent to stdout, so you can run `python build.py --device cuda --dry_run | bash` as well. Note, that build.py uses some dependencies from FastSurfer, so you will need to set the PYTHONPATH environment variable to the FastSurfer root (include of `FastSurferCNN` must be possible) and we only support Python 3.10.

By default, the build script will tag your image as "fastsurfer:{version_tag}[-{device}]", where {version_tag} is {version-identifer from pyproject.toml}_{current git-hash} and {device} is the value to --device (and omitted for cuda), but a custom tag can be specified by `--tag {tag_name}`.
By default, the build script will tag your image as `"fastsurfer:{version_tag}[-{device}]"`, where `{version_tag}` is `{version-identifer from pyproject.toml}_{current git-hash}` and `{device}` is the value to `--device` (and omitted for cuda), but a custom tag can be specified by `--tag {tag_name}`.

#### BuildKit
Note, we recommend using BuildKit to build docker images (e.g. `DOCKER_BUILDKIT=1` -- the build.py script already always adds this). To install BuildKit, run `wget -qO ~/.docker/cli-plugins/docker-buildx https://github.com/docker/buildx/releases/download/<version>/buildx-<version>.<platform>`, for example `wget -qO ~/.docker/cli-plugins/docker-buildx https://github.com/docker/buildx/releases/download/v0.12.1/buildx-v0.12.1.linux-amd64`. See also https://github.com/docker/buildx#manual-download.

### Example 1: Build GPU FastSurfer Image

Expand Down Expand Up @@ -156,3 +162,42 @@ docker run --rm --cap-add=SYS_PTRACE --security-opt seccomp=unconfined \
--sid subjectX --sd /output
```

## Build docker image with attestation and provenance

To build a docker image with attestation and provenance, i.e. Software Bill Of Materials (SBOM) information, several requirements have to be met:

1. The image must be built with version v0.11+ of BuildKit (we recommend you [install BuildKit](#buildkit) independent of attestation).
2. You must configure a docker-container builder in buildx (`docker buildx create --use --bootstrap --name fastsurfer-bctx --driver docker-container`). Here, you can add additional configuration options such as safe registries to the builder configuration (add `--config /etc/buildkitd.toml`).
```toml
root = "/path/to/data/for/buildkit"
[worker.containerd]
gckeepstorage=9000
[[worker.containerd.gcpolicy]]
keepBytes = 512000000
keepDuration = 172800
filters = [ "type==source.local", "type==exec.cachemount", "type==source.git.checkout"]
[[worker.containerd.gcpolicy]]
all = true
keepBytes = 1024000000
# settings to push to a "local", registry with self-signed certificates
# see for example https://tech.paulcz.net/2016/01/secure-docker-with-tls/ https://github.com/paulczar/omgwtfssl
[registry."host:5000"]
ca=["/path/to/registry/ssl/ca.pem"]
[[registry."landau.dzne.ds:5000".keypair]]
key="/path/to/registry/ssl/key.pem"
cert="/path/to/registry/ssl/cert.pem"
```
3. Attestation files are not supported by the standard docker image storage driver. Therefore, images cannot be tested locally.
There are two solutions to this limitation.
1. Directly push to the registry:
Add `--action push` to the build script (the default is `--action load`, which loads the created image into the current docker context, and for the image name, also add the registry name. For example `... python Docker/build.py ... --attest --action push --tag docker.io/<myaccount>/fastsurfer:latest`.
2. [Install the containerd image storage driver](https://docs.docker.com/storage/containerd/#enable-containerd-image-store-on-docker-engine), which supports attestation: To implement this on Linux, make sure your docker daemon config file `/etc/docker/daemon.json` includes
```json
{
"features": {
"containerd-snapshotter": true
}
}
```
Also note, that the image storage location with containerd is not defined by the docker config file `/etc/docker/daemon.json`, but by the containerd config `/etc/containerd/config.toml`, which will likely not exist. You can [create a default config](https://github.com/containerd/containerd/blob/main/docs/getting-started.md#customizing-containerd) file with `containerd config default > /etc/containerd/config.toml`, in this config file edit the `"root"`-entry (default value is `/var/lib/containerd`).
4. Finally, you can now build the FastSurfer image with `python Docker/build.py ... --attest`. This will add the additional flags to the docker build command.
Loading