This package collects various ways to build and use containers with Spack to build or run Wire-Cell Toolkit (WCT) software.
Here gives a menu of how to achieve various objectives.
sudo mkdir /srv/spack sudo chown $USER:$USER /srv/spack git clone https://github.com/spack/spack /srv/spack/local bash --rcfile /srv/spack/local/share/spack/setup-env.sh spack -h
Anticipating some future goals, do some configuration
spack config --scope defaults edit config
Consider:
padded_length:128- to give room in binaries for relocatable paths.
install_tree:root:- change to install outside of the git clone.
- build_stage:
- change if default temp storage areas are too small.
If using Spack on multiple hosts, it’s nice share the spoils of a build with a source mirror and binary cache. This takes some finagling with GPG keys and an extra command after a local build to populate the buildcache but will help you or others doing subsequent, identical builds.
spack mirror create -d /srv/spack/mirror xz spack mirror add local /srv/spack/mirror
See tutorial to create and add signing keys.
spack buildcache create --allow-root --force -d /srv/spack/mirror --only=package xz
To do this wholesale, see the for loop on spack find given in the
tutorial linked above.
A remote server is out of scope of the tutorial but easy enough. Here’s an example assuming Debian:
ssh server sudo mkdir -p /var/www/html/spack/mirror sudo chown $USER:$USER /var/www/html/spack/mirror exit rsync -av /srv/spack/mirror/ server:/var/www/html/spack/mirror/ spack mirror add server http://server/spack/mirror spack mirror rm local # to check the remote server is working
Other users can use this as a source mirror with no hassle. To use
the buildcache requires the public key which is in the server at the
URL path: /spack/mirror/build_cache/_pgp/*.pub. See the tutorial for
details.
You may want to use an OS other than your native one. The example here is to build a minimal Debian 11 (bullseye) Docker image and extract that to a Singularity image. We then run the native Spack through the container to populate with Debian 11 builds.
This uses ./dockerfiles/debian-prespack.df from this repository.
docker build -t debian-prespack:11.2 -f dockerfiles/debian-prespack.df . singularity build /srv/spack/debian-bullseye.sif docker-daemon://debian-prespack:11.2 mkdir /srv/debian-bullseys-home singularity exec \ --writable-tmpfs \ --bind /srv/spack/debian-bullseye-home:/home/bv \ --bind /srv/spack /srv/spack/debian-bullseye.sif \ bash --rcfile /srv/spack/local/share/spack/setup-env.sh Singularity> spack install xz Singularity> spack buildcache create --allow-root --force -d /srv/spack/mirror --only=package xz
The empty home directory is mounted to avoid polluting the container
with the native OS’s ~/.spack.
This needs wire-cell-spack and can be run in the native OS or in the
Singularity container.
mkdir /srv/spack/repos git clone https://github.com/WireCell/wire-cell-spack.git /srv/spack/repos/wct-spack spack repo add /srv/spack/repos/wct-spack spack info wire-cell-toolkit spack install wire-cell-toolkit
$ docker pull brettviren:wct-core $ docker run -it wct-core root@<hash>:~# wire-cell --help
See Details for understanding the names to use.
Singularity images or file based and running a container “sees” you home directory.
$ singularity build spack-base.sif docker-daemon://spack-base:0.17.1 $ ls -lh spack-base.sif -rwxr-xr-x 1 bv bv 265M Mar 15 16:32 spack-base.sif $ singularity run --env SPACK_ROOT=/opt/spack spack-base.sif interactive-shell $ spack -h
$ docker build -t <output-name> -f dockerfiles/<input-name>.df
The containers are built in major groups of layers starting with a Debian base and each with a Dockerfile under ./dockerfiles/.
This adds some Debian packages and adds Spack at the git submodule
version pointed to in this repo. In the image, Spack files are placed
under /opt/spack. This image could be used as a base for others built
with Spack.
$ docker build -t spack-base:0.17.1 -f dockerfiles/spack-base.df . $ docker run -it spack-base:0.17.1 root@909ec8b19b45:~# spack list wire-cell-toolkit ==> 0 packages.
This adds WCT Spack package.py’s and builds the packages needed for
WCT “core” including default options. This could be used as a base
for developing WCT core packages.
$ docker build -t wct-base:0.18.0 -f dockerfiles/wct-base.df . $ docker run -it wct-base:0.18.0 root@33c1283fb71a:~# spack load go-jsonnet root@33c1283fb71a:~# jsonnet --version Jsonnet commandline interpreter (Go implementation) v0.18.0
This builds a WCT release via Spack. It may be used to run core WCT jobs or as a basis for WCT core development.
$ docker build -t wct-core:0.18.0 -f dockerfiles/wct-core.df .
$ docker run -it wct-core:0.18.0
root@bf8cc5fd7982:~# spack load wire-cell-toolkit
root@bf8cc5fd7982:~# wire-cell --help
Options:
-h [ --help ] wire-cell [options] [arguments]
-l [ --logsink ] arg set log sink as <filename> or 'stdout' or 'stderr', a
log level for the sink may be given by appending
':<level>'
-L [ --loglevel ] arg set lowest log level for a log in form 'name:level' or
just give 'level' value for all (level one of:
critical,error,warn,info,debug,trace)
-a [ --app ] arg application component to invoke
-c [ --config ] arg provide a configuration file
-p [ --plugin ] arg specify a plugin as name[:lib]
-V [ --ext-str ] arg specify a Jsonnet external variable=<string>
-C [ --ext-code ] arg specify a Jsonnet external variable=<code>
-A [ --tla-str ] arg specify a Jsonnet top level arguments variable=<string>
--tla-code arg specify a Jsonnet top level arguments variable=<code>
-P [ --path ] arg add to JSON/Jsonnet search path
-t [ --threads ] arg limit number of threads used
-v [ --version ] print the compiled version to stdout
This builds the full set of non-default optional WCT dependencies using Spack but does not build WCT itself. It includes CUDA, HDF5, ZeroMQ, ROOT.
Builds WCT with all options.
This builds externals WCT extensions developed as part of the experimental HEP-CCE/PPS work. It includes Kokkos with various backends.
Breaking up the layers as above helps to produce images which may have multiple purposes and ease (re)build issue during their definition.
However, a more direct and monolithic construction can be created with
a Dockerfile like:
FROM spack-base RUN spack install wire-cell-toolkit@0.18.0+root+hdf+zmq+kokkos
This example will build the “full” WCT as well as all dependencies in a single, if rather long, construction.