Deployment scripts for CCTBX on various systems.
[[TOC]]
This repo makes extensive use of git submodules -- in order to ensure that all submodules are up-to-date, run:
git submodule update --init --recursive
Large resource files are tracked using git large file storage (LFS) -- in order to update all LFS objects, run:
git lfs install
git lfs pull
Note: in order to use LFS, you might need to install it:
- On macOS (using homebrew) run:
brew install git-lfs
- On NERSC Cori run:
module load git-lfs
CCTBX can be linked into the $CONDA_PREFIX
-- avoiding the need for the
dispatcher scripts. This is necessary if the computing environment reacts
badly to modifications of $LD_LIBRARY_PATH
. In this case, you need to run:
./opt/link_to_conda.sh
NOTE: you need to run source opt/cctbx/build/unsetpaths.sh
whenever you
source'ed opt/activate.sh
IF you don't want the have the dispatchers in your
$PATH
-- but this is not strictly necessary.
This deployment script is a little different for containerized workflows. Building (and running) docker images is described here in more detail.
To build, run:
./install_all.sh
This does the following:
- Ensure that all git submodules have been initialized
- Load site-specific modules/settings
- Install the local miniconda app
- Install the pipeline's conda environment
- Install CCTBX and psana2
If you're seeing libreadline.so
warnings, you can run the following to
silence these warnings:
pushd $CONDA_PREFIX/lib
ln -sf /lib64/libtinfo.so.6
popd
This is done automatically on NERSC's Cori system
For example, running the xtc processing pipeline involves 2 steps:
- Activate local environment (from repository root):
source opt/activate.sh
- Denpending on your compute environment, you might also need to load the required modules (this does not happen automatically as it is not always necessary):
source opt/env/load_modules.sh
- Run the xtc processing pipeline (from the
run
folder):
pushd
[srun|mpirun] ./index_lite.sh cxid9114 1 12 none 1000 ../data/tmp
popd
cf README_DOCKER.md for instructions on running inside a docker/shifter container.