Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add choice of run or exec wrapper behaviours #60

Merged
merged 1 commit into from
Apr 27, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docker/irods_clients/ubuntu/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,7 @@ COPY --from=installer /usr/local /usr/local
COPY --from=singularity ./scripts/* /usr/local/bin/
COPY --from=installer /opt/docker/irods_clients/manifest.json /usr/local/irods_clients/etc/manifest.json
COPY --from=installer /opt/docker/irods_clients/scripts/docker-entrypoint.sh /usr/local/irods_clients/bin/docker-entrypoint.sh
COPY --from=installer /opt/docker/irods_clients/scripts/singularity-* /usr/local/bin/

# Configure the singularity-wrapper script
ENV MANIFEST_PATH="/usr/local/irods_clients/etc/manifest.json"
Expand Down
10 changes: 10 additions & 0 deletions singularity/scripts/singularity-exec-docker
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/bin/bash

set -euo pipefail

DOCKER_REGISTRY=${DOCKER_REGISTRY:?required, but not set}
DOCKER_USER=${DOCKER_USER:?required, but not set}
DOCKER_IMAGE=${DOCKER_IMAGE:?required, but was not set}
DOCKER_TAG=${DOCKER_TAG:?required, but not set}

singularity exec "docker://$DOCKER_REGISTRY/$DOCKER_USER/$DOCKER_IMAGE:$DOCKER_TAG" "$@"
16 changes: 2 additions & 14 deletions singularity/scripts/singularity-run-docker
Original file line number Diff line number Diff line change
@@ -1,22 +1,10 @@
#!/bin/bash
#
# Running this script will start an instance of the container, if one is not
# already running, and then execute the command in the instance.
#
# Cleaning up unwanted instances afterwards is outside the scope of this script.

set -euo pipefail

DOCKER_REGISTRY=${DOCKER_REGISTRY:?required, but not set}
DOCKER_USER=${DOCKER_USER:?required, but not set}
DOCKER_IMAGE=${DOCKER_IMAGE:?required, but was not set}
DOCKER_TAG=${DOCKER_TAG:?required, but not set}

# Colons and slashes are not allowed in Singularity instance names
instance="$DOCKER_REGISTRY--$DOCKER_USER--$DOCKER_IMAGE--$DOCKER_TAG"

if ! singularity instance list | grep "$instance" 2>&1 >/dev/null ; then
singularity instance start \
"docker://$DOCKER_REGISTRY/$DOCKER_USER/$DOCKER_IMAGE:$DOCKER_TAG" "$instance" 2>&1 >/dev/null
fi

singularity exec "instance://$instance" "$@"
singularity run "docker://$DOCKER_REGISTRY/$DOCKER_USER/$DOCKER_IMAGE:$DOCKER_TAG" "$@"
22 changes: 22 additions & 0 deletions singularity/scripts/singularity-service-docker
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#!/bin/bash
#
# Running this script will start an instance of the container, if one is not
# already running, and then execute the command in the instance.
#
# Cleaning up unwanted instances afterwards is outside the scope of this script.
set -euo pipefail

DOCKER_REGISTRY=${DOCKER_REGISTRY:?required, but not set}
DOCKER_USER=${DOCKER_USER:?required, but not set}
DOCKER_IMAGE=${DOCKER_IMAGE:?required, but was not set}
DOCKER_TAG=${DOCKER_TAG:?required, but not set}

# Colons and slashes are not allowed in Singularity instance names
instance="$DOCKER_REGISTRY--$DOCKER_USER--$DOCKER_IMAGE--$DOCKER_TAG"

if ! singularity instance list | grep "$instance" 2>&1 | logger -p user.notice -t singularity-service-docker ; then
singularity instance start \
"docker://$DOCKER_REGISTRY/$DOCKER_USER/$DOCKER_IMAGE:$DOCKER_TAG" "$instance" 2>&1 | logger -p user.notice -t singularity-service-docker
fi

singularity exec "instance://$instance" "$@"
39 changes: 31 additions & 8 deletions singularity/scripts/singularity-wrapper
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,22 @@ usage() {
Install singularity proxy wrappers for the executables listed in a
container's manifest to a nominated directory.

Usage: $0 [-h] [-i <Docker image name>]
Two different types of wrapper are available which differ in how
they run singularity. The default type uses "singularity run",
while the alternative type uses "singularity exec".

If the "exec" type is used, the wrapper can additionally be made
to create a long-running service instance on first use and
subsequently "exec" within that instance.

Usage: $0
[-e]
[-h]
[-i <Docker image name>]
[-m <JSON manifest path>]
[-p <wrapper install prefix>]
[-r <Docker registry name>]
[-s]
[-t <Docker image tag>]
[-u <Docker user name>] [-v] <operation>

Expand All @@ -40,6 +52,8 @@ Operation may be one of:
Options:

-h Print usage and exit.
-e Use "singularity exec", rather than "singularity run" in the
generated wrappers.
-i Docker image name. Required, defaults to the value of the
environment variable "\$DOCKER_IMAGE" ("$DOCKER_IMAGE").
-m Manifest file path. Optional, defaults to the value of the
Expand All @@ -48,6 +62,7 @@ Options:
environment variable \$PREFIX ("$PREFIX").
-r Docker registry name. Optional, defaults to the value of the
environment variable \$DOCKER_REGISTRY ("$DOCKER_REGISTRY").
-s Start a long-running service instance (implies use of exec).
-t Docker image tag. Optional, defaults to the value of the
environment variable \$DOCKER_TAG ("$DOCKER_TAG").
-u Docker user name. Optional, defaults to the value of the
Expand Down Expand Up @@ -86,7 +101,7 @@ export DOCKER_TAG="$DOCKER_TAG"
# time, be permanently set in the installed wrapper. E.g. a candidate
# is SINGULARITY_CACHEDIR.

"\$(dirname "\${BASH_SOURCE[0]}")/singularity-run-docker" "$exe" "\$@"
"\$(dirname "\${BASH_SOURCE[0]}")/$singularity_wrap_impl" "$exe" "\$@"
EOF

chmod +x "$dir/$exe"
Expand All @@ -96,7 +111,7 @@ chmod +x "$dir/$exe"
install_wrappers() {
local dir="$PREFIX/bin"
install -d "$dir"
cp /usr/local/bin/singularity-run-docker "$PREFIX/bin"
cp "/usr/local/bin/$singularity_wrap_impl" "$PREFIX/bin"

for exe in "${wrappers[@]}" ; do
write_wrapper "$dir" "$exe"
Expand All @@ -111,8 +126,13 @@ DOCKER_TAG=${DOCKER_TAG:-latest}
PREFIX=${PREFIX:-/opt/wtsi-npg}
MANIFEST_PATH=${MANIFEST_PATH:-"$PREFIX/etc/manifest.json"}

while getopts "hi:m:p:r:t:u:v" option; do
singularity_wrap_impl="singularity-run-docker"

while getopts "hei:m:p:r:st:u:v" option; do
case "$option" in
e)
singularity_wrap_impl="singularity-exec-docker"
;;
h)
usage
exit 0
Expand All @@ -129,6 +149,9 @@ while getopts "hi:m:p:r:t:u:v" option; do
r)
DOCKER_REGISTRY="$OPTARG"
;;
s)
singularity_wrap_impl="singularity-service-docker"
;;
t)
DOCKER_TAG="$OPTARG"
;;
Expand Down Expand Up @@ -156,14 +179,14 @@ fi

wrappers=($(jq -j '.executable[] + " "' "$MANIFEST_PATH"))

OPERATION="$@"
if [ -z "$OPERATION" ] ; then
operation="$@"
if [ -z "$operation" ] ; then
usage
echo -e "\nERROR:\n An operation argument is required"
exit 4
fi

case "$OPERATION" in
case "$operation" in
list)
print_manifest
exit 0
Expand All @@ -173,7 +196,7 @@ case "$OPERATION" in
;;
*)
usage
echo -e "\nERROR:\n Invalid wrapper operation '$OPERATION'"
echo -e "\nERROR:\n Invalid wrapper operation '$operation'"
exit 4
;;
esac