eXist-db Docker Image

Minimal Docker Image of eXist-db NoSQL Database Client/Server with FO support

This module holds the source files for building a minimal docker image of the exist-db xml database, images are automatically updated as part of the build-test life-cycle. The images are based on Google Cloud Platform's "Distroless" Docker Images.

Requirements

• Docker: 18-stable

Build Requirements

• Docker: 19.03
• java: 11
• bats: ^1.1.0 (for testing)
• buildx

How to use

Pre-build images are available on DockerHub. There are two continuously updated channels:

• release for the stable releases based on the master branch
• latest for the latest commit to the develop branch.

Multi-arch Images are currently produced for linux/amd64 and linux/arm64.

To download tanhe image run:

docker pull duncdrum/existdb:latest

Once the download is complete, you can run the image

docker run -dit -p 8080:8080 -p 8443:8443 --name exist duncdrum/existdb:latest

What does this do?

• -it allocates a TTY and keeps STDIN open. This allows you to interact with the running Docker container via your console.
• -d detaches the container from the terminal that started it. So your container won't stop when you close the terminal.
• -p maps the Containers internal and external port assignments (we recommend sticking with matching pairs). This allows you to connect to the eXist-db Web Server running in the Docker container.
• --name lets you provide a name (instead of using a randomly generated one)

The only required parts are docker run duncdrum/existdb. For a full list of available options see the official Docker documentation

After running the pull and run commands, you can access eXist-db via localhost:8080 in your browser.

To stop the container issue:

docker stop exist

or if you omitted the -d flag earlier press CTRL-C inside the terminal showing the exist logs.

Interacting with the running container

You can interact with a running container as if it were a regular Linux host (without a shell in our case). You can issue shell-like commands to the Java admin client, as we do throughout this readme, but you can't open the shell in interactive mode.

The name of the container in this readme is exist, adjust the name in the commands to suit your needs:

# Using java syntax on a running eXist-db instances
docker exec exist java org.exist.start.Main client --no-gui --xpath "system:get-version()"

# Interacting with the JVM
docker exec exist java -version

Containers build from this image run periodical healthchecks to ensure that eXist-db is operating normally. If docker ps reports unhealthy you can get a more detailed report with this command:

docker inspect --format='{{json .State.Health}}' exist

Logging

There is a slight modification to eXist's logger to ease access to the logs via:

docker logs exist

This works best when providing the -t flag when running an image.

Use as base image

A common usage of these images is as a base image for your own applications. We'll take a quick look at three scenarios of increasing complexity, to demonstrate how to achieve common tasks from inside Dockerfile.

A simple app image

The simplest and straightforward case assumes that you have a .xar app inside a build folder on the same level as the Dockerfile. To get an image of an eXist-db instance with your app installed and running, simply adopt the docker cp ... command to the appropriate Dockerfile syntax.

FROM duncdrum/existdb:6.0.1

COPY build/*.xar /exist/autodeploy

You should see something like this:

Sending build context to Docker daemon  4.337MB
Step 1/2 : FROM duncdrum/existdb:6.0.1
 ---> 3f4dbbce9afa
Step 2/2 : COPY build/*.xar /exist/autodeploy
 ---> ace38b0809de

The result is a new image of your app installed into eXist-db. Since you didn't provide further instructions it will simply reuse the EXPOSE, CMD, HEALTHCHECK, etc instructions defined by the base image. You can now publish this image to a docker registry and share it with others.

A slightly more complex single stage image

The following example will install your app, but also modify the underlying eXist-db instance in which your app is running. Instead of a local build directory, we'll download the .xar from the web, and copy a modified conf.xml from a src/ directory along side your Dockerfile. To execute any of the docker exec … style commands from this readme, we need to use RUN.

FROM duncdrum/existdb

# NOTE: this is for syntax demo purposes only
RUN [ "java", "org.exist.start.Main", "client", "--no-gui",  "-l", "-u", "admin", "-P", "", "-x", "sm:passwd('admin','123')" ]

# use a modified conf.xml
COPY src/conf.xml /exist/etc

ADD https://github.com/eXist-db/documentation/releases/download/4.0.4/exist-documentation-4.0.4.xar /exist/autodeploy

The above is intended to demonstrate the kind of operations available to you in a single stage build. For security reasons more elaborate techniques for not sharing your password in the clear are highly recommended, such as the use of secure variables inside your CI environment. However, the above shows you how to execute the Java Admin Client from inside a Dockerfile, which in turn allows you to run any XQuery code you want when modifying the eXist-db instance that will ship with your images. You can also chain multiple RUN commands.

As for the sequence of the commands, those with the most frequent changes should come last to avoid cache busting. Chances are, you wouldn't change the admin password very often, but the .xar might change more frequently.

Multi-stage build with ant

Lastly, you can eliminate external dependencies even further by using a multi-stage build. To ensure compatibility between different Java engines we recommend sticking with debian based images for the builder stage.

The following 2-stage build will download and install ant and nodeJS into a builder stage which then downloads frontend dependencies before building the .xar file. The second stage (each FROM begins a stage) is just the simple example from above. Such a setup ensures that non of your collaborators has to have java or nodeJS installed, and is great for fully automated builds and deployment.

# START STAGE 1
FROM openjdk:8-jdk-slim as builder

USER root

ENV ANT_VERSION 1.10.5
ENV ANT_HOME /etc/ant-${ANT_VERSION}

WORKDIR /tmp

RUN wget http://www-us.apache.org/dist/ant/binaries/apache-ant-${ANT_VERSION}-bin.tar.gz \
    && mkdir ant-${ANT_VERSION} \
    && tar -zxvf apache-ant-${ANT_VERSION}-bin.tar.gz \
    && mv apache-ant-${ANT_VERSION} ${ANT_HOME} \
    && rm apache-ant-${ANT_VERSION}-bin.tar.gz \
    && rm -rf ant-${ANT_VERSION} \
    && rm -rf ${ANT_HOME}/manual \
    && unset ANT_VERSION

ENV PATH ${PATH}:${ANT_HOME}/bin

WORKDIR /home/my-app
COPY . .
RUN apk add --no-cache --virtual .build-deps \
 nodejs \
 nodejs-npm \
 git \
 && npm i npm@latest -g \
 && ant


# START STAGE 2
FROM duncdrum/existdb:release

COPY --from=builder /home/my-app/build/*.xar /exist/autodeploy

EXPOSE 8080 8443

CMD [ "java", "org.exist.start.Main", "jetty" ]

The basic idea of the multi-staging is that everything you need for building your software should be managed by docker, so that all collaborators can rely on one stable environment. In the end, and after how ever many stages you need, only the files necessary to run your app should go into the final stage. The possibilities are virtually endless, but with this example and the Dockerfile in this repo you should get a pretty good idea of how you might apply this idea to your own projects.

Development use via docker-compose

We highly recommend use of a docker-compose.yml for use with docker-compose. docker-compose for local development or integration into multi-container environments. For options on how to configure your own compose file, follow the link at the beginning of this paragraph.

To start exist using a compose file, type:

# starting eXist-db
docker-compose up -d
# stop eXist-db
docker-compose down

Volumes let you ensure data persistence between reboots, in particular:

• exist/data so that any database changes persist through reboots and updates.
• exist/etc so you can configure eXist startup options.

can be declared as mount volumes.

You can configure additional volumes e.g. for backups, or additional services such as an nginx reverse proxy via a docker-compose.yml, to suite your needs.

To update the exist-docker image from a newer version

docker-compose pull

Caveat

As with normal installations, the password for the default dba user admin is empty. Change it via the usermanager or from CLI (s.a.).

Building the Image

Testing

There are unit tests for our images that run on CI using the bats framework. The test are located in exist-docker/src/test/bats.

To execute them run:

bats exist-docker/src/test/bats/*.bats 

The tests use fixtures and are creating a modified image call ex-mod. By default they expect a name container exist-ci to be up and running. When running test locally you must ensure that no previous image ex-mod exists, and that exist-ci is running before starting the testsuite.

Available Arguments and Defaults

eXist-db's cache size and maximum brokers can be configured at build time using the following syntax.

mvn -DskipTests clean package docker:build --build-arg MAX_CACHE=312 MAX_BROKER=15 .

NOTE: Due to the fact that the final images does not provide a shell, setting ENV variables via docker does not work.

# !This has no effect!
docker run -dit -p8080:8080 -e MAX_BROKER=10 ae4d6d653d30

If you wish to permanently adopt a customized cache or broker configuration, you can either make a local copy of the Dockerfile and edit the default values there.

ARG MAX_BROKER=10

Or modify eXist-db's configuration files via xslt scripts located at exist-docker/src/main/xslt/. For multi-stage builds e.g. xmlstarlet let's you modify the default config files from within the builder stage, e.g.:

# Config files are modified here
RUN echo 'modifying conf files'\
&& cd $EXIST_HOME/etc \
&& xmlstarlet ed  -L -s '/Configuration/Loggers/Root' -t elem -n 'AppenderRefTMP' -v '' \
 -i //AppenderRefTMP -t attr -n 'ref' -v 'STDOUT'\
 -r //AppenderRefTMP -v AppenderRef \
 log4j2.xml

JVM configuration

This image uses an advanced JVM configuration, via the JAVA_TOOL_OPTIONS env variable inside the Dockerfile. You should avoid the traditional way of setting the heap size via -Xmx arguments, this can lead to frequent crashes since Java8 and Docker are (literally) not on the same page concerning available memory.

Instead, use the -XX:MaxRAMPercentage argument to modify the memory available to the JVM inside the container. The default of 75% should be save for production use. To allocate e.g. 600mb to the container around the JVM use, however, you can tweak this at buildtime via the JVM_MAX_RAM_PERCENTAGE argument. Note that percentages should include a decimal point 80.0 = 80%.

To allocate e.g. 600mb to the container around the JVM use:

docker run -m 600m …

Lastly, this image uses a new garbage collection mechanism garbage first (G1) -XX:+UseG1GC and string deduplication -XX:+UseStringDeduplication to improve performance.

Customizing individual arguments is not possible at run time. To adjust individual parameters the the whole JAVA_TOOL_OPTIONS environment variable must be passed to docker run …