CWL Viewer

This is a Spring Boot MVC application which fetches Common Workflow Language files from a Github repository and creates a page for it detailing the main workflow and its inputs, outputs and steps.

Using CWL Viewer

You are recommended to use the production instance of CWL Viewer at https://view.commonwl.org/ which runs the latest release. Any downtime should be reported on the gitter chat for cwlviewer.

Running

If you are a developer, or you want to use the CWL Viewer in a closed environment, then you can run your own instance.

Recommended - Running with Docker

This application can be started with Docker and Docker Compose and Docker Compose is the recommended method of running or developing this codebase.

Then run the following commands to clone the project in your local system.

 git clone https://github.com/common-workflow-language/cwlviewer.git
 cd cwlviewer

In the project directory, to start CWLViewer exposed on port 8080, run:

docker-compose up

The web server will connect to a local host, you'll see the message saying "Tomcat started on port(s):8080".

To see the locally running CWL Viewer app, visit http://localhost:8080/ in your web browser.

To stop and remove:

docker-compose down

If you change the source code, then use this docker-compose.override.yml and re-build with docker-compose build:

version: '3.2'
services:
  spring:
    build: .

See the docker-compose.yml file for details.

If you have modified the source code, then you may want to build the docker image locally first:

docker build -t commonworkflowlanguage/cwlviewer .

Running Spring Boot locally for development, with MongoDB and Jena Fuseki in Docker

Create docker-compose.override.yml:

version: '3.2'
services:
  mongo:
    ports:
     - "27017:27017"
  sparql:
    ports:
     - "3030:3030"

Then start the containers:

docker-compose up

Then start Spring Boot locally:

mvn spring-boot:run -Dserver.port=7999

Now you can connect to http://localhost:7999 in your browser.

Deleting the data volumes to reset state

To completely reset the state, you must delete the data volumes:

docker-compose down
docker volume rm  cwlviewer_bundle cwlviewer_git cwlviewer_graphviz cwlviewer_mongo cwlviewer_sparql

Running without Docker

Requirements

MongoDB

You will need to have MongoDB running, by default on localhost:27017

If you are running from the command line, you can override this by supplying system properties like -Dspring.data.mongodb.host=mongo.example.org and -Dspring.data.mongodb.port=1337

Apache Jena Fuseki (or alternative SPARQL server)

You will also need to have a SPARQL server such as Apache Jena Fuseki running, by default on localhost:3030

Compiling and Running

To compile you will need Java 8 or OpenJDK 8 (apt install openjdk-8-jdk), as well as Apache Maven 3 (apt install maven).

Spring Boot uses an embedded HTTP server. The Spring Boot Maven plugin includes a run goal which can be used to quickly compile and run it:

$ mvn spring-boot:run

Alternatively, you can run the application from your IDE as a simple Java application by importing the Maven project.

You need to install Graphviz for all unit tests to pass.

You can create an executable JAR file by using:

mvn clean install

Afterwards, run:

java -jar target/cwlviewer*.jar

(The exact filename will vary per version)

Once CWL Viewer is running, you should see log output somewhat like:

()..)
s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat started on port(s): 8080 (http)
org.researchobject.CwlViewerApplication  : Started CwlViewerApplication in 28.604 seconds

Now check out http://localhost:8080/ to access CWL Viewer.

Configuration

There are a variety of configuration options detailed in the application configuration file which can be adjusted.

When deploying with docker, these can be overridden externally by creating/modifying docker-compose.override.yml as follows:

version: '3.2'
services:
  spring:
    environment:
            applicationName: Common Workflow Language Viewer
            applicationURL: https://view.commonwl.org
            cacheDays: 1

The properties can alternatively be provided as system properties on the command line, e.g. -DcacheDays=1 or via a variety of other methods supported by Spring Boot

Dump/restore

While you can perform backup of the Docker volumes, for larger upgrades of CWL Viewer it is recommended instead to do a JSON dump and re-load, which will force CWL Viewer to fetch and parse again.

The script dump.sh can be used for regular backups, it will store the full output of /workflows as a timestamped gzip-compressed JSON file:

$ ./dump.sh https://view.commonwl.org/ /var/backups/cwl
/var/backups/cwl/2018-06-06T135133+0000.json.gz

The script load.py (requires Python 3) can be used to restore from such JSON dumps:

./load.py /var/backups/cwl/2018-06-06T135133+0000.json.gz https://view.commonwl.org/

The optional parameter --no-commits can be added to skip those entries that look like a commit ID. Note that this might break previous permalinks.

Documentation

2017 Poster https://doi.org/10.7490/f1000research.1114375.1

2017 Video overview https://youtu.be/_yjhVTmvxLU

2017 Technical Report https://doi.org/10.5281/zenodo.823295

License

Distributed under the Apache License, Version 2.0. See the file LICENSE.md for details, and NOTICE.md for required attribution notices.

Contribute

Feel free to contribute! You may raise an issue, provide a pull request or join the gitter chat for cwlviewer!

Changelog

See CHANGELOG

Thanks

Developers and contributors include:

• Mark Robinson http://orcid.org/0000-0002-8184-7507
• Stian Soiland-Reyes http://orcid.org/0000-0001-9842-9718
• Michael Crusoe http://orcid.org/0000-0002-2961-9670
• Carole Goble http://orcid.org/0000-0003-1219-2137
• Charles Overbeck https://github.com/coverbeck
• Finn Bacall http://orcid.org/0000-0002-0048-3300

Thanks to:

• eScience Lab at The University of Manchester
• BioExcel Center of Excellence for Computational Biomolecular Research
• • European Commission's H2020 grant 675728
• Common Workflow Language community and the CWL Gitter room