GitHub Action - OPTIMADE validator

This action runs optimade-validator from the optimade package found in the optimade-python-tools repository on either a locally running server or a public server.

All minor and patch updates to v2 will be folded into the v2 tag, so that using the action @v2 is recommended, since it results in using the latest v2.minor.patch.

Latest versions:

Used tag	Effective version
v2	v2.8.0
v1	v1.2.0

Example usage

To run optimade-validator for an index meta-database at http://gh_actions_host:5001/ do the following:
Within the same job, first, start a server, e.g., using the ghcr.io/materials-consortia/optimade container image as a service, and then add the step

uses: Materials-Consortia/optimade-validator-action@v2
with:
  port: 5001
  path: /
  index: yes

Note: The service should be used with a set OPTIMADE_BASE_URL:

services:
  optimade_index:
    image: ghcr.io/materials-consortia/optimade
    ports:
      - 5001:5000
    env:
      OPTIMADE_BASE_URL: http://gh_actions_host:5001

To run optimade-validator for a regular OPTIMADE deployed implementation, testing all possible versioned base URLs, for example:

• https://example.org:443/optimade/example/v1
• https://example.org:443/optimade/example/v1.0
• https://example.org:443/optimade/example/v1.0.0

uses: Materials-Consortia/optimade-validator-action@v2
with:
  protocol: https
  domain: example.org
  port: 443
  path: /optimade/example
  all versioned paths: True

Note: This will also run optimade-validator for the unversioned base URL.

By default, the validator follows the OPTIMADE specification. This means it will always check the mandatory base URLs:

• Unversioned base URL (/)
• Major-versioned base URL (/vMAJOR, e.g. /v1)

The major version number will be determined based on the validator version used, i.e., the supported OPTIMADE API version in the OPTIMADE Python tools repository. This can be chosen using the input validator version.

Inputs

Input	Description	Usage	Default	Action version
all versioned paths	Whether to test the optional versioned base URLs: /vMAJOR /vMAJOR.MINOR /vMAJOR.MINOR.PATCH If false, only the mandatory /vMAJOR URL will be tested.	Optional	false	v1+
validate unversioned path	Whether to validate the input URL as a full OPTIMADE implementation without appending any version. This action assumes that the 'path' parameter will contain an unversioned URL. As it is not mandatory to run a full OPTIMADE implementation from the unversioned URL, by default, this action will not perform any validation of this URL, beyond checking the mandatory /versions endpoint.	Optional	false	v2.3+
as type	Validate the request URL with the provided type, rather than scanning the entire implementation. Example values: structures, reference. For a full list of values see the OPTIMADE Python tools documentation. It is expected that path points to the exact endpoint to be validated. You must include any versions or similar if desired, as the path will be used as given. NB: This input takes precedence over all versioned paths, validate unversioned path, and index, meaning these will be ignored if as type is defined.	Optional	-	v1+
domain	Domain for the OPTIMADE URL (defaults to the GitHub Actions runner host).	Optional	gh_actions_host	v1+
fail fast	Whether or not to exit and return a non-zero error code on first failure.	Optional	false	v2+
index	Whether or not this is an index meta-database.	Optional	false	v1+
path	Path to append to the domain to reach the OPTIMADE unversioned base URL - MUST start with /. The path MUST NOT include the versioned part of the base URL. Rather, it MUST point to the unversioned base URL of your OPTIMADE implementation (except if as type is defined, then path should define the full path to the endpoint to be validated, including also the versioned part if desired, e.g., /v1.0/structures).	Optional	/	v1+
port	Port for the OPTIMADE URL.	Optional	5000	v1+
protocol	Protocol for the OPTIMADE URL.	Optional	http	v1+
skip optional	Whether or not to skip tests for optional features.	Optional	false	v2+
minimal	Whether or not to reduce the validation to a minimal test set that only checks responses and not filters.	Optional	false	v2.4.1+
validator version	Full version of an OPTIMADE Python tools release to PyPI, e.g., 'v0.6.0' or '0.3.4', which hosts the optimade-validator. It can also be a branch, tag, or git commit to use from the GitHub repository, e.g., 'master' or '5a5e903'. See the pip documentation for more information of what is allowed here. Finally, it may also be 'latest' (default), which is understood to be the latest official release of the optimade package on PyPI. Note, for the latest development version, choose 'master'.	Required	latest	v1+
verbosity	The verbosity of the output. 0: minimalistic, 1: informational, 2+: debug	Optional	1	v1+

Running test suite (developers)

The test suite is based on BATS and should be run in Docker. The reason to run in Docker, is that the entrypoint.sh file will install the optimade package. And to not pollute your local environment, it is safer to run it in a Docker container.

Furthermore, for each test run, the optimade package will be installed in a virtual environment within the Docker file, since changing the optimade version is part of the test suite.

Steps to setup your test environment:

1. Git clone this repository to your local environment
2. Install Docker (this depends on your OS, see the Docker documentation)
3. Ensure you are a member of the GitHub organization Materials-Consortia. This is necessary to pull the base image for the Docker image for testing.
4. Build the Docker image for testing (based on a Unix system):

docker login ghcr.io
# You will be prompted for your GitHub username and password

cd /path/to/optimade-validator-action
docker build --tag optimade_bats ./tests

Now you can run

docker run --rm -it -v "$(pwd):/code" --workdir /code optimade_bats ./tests

or

./run_tests.sh

This is a file that contains the previous line and runs it, for posterity. Furthermore, it takes two arguments.

One is the Docker tag. So if you named the built Docker image something different from optimade_bats, you can pass the name to run_tests.sh and it will work, e.g.,

./run_tests.sh my_custom_image_name

will run the tests with image my_custom_image_name instead of optimade_bats.

Another is the relative or absolute path to the .bats files directory or a single .bats file, e.g.,

./run_tests.sh ./tests/test_verbosity.bats

will run the tests in the test_verbosity.bats file under the tests directory relative to where you're running the run_tests.sh file.

Concerning action versions and optimade-python-tools

Since this action installs optimade-python-tools and runs the validator from this repository, the versions of this Action relies on different versions of the optimade package in the optimade-python-tools repository.

To keep it simple, this overview will only consider the major versions of this Action:

Action version	optimade package versions	Supported OPTIMADE API specification version(s)
v2	v0.10.0+	v1.0.0 - latest
v1	v0.7.0 - v0.9.8	v0.10.0 - v1.0.0-rc2