Skip to content

Latest commit

 

History

History
182 lines (111 loc) · 13.4 KB

README.md

File metadata and controls

182 lines (111 loc) · 13.4 KB
Raw

OSCAL Build Tools

This subdirectory contains infrastructure used to create OSCAL-related artifacts (e.g., schemas, converters) and perform status checks. Below are instructions for using these tools.

Prerequisites

The build tools in this directory require a Unix environment with the following software:

  • Maven: Required to generate artifacts
  • NodeJS (ensure the version matches .nvmrc): Required to perform link checking
  • libxml: Provides xmllint which is required to validate generated XML artifacts

Additionally ensure your environment has standard build tools such as git 2.34.1 or newer, make 4.3 or newer, zip 3.0 or newer, and bash 3.2 or newer for syntax we require.

Building

Overview

All OSCAL build targets are defined in a Makefile. To summarize the build targets, run make help. To generate all artifacts and run all checks, simply run make all.

To run targets in parallel, pass a --jobs <n> flag, where <n> is the number of concurrent jobs and should be <= your machine's core count. Note that running jobs concurrently will make the output harder to follow and debug.

Artifact Generation

As of v1.1.0, artifacts are no longer tracked in the OSCAL git repository (discussion link). Instead artifacts are generated as part of a release.

Developers can generate schemas locally using the make artifacts command.

Developers can also generate individual artifacts using the following commands:

  • make schemas: Generates the JSON Schemas and XSDs off of the source Metaschemas;
  • make converters: Generates the XSLT stylesheets for JSON<->XML conversion off of the source Metaschemas;
  • make resolved-metaschemas: Resolves external entities (XXE) present in the source OSCAL Metaschema modules for use with tools that do not support XXEs.

Checks

Developers can run checks using the make checks command.

The checks can also be run individually:

  • make linkcheck: Checks each markdown file in the repository for bad links;
  • make validate-jsonschemas: Validates that the JSON schemas generated by make schemas are schema valid against the JSON Schema specification;
  • make validate-xsds: Validates that the XSD files generated by make schemas are schema valid against the XSD specification;
  • make test-profile-resolution: Runs the test suite in /src/utils/resolver-pipeline.

Archive Generation

make archives generates .zip and .tar.bz2 archives with the generated artifacts.

By default, archives will have the suffix SNAPSHOT, but this can be overridden using the RELEASE variable:

$ make archive RELEASE=my_release

Artifact Usage

Schemas

The OSCAL project provides JSON Schemas based on the JSON Schema Draft-07 and XML Schema based on the XML Schema Definition Language (XSD) 1.1 for all OSCAL models.

After running make schemas, JSON Schemas and XSD files will be created in the generated/ directory, following the general pattern oscal_[MODEL_NAME]_schema.json and oscal_[MODEL_NAME]_schema.xsd.

Pre-built schemas are also distributed as part of a release.

Validating OSCAL JSON Content

The OSCAL project uses the Another Json Validator (AJV) command line interface (CLI) to perform automated validation of all JSON-based OSCAL content provided in this repository. ajv-cli is an open source tool that can be installed, and run on OSX, Linux, and Windows environments, with a Node.js environment installed. Node.js is the JavaScript runtime environment that is required to run AJV.

The following example uses the ajv-cli to perform validation of an OSCAL catalog JSON file.

ajv validate -s "oscal_catalog_schema.json" -d "catalog.json" --extend-refs=true --verbose

The online documentation for ajv-cli provides more information on the command-line arguments.

Validating OSCAL XML Content

The OSCAL project uses xmllint to perform automated validation of all XML-based OSCAL content provided in this repository. xmllint is an open source tool that can be downloaded, installed, and run on OSX, Linux, and Windows environments.

The following example uses xmllint to perform validation of an OSCAL catalog XML file.

xmllint --noout --schema "oscal_catalog_schema.xsd" "catalog.xml"

The online documentation for xmllint provides more information on the command-line arguments.

Converters

The OSCAL project provides Extensible Stylesheet Language Transformation (XSLT) templates based on XSLT 3.0 and XPath 3.1 for all OSCAL models.

IMPORTANT : XSLT 3.0 and XPath 3.1 are only required when using the NIST provided tools for converting OSCAL content between JSON and XML. Any version of XSLT or XPath may be used when transforming or querying OSCAL files for other reasons. These newer versions of XSLT and XPath offer more robust capabilities, that are needed to support the OSCAL converters.

After running make converters, converter stylesheets will be created in the generated/ directory, following the general pattern oscal_[MODEL_NAME]_json-to-xml-converter.xsl and oscal_[MODEL_NAME]_xml-to-json-converter.xsl.

Pre-built converters are also distributed as part of a release.

Converting OSCAL XML Content to JSON

The OSCAL project uses Saxon-HE (Saxon Home Edition) to evaluate the XSLT templates supporting conversion of OSCAL XML and JSON content provided in this repository. Saxon-HE is an open source implementation of XSLT 3.0, XPath 2.0 and 3.1, and XQuery 3.1 supporting Java and .NET programming environments. These versions of Saxon-HE can be downloaded directly or the Java version can be downloaded using Apache Maven. Saxonica also offers Saxon PE and EE versions, which are commercial products with technical support and redistribution rights.

The OSCAL project uses Saxon-HE with Java version 8 or greater.

The following example uses Saxon HE to convert an OSCAL catalog XML file to JSON using one of the NIST-provided JSON to XML XSLT converters. This example assumes that Java 8+ has been installed and the Saxon-HE jar files have already unzipped.

java -jar "saxon10he.jar" -xsl:"oscal_catalog_xml-to-json-converter.xsl" -s:"oscal-catalog.xml" -o:"oscal-catalog.json" json-indent=yes

Note: at time of writing, Saxon 9 users are being encouraged to upgrade systems to use Saxon 10, and the stylesheets provided should function equally well or better with the later software. Please feel free to use the latest Saxon or indeed any conformant XSLT 3.0 processor.

Operators of XSLT-based platforms should by all means test these processes with any XSLT 3.0 conformant processor, and report problems to us via Github Issues.

Paths\names of these files need to be provided based on the location of the files on your computer:

  • The Saxon JAR file is named saxon10he.jar.
  • The catalog converter is specified as -xsl:"oscal_catalog_xml-to-json-converter.xsl"
  • The source catalog XML file is specified as -s:"oscal-catalog.xml"
  • The destination catalog JSON file is specified as -o:"oscal-catalog.json".

The online documentation for Saxon provides more information on the command line arguments.

Alternate invocations

The configuration just provided will convert a JSON file given as a file reference, into OSCAL XML. There are also different configurations available for debugging:

  • -it:from-xml (indicating initial template) - provides the default XSLT entry point explicitly.
  • -file:mycatalog.xml used with explicit -it:from-xml will look for the XML at the location given by the parameter, instead of on the source port (given by -s). This configuration parallels the JSON-to-XML converter.
  • produce=supermodel as a runtime parameter will emit not OSCAL XML, but the intermediate format produced by the converter (a so-called 'OSCAL supermodel' derived from its metaschema): useful for debugging or as a pivot to other serializations.
  • produce=xpath-json will produce the results in an XML format defined by the XPath function json-to-xml(), which when consumed by the complementary function xml-to-json() can deterministically provide syntactically correct JSON. This XML format is used internally to provide the JSON data description, to be cast into JSON as a (terminal) serialization step. Expressing the transformation results in this format directly can provide insight for debugging, especially in failure conditions (when the syntax cannot be written). See https://www.w3.org/TR/xpath-functions-31/#func-json-to-xml for more details on this format.

Converting OSCAL JSON Content to XML

The OSCAL project uses Saxon-HE (Saxon Home Edition) to evaluate the XSLT templates supporting conversion of OSCAL XML and JSON content provided in this repository. Saxon-HE is an open source implementation of XSLT 3.0, XPath 2.0 and 3.1, and XQuery 3.1 supporting Java and .NET programming environments. These versions of Saxon-HE can be downloaded directly or the Java version can be downloaded using Apache Maven. Saxonica also offers Saxon PE and EE versions, which are commercial products with technical support and redistribution rights.

The OSCAL project uses Saxon-HE with Java version 8 or greater.

The following example uses Saxon HE to convert an OSCAL catalog JSON file to XML using one of the NIST-provided JSON to XML XSLT converters. This example assumes that has been installed and the Saxon-HE jar files have already unzipped.

java -jar "saxon10he.jar" -xsl:"oscal_catalog_json-to-xml-converter.xsl" -o:"oscal-catalog.xml" -it:from-json file="oscal-catalog.json"

Note: at time of writing, Saxon 9 users are being encouraged to upgrade systems to use Saxon 10, and the stylesheets provided should function equally well or better with the later software. Please feel free to use the latest Saxon or indeed any conformant XSLT 3.0 processor.

Operators of XSLT-based platforms should by all means test these processes with any XSLT 3.0 conformant processor, and report problems to us via Github Issues.

-it indicates the initial template (XSLT entry point) should be from-json. (As documented in the source code, other entry points are offered for diagnostics or integration.)

Paths/names given to other settings are offered as relative or absolute system paths or URIs:

  • The Saxon JAR file is named saxon9he.jar (system path).
  • The catalog converter is specified as -xsl:"oscal_catalog_json-to-xml-converter.xsl" (relative or absolute URI)
  • The source catalog JSON file is specified as file="oscal-catalog.json" (URI)
  • The destination catalog XML file is specified as -o:"oscal-catalog.xml" (URI)

The online documentation for Saxon provides more information on the command line arguments.

Alternate invocations

The configuration just provided will convert a JSON file given as a file reference, into OSCAL XML. However it may sometimes be convenient (for example when invoking programmatically) to pass the JSON into the converter as a literal.

Use the json runtime parameter to do this. On the command line this might be:

java -jar "saxon10he.jar" -xsl:"oscal_catalog_json-to-xml-converter.xsl" -o:"oscal-catalog.xml" -it:from-json json="{ "catalog": {} }"

(With allowances made for quote marks etc.)

There are also different configurations available for debugging:

  • -it (initial template) from-xdm-json-xml - assume the source is not given as a URI reference to a file, but as XML conformant to the model returned by the XPath function 'json-to-xml()'. In this case, the file parameter must point to this XML file not a JSON file.
  • Alternatively, -s:file.xml (with or instead of -it) will operate the same way, except finding the XML at file.xml.
  • produce=supermodel as a runtime parameter will emit not OSCAL XML, but the intermediate format produced by the converter (a so-called 'OSCAL supermodel' derived from its metaschema): useful for debugging or as a pivot to other serializations.