Added loop support (#5)

* conformance tests: report the names of the failures
---------

Co-authored-by: Michael R. Crusoe <michael.crusoe@gmail.com>
Co-authored-by: Peter Amstutz <peter.amstutz@curii.com>

.github/config/wordlist.txt

@@ -1,3 +1,4 @@
+abstractworkflowstep
 acyclic
 amstutz
 arrayschema
@@ -160,6 +161,9 @@ loadlistingrequirement
 localhost
 lookahead
 loopback
+loopinput
+loopoutputmethod
+loopworkflowstep
 maccallum
 macos
 mappredicate
@@ -194,6 +198,7 @@ outputbinding
 outputenumschema
 outputeval
 outputformat
+outputmethod
 outputschema
 outputsink
 outputsource
@@ -231,6 +236,7 @@ saladversion
 sbgenomics
 scatterfeaturerequirement
 scattermethod
+scatterworkflowstep
 schemadefrequirement
 schemaorg
 schemas

.github/workflows/cwltool.yml

@@ -24,7 +24,7 @@ jobs:
           python-version: '3.9.x'
 
       - name: Setup prerequirements
-        run: pip install "cwltest>=2.3" pytest-xdist "git+https://github.com/common-workflow-language/cwltool.git@main#egg=cwltool"
+        run: pip install "cwltest>=2.3" pytest-xdist "cwltool>=3.1.20240909164951"
 
       - name: Copy in cwltool-specific configuration for the cwltest pytest plugin
         run: cp "$(python -c 'from cwltool.tests.util import get_data; print(get_data("tests/cwl-conformance/cwltool-conftest.py"))')" conftest.py
@@ -33,4 +33,4 @@ jobs:
         run: cp conformance_tests.yaml conformance_tests.cwltest.yaml
 
       - name: Run tests against the reference runner
-        run: python -m pytest conformance_tests.cwltest.yaml -n auto -rs
+        run: python -m pytest conformance_tests.cwltest.yaml -n auto -rsfE

CONFORMANCE_TESTS.md

@@ -144,7 +144,7 @@ We will use this single entry to explain the format
      Must include one or more of the following tags: `command_line_tool`, `expression_tool` or `workflow`.
      If the test does not test any optional features, the tag `required` is required.
 
-Because `conformance_tests.yaml` is a `schema-salad` processed document, [`$import`](https://www.commonwl.org/v1.2/SchemaSalad.html#Import)
+Because `conformance_tests.yaml` is a `schema-salad` processed document, [`$import`](https://www.commonwl.org/v1.3/SchemaSalad.html#Import)
 can be used to organize the tests into separate files.
 
 Currently, the main file is too big (over 3400 lines); we are slowly re-organizing it.
@@ -166,7 +166,7 @@ At any level, if there is an extra field, then that will be considered an error.
 An exception to this is `class: File` and `class: Directory` objects, the `cwl-runner` under test can add additional fields here without causing a test to fail.
 Likewise, if you don't want to test some aspect of a `class: File` or `class: Directory` object (like `nameext`) you can just omit it.
 
-[According to the CWL standards](https://www.commonwl.org/v1.2/CommandLineTool.html#File), the format of the `location` field in 
+[According to the CWL standards](https://www.commonwl.org/v1.3/CommandLineTool.html#File), the format of the `location` field in 
 `class: File` and `class: Directory` is implementation specific and we should not be testing them.
 Please remember to use `location: Any` for them.