CONFORMANCE_TESTS.md

Common Workflow Language standards v1.0 conformance test suite

The conformance tests are intended to test feature coverage of a CWL implementation. It uses the Python module "cwltest" from https://github.com/common-workflow-language/cwltest/

For CWL v1.1 conformance tests see: https://github.com/common-workflow-language/cwl-v1.1/blob/main/CONFORMANCE_TESTS.md and for CWL v1.2 conformance tests see: https://github.com/common-workflow-language/cwl-v1.2/blob/main/CONFORMANCE_TESTS.md

Usage

$ ./run_test.sh
--- Running conformance test draft-3 on cwl-runner ---
Test [49/49]
All tests passed

By default, run_test.sh will test cwl-runner in $PATH against the current stable CWL draft.

Options

RUNNER=other-cwl-runner

The CWL implementation to be tested.

EXTRA=--parallel

Extra options to pass to the CWL runner

-nN

Run a single specific test number N.

For example, to run conformance test 15 against the "cwltool" reference implementation with --parallel:

$ ./run_test.sh RUNNER=cwltool EXTRA=--parallel -n15
Test [15/49]
All tests passed

Notes

NOTE: For running on OSX systems, you'll need to install coreutils via brew. This will add to your system some needed GNU-like tools like greadlink.

1. If you haven't already, install brew package manager in your mac
2. Run brew install coreutils