Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow configuration of a set of validation tests to be defined in a config file #10

Open
jennet opened this issue Sep 13, 2018 · 1 comment
Labels
enhancement New feature or request

Comments

@jennet
Copy link

jennet commented Sep 13, 2018

Handle a config file that can specify a subset of query files

@jennet jennet added the enhancement New feature or request label Sep 13, 2018
@jennet jennet changed the title Add a different type for the --suite option to handle Allow configuration of a set of validation tests to be defined in a config file Oct 2, 2018
@jennet
Copy link
Author

jennet commented Oct 2, 2018

Here's how I envision it looking... I'd write a config file a bit like this:

{ 
    "validations" : [
      {
          "name" : "rdf-cube",
          "description" : "A set of validations that check the integrity of the data against the RDF Cube specifications",
          "queries" : "queries/rdf-cube/",
          "dependencies" : "cube-exists"
      },
      {
        "name" : "cube-exists",
        "description" : "Checks the data contains at least one entity that describes itself as a RDF cube",
        "queries" : "queries/at_least_one_rdf_cube.sparql"
        },
      {
        "name" : "atlas-hierarchy",
        "description" : "Checks the data contains at least one POI and that all POIs contained in the file are connected to the root atlas node (parameterised query)",
        "queries" : "queries/atlas/hierarchy/",
        "variables" : "queries/atlas/hierarchy/vars.edn",
        "dependencies" : "pmd-vocab-exists"
        },
    {
        "name" : "pmd-vocab-exists",
        "description" : "Check the vocabs necessary for running PMD atlas are available at the supplied data source",
        "queries" : "/queries/atlas/vocabs/",
        "dependencies" : ""
        }
    ]
  }

and I'd want to run this pre-written config against a data source instead of writing out multiple calls to rdf-validator.

Initially, the whole "dependency" part of this is a nice idea but might over complicate the process at this stage. Instead perhaps each object can have a "run" : true/false added so that that validator can skip that step rather than a user having to delete / add chunks of the file content.

The idea is that depending on the use case, a pre-written config file ("validation suite"?) can be used and called by name.

A validation suite could be easily added to address the "vertical" validations for UI building, which can be expanded on in the future to handle SHACL definitions rather than SPARQL queries.

Does this make sense @lkitching ?

lkitching added a commit that referenced this issue Oct 4, 2018
Issue #10 - Allow test suites to be specified within an EDN file and
executed through the CLI. EDN test suites explicitly list the test
files to execute and can also specify the name and test type. Existing
directory and file test suite types are also supported. In all cases,
a data structure is built representing the test suite(s) defined on
the command-line.

Test suites can also include tests from other suites, and exclude
imported tests which are not required.

Add reporting namespace containing TestReporter protocol for reporting
individual test results and the summary for all test executions.

Move repository loading into endpoint namespace.

Remove the two-stage loading and running of test cases and move the
loading into the test execution.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant