Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dimension relation validations #199

Merged
merged 16 commits into from
Sep 30, 2024
Merged

Dimension relation validations #199

Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
99bd903
feat: margin of error
tpluscode Jun 7, 2024
56033a3
refactor: term is confidence
tpluscode Jun 28, 2024
7dda5fc
fix: typing errors
tpluscode Jul 8, 2024
7c58565
refactor: review suggestions
tpluscode Jul 18, 2024
2051a36
refactor: better snippet
tpluscode Jul 18, 2024
792fdc0
test: dimension relation basic tests
tpluscode Sep 18, 2024
c911e11
test: validate `meta:relatesTo`
tpluscode Sep 18, 2024
a41f716
chore: changeset
tpluscode Sep 18, 2024
e0971fd
Merge branch 'shacl-test' into error-margin-validation
tpluscode Sep 18, 2024
35f7f7a
ci: running script on windows
tpluscode Sep 18, 2024
ee891f7
Merge branch 'main' into error-margin-validation
tpluscode Sep 19, 2024
f514843
Merge remote-tracking branch 'origin/main' into error-margin-validation
tpluscode Sep 19, 2024
bb83580
chore: update blank nodes in tests
tpluscode Sep 19, 2024
49c2df2
fix typo
tpluscode Sep 24, 2024
a31482a
feat: require related dimension to be a cube:MeasureDimension
tpluscode Sep 30, 2024
7cb9da8
chore: approve bnodes
tpluscode Sep 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .changeset/lucky-doors-brush.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
"cube-link": patch
---

Improve validation of `meta:dimensionRelation`:

1. Check that upper/lower bound has at most one `dcterms:type`
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kronmar I'm a little creative here since this was not really discussed. WDYT?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like reasonable checks

2. Check that `meta:relatesTo` is actually a dimension
62 changes: 62 additions & 0 deletions test/standalone-constraint-constraint/invalid.dimensionRelation-multipleDcType.ttl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
PREFIX dcterms: <http://purl.org/dc/terms/>
@prefix relation: <https://cube.link/relation/> .
@prefix meta: <https://cube.link/meta/> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix cube: <https://cube.link/> .
@prefix observation: <https://environment.ld.admin.ch/foen/nfi/observation/max_min_undefined> .
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix schema: <http://schema.org/> .
@base <https://example.org/> .

<cube> a cube:Cube ;
cube:observationConstraint <shape> ;
cube:observationSet <observationSet> .

<observationSet> cube:observation <observationA>, <observationB>, <observationC> .

<observationA> a cube:Observation ;
cube:observedBy <observer> ;
<dimension> 4.9 ;
<upperConfidence> 0.1 ;
.

<shape> a cube:Constraint ;
sh:targetClass cube:Observation ;
sh:closed true ;
sh:property
[
sh:path rdf:type ;
sh:nodeKind sh:IRI ;
sh:minCount 1 ;
sh:maxCount 1
] ;
sh:property
[
sh:path cube:observedBy ; ;
sh:nodeKind sh:IRI ;
sh:minCount 1 ;
sh:maxCount 1
] ;
sh:property
[
sh:datatype xsd:decimal ;
sh:path <dimension> ;
schema:name "dimension" ;
sh:minCount 1 ;
sh:maxCount 1 ;
],
[
sh:datatype xsd:decimal ;
sh:path <upperConfidence> ;
schema:name "upper confidence" ;
sh:minCount 1 ;
sh:maxCount 1 ;
meta:dimensionRelation
[
a relation:ConfidenceUpperBound ;
dcterms:type "Confidence interval", "bogus" ;
meta:relatesTo <dimension> ;
] ;
];
.
21 changes: 21 additions & 0 deletions ...tandalone-constraint-constraint/invalid.dimensionRelation-multipleDcType.ttl.approved.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix schema: <http://schema.org/> .
@prefix cube: <https://cube.link/> .

_:report a sh:ValidationReport ;
sh:result [
rdf:type sh:ValidationResult ;
sh:resultSeverity sh:Violation ;
sh:sourceConstraintComponent sh:MaxCountConstraintComponent ;
sh:sourceShape [
sh:path <http://purl.org/dc/terms/type> ;
sh:maxCount 1 ;
] ;
sh:focusNode _:b5 ;
sh:resultPath <http://purl.org/dc/terms/type> ;
sh:resultMessage "More than 1 values" ;
] ;
sh:conforms false .
62 changes: 62 additions & 0 deletions test/standalone-constraint-constraint/invalid.dimensionRelation-relatesToNotMatching.ttl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
@prefix dcterms: <http://purl.org/dc/terms/> .
@prefix relation: <https://cube.link/relation/> .
@prefix meta: <https://cube.link/meta/> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix cube: <https://cube.link/> .
@prefix observation: <https://environment.ld.admin.ch/foen/nfi/observation/max_min_undefined> .
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix schema: <http://schema.org/> .
@base <https://example.org/> .

<cube> a cube:Cube ;
cube:observationConstraint <shape> ;
cube:observationSet <observationSet> .

<observationSet> cube:observation <observationA>, <observationB>, <observationC> .

<observationA> a cube:Observation ;
cube:observedBy <observer> ;
<dimension> 4.9 ;
<upperConfidence> 0.1 ;
.

<shape> a cube:Constraint ;
sh:targetClass cube:Observation ;
sh:closed true ;
sh:property
[
sh:path rdf:type ;
sh:nodeKind sh:IRI ;
sh:minCount 1 ;
sh:maxCount 1
] ;
sh:property
[
sh:path cube:observedBy ; ;
sh:nodeKind sh:IRI ;
sh:minCount 1 ;
sh:maxCount 1
] ;
sh:property
[
sh:datatype xsd:decimal ;
sh:path <dimension> ;
schema:name "dimension" ;
sh:minCount 1 ;
sh:maxCount 1 ;
],
[
sh:datatype xsd:decimal ;
sh:path <upperConfidence> ;
schema:name "upper confidence" ;
sh:minCount 1 ;
sh:maxCount 1 ;
meta:dimensionRelation
[
a relation:ConfidenceUpperBound ;
dcterms:type "Confidence interval" ;
meta:relatesTo <non-dimension> ;
] ;
];
.
46 changes: 46 additions & 0 deletions ...one-constraint-constraint/invalid.dimensionRelation-relatesToNotMatching.ttl.approved.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix schema: <http://schema.org/> .
@prefix cube: <https://cube.link/> .

_:report a sh:ValidationReport ;
sh:result [
rdf:type sh:ValidationResult ;
sh:resultSeverity sh:Violation ;
sh:sourceConstraintComponent sh:NodeConstraintComponent ;
sh:sourceShape [
sh:path sh:property ;
sh:node <https://cube.link/shape/standalone-constraint-constraint#DimensionRelation> ;
] ;
sh:focusNode <https://example.org/shape> ;
sh:resultPath sh:property ;
sh:resultMessage "Value does not have shape <https://cube.link/shape/standalone-constraint-constraint#DimensionRelation>" ;
sh:detail [
rdf:type sh:ValidationResult ;
sh:resultSeverity sh:Violation ;
sh:sourceConstraintComponent sh:MinCountConstraintComponent ;
sh:sourceShape [
sh:path _:b675 ;
sh:class cube:Constraint ;
sh:minCount 1 ;
sh:message "value of meta:relatesTo must be a cube dimension" ;
] ;
sh:focusNode _:b5 ;
sh:resultPath _:b675 ;
sh:resultMessage "value of meta:relatesTo must be a cube dimension" ;
] ;
sh:value _:b4 ;
] ;
sh:conforms false .

_:b675 rdf:first <https://cube.link/meta/relatesTo> ;
rdf:rest (
[
sh:inversePath sh:path ;
]
[
sh:inversePath sh:property ;
]
) .
76 changes: 76 additions & 0 deletions test/standalone-constraint-constraint/valid.dimensionRelation.ttl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
PREFIX dcterms: <http://purl.org/dc/terms/>
@prefix relation: <https://cube.link/relation/> .
@prefix meta: <https://cube.link/meta/> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix cube: <https://cube.link/> .
@prefix observation: <https://environment.ld.admin.ch/foen/nfi/observation/max_min_undefined> .
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix schema: <http://schema.org/> .
@base <https://example.org/> .

<cube> a cube:Cube ;
cube:observationConstraint <shape> ;
cube:observationSet <observationSet> .

<observationSet> cube:observation <observationA>, <observationB>, <observationC> .

<observationA> a cube:Observation ;
cube:observedBy <observer> ;
<dimension> 4.9 ;
<upperConfidence> 0.1 ;
<lowerConfidence> 0.15 ;
.

<shape> a cube:Constraint ;
sh:targetClass cube:Observation ;
sh:closed true ;
sh:property
[
sh:path rdf:type ;
sh:nodeKind sh:IRI ;
sh:minCount 1 ;
sh:maxCount 1
] ;
sh:property
[
sh:path cube:observedBy ; ;
sh:nodeKind sh:IRI ;
sh:minCount 1 ;
sh:maxCount 1
] ;
sh:property
[
sh:datatype xsd:decimal ;
sh:path <dimension> ;
schema:name "dimension" ;
sh:minCount 1 ;
sh:maxCount 1 ;
],
[
sh:datatype xsd:decimal ;
sh:path <upperConfidence> ;
schema:name "upper confidence" ;
sh:minCount 1 ;
sh:maxCount 1 ;
meta:dimensionRelation
[
a relation:ConfidenceUpperBound ;
dcterms:type "Confidence interval" ;
meta:relatesTo <dimension> ;
] ;
],
[
sh:datatype xsd:decimal ;
sh:path <lowerConfidence> ;
schema:name "lower confidence" ;
sh:minCount 1 ;
sh:maxCount 1 ;
meta:dimensionRelation
[
a relation:ConfidenceLowerBound ;
dcterms:type "Confidence interval" ;
meta:relatesTo <dimension> ;
] ;
] ;
.
37 changes: 30 additions & 7 deletions validation/standalone-constraint-constraint.ttl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
@prefix dcterms: <http://purl.org/dc/terms/> .
@prefix : <https://cube.link/shape/standalone-constraint-constraint#> .
@prefix dash: <http://datashapes.org/dash#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
Expand All @@ -8,6 +9,7 @@
@prefix cube: <https://cube.link/> .
@prefix meta: <https://cube.link/meta/> .
@prefix qudt: <http://qudt.org/schema/qudt/> .
@prefix relation: <https://cube.link/relation/> .

#
# This is the bare minimal SHACL shape for validating a Cube Constraint.
Expand Down Expand Up @@ -179,17 +181,38 @@
.

:DimensionRelation a sh:NodeShape ;
sh:property [
sh:property
[
sh:path meta:dimensionRelation;
sh:property [
sh:path meta:relatesTo;
sh:nodeKind sh:IRI ;
sh:minCount 1;
sh:message "meta:dimensionRelation requires at least one meta:relatesTo";
] ;
sh:property
[
sh:path meta:relatesTo ;
sh:nodeKind sh:IRI ;
sh:minCount 1 ;
sh:message "meta:dimensionRelation requires at least one meta:relatesTo" ;
],
[
sh:path
(
meta:relatesTo
[ sh:inversePath sh:path ]
[ sh:inversePath sh:property ]
) ;
sh:minCount 1 ;
sh:class cube:Constraint ;
sh:message "value of meta:relatesTo must be a cube dimension" ;
] ;
Copy link
Contributor Author

@tpluscode tpluscode Sep 19, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With SHACL-SPARQL we would be able to check that the two dimensions actually belong to the same cube. As above, we merely verify that they the related dimension is use in a cube, not necessarily the same cube. That would potentially cause false-positive results if a graph contained multiple cubes. But since we typically only validate cubes individually, we should be covered

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like a reasonable restriction. But I have an additional sh:property here:

 [
      sh:path
          (
              meta:relatesTo
              [ sh:inversePath sh:path ] 
           ) ;
      sh:class cube:MeasureDimension ;
      sh:message "value of meta:relatesTo must point to measure dimension " ;
   ]

The dimension which is related to should be a cube:MeasureDimension. I can't see a reasonable case, where that would not be the case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Rdataflow @l00mi WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

] ;
.

:Confidende a sh:NodeShape ;
tpluscode marked this conversation as resolved.
Show resolved Hide resolved
sh:targetClass relation:ConfidenceLowerBound, relation:ConfidenceUpperBound ;
sh:property
[
sh:path dcterms:type ;
sh:maxCount 1 ;
] ;
.

# Testing proper rdf:list construction

Expand Down
Loading