-
-
Notifications
You must be signed in to change notification settings - Fork 230
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Propagate input annotations to primary.cwlprov files #1678
base: main
Are you sure you want to change the base?
Conversation
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #1678 +/- ##
==========================================
+ Coverage 83.80% 83.92% +0.11%
==========================================
Files 46 46
Lines 8221 8266 +45
Branches 2182 2201 +19
==========================================
+ Hits 6890 6937 +47
+ Misses 854 850 -4
- Partials 477 479 +2 ☔ View full report in Codecov by Sentry. |
Thank you for this @RenskeW ! can you |
@RenskeW Thanks! Checking https://www.commonwl.org/v1.2/CommandLineTool.html#File we see that After you make that change, can you add a |
(maybe in CWL v1.3 we will add a label field to |
This pull request introduces 1 alert when merging 99fc196 into c23a9ed - view on LGTM.com new alerts:
|
This pull request introduces 1 alert when merging c6e7385 into b21b0c1 - view on LGTM.com new alerts:
|
cwltool/provenance_profile.py
Outdated
|
||
def _add_nested_annotations(dataset, e: ProvEntity) -> ProvEntity: | ||
for annotation in dataset: | ||
if isinstance(dataset[annotation], (str, bool, int, float)): # check if these are all allowed types |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
isinstance(dataset[annotation], list), isinstance(dataset[annotation], MutableMapping)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
MutableSequence = list
This pull request introduces 2 alerts when merging bf34ca6 into bb41504 - view on LGTM.com new alerts:
|
cwltool/provenance_profile.py
Outdated
# # how do we identify the correct file to write to? self.workflow_run_uri? | ||
# # | ||
# pass | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
First step in propagating metadata under cwlprov:prov
to provenance as well.
This pull request introduces 2 alerts when merging 70d184f into bb41504 - view on LGTM.com new alerts:
|
e5c9176
to
00cf177
Compare
00cf177
to
e2009ac
Compare
09193f7
to
5466f95
Compare
This pull request has been mentioned on Common Workflow Language Discourse. There might be relevant details there: https://cwl.discourse.group/t/equivalent-to-file-metadata-for-local-environment/782/2 |
95def37
to
faddc3a
Compare
25b3ddb
to
fad5e8e
Compare
fad5e8e
to
9e10585
Compare
TODO: fix intent list add/amend tests
9e10585
to
dea0b8e
Compare
@mr-c here is a hand-annotated example of the new RDF provenance graph, including the CWL metadata fields: There is a separate example, containing propagation of annotation of input parameter values (e.g. an input File), to CWLprov RDF, here: |
Propagate structured annotations for files and directories (e.g. Schema.org, but also CWL-specific metadata fields) to the provenance graph. In addition, annotations under the
cwlprov:prov
field should be incorporated in the RDF graph as well.Step 1: If 'label' is specified in input object, it is added as an annotation to the file entity it corresponds to.
Example:
This would be represented in the cwlprov provenance like this:
See here for the full example RO.
Step 2: Values of
doc
,format
,label
,intent
are propagated to RDF provenance graph.EXAMPLE RO: https://github.com/RenskeW/cwlprov-provenance/tree/930cc268da77c5aa3739a2f7f87b94e076b144e6/cwlprov_rdf_examples
Mapping to Schema.org terms:
doc
: https://schema.org/descriptionlabel
: https://schema.org/nameformat
: https://schema.org/encodingFormatintent
: https://schema.org/featureListTODO:
cwlprov:prov
doc
,intent
,label
, andintent
fields to RDFdata/
, not be broken symlinks