Tuesday, 2024/06/18
10:00 am ET
Terrell Russell, Kory Draughn, Lazlo Westerhof (Utrecht), Mariana Montes (KU Leuven), Mike Conway (NIEHS)
- Python microservices
- Saved to repository
- Four bats tests, add / remove / validate collection with two objects
- Presented at UGM2024
- Conversion from JSON to AVUs and back?
- Should this be a consortium-provided set of functions?
- If so... now we have to agree.
- OR, plugin!
- Arrays instead of stomping identical a/v combinations
- https://github.com/irods-contrib/irods_working_group_metadata_templates/blob/main/metadata_templates.py#L132C29-L132C76
- Could always build an array, and then collapse?
- Leuven using schema name IN the Attribute to know which schema is in play
- Can determine whether to build an array or just single element
- Requires 'naming' the schemas, and managing that namespace (and versioning)
- Converting/representing schema to JSON can be a pluggable interface
- Default can do what we've got here today
- Plugin / override could define Leuven way of using schema / array information
- Would be called before validate to prepare the JSON
- Only attaching schemas to collections today
- Now, for … users / groups / resources / data objects?
- Data object validate would validate against schema on data object AND all parents (if recursive)
- Allows for 'reuse' because schemas are not in iRODS, references to external
- Versioning
- Different schema versions cannot be in play at the same time with current design
- Leuven allows for multiple versions in flight at the same time because each data object declares which schemas
- Yoda, on update of a schema, walks the affected data objects / collections to 'convert' to the new version, if a transformation is available.
- And one should be available, or an 'error' by the admin.
- JSON-Schema does not have an inherent notion of versioning
- Depends on the schema just having a different name that 'means' a different version
- This might not be the consortium's job
- To be handled 'up there', per namespace / application
- Leuven does not 'attach' to the collections themselves
- Instead, things are more 'composed'
- The metadata is declaring which schema it's a part of
- Mango-specific, until now a Python module presented at UGM2024
- Handles versioning
- Weakness / loophole for now is how to indicate metadata applying to parent / grandparent applies to children / data objects
- Yoda attaches schema to a user group
- Singular(!) collection managed/associated to that group
- But also groups in groups!
- Data objects under that collection are 'validated' against that schema
- Leuven using 'realms' like group, schemas 'owned' by a realm today
- Want to share across realms, potentially soon
- Singular(!) collection managed/associated to that group
- Next Steps
- Plugin/override mechanism for JSON representation
- Add schema attachment to data objects
- Validate data object will take into consideration any schemas attached to that data object (requires gather for data object)
- Next Meeting
- July 2024