incorporating NeuroLibre data into CONP

[emmetaobrien]

The proposed initial approach is that individual NeuroLibre datasets will show up on Zenodo with the appropriate flag for ingestion into CONP. Alternatively, they could be loaded manually using standard procedures. Examples for testing this process would be useful.

The NeuroLibre publication information will go in the primaryPublication field and subfields in the DATS model as usual. Currently publication information in CONP pages links back to the online primary publication via DOI in many cases; we should confirm that this will work for NeuroLibre.

The connection between a NeuroLibre dataset and the source dataset should use the extraProperties->derivedFrom relationship in DATS.json, and the GUI and/or user-facing documentation should make this clear. As this is a property we added ourselves, I see no difficulty with extending it to include multiple values as needed in the longer term (though it is potentially confusing nomenclature, as the core DATS schema has a property derivesFrom referring to the biological material from which a sample is taken).

Linking from a parent dataset to derived datasets is already managed in a back-end table so should not need any changes to DATS.json. (@cmadjar could you link to some documentation of this functionality here?)

Once we get this more clearly sorted out and tested I will update user and developer-facing documentation as necessary.

[cmadjar]

In the backend database, when a dataset is added to CONP, there is a table called dataset_ancestry that stores the dataset relationship based on the DATS extra property derivedFrom. The table structure is as follows:

• id
• parent_dataset_id (example: projects/bigbrain-datalad)
• child_dataset_id (example: projects/BigBrain_3DROIs)
• date_created
• date_updated

The code of the portal will then query that table to display the relationship between parent datasets and child datasets if there are any linked to the dataset displayed.

[emmetaobrien]

Does that all work automatically, or is any manual intervention needed?

[cmadjar]

There is a cronjob on the portal that automatically runs flask update_datasets every 15 minutes and that command will update the dataset_ancestry table when there are relationships between datasets.

Then the code reads that table for proper display of the relationships between datasets.

[emmetaobrien]

Excellent.

[github-actions]

This issue is stale because it has been open 5 months with no activity. Remove stale label or comment or this will be closed in 3 months.

[github-actions]

This issue was closed because it has been stalled for 3 months with no activity.

[github-actions]

This issue is stale because it has been open 5 months with no activity. Remove stale label or comment or this will be closed in 3 months.

[github-actions]

This issue was closed because it has been stalled for 3 months with no activity.

[github-actions]

This issue is stale because it has been open 5 months with no activity. Remove stale label or comment or this will be closed in 3 months.

[github-actions]

This issue is stale because it has been open 5 months with no activity. Remove stale label or comment or this will be closed in 3 months.

[github-actions]

This issue was closed because it has been stalled for 3 months with no activity.