JSON Schema creator and validator

[sekmiller]

What this PR does / why we need it: Creates a custom json schema for a given Dataverse Collection that can be used to validate a dataset.json file prior to uploading to create the dataset

Which issue(s) this PR closes:

Closes #9464 Dataset json validation update code to use schema
Closes #9465 Create api.endpoint for Dataset json validation.
Closes #6978 Query Dataverse for mandatory metadata fields via API

Special notes for your reviewer: The api returns a json string that includes the escape characters. wasn't sure if that that is OK or if I should strip them out before returning

Suggestions on how to test this:

Does this PR introduce a user interface change? If mockups are available, please link/include them here:

Is there a release notes update needed for this change?: Release notes are included

Additional documentation: doc updated to include the new endpoints

https://dataverse-guide--10109.org.readthedocs.build/en/10109/api/native-api.html#retrieve-a-dataset-json-schema-for-a-collection

[landreev]

Copy-and-paste from the other command, probably.

[landreev]

I'm not seeing this discussed, in the pr or the linked issues - sorry if I missed it - but was there a specific reason to only use this validation code in the dedicated "validate" api? - and not always use it in the "create" api as well, before attempting to import the json uploaded by the user?
(are we just assuming that the attempt to import is validation enough, so there's no reason to spend extra cycles pre-validating? ... and if the user needs to know more details about why it failed, then they can run the validate api? - I may have answered my own question already, but please let me know what you think)

[JR-1991]

I tested the build and endpoint, and everything went smoothly during installation/testing. The JSON schema I received was helpful, and I included a screenshot of the request and response for you to check out.

I noticed that controlled vocabularies are missing. For instance, the subject field in Citation has one, but it is missing here. It would be great to make this field an enum field to ensure consistent and accurate data entry. What do you think?

[pdurbin]

@JR-1991 are you thinking the values themselves (Chemistry, Law, etc.) should be returned in the JSON Schema?

That would certainly be more convenient than hitting another API to get them. Thanks again for adding that in this PR:

• #8944 - Extend 'metadatablocks/{block_id}' endpoint JSON output #9213

Also, I'm wondering if this PR closes the following issue:

• Query Dataverse for mandatory metadata fields via API #6978

It sounds so similar: "If this list [of five required fields] never changes, then RSpace could develop a solution where it reads a list of mandatory fields from a configuration file. But if it does change from time to time, it would be great if there was an API method in Dataverse to get a list of mandatory metadata fields .Then, a client could programmatically generate input fields for these properties so that the end-user could make a valid submission."

I'll go ask. 😄

[johannes-darms]

I'm not sure if we should address this in this MR. However, IMHO its crucial that the json schema encodes the correct datatypes as defined within the metadatablock and the API returns data that is valid wrt. to the schema. So it would be nice if these two issues #9494. #9495 were somehow addressed.

[pdurbin]

crucial that the json schema encodes the correct datatypes as defined within the metadatablock

A good example of this is that the geospatial block expects floats for north, south, east, west (latitude and longitude). See this related pull request by @stevenwinship and @landreev merged yesterday:

• Add validation for entered coordinate values in Geospatial metadata #10142

[JR-1991]

@JR-1991 are you thinking the values themselves (Chemistry, Law, etc.) should be returned in the JSON Schema?

Sorry for missing your comment earlier. I believe this addition would be valuable and allow for client-side controlled vocabulary validation.

[github-actions]

📦 Pushed preview images as

ghcr.io/gdcc/dataverse:9464-schema-creator-validator

ghcr.io/gdcc/configbaker:9464-schema-creator-validator

🚢 See on GHCR. Use by referencing with full name as printed above, mind the registry name.