Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release Notes related to HDC 1 and 3A/3B #8894

Conversation

qqmyers
Copy link
Member

@qqmyers qqmyers commented Aug 4, 2022

What this PR does / why we need it: With the interactions between HDC PRs, it makes more sense to have one integrated release notes - this is it.

Which issue(s) this PR closes:

Closes #8611

Special notes for your reviewer:

Suggestions on how to test this:

Does this PR introduce a user interface change? If mockups are available, please link/include them here:

Is there a release notes update needed for this change?:

Additional documentation:

@qqmyers qqmyers added HDC Harvard Data Commons HDC: 3a Harvard Data Commons Obj. 3A HDC: 3b Harvard Data Commons Objective 3B HDC: 1 Harvard Data Commons Obj. 1 labels Aug 4, 2022
@scolapasta scolapasta added this to the 5.12 milestone Aug 5, 2022
@pdurbin pdurbin self-assigned this Aug 8, 2022
Copy link
Member

@pdurbin pdurbin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall these look great. 5.12 is going to be a huge release! I left some suggestions.

doc/release-notes/8611-DataCommons-related-notes.md Outdated Show resolved Hide resolved
doc/release-notes/8611-DataCommons-related-notes.md Outdated Show resolved Hide resolved

- Initial support for Globus file transfer to upload/download to/from a Dataverse managed S3 store. The current implementation disables file restriction and embargo on Globus-enabled stores.
- Initial support for Remote File Storage. This capability, enabled via a new RemoteOverlay store type, allows a file stored in a remote system to be added to a dataset (currently only via API) with download requests redirected to the remote system. Use cases include referencing public files hosted on external web servers as well as support for controlled access managed by Dataverse (e.g. via restricted and embargoed status) and/or by the remote store.
- Workflow (add Aday's notes here or reword to separate the Objective 2 work)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adaybujeda do you have notes to add here?

doc/release-notes/8611-DataCommons-related-notes.md Outdated Show resolved Hide resolved
doc/release-notes/8611-DataCommons-related-notes.md Outdated Show resolved Hide resolved
doc/release-notes/8611-DataCommons-related-notes.md Outdated Show resolved Hide resolved
doc/release-notes/8611-DataCommons-related-notes.md Outdated Show resolved Hide resolved
doc/release-notes/8611-DataCommons-related-notes.md Outdated Show resolved Hide resolved
doc/release-notes/8611-DataCommons-related-notes.md Outdated Show resolved Hide resolved
doc/release-notes/8611-DataCommons-related-notes.md Outdated Show resolved Hide resolved
Copy link
Member

@pdurbin pdurbin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should point out the verb and path change to the bagit API.


### OAI-ORE and Archival Bag Changes

Earlier versions of the archival bags included the ingested (tab-separated-value) version of tabular files while providing the checksum of the original file (Issue #8449). This release fixes that by including the original file and its metadata in the archival bag. This means that archival bags created prior to this version do not include a way to validate ingested files. Further, it is likely that capabilities in development (i.e. as part of the [Dataverse Uploader](https://github/org/GlobalDataverseCommunityConsortium/dataverse-uploader) to allow re-creation of a dataset version from an archival bag will only be fully compatible with archival bags generated by a Dataverse instance at a release > v5.12. (Specifically, at a minimum, since only the ingested file is included in earlier archival bags, an upload via DVUploader would not result in the same original file/ingested version as in the original dataset.) Administrators should be aware that re-creating archival bags, i.e. via the new batch archiving API, may be advisable now and will be recommended at some point in the future (i.e. there will be a point where we will start versioning archival bags and will start maintaining backward compatibility for older versions as part of transitioning this from being an experimental capability).
Copy link
Member

@pdurbin pdurbin Aug 9, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Under backward incompatibilities can we please make it super clear that

GET /admin/submitDataVersionToArchive

has changed to

POST /admin/submitDatasetVersionToArchive

? This is very likely to trip people up, like I was when reviewing a PR: #8901 (comment)

@pdurbin pdurbin removed their assignment Aug 10, 2022
@qqmyers qqmyers removed their assignment Sep 19, 2022
@pdurbin pdurbin self-assigned this Sep 19, 2022
@pdurbin pdurbin merged commit 9ac2244 into IQSS:develop Sep 19, 2022
@pdurbin pdurbin removed their assignment Sep 19, 2022
@qqmyers qqmyers mentioned this pull request Sep 26, 2022
@mreekie mreekie added the NIH OTA: 1.1.1 1 | 1.1.1 | Minimum Viable Product (MVP) for registering metadata in the repository and connectin... label Oct 8, 2022
@mreekie mreekie added the pm.GREI-d-1.1.1 NIH, yr1, aim1, task1: MVP for registering metadata in the repository label Mar 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
HDC Harvard Data Commons HDC: 1 Harvard Data Commons Obj. 1 HDC: 3a Harvard Data Commons Obj. 3A HDC: 3b Harvard Data Commons Objective 3B NIH OTA: 1.1.1 1 | 1.1.1 | Minimum Viable Product (MVP) for registering metadata in the repository and connectin... pm.GREI-d-1.1.1 NIH, yr1, aim1, task1: MVP for registering metadata in the repository
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Archiving/Bag/OAI_ORE/DataCommons Release notes
4 participants