Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

5050 Parse all dc identifier elements and allow identifiers that don'… #7214

Merged
merged 79 commits into from
Sep 30, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
a018055
cleanup - add explicit type
qqmyers Apr 11, 2020
d314431
add multipart upload api calls and add to S3StorageIO class
qqmyers Apr 11, 2020
dbbd569
Merge remote-tracking branch 'IQSS/develop' into IQSS/6763
qqmyers Apr 17, 2020
61448a8
update error msgs
qqmyers Apr 17, 2020
cd7e345
try test update
qqmyers Apr 17, 2020
8b88b39
Restore driverId check in getMainKey to pass tests
qqmyers Apr 17, 2020
7b91c07
remove unused imports
qqmyers Apr 24, 2020
6b03884
Merge remote-tracking branch 'IQSS/develop' into IQSS/6763
qqmyers Apr 24, 2020
e707519
report the exception message when can't access bucket
qqmyers Apr 24, 2020
dab4af6
typo on comparison
qqmyers Apr 24, 2020
d322809
IQSS/6829 - account for file failures
qqmyers Apr 24, 2020
6ca407c
remove ~duplicate code
qqmyers Apr 24, 2020
b0557aa
IQSS-6829 - avoid race with 2+ files uploading
qqmyers Apr 27, 2020
beb419e
IQSS/6829 - create mode uses DatasetPage - initialize identifier there
qqmyers Apr 27, 2020
8400c20
move directUploadEnabled to systemconfig
qqmyers Apr 27, 2020
6f9baed
typo/fix method calls
qqmyers Apr 27, 2020
f841d59
set minimum part size
qqmyers Apr 27, 2020
f844914
add convenience urls
qqmyers Apr 27, 2020
db00c6f
add dv_status=temp tag in multipart uploads
qqmyers Apr 29, 2020
f473fab
return no content for delete call
qqmyers Apr 30, 2020
088efb0
add request for upload in parts method
qqmyers Apr 30, 2020
be27169
handle net:ERR_NETWORK_CHANGED errors w/o 'undefined' error
qqmyers May 1, 2020
ef6ef13
remove draft code from other branch
qqmyers May 1, 2020
a666a7b
handle dataverse change - keep GlobalId in direct upload case
qqmyers May 1, 2020
4bb1b66
IQSS/6881 delete temp files when cancelling dataset create
qqmyers May 1, 2020
45245b3
IQSS 6881 cleanup temp files on cancel when creating dataset
qqmyers May 4, 2020
e2d7207
get/set for uploadInProgress
qqmyers May 4, 2020
56bd6a7
actually initialize uploadInProgress
qqmyers May 4, 2020
40f2cd1
limit processing on cancelCreate
qqmyers May 5, 2020
dc3f130
catch all files in direct upload cancel + debug statements
qqmyers May 6, 2020
7d51511
split cancel into two parts, allow stopping reports in edit mode
qqmyers May 6, 2020
0ee39a9
use redirect
qqmyers May 6, 2020
7ccd4b9
fix remotecommand and disable buttons if waiting
qqmyers May 6, 2020
4974cdf
don't bubble click by returning false
qqmyers May 11, 2020
2737f43
one handleExternalUpload call at a time
qqmyers May 11, 2020
a2b47ec
Merge remote-tracking branch 'IQSS/develop' into IQSS/6763
qqmyers May 11, 2020
1963f09
Merge remote-tracking branch 'IQSS/develop' into IQSS/6763
qqmyers May 22, 2020
dbecab9
add a delay for tiny files
qqmyers May 22, 2020
e1993b2
more changes to fix concurrency issues
qqmyers May 28, 2020
a1c2c92
bug - requests for URL weren't queued after initial 4
qqmyers Jun 5, 2020
a3b775b
update version console entry
qqmyers Jun 5, 2020
b4f7178
make it easier to adjust delay
qqmyers Jun 5, 2020
0a546d9
decrease to 100 ms delay
qqmyers Jun 5, 2020
c438219
correct save button selector for disbale during cancel
qqmyers Jun 5, 2020
6bae264
fix selector
qqmyers Jun 5, 2020
37ab57a
update console version info
qqmyers Jun 5, 2020
6d6d2b0
Merge remote-tracking branch 'IQSS/develop' into IQSS/6763
qqmyers Jun 15, 2020
d4ae322
Merge remote-tracking branch 'IQSS/develop' into IQSS/6763
qqmyers Jun 17, 2020
050b59d
removing whitespace changes
qqmyers Jun 17, 2020
66d84c1
removing mp-upload-specific changes
qqmyers Jun 17, 2020
a792363
Merge remote-tracking branch 'IQSS/develop' into IQSS/6993
qqmyers Aug 10, 2020
319b394
fix for #7184 spaces changed in s3 direct download
qqmyers Aug 11, 2020
76c0b3c
The version pf hibernate-validator that we were using, 5.0.3.Final wa…
rtreacy Aug 20, 2020
438a7d0
code cleanup
rtreacy Aug 20, 2020
ab33496
sorry, one more line of commented code removed
rtreacy Aug 20, 2020
0caf09e
+ note
djbrooke Aug 20, 2020
e7638c1
adding character
djbrooke Aug 20, 2020
10c2fb7
Merge remote-tracking branch 'IQSS/develop' into IQSS/6993
qqmyers Aug 20, 2020
df442c5
Change ek dependency to jakarta.el provided by Payara
rtreacy Aug 20, 2020
a7fcdfa
Use hibernate-validator provided with Payara (still a 6.1.x -6.1.2 sp…
rtreacy Aug 20, 2020
2caf302
5050 Parse all dc identifier elements and allow identifiers that don'…
JingMa87 Aug 21, 2020
5a547a6
#7215 correct dataciterestapiurlstrings, add mdcbaseurlstring deletio…
Aug 21, 2020
cdbb7f2
Merge pull request #6994 from GlobalDataverseCommunityConsortium/IQSS…
kcondon Aug 24, 2020
37433c8
Merge remote-tracking branch 'IQSS/develop' into IQSS/7184-encode_spa…
qqmyers Aug 24, 2020
fb38e66
Merge remote-tracking branch 'origin/IQSS/7184-encode_space_in_filena…
qqmyers Aug 24, 2020
6e1fe74
Merge pull request #7186 from QualitativeDataRepository/IQSS/7184-enc…
kcondon Aug 24, 2020
1d13880
Merge pull request #7216 from OdumInstitute/develop
kcondon Aug 25, 2020
1dd2431
changed hibernat-validator scope from provided to compile, because we…
rtreacy Aug 25, 2020
7f3051a
Remove compile scope as unneccessary because it is the default scope
rtreacy Aug 26, 2020
5e3b6ac
Last commit included experimental removal of jakarta.el, which does n…
rtreacy Aug 26, 2020
796e612
Merge pull request #7222 from IQSS/security18-hibernate-validator-vul…
kcondon Aug 26, 2020
89cb145
Use handle library to resolve handle.
JingMa87 Aug 26, 2020
c84ca49
5050 Parse all dc identifier elements and allow identifiers that don'…
JingMa87 Aug 21, 2020
ab1375b
Use handle library to resolve handle.
JingMa87 Aug 26, 2020
06554b0
Make identifier lower case.
JingMa87 Sep 28, 2020
6d5fd09
Make identifier lower case.
JingMa87 Sep 28, 2020
3d0524b
Remove unused variable.
JingMa87 Sep 28, 2020
20a1a08
Change to startsWith() and check for doi and hdl URLs.
JingMa87 Sep 29, 2020
23a0252
Remove toLowerCase().
JingMa87 Sep 29, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions doc/release-notes/5.0-release-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -302,13 +302,15 @@ Add the below JVM options beneath the -Ddataverse settings:

For production environments:

`/usr/local/payara5/bin/asadmin create-jvm-options "\-Ddoi.dataciterestapiurlstring=https://api.datacite.org"`
`/usr/local/payara5/bin/asadmin create-jvm-options "\-Ddoi.dataciterestapiurlstring=https\://api.datacite.org"`

For test environments:

`/usr/local/payara5/bin/asadmin create-jvm-options "\-Ddoi.dataciterestapiurlstring=https://api.test.datacite.org"`
`/usr/local/payara5/bin/asadmin create-jvm-options "\-Ddoi.dataciterestapiurlstring=https\://api.test.datacite.org"`

The JVM option `doi.mdcbaseurlstring` should be deleted if it was previously set.
The JVM option `doi.mdcbaseurlstring` should be deleted if it was previously set, for example:

`/usr/local/payara5/bin/asadmin delete-jvm-options "\-Ddoi.mdcbaseurlstring=https\://api.test.datacite.org"`

4. (Recommended for installations using DataCite) Pre-register DOIs

Expand Down
7 changes: 7 additions & 0 deletions doc/release-notes/7184-spaces-in-filenames.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
## Notes for Tool Developers and Integrators

### Filenames

Dataverse Installations using S3 storage will no longer replace spaces in file names with the + character. If your tool or integration has any special handling around this character change, you can remove it.

(review this note if this is in the same release as the fix for #7188)
8 changes: 6 additions & 2 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -294,9 +294,13 @@
<version>1.7</version> <!-- Or 1.8-SNAPSHOT -->
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<groupId>org.hibernate.validator</groupId>
<artifactId>hibernate-validator</artifactId>
<version>5.0.3.Final</version>
</dependency>
<dependency>
<groupId>org.glassfish</groupId>
<artifactId>jakarta.el</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>commons-lang</groupId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@
import javax.persistence.NoResultException;
import javax.persistence.PersistenceContext;
import javax.xml.stream.XMLInputFactory;
import net.handle.hdllib.HandleException;
import net.handle.hdllib.HandleResolver;


/**
Expand Down Expand Up @@ -366,27 +368,42 @@ private FieldDTO makeDTO(DatasetFieldType dataverseFieldType, FieldDTO value, St
}

private String getOtherIdFromDTO(DatasetVersionDTO datasetVersionDTO) {
List<String> otherIds = new ArrayList<>();
for (Map.Entry<String, MetadataBlockDTO> entry : datasetVersionDTO.getMetadataBlocks().entrySet()) {
String key = entry.getKey();
MetadataBlockDTO value = entry.getValue();
if ("citation".equals(key)) {
for (FieldDTO fieldDTO : value.getFields()) {
if (DatasetFieldConstant.otherId.equals(fieldDTO.getTypeName())) {
String otherId = "";
for (HashSet<FieldDTO> foo : fieldDTO.getMultipleCompound()) {
for (FieldDTO next : foo) {
if (DatasetFieldConstant.otherIdValue.equals(next.getTypeName())) {
otherId = next.getSinglePrimitive();
otherIds.add(next.getSinglePrimitive());
}
}
if (!otherId.isEmpty()){
return otherId;
}
}
}
}
}
}
if (!otherIds.isEmpty()) {
// We prefer doi or hdl identifiers like "doi:10.7910/DVN/1HE30F"
for (String otherId : otherIds) {
if (otherId.startsWith(GlobalId.DOI_PROTOCOL) || otherId.startsWith(GlobalId.HDL_PROTOCOL) || otherId.startsWith(GlobalId.DOI_RESOLVER_URL) || otherId.startsWith(GlobalId.HDL_RESOLVER_URL)) {
return otherId;
}
}
// But identifiers without hdl or doi like "10.6084/m9.figshare.12725075.v1" are also allowed
for (String otherId : otherIds) {
try {
HandleResolver hr = new HandleResolver();
hr.resolveHandle(otherId);
return GlobalId.HDL_PROTOCOL + ":" + otherId;
} catch (HandleException e) {
logger.fine("Not a valid handle: " + e.toString());
}
}
}
return null;
}

Expand Down
Loading