Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

standardize image url #10855

Merged
merged 7 commits into from
Sep 24, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions doc/release-notes/10831-standardize-image-url-of-search-api.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
Search API (/api/search) response will now include new image_url format for the Datafile and Dataverse logo.
Note to release note writer: this supersedes the release note 10810-search-api-payload-extensions.md

For Dataverse:

- "image_url" (optional)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the image_url is always returned, even though the URL returns a 404 for files and a 204 for collections, then it is not an optional field. Anyway, check my final review comment, where I suggest not adding the fields to the payload in these cases.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GPortas what if when no image is available the endpoint just returns image_url: null ?
In that way we will know that image_url could be either a string(image exists) or null.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed image_url if none exists


```javascript
"items": [
{
"name": "Darwin's Finches",
...
"image_url":"/api/access/dvCardImage/{identifier}"
(etc, etc)
```

For DataFile:

- "image_url" (optional)

```javascript
"items": [
{
"name": "test.txt",
...
"image_url":"/api/access/datafile/{identifier}?imageThumb=true"
(etc, etc)
```
5 changes: 5 additions & 0 deletions doc/sphinx-guides/source/api/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,11 @@ This API changelog is experimental and we would love feedback on its usefulness.
:local:
:depth: 1

v6.4
----

- ** /api/search?q=**: Json values for image_url in DataFiles and Collections have changed from Base64URL ("data:image/png;base64,...) to "/api/access/datafile/{identifier}?imageThumb=true" and "/api/access/dvCardImage/{identifier}" respectively. This was done to match the image_url of Dataset.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need this release note if we merge this before we release 6.4?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed for now. can be re-added if this doesn't make 6.4


v6.3
----

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In #10811 the following line was added:

"Note that the image_url field, if exists, will be returned as a regular URL for Datasets, while for Files and Dataverses, it will be returned as a Base64 URL. We plan to standardize this behavior so that the field always returns a regular URL. (See: #10831)"

Can we remove it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed. also modified the example json

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -361,7 +361,15 @@ public String getDataverseLogoThumbnailAsBase64ById(Long dvId) {
}
return null;
}


public String getDataverseLogoThumbnailAsUrl(Long dvId) {
File dataverseLogoFile = getLogoById(dvId);
if (dataverseLogoFile != null) {
return SystemConfig.getDataverseSiteUrlStatic() + "/api/access/dvCardImage/" + dvId;
}
return null;
}

private File getLogo(Dataverse dataverse) {
if (dataverse.getId() == null) {
return null;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,18 @@ public class ThumbnailServiceWrapper implements java.io.Serializable {
private Map<Long, DvObject> dvobjectViewMap = new HashMap<>();
private Map<Long, Boolean> hasThumbMap = new HashMap<>();

public String getFileCardImageAsUrl(SolrSearchResult result) {
if (result.isHarvested()) {
return null;
}

if (result.getEntity() == null) {
return null;
}

return SystemConfig.getDataverseSiteUrlStatic() + "/api/access/datafile/" + result.getEntity().getId() + "?imageThumb=true";
}

// it's the responsibility of the user - to make sure the search result
// passed to this method is of the Datafile type!
public String getFileCardImageAsBase64Url(SolrSearchResult result) {
Expand Down Expand Up @@ -208,7 +220,13 @@ public String getDatasetCardImageAsUrl(Dataset dataset, Long versionId, boolean
public String getDataverseCardImageAsBase64Url(SolrSearchResult result) {
return dataverseService.getDataverseLogoThumbnailAsBase64ById(result.getEntityId());
}


// it's the responsibility of the user - to make sure the search result
// passed to this method is of the Dataverse type!
public String getDataverseCardImageAsUrl(SolrSearchResult result) {
return dataverseService.getDataverseLogoThumbnailAsUrl(result.getEntityId());
}

public void resetObjectMaps() {
dvobjectThumbnailsMap = new HashMap<>();
dvobjectViewMap = new HashMap<>();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -593,7 +593,7 @@ public SolrQueryResponse search(
solrSearchResult.setDataverseAffiliation(dataverseAffiliation);
solrSearchResult.setDataverseParentAlias(dataverseParentAlias);
solrSearchResult.setDataverseParentName(dataverseParentName);
solrSearchResult.setImageUrl(thumbnailServiceWrapper.getDataverseCardImageAsBase64Url(solrSearchResult));
solrSearchResult.setImageUrl(thumbnailServiceWrapper.getDataverseCardImageAsUrl(solrSearchResult));
/**
* @todo Expose this API URL after "dvs" is changed to
* "dataverses". Also, is an API token required for published
Expand Down Expand Up @@ -652,7 +652,7 @@ public SolrQueryResponse search(
}
solrSearchResult.setHtmlUrl(baseUrl + "/dataset.xhtml?persistentId=" + parentGlobalId);
solrSearchResult.setDownloadUrl(baseUrl + "/api/access/datafile/" + entityid);
solrSearchResult.setImageUrl(thumbnailServiceWrapper.getFileCardImageAsBase64Url(solrSearchResult));
solrSearchResult.setImageUrl(thumbnailServiceWrapper.getFileCardImageAsUrl(solrSearchResult));
/**
* @todo We are not yet setting the API URL for files because
* not all files have metadata. Only subsettable files (those
Expand Down
43 changes: 43 additions & 0 deletions src/test/java/edu/harvard/iq/dataverse/api/SearchIT.java
Original file line number Diff line number Diff line change
Expand Up @@ -1283,4 +1283,47 @@ public void tearDownDataverse() {
public static void cleanup() {
}

@Test
public void testSearchFiles() {
Response createUser = UtilIT.createRandomUser();
createUser.prettyPrint();
String username = UtilIT.getUsernameFromResponse(createUser);
String apiToken = UtilIT.getApiTokenFromResponse(createUser);

Response createDataverseResponse = UtilIT.createRandomDataverse(apiToken);
createDataverseResponse.prettyPrint();
String dataverseAlias = UtilIT.getAliasFromResponse(createDataverseResponse);

Response createDatasetResponse = UtilIT.createRandomDatasetViaNativeApi(dataverseAlias, apiToken);
createDatasetResponse.prettyPrint();
Integer datasetId = UtilIT.getDatasetIdFromResponse(createDatasetResponse);
System.out.println("id: " + datasetId);
String datasetPid = JsonPath.from(createDatasetResponse.getBody().asString()).getString("data.persistentId");
System.out.println("datasetPid: " + datasetPid);

String pathToFile = "src/main/webapp/resources/images/dataverseproject.png";
Response uploadImage = UtilIT.uploadFileViaNative(datasetId.toString(), pathToFile, apiToken);
uploadImage.prettyPrint();
uploadImage.then().assertThat()
.statusCode(200);

Response publishDataverse = UtilIT.publishDataverseViaSword(dataverseAlias, apiToken);
publishDataverse.prettyPrint();
publishDataverse.then().assertThat()
.statusCode(OK.getStatusCode());
Response publishDataset = UtilIT.publishDatasetViaNativeApi(datasetId, "major", apiToken);
publishDataset.prettyPrint();
publishDataset.then().assertThat()
.statusCode(OK.getStatusCode());

Response searchResp = UtilIT.search("dataverseproject", apiToken);
searchResp.prettyPrint();
searchResp.then().assertThat()
.statusCode(OK.getStatusCode())
.body("data.items[0].type", CoreMatchers.is("file"))
.body("data.items[0].file_content_type", CoreMatchers.is("image/png"))
.body("data.items[0].url", CoreMatchers.containsString("/api/access/datafile/"))
.body("data.items[0].image_url", CoreMatchers.containsString("/api/access/datafile/"))
.body("data.items[0].image_url", CoreMatchers.containsString("imageThumb=true"));
}
}
Loading