Merge branch 'develop' into 2419-4475-7575-disable-users #2419 #4475

doc/release-notes/7337-replace-draft-file.md

@@ -0,0 +1,3 @@
+## Major Use Cases
+
+- Users can now replace files in draft datasets. This functionality was previously only available on published datasets. Issue #7149/PR #7337

doc/release-notes/7638-citation-metadatablock-update.md

@@ -0,0 +1,7 @@
+### Citation metadatablock update
+
+Due to a minor update in the citation metadata block (extra ISO-639-3 language codes added) a block upgrade is required:
+
+`wget https://github.com/IQSS/dataverse/releases/download/v5.4/citation.tsv`
+`curl http://localhost:8080/api/admin/datasetfield/load -X POST --data-binary @citation.tsv -H "Content-type: text/tab-separated-values"`
+

doc/release-notes/7687-curate-command.md

@@ -0,0 +1,9 @@
+## Notes for Dataverse Installation Administrators
+
+### DB Cleanup for Superusers Releasing without Version Updates
+
+In datasets where a superuser has run the Curate command and the update included a change to the fileaccessrequest flag, those changes would not be reflected appropriately in the published version. This should be a rare occurrence.
+
+Instead of an automated solution, we recommend inspecting the affected datasets and correcting the fileaccessrequest flag as appropriate. You can identify the affected datasets this via a query, which is available in the folder here:
+
+https://github.com/IQSS/dataverse/raw/develop/scripts/issues/7687/

doc/sphinx-guides/source/admin/harvestserver.rst

@@ -73,6 +73,14 @@ Some useful examples of search queries to define OAI sets:
 
   ``parentId:NNN``
 
+  where NNN is the database id of the Dataverse collection object (consult the Dataverse table of the SQL database used by the application to verify the database id).
+
+  Note that this query does **not** provide datasets that are linked into the specified Dataverse collection.
+
+- A query to create a set to include the datasets from a specific Dataverse collection including datasets that have been deposited into other Dataverse collections but linked into the specified Dataverse collection: 
+
+  ``subtreePaths:"/NNN"``
+
   where NNN is the database id of the Dataverse collection object (consult the Dataverse table of the SQL database used by the application to verify the database id). 
 
 - A query to find all the dataset by a certain author: 

doc/sphinx-guides/source/developers/workflows.rst

@@ -18,12 +18,14 @@ Steps can be internal (say, writing some data to the log) or external. External
 
 The external system reports the step result back to the Dataverse installation, by sending a HTTP ``POST`` command to ``api/workflows/{invocation-id}`` with Content-Type: text/plain. The body of the request is passed to the paused step for further processing.
 
+Steps can define messages to send to the log and to users. If defined, the message to users is sent as a user notification (creating an email and showing in the user notification tab) and will show once for the given user if/when they view the relevant dataset page. The latter provides a means for the asynchronous workflow execution to report success or failure analogous to the way the publication and other processes report on the page.
+
 If a step in a workflow fails, the Dataverse installation makes an effort to roll back all the steps that preceded it. Some actions, such as writing to the log, cannot be rolled back. If such an action has a public external effect (e.g. send an EMail to a mailing list) it is advisable to put it in the post-release workflow.
 
 .. tip::
   For invoking external systems using a REST api, the Dataverse Software's internal step
   provider offers two steps for sending and receiving customizable HTTP requests.
-  *http/sr* and *http/authExt*, detailed below, with the latter able to use the API to make changes to the dataset being processed. Both lock the dataset to prevent other processes from changing the dataset between the time the step is launched to when the external process responds to the Dataverse instance.
+  *http/sr* and *http/authExt*, detailed below, with the latter able to use the API to make changes to the dataset being processed. (Both lock the dataset to prevent other processes from changing the dataset between the time the step is launched to when the external process responds to the Dataverse instance.)
 
 Administration
 ~~~~~~~~~~~~~~
@@ -70,6 +72,23 @@ The pause step is intended for testing - the invocationId required to end the pa
       "stepType":"pause"
   }
 
+pause/message
++++++++++++++
+
+A variant of the  pause step that pauses the workflow and allows the external process to send a success/failure message. The workflow is paused until a POST request is sent to ``/api/workflows/{invocation-id}``. 
+The response in the POST body (Content-type:application/json) should be a json object (the same as for the http/extauth step) containing:
+- "status" - can be "success" or "failure"
+- "reason" - a message that will be logged
+- "message" - a message to send to the user that will be sent as a notification and as a banner on the relevant dataset page.
+An unparsable reponse will be considered a Failure that will be logged with no user message. (See the http/authext step for an example POST call)
+
+.. code:: json
+
+  {
+      "provider":":internal",
+      "stepType":"pause/message"
+  }
+
 
 http/sr
 +++++++
@@ -113,11 +132,20 @@ The invocationId must be sent as an 'X-Dataverse-invocationId' HTTP Header or as
 Once this step completes and responds, the invocationId is invalidated and will not allow further access.
 
 The url, content type, and message body can use data from the workflow context, using a simple markup language. This step has specific parameters for rollback.
-The workflow is restarted when the external system replies with a POST request  to ``/api/workflows/{invocation-id}``.
+The workflow is restarted when the external system replies with a POST request  to ``/api/workflows/{invocation-id}`` (Content-Type: application/json).
+
 The response has is expected to be a json object with three keys:
-- "Status" - can be "Success" or "Failure"
-- "Reason" - a message that will be logged
-- "Message" - a message to send to the user (message sending is not yet implemented).
+- "status" - can be "success" or "failure"
+- "reason" - a message that will be logged
+- "message" - a message to send to the user that will be sent as a notification and as a banner on the relevant dataset page.
+
+.. code-block:: bash
+
+  export INVOCATION_ID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
+  export SERVER_URL=https://demo.dataverse.org
+  export MESSAGE={"status":"success", "reason":"Workflow completed in 10 seconds", "message":"An external workflow to virus check your data was successfully run prior to publication of your data"}
+ 
+  curl -H 'Content-Type:application/json' -X POST -d $MESSAGE "$SERVER_URL/api/workflows/$INVOCATION_ID"
 
 .. code:: json