Update the docs for latest release and improve the publish flow and documentation

PUBLISHING.md

@@ -8,9 +8,18 @@ Once ready to do a release, create a local branch that includes the following up
 
 1. Create a PR branch from an updated `main` branch.
 
-2. Update the CHANGELOG.md to add the new release.  Only create a new section when working on the first release candidate for a new release. When transitioning from one release candidate to the next, or to an official release, just update the title and date of the change log section.
+2. See if there are any Document Site `mkdocs` changes needed. Run the script
+   `./scripts/prepmkdocs.sh; mkdocs`. Watch the log, noting particularly if
+   there are new documentation files that are in the docs folder and not
+   referenced in the mkdocs navigation. If there is, update the `mkdocs.yml`
+   file as necessary. On completion of the testing, run the script
+   `./scripts/prepmkdocs clean` to undo the temporary changes to the docs. Be
+   sure to do the last `clean` step -- **DO NOT MERGE THE TEMPORARY DOC
+   CHANGES.**
 
-3. Include details of the merged PRs included in this release. General process to follow:
+3. Update the CHANGELOG.md to add the new release.  Only create a new section when working on the first release candidate for a new release. When transitioning from one release candidate to the next, or to an official release, just update the title and date of the change log section.
+
+4. Include details of the merged PRs included in this release. General process to follow:
 
 - Gather the set of PRs since the last release and put them into a list. A good
   tool to use for this is the
@@ -91,17 +100,17 @@ Once you have the list of PRs:
 - Organize the list into suitable categories, update (if necessary) the PR description and add notes to clarify the changes. See previous release entries to understand the style -- a format that should help developers.
 - Add a narrative about the release above the PR that highlights what has gone into the release.
 
-4. Check to see if there are any other PRs that should be included in the release.
+5. Check to see if there are any other PRs that should be included in the release.
 
-5. Update the ReadTheDocs in the `/docs` folder by following the instructions in
+6. Update the ReadTheDocs in the `/docs` folder by following the instructions in
    the `docs/UpdateRTD.md` file. That will likely add a number of new and modified
    files to the PR. Eliminate all of the errors in the generation process,
    either by mocking external dependencies or by fixing ACA-Py code. If
    necessary, create an issue with the errors and assign it to the appropriate
    developer. Experience has demonstrated to use that documentation generation
    errors should be fixed in the code.
 
-6. Search across the repository for the previous version number and update it
+7. Search across the repository for the previous version number and update it
    everywhere that makes sense. The CHANGELOG.md entry for the previous release
    is a likely exception, and the `pyproject.toml` in the root **MUST** be
    updated. You can skip (although it won't hurt) to update the files in the
@@ -114,26 +123,26 @@ Once you have the list of PRs:
    have dropped the previously used `-` in the release candidate version string
    to better follow the semver rules.
 
-7. Regenerate openapi.json and swagger.json by running
+8. Regenerate openapi.json and swagger.json by running
    `../scripts/generate-open-api-spec` from within the `aries_cloudagent` folder.
 
    Command: `cd aries_cloudagent;../scripts/generate-open-api-spec;cd ..`
 
-8.  Double check all of these steps above, and then submit a PR from the branch.
+9.  Double check all of these steps above, and then submit a PR from the branch.
    Add this new PR to CHANGELOG.md so that all the PRs are included.
    If there are still further changes to be merged, mark the PR as "Draft",
    repeat **ALL** of the steps again, and then mark this PR as ready and then
    wait until it is merged. It's embarrassing when you have to do a whole new
    release just because you missed something silly...I know!
 
-9.  Immediately after it is merged, create a new GitHub tag representing the
+10. Immediately after it is merged, create a new GitHub tag representing the
    version. The tag name and title of the release should be the same as the
    version in [pyproject.toml](https://github.com/hyperledger/aries-cloudagent-python/tree/main/pyproject.toml). Use
    the "Generate Release Notes" capability to get a sequential listing of the
    PRs in the release, to complement the manually curated Changelog. Verify on
    PyPi that the version is published.
 
-10. New images for the release are automatically published by the GitHubAction
+11. New images for the release are automatically published by the GitHubAction
    Workflows: [publish.yml] and [publish-indy.yml]. The actions are triggered
    when a release is tagged, so no manual action is needed. The images are
    published in the [Hyperledger Package Repository under
@@ -147,11 +156,11 @@ Once you have the list of PRs:
 [publish.yml]: https://github.com/hyperledger/aries-cloudagent-python/blob/main/.github/workflows/publish.yml
 [publish-indy.yml]: https://github.com/hyperledger/aries-cloudagent-python/blob/main/.github/workflows/publish-indy.yml
 
-11. Update the ACA-Py Read The Docs site by building the new "latest" (main
+12. Update the ACA-Py Read The Docs site by building the new "latest" (main
     branch) and activating and building the new release. Appropriate permissions
     are required to publish the new documentation version.
 
-12. Update the [https://aca-py.org] website with the latest documentation by
+13. Update the [https://aca-py.org] website with the latest documentation by
     creating a PR and tag of the latest documentation from this site. Details
     are provided in the [aries-acapy-docs] repository.
 

docs/features/AnoncredsControllerMigration.md → docs/deploying/AnoncredsControllerMigration.md

@@ -1,6 +1,6 @@
-# Anoncreds Controller Migration
+# AnonCreds Controller Migration
 
-To upgrade an agent to use anoncreds a controller should implement the required changes to endpoints and payloads in a way that is backwards compatible. The controller can then trigger the upgrade via the upgrade endpoint.
+To upgrade an agent to use AnonCreds a controller should implement the required changes to endpoints and payloads in a way that is backwards compatible. The controller can then trigger the upgrade via the upgrade endpoint.
 
 ## Step 1 - Endpoint Payload and Response Changes
 
@@ -10,12 +10,12 @@ A good way to implement this with backwards compatibility is to get the wallet t
 
 ## Schemas
 
-#### Creating a Schema:
+### Creating a Schema:
 
 - Change endpoint from **POST /schemas** to **POST /anoncreds/schema**
 - Change payload and parameters from
 
-```
+```yml
 params
  - conn_id
  - create_transaction_for_endorser
@@ -144,12 +144,12 @@ to
 }
 ```
 
-#### Getting schemas:
+#### Getting schemas
 
 - Change endpoint from **GET /schemas/created** to **GET /anoncreds/schemas**
 - Response payloads have no change
 
-#### Getting a schema:
+#### Getting a schema
 
 - Change endpoint from **GET /schemas/{schema_id}** to **GET /anoncreds/schema/{schema_id}**
 - Response payload changed from
@@ -185,12 +185,12 @@ to
 
 ## Credential Definitions
 
-#### Creating a credential definition:
+### Creating a credential definition
 
 - Change endpoint from **POST /credential-definitions** to **POST /anoncreds/credential-definition**
 - Change payload and parameters from
 
-```
+```yml
 params
  - conn_id
  - create_transaction_for_endorser
@@ -297,14 +297,14 @@ With Endorsement:
 }
 ```
 
-#### Getting credential definitions:
+### Getting credential definitions
 
-- Change endpoint from **GET /credential-definitons/created** to **GET /anoncreds/credential-defintions**
+- Change endpoint from **GET /credential-definitions/created** to **GET /anoncreds/credential-definitions**
 - Response payloads have no change
 
-#### Getting a credential definition:
+### Getting a credential definition
 
-- Change endpoint from **GET /credential-definitons/{schema_id}** to **GET /anoncreds/credential-defintion/{cred_def_id}**
+- Change endpoint from **GET /credential-definitions/{schema_id}** to **GET /anoncreds/credential-definition/{cred_def_id}**
 - Response payload changed from
 
 ```json
@@ -345,7 +345,7 @@ to
 
 Most of the changes with revocation endpoints only require prepending `/anoncreds` to the path. There are some other subtle changes listed below.
 
-#### Create and publish registry definition
+### Create and publish registry definition
 
 - The endpoints **POST /revocation/create-registry** and **POST /revocation/registry/{rev_reg_id}/definition** have been replaced by the single endpoint **POST /anoncreds/revocation-registry-definition**
 - Instead of creating the registry with **POST /revocation/create-registry** and payload
@@ -359,7 +359,7 @@ Most of the changes with revocation endpoints only require prepending `/anoncred
 
 - And then publishing with **POST /revocation/registry/{rev_reg_id}/definition**
 
-```
+```yml
 params
  - conn_id
  - create_transaction_for_endorser
@@ -483,12 +483,12 @@ to
 }
 ```
 
-#### Send revocation entry or list to ledger
+### Send revocation entry or list to ledger
 
 - Changes from **POST /revocation/registry/{rev_reg_id}/entry** to **POST /anoncreds/revocation-list**
 - Change from
 
-```
+```yml
 params
  - conn_id
  - create_transaction_for_endorser
@@ -529,27 +529,27 @@ to
 
 ```
 
-#### Get current active registry:
+### Get current active registry:
 
 - Change from **GET /revocation/active-registry/{cred_def_id}** to **GET /anoncreds/revocation/active-registry/{cred_def_id}**
 - No payload changes
 
-#### Rotate active registry
+### Rotate active registry
 
 - Change from **POST /revocation/active-registry/{cred_def_id}/rotate** to **POST /anoncreds/revocation/active-registry/{cred_def_id}/rotate**
 - No payload changes
 
-#### Get credential revocation status
+### Get credential revocation status
 
 - Change from **GET /revocation/credential-record** to **GET /anoncreds/revocation/credential-record**
 - No payload changes
 
-#### Publish revocations
+### Publish revocations
 
 - Change from **POST /revocation/publish-revocations** to **POST /anoncreds/revocation/publish-revocations**
 - Change payload and parameters from
 
-```
+```yml
 params
  - conn_id
  - create_transaction_for_endorser
@@ -583,69 +583,69 @@ to
 
 - options are not required
 
-#### Get registries
+### Get registries
 
 - Change from **GET /revocation/registries/created** to **GET /anoncreds/revocation/registries**
 - No payload changes
 
-#### Get registry
+### Get registry
 
 - Changes from **GET /revocation/registry/{rev_reg_id}** to **GET /anoncreds/revocation/registry/{rev_reg_id}**
 - No payload changes
 
-#### Fix reocation state
+### Fix reocation state
 
 - Changes from **POST /revocation/registry/{rev_reg_id}/fix-revocation-entry-state** to **POST /anoncreds/revocation/registry/{rev_reg_id}/fix-revocation-state**
 - No payload changes
 
-#### Get number of issued credentials
+### Get number of issued credentials
 
 - Changes from **GET /revocation/registry/{rev_reg_id}/issued** to **GET /anoncreds/revocation/registry/{rev_reg_id}/issued**
 - No payload changes
 
-#### Get credential details
+### Get credential details
 
 - Changes from **GET /revocation/registry/{rev_reg_id}/issued/details** to **GET /anoncreds/revocation/registry/{rev_reg_id}/issued/details**
 - No payload changes
 
-#### Get revoked credential details
+### Get revoked credential details
 
 - Changes from **GET /revocation/registry/{rev_reg_id}/issued/indy_recs** to **GET /anoncreds/revocation/registry/{rev_reg_id}/issued/indy_recs**
 - No payload changes
 
-#### Set state manually
+### Set state manually
 
 - Changes from **PATCH /revocation/registry/{rev_reg_id}/set-state** to **PATCH /anoncreds/revocation/registry/{rev_reg_id}/set-state**
 - No payload changes
 
-#### Upload tails file
+### Upload tails file
 
 - Changes from **PUT /revocation/registry/{rev_reg_id}/tails-file** to **PUT /anoncreds/registry/{rev_reg_id}/tails-file**
 - No payload changes
 
-#### Download tails file
+### Download tails file
 
 - Changes from **GET /revocation/registry/{rev_reg_id}/tails-file** to **GET /anoncreds/revocation/registry/{rev_reg_id}/tails-file**
 - No payload changes
 
-#### Revoke a credential
+### Revoke a credential
 
 - Changes from **POST /revocation/revoke** to **POST /anoncreds/revocation/revoke**
 - Change payload and parameters from
 
-#### Clear pending revocations
+### Clear pending revocations
 
 - **POST /revocation/clear-pending-revocations** has been removed.
 
-#### Delete tails file
+### Delete tails file
 
 - Endpoint **DELETE /revocation/delete-tails-server** has been removed
 
-#### Update tails file
+### Update tails file
 
 - Endpoint **PATCH /revocation/registry/{rev_reg_id}** has been removed
 
-#### Additional Endpoints
+### Additional Endpoints
 
 - **PUT /anoncreds/registry/{rev_reg_id}/active** is available to set the active registry
 
@@ -664,12 +664,12 @@ The behavior for a base wallet (standalone) or admin wallet in multitenant mode
 3. Call the upgrade endpoint
 4. Scale up the controller instances to handle new endpoints
 
-### Base wallet (standalone) or admin wallet in multitenant mode:
+### Base wallet (standalone) or admin wallet in multitenant mode
 
 The agent will get a 503 error during the upgrade process. Any agent instance will shut down when the upgrade is complete. It is up to the aca-py agent to start up again. After the upgrade is complete the old endpoints will no longer be available and result in a 400 error.
 
 The aca-py agent will work after the restart. However, it will receive a warning for having the wrong wallet type configured. It is recommended to change the `wallet-type` to `askar-anoncreds` in the agent configuration file or start-up command.
 
-### Subwallet (tenant) in multitenancy mode:
+### Subwallet (tenant) in multitenancy mode
 
 The sub-tenant which is in the process of being upgraded will get a 503 error during the upgrade process. All other sub-tenants will continue to operate normally. After the upgrade is complete the sub-tenant will be able to use the new endpoints. The old endpoints will no longer be available and result in a 403 error. Any aca-py agents will remain running after the upgrade and it's not required that the aca-py agent restarts. 

docs/design/UpgradeViaApi.md

@@ -1,8 +1,12 @@
 # Upgrade via API Design
 
-#### To isolate an upgrade process and trigger it via API the following pattern was designed to handle multitenant scenarios. It includes an is_upgrading record in the wallet(DB) and a middleware to prevent requests during the upgrade process.
+## Design Goals
 
-#### The diagam below descripes the sequence of events for the anoncreds upgrade process which it was designed for, but the architecture can be used for any upgrade process.
+To isolate an upgrade process and trigger it via API the following pattern was designed to handle multitenant scenarios. It includes an is_upgrading record in the wallet(DB) and a middleware to prevent requests during the upgrade process.
+
+## Flow
+
+The diagram below describes the sequence of events for the anoncreds upgrade process which it was designed, but the architecture can be used for any upgrade process.
 
 ```mermaid
 sequenceDiagram
@@ -94,10 +98,12 @@ sequenceDiagram
     Note over A2: Same as Agent 1
 ```
 
+## Example
+
+An example of the implementation can be found via the anoncreds upgrade components.
 
-##### An example of the implementation can be found via the anoncreds upgrade components.
-    - `aries_cloudagent/wallet/routes.py` in the `upgrade_anoncreds` controller 
-    - the upgrade code in `wallet/anoncreds_upgrade.py`
-    - the middleware in `admin/server.py` in the `upgrade_middleware` function
-    - the singleton sets in `wallet/singletons.py`
-    - the startup process in `core/conductor.py` in the `check_for_wallet_upgrades_in_progress` function
+- `aries_cloudagent/wallet/routes.py` in the `upgrade_anoncreds` controller 
+- the upgrade code in `wallet/anoncreds_upgrade.py`
+- the middleware in `admin/server.py` in the `upgrade_middleware` function
+- the singleton sets in `wallet/singletons.py`
+- the startup process in `core/conductor.py` in the `check_for_wallet_upgrades_in_progress` function

mkdocs.yml

@@ -134,6 +134,7 @@ nav:
     - Deployment Model: deploying/deploymentModel.md
     - Upgrading ACA-Py: deploying/UpgradingACA-Py.md
     - Indy SDK to Askar Migration: deploying/IndySDKtoAskarMigration.md
+    - Controller Migration to use AnonCreds Rust: deploying/AnoncredsControllerMigration.md
     - The Use of Poetry in ACA-Py: deploying/Poetry.md
     - ACA-Py Container Images: deploying/ContainerImagesAndGithubActions.md
     - Databases: deploying/Databases.md

scripts/prepmkdocs.sh

@@ -1,8 +1,7 @@
 #!/bin/bash
 
 # A script to prep for testing of mkdocs generation, and then to clean up after
-# Replicates the file preparation done in the .github/workflows/publish-docs Git Hub Action
-# TODO: Move all the prepaanupCalled from the GHA to do all of the work being done there.
+# Called as the file preparation step in the .github/workflows/publish-docs Git Hub Action
 #
 # Add argument "clean" to undo these changes when just being used for testing.
 #    WARNING -- does a `git checkout -- docs` so you will lose any others changes you make!!!
@@ -14,12 +13,13 @@ if [[ "$1" == "clean" ]]; then
         docs/MAINTAINERS.md \
         docs/PUBLISHING.md \
         docs/SECURITY.md
-    git checkout -- docs
+    ## Update the following line to "clean" any changes made below to files that remain in the `docs` folder
+    git checkout -- docs/README.md docs/demo/AriesOpenAPIDemo.md docs/demo/AliceGetsAPhone.md docs/features/DevReadMe.md
 else
     # Copy all of the root level md files into the docs folder for deployment, tweaking the relative paths
     for i in *.md; do sed -e "s#docs/#./#g" $i >docs/$i; done
     # Fix references in DevReadMe.md to moved files
-    sed -e "s#\.\./\.\./#../#" docs/features/DevReadMe.md >tmp.md; mv tmp.md docs/features/DevReadMe.md
+    sed -e "s#\.\./\.\./#../#g" docs/features/DevReadMe.md >tmp.md; mv tmp.md docs/features/DevReadMe.md
     # Fix image references in demo documents so they work in GitHub and mkdocs
     for i in docs/demo/AriesOpenAPIDemo.md docs/demo/AliceGetsAPhone.md; do sed -e "s#src=.collateral#src=\"../collateral#" $i >$i.tmp; mv $i.tmp $i; done
     # Cleanup indented bullets in at least the CHANGELOG.md so they look right when published