Feat/prep 2020.9

CHANGELOG.md

@@ -7,6 +7,25 @@ Our versioning scheme is `YEAR.N` where `N` is incremented whenever a new releas
 ## [upcoming release]
 
 
+## [2020.9] - 2020-08-26
+
+### Added
+
+- Exceptions thrown through the API are now [documented](include/libaktualizr/aktualizr.h): [PR](https://github.com/advancedtelematic/aktualizr/pull/1737)
+- The client TLS certifcate and key can be re-imported from the filesystem as long as the device ID is unchanged: [PR](https://github.com/advancedtelematic/aktualizr/pull/1743)
+
+### Changed
+
+- More required headers for libaktualizr usage have been refactored for easier use: [PR](https://github.com/advancedtelematic/aktualizr/pull/1719)
+- All code is now checked with clang-tidy-10: [PR](https://github.com/advancedtelematic/aktualizr/pull/1724)
+
+### Removed
+
+- The Debain package manager has been removed as it was never fully functional: [PR](https://github.com/advancedtelematic/aktualizr/pull/1739)
+- Android support has been removed as it was an unfinished prototype: [PR](https://github.com/advancedtelematic/aktualizr/pull/1732)
+- The ISO-TP Secondary has been removed as it was an unmaintained prototype: [PR](https://github.com/advancedtelematic/aktualizr/pull/1732)
+
+
 ## [2020.8] - 2020-07-09
 
 ### Special considerations
@@ -38,7 +57,7 @@ As a result of changes to the IP/POSIX Secondary protocol (see below), users of
 
 ### Fixed
 
-- Various docker-app fixes: [PR](https://github.com/advancedtelematic/aktualizr/pull/1664) [PR](https://github.com/advancedtelematic/aktualizr/pull/1665)
+- Various docker-app fixes: [PR #1664](https://github.com/advancedtelematic/aktualizr/pull/1664) and [PR #1665](https://github.com/advancedtelematic/aktualizr/pull/1665)
 - Use ED25519 to sign manifests when set as key type: [PR](https://github.com/advancedtelematic/aktualizr/pull/1608)
 
 ## [2020.6] - 2020-04-30
@@ -252,9 +271,9 @@ As a result of changes to the IP/POSIX Secondary protocol (see below), users of
 - Target object equality requires that hardware IDs match: [PR](https://github.com/advancedtelematic/aktualizr/pull/1258)
 - aktualizr-repo image command now requires a hardware ID: [PR](https://github.com/advancedtelematic/aktualizr/pull/1258)
 - `GetStoredTargets` and `DeleteStoredTarget` aktualizr API methods: [PR](https://github.com/advancedtelematic/aktualizr/pull/1290)
-- [aktualizr-get](https://github.com/advancedtelematic/aktualizr/blob/master/src/aktualizr_get/main.cc) debugging tool: [PR](https://github.com/advancedtelematic/aktualizr/pull/1276)
+- [aktualizr-get](src/aktualizr_get/main.cc) debugging tool: [PR](https://github.com/advancedtelematic/aktualizr/pull/1276)
 - Automatic reboot command is now customizable: [PR](https://github.com/advancedtelematic/aktualizr/pull/1274)
-- Basic [C API](https://github.com/advancedtelematic/aktualizr/blob/master/include/libaktualizr-c.h): [PR](https://github.com/advancedtelematic/aktualizr/pull/1263)
+- Basic [C API](include/libaktualizr-c.h): [PR](https://github.com/advancedtelematic/aktualizr/pull/1263)
 - Ability to pass custom headers in HTTP requests: [PR](https://github.com/advancedtelematic/aktualizr/pull/1251)
 - Mutual TLS support in garage tools: [PR #1243](https://github.com/advancedtelematic/aktualizr/pull/1243) and [PR #1288](https://github.com/advancedtelematic/aktualizr/pull/1288)
 
@@ -323,10 +342,10 @@ As a result of changes to the IP/POSIX Secondary protocol (see below), users of
 ### Added
 
 - New tool aktualizr-lite for anonymous TUF-only updates: [PR](https://github.com/advancedtelematic/aktualizr/pull/1107)
-- [Abort() API call](src/libaktualizr/primary/aktualizr.h): [PR](https://github.com/advancedtelematic/aktualizr/pull/1122)
+- [Abort() API call](include/libaktualizr/aktualizr.h): [PR](https://github.com/advancedtelematic/aktualizr/pull/1122)
 - [Option to print delegation metadata with aktualizr-info](src/aktualizr_info/main.cc): [PR](https://github.com/advancedtelematic/aktualizr/pull/1138)
 - Support for custom URIs for downloading targets: [PR](https://github.com/advancedtelematic/aktualizr/pull/1147)
-- [SendManifest() API call](src/libaktualizr/primary/aktualizr.h): [PR](https://github.com/advancedtelematic/aktualizr/pull/1176)
+- [SendManifest() API call](include/libaktualizr/aktualizr.h): [PR](https://github.com/advancedtelematic/aktualizr/pull/1176)
 - [Support for Android package management](src/libaktualizr/package_manager/androidmanager.h): [PR](https://github.com/advancedtelematic/aktualizr/pull/1034)
 
 ### Changed
@@ -367,7 +386,7 @@ As a result of changes to the IP/POSIX Secondary protocol (see below), users of
 
 ### Changed
 
-- [Most API calls refactored to return immediately with a future](src/libaktualizr/primary/aktualizr.h)
+- [Most API calls refactored to return immediately with a future](include/libaktualizr/aktualizr.h)
 - With an OSTree Primary, an installation is now considered successful when the device reboots with the new file system version. Before that, the installation is still considered in progress.
 - [Running modes in libaktualizr have been replaced by simpler logic in the aktualizr wrapper](src/aktualizr_primary/main.cc): [PR](https://github.com/advancedtelematic/aktualizr/pull/1039)
 - Tests now use ed25519 as the default key type: [PR](https://github.com/advancedtelematic/aktualizr/pull/1038)
@@ -393,7 +412,7 @@ As a result of changes to the IP/POSIX Secondary protocol (see below), users of
 
 ### Added
 
-- [Ability to pause and resume binary update downloads](src/libaktualizr/primary/aktualizr.h)
+- [Ability to pause and resume binary update downloads](include/libaktualizr/aktualizr.h)
 - Expose download binary targets in API
 
 ### Changed

docs/README.adoc

@@ -40,6 +40,7 @@ The link above is for the doxygen docs on master. Doxygen docs for the following
 * https://advancedtelematic.github.io/aktualizr/2020.6/index.html[2020.6]
 * https://advancedtelematic.github.io/aktualizr/2020.7/index.html[2020.7]
 * https://advancedtelematic.github.io/aktualizr/2020.8/index.html[2020.8]
+* https://advancedtelematic.github.io/aktualizr/2020.9/index.html[2020.9]
 ====
 
 == Release process

docs/ota-client-guide/antora.yml

@@ -1,6 +1,6 @@
 name: ota-client
 title: OTA Connect Developer Guide
 version: latest
-display_version: 2020.8 (latest)
+display_version: 2020.9 (latest)
 nav:
 - modules/ROOT/nav.adoc

docs/ota-client-guide/modules/ROOT/examples/sota-local.toml

@@ -1,6 +1,8 @@
 [provision]
 provision_path = "credentials.zip"
 primary_ecu_hardware_id = "local-fake"
+# This setting is for testing purposes only. Do not use in a real device.
+mode = "SharedCredReuse"
 
 [logger]
 loglevel = 1
@@ -10,6 +12,7 @@ path = "storage"
 
 [pacman]
 type = "none"
+images_path = "storage/images"
 
 [uptane]
 secondary_config_file = "virtualsec.json"

docs/ota-client-guide/modules/ROOT/nav.adoc

@@ -71,7 +71,9 @@ ifndef::env-github[:pageroot:]
 
 * xref:{pageroot}secure-software-updates.adoc[Secure your software repository]
 ** xref:{pageroot}install-garage-sign-deploy.adoc[Install the garage-deploy tool]
+** xref:{pageroot}keep-local-repo-on-external-storage.adoc[Keep your repository on external storage]
 ** xref:{pageroot}rotating-signing-keys.adoc[Rotate keys for Root and Targets metadata]
+** xref:{pageroot}change-signature-thresholds.adoc[Change signature thresholds]
 ** xref:{pageroot}metadata-expiry.adoc[Manage metadata expiry dates]
 
 .Deploy your OTA-enabled solution
@@ -86,6 +88,7 @@ ifndef::env-github[:pageroot:]
 * xref:{pageroot}rollback.adoc[Set up rollback behavior]
 * xref:{pageroot}deb-package-install.adoc[Install the client from a deb package]
 * xref:{pageroot}upload-large-binary.adoc[Upload a binary file]
+* xref:{pageroot}remove-sw-version.adoc[Remove a software version]
 * xref:{pageroot}push-images-with-bitbake.adoc[Upload Yocto images using offline credentials]
 * xref:{pageroot}virtual-secondaries.adoc[Use Virtual Secondaries]
 

docs/ota-client-guide/modules/ROOT/pages/_partials/aktualizr-version.adoc

@@ -3,8 +3,8 @@
 // the version being viewed, but when we are referencing aktualizr from
 // the other, non-versioned docs, we want to make sure we're using the
 // latest version.
-:aktualizr-version: 2020.8
+:aktualizr-version: 2020.9
 
-:yocto-version: 3.0
+:yocto-version: 3.1
 
-:yocto-branch: zeus
+:yocto-branch: dunfell

docs/ota-client-guide/modules/ROOT/pages/aktualizr-config-options.adoc

@@ -153,7 +153,10 @@ Options for how aktualizr stores data locally.
 |==========================================================================================
 | Name                      | Default                   | Description
 | `type`                    | `"sqlite"`                | What type of storage driver to use. Options: `"sqlite"`. The former `"filesystem"` option is now disabled, existing devices will be migrated (see note below)
-| `path`                    | `"/var/sota"`             | Directory for storage
+| `path`                    | `"/var/sota"`             | Directory for storage.
+
+This should be a directory dedicated to aktualizr data. Aktualizr will attempt to set permissions on this directory, so this option should not be set to anything that is used for another purpose. In particular, do not set it to `/` or to your home directory, as this may render your system unusable.
+
 | `sqldb_path`              | `"sql.db"`                | Relative path to the database file.
 | `uptane_metadata_path`    | `"metadata"`              | Path to the uptane metadata store, for migration from `filesystem`.
 | `uptane_private_key_path` | `"ecukey.der"`            | Relative path to the Uptane specific private key, for migration from `filesystem`.

docs/ota-client-guide/modules/ROOT/pages/change-signature-thresholds.adoc

@@ -0,0 +1,58 @@
+= Change signature thresholds
+ifdef::env-github[]
+
+[NOTE]
+====
+We recommend that you link:https://docs.ota.here.com/ota-client/latest/{docname}.html[view this article in our documentation portal]. Not all of our articles render correctly in GitHub.
+====
+endif::[]
+
+One way you can keep your software repository more secure is by adding extra signing keys, and then setting up *key thresholds* to require metadata to be signed by multiple different keys. By default, each role on the repository has one signing key and a threshold of one. For more security in the Root role, for example, you might want to create 5 different keys, keep them locked up at different locations around the world, and require that at least three of the keys sign off on any change to Root metadata. You can also use key thresholds for delegations.
+
+*To change the thresholds for signing Root metadata:*
+
+. Generate a new Root key.
++
+[source,bash]
+----
+garage-sign key generate --repo <localreponame> --name root-key-1 --type rsa
+----
+
+. Add the new Root key to the `root.json` file.
++
+[source,bash]
+----
+garage-sign root key add --repo <localreponame> --key-name </path/to/public-root-key-1>
+----
+
+. If you need more Root keys, repeat steps 1 and 2.
+. In the *Roles* folder of your local software repository, open the *Unsigned* folder, and then open the `root.json` file.
+. Depending on the threshold that you want to change, in the `signed` block, in the `root` subsection, for the `keyids` object, specify the list of valid keys that you want to use for signing.
++
+You can find the list of all your Root keys in the `keys` subsection.
++
+NOTE: The new version of Root metadata should be valid according to the rules of the previous and current versions. So you must sign Root metadata with the threshold of keys specified in the previous Root metadata file and the threshold of keys specified in the new Root metadata file. For more information, see the https://uptane.github.io/papers/uptane-standard.1.0.1.html#rfc.section.5.4.4.3[related section] of the Uptane standard.
+
+. For the `version` object, specify the version number of the new Root metadata.
+. Sign the `root.json` file with the same number of valid signatures that you specified in the Root key threshold.
++
+[source,bash]
+----
+garage-sign root sign \
+  --repo <localreponame> \
+  -k </path/to/public-root-key-1> \
+  -k </path/to/public-root-key-2> \
+  --expires <root-metadata-expiry-date>
+----
+
+. Push the modified `root.json` file to OTA Connect.
++
+[source,bash]
+----
+garage-sign root push \
+  --repo <localreponame>
+----
+
+To change the threshold of delegations, modify the .json file with the delegation metadata in the same way as the `root.json` file.
+
+To learn more about the `garage-sign` commands and options, see its xref:garage-sign-reference.adoc[reference] documentation.

docs/ota-client-guide/modules/ROOT/pages/garage-sign-reference.adoc

@@ -391,7 +391,10 @@ Note that this *will not* make the binary available on its own. After the upload
 
 To learn how to use the garage-sign tool, see the following documentation:
 
+* xref:keep-local-repo-on-external-storage.adoc[Keep your repository on external storage]
 * xref:rotating-signing-keys.adoc[Rotate keys for Root and Targets metadata]
+* xref:change-signature-thresholds.adoc[Change signature thresholds]
 * xref:metadata-expiry.adoc[Manage metadata expiry dates]
-* xref:upload-large-binary.adoc[Upload a binary file]
 * xref:customise-targets-metadata.adoc[Add custom metadata fields to Targets metadata]
+* xref:upload-large-binary.adoc[Upload a binary file]
+* xref:remove-sw-version.adoc[Remove a software version]

docs/ota-client-guide/modules/ROOT/pages/keep-local-repo-on-external-storage.adoc

@@ -0,0 +1,31 @@
+= Keep your repository on external storage
+ifdef::env-github[]
+
+[NOTE]
+====
+We recommend that you link:https://docs.ota.here.com/ota-client/latest/{docname}.html[view this article in our documentation portal]. Not all of our articles render correctly in GitHub.
+====
+endif::[]
+
+Your software repository contains information on your software images and packages, as well as your Root and Targets keys.
+For safety reasons, we recommend keeping your keys offline on an external storage device, and for convenience, we recommend to simply keep the entire repository structure (with the keys included) on external storage. This external device should be kept offline, in a securely locked location, and only plugged into a computer when you need to make changes to the repository.
+
+*To get a copy of a software repository on external storage:*
+
+. Connect the external storage to your computer.
+. Make sure you have the link:https://tuf-cli-releases.ota.here.com/index.html[latest version,window="_blank"] of the `garage-sign` tool.
+. Get the .zip file with your provisioning credentials and save it to your external storage.
++
+For instructions, see the xref:ota-client::generating-provisioning-credentials.adoc[related] section in this guide.
+. In the folder of your external storage, initialize a local repository.
++
+[source, bash]
+----
+garage-sign init \
+  --repo <localreponame> \
+  --credentials </path/to/credentials.zip>
+----
+
+Work with your local repository only on the external storage.
+
+To learn more about the `garage-sign` commands and options, see its xref:garage-sign-reference.adoc[reference] documentation.

docs/ota-client-guide/modules/ROOT/pages/release-process.adoc

@@ -169,6 +169,6 @@ Newly created releases automatically trigger an OTF pipeline in gitlab. Currentl
 
 == Update meta-updater
 
-The version of aktualizr used by link:https://github.com/advancedtelematic/meta-updater/[meta-updater] should be updated to match the new release. First, open a PR against master that updates aktualizr to the same commit used in the newly released tag. This is also a good time to update the aktualizr recipe to pull the latest version of link:https://ats-tuf-cli-releases.s3-eu-central-1.amazonaws.com/index.html[garage-sign].
+The version of aktualizr used by link:https://github.com/advancedtelematic/meta-updater/[meta-updater] should be updated to match the new release. First, open a PR against dunfell that updates aktualizr to the same commit used in the newly released tag. (Historically, we started with master, but we now skip straight to dunfell.) This is also a good time to update the aktualizr recipe to pull the latest version of link:https://ats-tuf-cli-releases.s3-eu-central-1.amazonaws.com/index.html[garage-sign].
 
-Once that PR has passed oe-selftest, successfully passed review, and gotten merged, you should then backport that change, along with anything else relevant since the last backport was done, to the xref:yocto-release-branches.adoc[currently supported release branches]. Note that while master is allowed to use arbitrary recent version of aktualizr, the release branches should only use released versions of aktualizr.
+Once that PR has passed oe-selftest, successfully passed review, and gotten merged, you should then backport that change, along with anything else relevant since the last backport was done, to the other xref:yocto-release-branches.adoc[currently supported release branches]. Note that while master is allowed to use arbitrary recent version of aktualizr, the release branches should only use released versions of aktualizr.

docs/ota-client-guide/modules/ROOT/pages/remove-sw-version.adoc

@@ -0,0 +1,38 @@
+= Remove a software version
+ifdef::env-github[]
+
+[NOTE]
+====
+We recommend that you link:https://docs.ota.here.com/ota-client/latest/{docname}.html[view this article in our documentation portal]. Not all of our articles render correctly in GitHub.
+====
+endif::[]
+
+To remove a software version, you need to delete a target in the Targets metadata file.
+include::garage-sign-reference.adoc[tags=target-term]
+
+Once you delete a target, you can no longer install it on devices.
+
+*To delete a target:*
+
+include::upload-large-binary.adoc[tags=gs-initialize]
+
+include::upload-large-binary.adoc[tags=gs-pull-targets]
+
+. To delete the target, depending on the type of image, specify the target name and version in one of the following forms:
+
+** For OSTree images, `<name>_<version>`
+** For binary images, `<name>-<version>`
++
+[source,bash]
+----
+garage-sign targets delete \
+  --filename <name>-<version>
+----
+
+include::upload-large-binary.adoc[tags=gs-sign-targets]
+
+include::upload-large-binary.adoc[tags=gs-push-targets]
+
+You can no longer see the deleted software version in your portal software repository.
+
+To learn more about the `garage-sign` commands and options, see its xref:garage-sign-reference.adoc[reference] documentation.

docs/ota-client-guide/modules/ROOT/pages/upload-large-binary.adoc

@@ -13,28 +13,33 @@ NOTE: You can upload software packages of up to 1 GB on the OTA Connect Portal.
 
 *To upload a binary file using `garage-sign`:*
 
+// tag::gs-initialize[]
 . Make sure you have the link:https://tuf-cli-releases.ota.here.com/index.html[latest version,window="_blank"] of the `garage-sign` tool.
 . Get the .zip file with your provisioning credentials.
 +
 For instructions, see the xref:ota-client::generating-provisioning-credentials.adoc[related] section in this guide.
 
 . Initialize a local repository.
 +
+NOTE: For safety reasons, we recommend xref:keep-local-repo-on-external-storage.adoc[keeping your local repository on an external storage device].
++
 [source, bash]
 ----
 garage-sign init \
   --repo <localreponame> \
   --credentials </path/to/credentials.zip>
 ----
-+
+// end::gs-initialize[]
 
+// tag::gs-pull-targets[]
 . Pull the latest version of the `targets.json` file.
 +
 [source,bash]
 ----
 garage-sign targets pull \
   --repo <localreponame>
 ----
+// end::gs-pull-targets[]
 
 . To upload your binary to OTA Connect, specify its name and version.
 +
@@ -63,6 +68,7 @@ garage-sign targets add-uploaded \
   --hardwareids <hardware-id1>,<hardware-id2>
 ----
 
+// tag::gs-sign-targets[]
 . Sign the new `targets.json` file with your Targets key.
 +
 [source,bash]
@@ -71,13 +77,16 @@ garage-sign targets sign \
   --repo <localreponame> \
   --key-name mytargets
 ----
+// end::gs-sign-targets[]
 
+// tag::gs-push-targets[]
 . Push the new `targets.json` file to OTA Connect.
 +
 [source,bash]
 ----
 garage-sign targets push \
   --repo <localreponame>
 ----
+// end::gs-push-targets[]
 
 To learn more about the `garage-sign` commands and options, see its xref:garage-sign-reference.adoc[reference] documentation.