Update libraries to LTS 2.0 version.

.gitmodules

@@ -1,6 +1,10 @@
-[submodule "components/esp-aws-iot"]
-	path = components/esp-aws-iot
-	url = https://github.com/espressif/esp-aws-iot.git
 [submodule "components/esp_secure_cert_mgr"]
 	path = components/esp_secure_cert_mgr
 	url = https://github.com/espressif/esp_secure_cert_mgr.git
+[submodule "components/esp-aws-iot"]
+	path = components/esp-aws-iot
+	url = https://github.com/ActoryOu/esp-aws-iot/
+[submodule "components/FreeRTOS-Libraries-Integration-Tests/FreeRTOS-Libraries-Integration-Tests"]
+	path = components/FreeRTOS-Libraries-Integration-Tests/FreeRTOS-Libraries-Integration-Tests
+	url = https://github.com/ActoryOu/FreeRTOS-Libraries-Integration-Tests
+	branch = lts-2.0

GettingStartedGuide.md

@@ -4,6 +4,37 @@ This guide contains instructions on how to setup, build and run the demo without
 
 Once completed, one can progress to the [Use Security Features](UseSecurityFeatures.md) guide.
 
+[1 Pre-requisites](#1-pre-requisites)<br>
+&nbsp;&nbsp;&nbsp;&nbsp;[1.1 Hardware Requirements](#11-hardware-requirements)<br>
+&nbsp;&nbsp;&nbsp;&nbsp;[1.2 Software Requirements](#12-software-requirements)<br>
+
+[2 Demo setup](#2-demo-setup)<br>
+&nbsp;&nbsp;&nbsp;&nbsp;[2.1 Setup AWS IoT Core](#21-setup-aws-iot-core)<br>
+&nbsp;&nbsp;&nbsp;&nbsp;[2.2 Configure the project with the AWS IoT Thing Name and AWS device Endpoint](#22-configure-the-project-with-the-aws-iot-thing-name-and-aws-device-endpoint)<br>
+&nbsp;&nbsp;&nbsp;&nbsp;[2.3 Provision the ESP32-C3 with the private key, device certificate and CA certificate in Development Mode](#23-provision-the-esp32-c3-with-the-private-key-device-certificate-and-ca-certificate-in-development-mode)<br>
+
+[3 Build and flash the demo project](#3-build-and-flash-the-demo-project)<br>
+
+[4 Monitoring the demo](#4-monitoring-the-demo)<br>
+
+[5 Perform firmware Over-the-Air Updates with AWS IoT](#5-perform-firmware-over-the-air-updates-with-aws-iot)<br>
+&nbsp;&nbsp;&nbsp;&nbsp;[5.1 Setup pre-requisites for OTA cloud resources](#51-setup-pre-requisites-for-ota-cloud-resources)<br>
+&nbsp;&nbsp;&nbsp;&nbsp;[5.2 Provision the project with the code-signing public key certificate](#52-provision-the-project-with-the-code-signing-public-key-certificate)<br>
+&nbsp;&nbsp;&nbsp;&nbsp;[5.3 Build an application binary with a higher version number, to be downloaded and activated on the device](#53-build-an-application-binary-with-a-higher-version-number-to-be-downloaded-and-activated-on-the-device)<br>
+&nbsp;&nbsp;&nbsp;&nbsp;[5.4 Build and flash the device with a binary with a lower version number](#54-build-and-flash-the-device-with-a-binary-with-a-lower-version-number)<br>
+&nbsp;&nbsp;&nbsp;&nbsp;[5.5 Upload the binary with the higher version number (created in step 5.3) and create an OTA Update Job](#55-upload-the-binary-with-the-higher-version-number-created-in-step-53-and-create-an-ota-update-job)<br>
+&nbsp;&nbsp;&nbsp;&nbsp;[5.6 Monitor OTA](#56-monitor-ota)<br>
+
+[6 Run AWS IoT Qualification Test](#6-run-aws-iot-qualification-test)<br>
+&nbsp;&nbsp;&nbsp;&nbsp;[6.1 Prerequisite](#61-prerequisite)<br>
+&nbsp;&nbsp;&nbsp;&nbsp;[6.2 Steps for each test case](#62-steps-for-each-test-case)<br>
+
+[7 Run AWS IoT Device Tester](#7-run-aws-iot-device-tester)<br>
+&nbsp;&nbsp;&nbsp;&nbsp;[7.1 Prerequisite](#71-prerequisite)<br>
+&nbsp;&nbsp;&nbsp;&nbsp;[7.2 Download AWS IoT Device Tester](#72-download-aws-iot-device-tester)<br>
+&nbsp;&nbsp;&nbsp;&nbsp;[7.3 Configure AWS IoT Device Tester](#73-configure-aws-iot-device-tester)<br>
+&nbsp;&nbsp;&nbsp;&nbsp;[7.4 Running AWS IoT Device Tester](#74-running-aws-iot-device-tester)<br>
+
 ## 1 Pre-requisites
 
 ### 1.1 Hardware Requirements
@@ -15,7 +46,7 @@ Once completed, one can progress to the [Use Security Features](UseSecurityFeatu
 
 ### 1.2 Software Requirements
 
-* ESP-IDF 4.4 or higher to configure, build, and flash the project. To setup for the ESP32-C3, follow Espressif's [Getting Started Guide for the ESP32-C3](https://docs.espressif.com/projects/esp-idf/en/latest/esp32c3/get-started/index.html).
+* ESP-IDF 4.4.2 or higher to configure, build, and flash the project. To setup for the ESP32-C3, follow Espressif's [Getting Started Guide for the ESP32-C3](https://docs.espressif.com/projects/esp-idf/en/latest/esp32c3/get-started/index.html).
 * [Python3](https://www.python.org/downloads/)
     and the Package Installer for Python [pip](https://pip.pypa.io/en/stable/installation/) to use the AWS CLI to import certificates and perform OTA Job set up. Pip is included when you install
     from Python 3.10.
@@ -32,7 +63,7 @@ Once completed, one can progress to the [Use Security Features](UseSecurityFeatu
 
 ## 2 Demo setup
 
-### 2.1 Setup AWS IoT Core:
+### 2.1 Setup AWS IoT Core
 
 To setup AWS IoT Core, follow the [AWS IoT Core Setup Guide](AWSSetup.md). The guide shows you how to sign up for an AWS account, create a user, and register your device with AWS IoT Core.
 After you have followed the instructions in the AWS IoT Core Setup Guide, you will have created a **device Endpoint**, an AWS IoT **thing**, a **PEM-encoded device certificate**, a **PEM-encoded private key**, and a **PEM-encoded root CA certificate**. (An explanation of these entities is given in the Setup Guide.) The root CA certificate can also be downloaded [here](https://www.amazontrust.com/repository/AmazonRootCA1.pem). Your ESP23-C3 board must now be provisioned with these entities in order for it to connect securely with AWS IoT Core.
@@ -421,3 +452,158 @@ I (3444) coreMQTT: Packet received. ReceivedBytes=3.
 I (3444) ota_over_mqtt_demo: Subscribed to topic $aws/things/thing_esp32c3_nonOta/jobs/notify-next. 
 ```
 
+## 6 Run AWS IoT Qualification Test
+
+### 6.1 Prerequisite
+- Run [OTA](#5-perform-firmware-over-the-air-updates-with-aws-iot) once manually.
+- Enable "Run qualification test" by menuconfig (Featured FreeRTOS IoT Integration -> Run qualification test).
+- Enable Unity and Unity/Fixture by menuconfig.
+*Note: The log of module `esp_ota_ops` and `esp-tls-mbedtls` will be disabled when `Run qualification test` is on. You can enable them back by commenting out `esp_log_level_set` in [main.c](./main/main.c).
+
+### 6.2 Steps for each test case
+
+1. DEVICE_ADVISOR_TEST_ENABLED - device advisor test
+    - Set DEVICE_ADVISOR_TEST_ENABLED to 1 in [test_execution_config.h](./components/FreeRTOS-Libraries-Integration-Tests/config/test_execution_config.h).
+    - Create a device advisor test on website. ( Iot Console -> Test -> Device Advisor )
+    - Create test suite.
+    - Run test suite and set the device advisor endpoint to MQTT_SERVER_ENDPOINT in [test_param_config.h](./components/FreeRTOS-Libraries-Integration-Tests/config/test_param_config.h).
+    - Set MQTT_SERVER_PORT and IOT_THING_NAME (Same as provisioned one) in [test_param_config.h](./components/FreeRTOS-Libraries-Integration-Tests/config/test_param_config.h).
+    - Build and run.
+    - See device advisor test result on website.
+1. MQTT_TEST_ENABLED - MQTT test
+    - Set MQTT_TEST_ENABLED to 1 in [test_execution_config.h](./components/FreeRTOS-Libraries-Integration-Tests/config/test_execution_config.h).
+    - Set the MQTT endpoint to MQTT_SERVER_ENDPOINT in [test_param_config.h](./components/FreeRTOS-Libraries-Integration-Tests/config/test_param_config.h).
+    - Set MQTT_SERVER_PORT and IOT_THING_NAME (Same as provisioned one) in [test_param_config.h](./components/FreeRTOS-Libraries-Integration-Tests/config/test_param_config.h).
+    - Build and run.
+    - See test result on target output.
+    - Example output
+        ```
+        I (821) qual_main: Run qualification test.
+        ...
+        -----------------------
+        7 Tests 0 Failures 0 Ignored
+        OK
+        I (84381) qual_main: End qualification test.
+        ```
+1. TRANSPORT_INTERFACE_TEST_ENABLED - Transport layer test
+    - Set TRANSPORT_INTERFACE_TEST_ENABLED to 1 [test_execution_config.h](./components/FreeRTOS-Libraries-Integration-Tests/config/test_execution_config.h).
+    - Follow [Run The Transport Interface Test](https://github.com/FreeRTOS/FreeRTOS-Libraries-Integration-Tests/tree/main/src/transport_interface#6-run-the-transport-interface-test) to start a echo server.
+    - Set ECHO_SERVER_ENDPOINT / ECHO_SERVER_PORT / ECHO_SERVER_ROOT_CA / TRANSPORT_CLIENT_CERTIFICATE and TRANSPORT_CLIENT_PRIVATE_KEY in [test_param_config.h](./components/FreeRTOS-Libraries-Integration-Tests/config/test_param_config.h).
+    - Build and run.
+    - See test result on target output.
+    - Example output
+        ```
+        I (855) qual_main: Run qualification test.
+        ...
+        -----------------------
+        14 Tests 0 Failures 0 Ignored
+        OK
+        I (612755) qual_main: End qualification test.
+        ```
+1. OTA_PAL_TEST_ENABLED - OTA PAL test
+    - Set OTA_PAL_TEST_ENABLED to 1 [test_execution_config.h](./components/FreeRTOS-Libraries-Integration-Tests/config/test_execution_config.h).
+    - Set OTA_PAL_FIRMWARE_FILE to "b_u585i_iot02a_ntz.bin" in [test_param_config.h](./components/FreeRTOS-Libraries-Integration-Tests/config/test_param_config.h).
+    - Build and run.
+    - See test result on target output.
+    - Example output
+        ```
+        I (905) qual_main: Run qualification test.
+        ...
+        -----------------------
+        15 Tests 0 Failures 0 Ignored
+        OK
+        I (113755) qual_main: End qualification test.
+        ```
+1. CORE_PKCS11_TEST_ENABLED - Core PKCS11 test
+    - Set CORE_PKCS11_TEST_ENABLED to 1 [test_execution_config.h](./components/FreeRTOS-Libraries-Integration-Tests/config/test_execution_config.h).
+    - Build and run.
+    - See test result on target output.
+    - Example output
+        ```
+        I (858) qual_main: Run qualification test.
+        ...
+        -----------------------
+        17 Tests 0 Failures 0 Ignored
+        OK
+        I (7518) qual_main: End qualification test.
+        ```
+1. OTA_E2E_TEST_ENABLED - OTA E2E test
+    - Disable all configurations in [test_execution_config.h](./components/FreeRTOS-Libraries-Integration-Tests/config/test_execution_config.h).
+    - Follow [FreeRTOS IDT 2.0](https://docs.aws.amazon.com/freertos/latest/userguide/lts-idt-freertos-qualification.html) to set-up tool.
+    - Run IDT OTA E2E test cases.
+    - See test result on tool output.
+
+## 7 Run AWS IoT Device Tester
+
+The reference integration can be tested using [AWS IoT Device Tester for FreeRTOS (IDT)](https://aws.amazon.com/freertos/device-tester/). IDT is a downloadable tool that can be used to exercise a device integration with FreeRTOS to validate functionality and compatibility with Amazon IoT cloud. Passing the test suite provided by IDT is also required to qualify a device for the [Amazon Partner Device Catalogue](https://devices.amazonaws.com/).
+
+IDT runs a suite of tests that include testing the device's transport interface layer implementation, PKCS11 functionality, and OTA capabilities. In IDT test cases, the IDT binary will make a copy of the source code, update the header files in the project, then compile the project and flash the resulting image to your board. Finally, IDT will read serial output from the board and communicate with the AWS IoT cloud to ensure that test cases are passing.
+
+### 7.1 Prerequisite
+- Run [OTA](#5-perform-firmware-over-the-air-updates-with-aws-iot) once manually.
+- Enable "Run qualification test" by menuconfig (Featured FreeRTOS IoT Integration -> Run qualification test).
+- Enable Unity and Unity/Fixture by menuconfig.
+*Note: The log of module `esp_ota_ops` and `esp-tls-mbedtls` will be disabled when `Run qualification test` is on. You can enable them back by commenting out `esp_log_level_set` in [main.c](./main/main.c).
+
+### 7.2 Download AWS IoT Device Tester
+
+The latest version of IDT can be downloaded from the [public documentation page](https://docs.aws.amazon.com/freertos/latest/userguide/dev-test-versions-afr.html). This reference implementation only supports test suites FRQ_2.2.0 or later.
+
+### 7.3 Configure AWS IoT Device Tester
+
+After downloading and unzipping IDT onto your file system, you should extract a file structure that includes the following directories:
+
+* The `bin` directory holds the devicetester binary, which is the entry point used to run IDT
+* The `results` directory holds logs that are generated every time you run IDT.
+* The `configs` directory holds configuration values that are needed to set up IDT
+
+Before we can run IDT, we have to update the files in `configs`. In this reference implementation, we have pre-defined configs available in the `idt_configs` directory. Copy these templates over into IDT, and the rest of this section will walk through the remaining values that need to be filled in.
+
+First, copy one of each file from `idt_configs` (based on host OS) in this reference repository to the `configs` directory inside the newly downloaded IDT project. This should provide you with the following files in `device_tester/configs` directory:
+
+```
+configs/dummyPublicKeyAsciiHex.txt
+configs/flash.bat or flash.sh
+configs/config.json
+configs/userdata.json
+configs/device.json
+configs/build.bat or build.sh
+```
+
+Next, we need to update some configuration values in these files.
+
+* In `build.bat` / `build.sh`, update ESP_IDF_PATH, and ESP_IDF_FRAMEWORK_PATH
+* In `flash.bat` / `flash.sh`, update ESP_IDF_PATH, ESP_IDF_FRAMEWORK_PATH, and NUM_COMPORT
+
+* In `config.json`, update the `profile` and `awsRegion` fields
+* In `device.json`, update `serialPort` to the serial port of your board as from [PORT](./GettingStartedGuide.md#23-provision-the-esp32-c3-with-the-private-key-device-certificate-and-ca-certificate-in-development-mode). Update `publicKeyAsciiHexFilePath` to the absolute path to `dummyPublicKeyAsciiHex.txt`. Update `publicDeviceCertificateArn` to the ARN of the certificate uploaded when [Setup AWS IoT Core](./GettingStartedGuide.md#21-setup-aws-iot-core).
+* In `userdata.json`, update `sourcePath` to the absolute path to the root of this reference implementation repository.
+* In `userdata.json`, update `signerCertificate` with the ARN of the [Setup pre-requisites for OTA cloud resources
+.](./GettingStartedGuide.md#51-setup-pre-requisites-for-ota-cloud-resources)
+* Run all the steps to create a [second code signing certificate](./GettingStartedGuide.md#51-setup-pre-requisites-for-ota-cloud-resources) but do NOT provision the key onto your board. Copy the ARN for this certificate in `userdata.json` for the field `untrustedSignerCertificate`.
+
+### 7.4 Running AWS IoT Device Tester
+
+With all the configuration out of the way, we can run IDT either from an individual test group or test case, or the entire qualification suite.
+
+To list the available test groups, run:
+
+```
+.\devicetester_win_x86-64.exe list-groups
+```
+
+To run any one test group, run e.g.:
+
+```
+.\devicetester_win_x86-64.exe run-suite -g FullCloudIoT -g OTACore
+```
+
+To run the entire qualification suite, run:
+
+```
+.\devicetester_win_x86-64.exe run-suite --skip-group-id FullPKCS11_PreProvisioned_RSA
+```
+
+For more information, `.\devicetester_win_x86-64.exe help` will show all available commands.
+
+When you run IDT, a `results/uuid` directory is generated that will contain all the logs and other information associated with your test run. This allows you to debug any failures.

components/FreeRTOS-Libraries-Integration-Tests/CMakeLists.txt

@@ -0,0 +1,51 @@
+# This gives QUALIFICATION_TEST_SOURCES, and QUALIFICATION_TEST_INCLUDE_DIRS
+include(${CMAKE_CURRENT_LIST_DIR}/FreeRTOS-Libraries-Integration-Tests/qualification_test.cmake)
+
+# This gives TRANSPORT_TEST_SOURCES, and TRANSPORT_TEST_INCLUDE_DIRS
+include(${CMAKE_CURRENT_LIST_DIR}/FreeRTOS-Libraries-Integration-Tests/src/transport_interface_test.cmake)
+
+# This gives MQTT_TEST_SOURCES, and MQTT_TEST_INCLUDE_DIRS
+include(${CMAKE_CURRENT_LIST_DIR}/FreeRTOS-Libraries-Integration-Tests/src/mqtt_test.cmake)
+
+# This gives OTA_PAL_TEST_SOURCES, and OTA_PAL_TEST_INCLUDE_DIRS
+include(${CMAKE_CURRENT_LIST_DIR}/FreeRTOS-Libraries-Integration-Tests/src/ota_pal_test.cmake)
+
+# This gives PKCS11_TEST_SOURCES, and PKCS11_TEST_INCLUDE_DIRS
+include(${CMAKE_CURRENT_LIST_DIR}/FreeRTOS-Libraries-Integration-Tests/src/pkcs11_test.cmake)
+
+set(IDT_INCLUDE_DIRS
+    ${QUALIFICATION_TEST_INCLUDE_DIRS}
+    ${TRANSPORT_TEST_INCLUDE_DIRS}
+    ${MQTT_TEST_INCLUDE_DIRS}
+    ${OTA_PAL_TEST_INCLUDE_DIRS}
+    ${PKCS11_TEST_INCLUDE_DIRS}
+    ${CMAKE_CURRENT_LIST_DIR}/config
+)
+
+set(IDT_SRCS
+    ${QUALIFICATION_TEST_SOURCES}
+    ${TRANSPORT_TEST_SOURCES}
+    ${MQTT_TEST_SOURCES}
+    ${OTA_PAL_TEST_SOURCES}
+    ${PKCS11_TEST_SOURCES}
+    ${CMAKE_CURRENT_LIST_DIR}/port/platform_function.c
+    ${CMAKE_CURRENT_LIST_DIR}/../esp-aws-iot/libraries/corePKCS11/corePKCS11/source/dependency/3rdparty/mbedtls_utils/mbedtls_utils.c
+)
+
+set(IDT_REQUIRES
+    unity
+    coreMQTT
+    coreHTTP
+    freertos
+    corePKCS11
+    ota-for-aws-iot-embedded-sdk
+)
+
+idf_component_register(
+    SRCS
+        ${IDT_SRCS}
+    INCLUDE_DIRS
+        ${IDT_INCLUDE_DIRS}
+    REQUIRES
+        ${IDT_REQUIRES}
+)