v0.2.1

Release v0.2.1

Welcome to the version 0.2.1 release of the pqc-evaluation-tools project!

Release Description

Overview

The version 0.2.0 release updated the pqc-evaluation-tools benchmark suite to utilise the most recent versions of the OQS project's PQC implementations. Including, the latest algorithms supported by Liboqs, the use of the improved OQS-Provider library compared to the previous OQS-OpenSSL library, and support for the testing and parsing of Hybrid-PQC algorithms within TLS. This version also improved the general setup of the suite alongside various optimisations and condensing of automation script files.

This small update in version 0.2.1 included a minor fix that resolved an issue with the Python parsing script where trying to parse both Liboqs and OQS-Provider results caused an error. This bug was sadly overlooked, and is now addressed in this minor release.

Features

The project provides automation for:

• Compiling and configuration of the OQS, ARM PMU, and OpenSSL dependency libraries.

• Gathering PQC computational performance data, including CPU and memory usage metrics using the Liboqs library.

• Gathering Networking performance data for the integration of PQC schemes in the TLS 1.3 protocol by utilising the OpenSSL 3.2.1 and OQS-Provider libraries.

• Coordinated testing of PQC TLS handshakes using either the loopback interface or a physical network connection between a server and client device.

• Parsing of the PQC performance data, where data from multiple machines can be parsed, averaged, and then compared against each other.

Change Log

As version 0.2.1 contains only one minor change to the code, to address a serious bug, the information on the changes in v0.2.0 compared to v0.1.0 are included here as well for reference. Future releases will return to a normal changelog format.

Version 0.2.0

A list of the key changes in this release are as follows:

• Refactored Liboqs auto testing scripts into single script and set Liboqs build to use custom OpenSSL-3.2.1

• Update to APT packages dependencies

• TLS handshake tests refactored for PQC testing using OQS-Provider and OpenSSL 3.2.1

• TLS handshake tests refactored for classic algorithm testing using OQS-Provider and OpenSSL 3.2.1

• TLS speed tests refactored for testing using OQS-Provider and OpenSSL 3.2.1

• Added support for the testing and parsing of Hybrid-PQC algorithms when integrated into TLS

• Optimisation of Python Parsing Scripts

• Added backwards comparability in parsing scripts for older versions of Python

• Added exception handling to account for missing Falcon memory metrics data on ARM devices

• Overall Improvements to Code Formatting and Commenting

• Added support for dynamically getting what algorithms are supported by the OQS libraries

• Added safe-setup functionality to main setup script to allow for a fallback to the last tested OQS libraries

• Resolved Issue where users had to be in the same directory when executing a script in order for it to function

A full list of the changes made can viewed here:

Full Changelog: v0.1.0-alpha...v0.2.0

Version 0.2.1

• Resolved bug where parsing both liboqs and oqs-provider results failed

Known Issues

• Manual copying of generated certificate and private key files for the TLS performance benchmarks required.

• Limited support for system architectures and operating systems.

• Issue with gathering full memory usage metrics for the Falcon Algorithm on ARM devices.

Important Notes

• This is an early version of the project, functionality is limited to debian based operating systems.

We look forward to your feedback and contributions to this project!

v0.2.0

Release v0.2.0

Welcome to the version 0.2.0 release of the pqc-evaluation-tools project!

Release Description

Overview

This release updates the pqc-evaluation-tools benchmark suite to utilise the most recent versions of the OQS project's PQC implementations. Including, the latest algorithms supported by Liboqs, the use of the improved OQS-Provider library compared to the previous OQS-OpenSSL library, and support for the testing and parsing of Hybrid-PQC algorithms within TLS. This version also improves the general setup of the suite alongside various optimisations and condensing of automation script files.

Features

The project provides automation for:

• Compiling and configuration of the OQS, ARM PMU, and OpenSSL dependency libraries.

• Gathering PQC computational performance data, including CPU and memory usage metrics using the Liboqs library.

• Gathering Networking performance data for the integration of PQC schemes in the TLS 1.3 protocol by utilising the OpenSSL 3.2.1 and OQS-Provider libraries.

• Coordinated testing of PQC TLS handshakes using either the loopback interface or a physical network connection between a server and client device.

• Parsing of the PQC performance data, where data from multiple machines can be parsed, averaged, and then compared against each other.

Change Log

A list of the key changes in this release are as follows:

• Refactored Liboqs auto testing scripts into single script and set Liboqs build to use custom OpenSSL-3.2.1

• Update to APT packages dependencies

• TLS handshake tests refactored for PQC testing using OQS-Provider and OpenSSL 3.2.1

• TLS handshake tests refactored for classic algorithm testing using OQS-Provider and OpenSSL 3.2.1

• TLS speed tests refactored for testing using OQS-Provider and OpenSSL 3.2.1

• Added support for the testing and parsing of Hybrid-PQC algorithms when integrated into TLS

• Optimisation of Python Parsing Scripts

• Added backwards comparability in parsing scripts for older versions of Python

• Added exception handling to account for missing Falcon memory metrics data on ARM devices

• Overall Improvements to Code Formatting and Commenting

• Added support for dynamically getting what algorithms are supported by the OQS libraries

• Added safe-setup functionality to main setup script to allow for a fallback to the last tested OQS libraries

• Resolved Issue where users had to be in the same directory when executing a script in order for it to function

A full list of the changes made can viewed here:

Full Changelog: v0.1.0-alpha...v0.2.0

Known Issues

• Manual copying of generated certificate and private key files for the TLS performance benchmarks required.

• Limited support for system architectures and operating systems.

• Issue with gathering full memory usage metrics for the Falcon Algorithm on ARM devices.

Important Notes

• This is an early version of the project, functionality is limited to x86 Linux devices and Raspberry Pis. However, future updates will address this issue.

We look forward to your feedback and contributions to this project!

v0.1.0-alpha

Release v0.1.0-alpha

Welcome to the first pre-release of the PQC Evaluation Tools Project!

Features

• Automated PQC performance benchmarking using Liboqs library.

• Automated PQC integration into TLS performance benchmarking using OQS-OpenSSL library.

• Parsing scripts to transform raw benchmark data into workable CSV files.

• Automated setup and build scripts for both Liboqs and OQS-OpenSSL libraries.

• Supports benchmarking on x86 Linux devices and Raspberry Pi's.

Bug Fixes

This being the first pre-release, there are no bug fixes relative to a previous version.

Known Issues

• Automated graph generation is currently under development and will be included in future releases.

• Manual copying of generated certificate and private key files for the TLS performance benchmarks required.

• Limited support for system architectures and operating systems

• Limited documentation

Important Notes

• This is an early version of the project, functionality is limited to x86 Linux devices and Raspberry Pis. However, future updates will address this issue.

• Activating the automated tools will delete all results currently stored in the up-results directory. To retain previous results, these should be moved to another location prior to re-running the automated test tool.

We look forward to your feedback and contributions to this project!

Full Changelog: https://github.com/crt26/pqc-evaluation-tools/commits/v0.1.0-alpha