Make low-level crypto algorithms implementation switchable

[ueno]

Currently liboqs uses OpenSSL as the default backend for low-level support algorithms, such as AES and SHA3 if the OQS_USE_OPENSSL build flag is ON; otherwise it will use the bundled implementation of those algorithms. While integrating liboqs in GnuTLS, we found neither of them optimal: it is not desirable to bring in OpenSSL as a dependency[1], while we also don't want to have the same algorithms implemented in multiple places. Therefore I would suggest providing a way to make those algorithm implementation pluggable at either compile time or runtime.

More concretely, what I have in mind is something along the following lines:

• liboqs could provide a build option OQS_COMMON_CRYPTO, which takes a combo value, either OPENSSL, DEFAULT, or NONE. OQS_COMMON_CRYPTO=OPENSSL is the same as what OQS_USE_OPENSSL=ON currently does, DEFAULT is equivalent to OQS_USE_OPENSSL=OFF, and NONE means only a stub implementation (all functions return error) is compiled in
• Consumer applications (such as oqs-provider) could register implementation of those algorithms in their favorite libraries at runtime, maybe through a new API, say OQS_AES_register_implementation, OQS_SHA3_register_implementation, etc.
• To make the previous step easier, maybe liboqs could provide the default implementation as a static helper library (e.g., liboqs_openssl.a, liboqs_default.a)

This is similar to the approach taken by tpm2-tss[2].

1. https://gitlab.com/gnutls/gnutls/-/merge_requests/1786#note_1611785601
2. esapi: support runtime switchable crypto backends tpm2-software/tpm2-tss#2365

[baentsch]

Thanks for this proposal. Eliminating this OpenSSL dependency definitely makes sense for liboqs integrations to other crypto libs.

I wonder though, to which level of algorithm detail this needs to go: Is a distinction AES/SHA2/SHA3/RAND sufficient? Or should there be "redirection stubs" for all detail APIs e.g., as currently documented here for AES -- and equivalent for the other common crypto elements? Would you want to make a strawman proposal based on how you could envision a mapping onto GnuTLS APIs?

In my personal opinion, it would be beneficial if this could be a compile-time switch (maybe macros based on the current common APIs, such as OQS_AES256_ECB_enc_sch?) as otherwise the currently single, standalone library liboqs would be split into different pieces/shared libs. This could be inconvenient for (or even be mis-handled by) users of the common case where OpenSSL is providing the "production quality" common code for standalone liboqs applications; oqsprovider surely could be adapted to any sensible configuration -- although rather clearly none other than an OpenSSL config makes sense for an OpenSSL provider.

[beldmit]

XMLSEc project https://github.com/lsh123/xmlsec uses multibackend scheme with compile-time choice, and I can't say it's easily maintained by a person foreign to the project but it's definitely not the only possible architecture

[ueno]

Thanks for the prompt response!

I wonder though, to which level of algorithm detail this needs to go: Is a distinction AES/SHA2/SHA3/RAND sufficient? Or should there be "redirection stubs" for all detail APIs e.g., as currently documented here for AES -- and equivalent for the other common crypto elements? Would you want to make a strawman proposal based on how you could envision a mapping onto GnuTLS APIs?

Yes; I think that'd be sufficient. Let me try to create a PoC.

In my personal opinion, it would be beneficial if this could be a compile-time switch (maybe macros based on the current common APIs, such as OQS_AES256_ECB_enc_sch?) as otherwise the currently single, standalone library liboqs would be split into different pieces/shared libs.

One idea to make this transition easier might be to delay the loading of libcrypto.so.* with dlopen, so the application can swap the implementation before OpenSSL is actually being used. I guess that wouldn't be too hard, as I see only 18 functions from libcrypto.so.* are used.

[baentsch]

One idea to make this transition easier might be to delay the loading of libcrypto.so.* with dlopen, so the application can swap the implementation before OpenSSL is actually being used. I guess that wouldn't be too hard, as I see only 18 functions from libcrypto.so.* are used.

Agreed, there's not so many entry points. Would a dlopen-based solution work equally well for static and shared builds of the lib? On all platforms? Looking forward to the PoC/PR!