Segmentation fault while loading private key using the wrong password [12974]

[lexamor]

Crash happens when the library tries to load an encrypted private key, but the password provided by the user is incorrect or empty.

Expected Behavior

The error should be handled correctly (return error code or throw an exception)

Current Behavior

Crash happens while domain participant creation.

Steps to Reproduce

1. Configure Auth plugin using correct CA, identity_cert and encrypted(!) private key (dds.sec.auth.builtin.PKI-DH.private_key parameter).
2. Use incorrect or empty password for the dds.sec.auth.builtin.PKI-DH.password property.
3. Try to create domain participant using DomainParticipantFactory (DomainParticipantFactory::get_instance()->create_participant).

System information

• Fast-RTPS version: 2.4.0
• OS: Ubuntu 20.04

Additional context

Library tries to load private key (PKIDH.cpp:393) using PEM_read_bio_PrivateKey OpenSSL library function. But the return value of the function is not checked and just passed to the next step (X509_check_private_key).

However, in the OpenSSL docs, it is mentioned that in case of error this function can return nullptr (example at the bottom of page).

Unfortunately, I didn't find any docs if it's allowed to pass nullptr to the X509_check_private_key input, but according to the stack trace, seems, it is an issue.

Hotfix which is worked for me is just to check the return value of PEM_read_bio_PrivateKey and stop function execution in case of error (nullptr).

diff --git a/src/cpp/security/authentication/PKIDH.cpp b/src/cpp/security/authentication/PKIDH.cpp
index 2dfcbf32d..1f6aa9c1d 100644
--- a/src/cpp/security/authentication/PKIDH.cpp
+++ b/src/cpp/security/authentication/PKIDH.cpp
@@ -393,6 +393,11 @@ static EVP_PKEY* load_private_key(
                         PEM_read_bio_PrivateKey(in, NULL, (pem_password_cb*)private_key_password_callback,
                                 (void*)password.c_str());
 
+                if(returnedValue == nullptr){
+                    exception = _SecurityException_(std::string("Error while reading private key ") + file.substr(7));
+                    return returnedValue;
+                }
+
                 // Verify private key.
                 if (!X509_check_private_key(certificate, returnedValue))
                 {

Additional resources

• Stack trace

Thread 1 "app" received signal SIGSEGV, Segmentation fault.
--Type <RET> for more, q to quit, c to continue without paging--
0x00007ffff6e39604 in EVP_PKEY_cmp () from /lib/x86_64-linux-gnu/libcrypto.so.1.1
(gdb) bt
#0  0x00007ffff6e39604 in EVP_PKEY_cmp () from /lib/x86_64-linux-gnu/libcrypto.so.1.1
#1  0x00007ffff6eb8ef1 in X509_check_private_key () from /lib/x86_64-linux-gnu/libcrypto.so.1.1
#2  0x00007ffff7cd0391 in load_private_key(x509_st*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, eprosima::fastrtps::rtps::security::SecurityException&) () from /home/test/build/lib/libfastrtps.so.2.4
#3  0x00007ffff7ce004e in eprosima::fastrtps::rtps::security::PKIDH::validate_local_identity(eprosima::fastrtps::rtps::security::Handle**, eprosima::fastrtps::rtps::GUID_t&, unsigned int, eprosima::fastrtps::rtps::RTPSParticipantAttributes const&, eprosima::fastrtps::rtps::GUID_t const&, eprosima::fastrtps::rtps::security::SecurityException&) () from /home/test/build/lib/libfastrtps.so.2.4
#4  0x00007ffff7cb4520 in eprosima::fastrtps::rtps::security::SecurityManager::init(eprosima::fastrtps::rtps::security::ParticipantSecurityAttributes&, eprosima::fastrtps::rtps::PropertyPolicy const&, bool&) ()
   from /home/test/build/lib/libfastrtps.so.2.4
#5  0x00007ffff79a5fe6 in eprosima::fastrtps::rtps::RTPSParticipantImpl::RTPSParticipantImpl(unsigned int, eprosima::fastrtps::rtps::RTPSParticipantAttributes const&, eprosima::fastrtps::rtps::GuidPrefix_t const&, eprosima::fastrtps::rtps::GuidPrefix_t const&, eprosima::fastrtps::rtps::RTPSParticipant*, eprosima::fastrtps::rtps::RTPSParticipantListener*) () from /home/test/build/lib/libfastrtps.so.2.4
#6  0x00007ffff79a719d in eprosima::fastrtps::rtps::RTPSParticipantImpl::RTPSParticipantImpl(unsigned int, eprosima::fastrtps::rtps::RTPSParticipantAttributes const&, eprosima::fastrtps::rtps::GuidPrefix_t const&, eprosima::fastrtps::rtps::RTPSParticipant*, eprosima::fastrtps::rtps::RTPSParticipantListener*) () from /home/test/build/lib/libfastrtps.so.2.4
#7  0x00007ffff79b29c1 in eprosima::fastrtps::rtps::RTPSDomain::createParticipant(unsigned int, bool, eprosima::fastrtps::rtps::RTPSParticipantAttributes const&, eprosima::fastrtps::rtps::RTPSParticipantListener*) ()
   from /home/test/build/lib/libfastrtps.so.2.4
#8  0x00007ffff7a30f70 in eprosima::fastdds::dds::DomainParticipantImpl::enable() () from /home/test/build/lib/libfastrtps.so.2.4
#9  0x00007ffff7a395d2 in eprosima::fastdds::dds::DomainParticipant::enable() () from /home/test/build/lib/libfastrtps.so.2.4
#10 0x00007ffff7a1f971 in eprosima::fastdds::dds::DomainParticipantFactory::create_participant(unsigned int, eprosima::fastdds::dds::DomainParticipantQos const&, eprosima::fastdds::dds::DomainParticipantListener*, eprosima::fastdds::dds::StatusMask const&) () from /home/test/build/lib/libfastrtps.so.2.4