Question: does Fast-RTPS support OpenSSL 1.1.1d ? [8117]

[mikaelarguedas]

Trying to run the ros2 security example on Ubuntu Focal (OpenSSL 1.1.1d) fails with Fast-RTPS (branch 1.9.x): more details at ros2/sros2#180

Running the provided SecureHelloWorld example also fails with:

root@c2236a03ccbd:/opt/ros2_ws/install/fastrtps/examples/C++/SecureHelloWorldExample# ./bin/SecureHelloWorldExample subscriber
Starting 
Subscriber running. Please press enter to stop the Subscriber
2020-03-23 16:02:05.004 [SECURITY_AUTHENTICATION Error] OpenSSL library cannot set peer (/opt/ros2_ws/src/eProsima/Fast-RTPS/src/cpp/security/authentication/PKIDH.cpp:1041) -> Function on_process_handshake
2020-03-23 16:02:05.503 [SECURITY_AUTHENTICATION Error] OpenSSL library cannot set peer (/opt/ros2_ws/src/eProsima/Fast-RTPS/src/cpp/security/authentication/PKIDH.cpp:1041) -> Function on_process_handshake

Does Fast-RTPS support OpenSSL 1.1.1d? and if yes could you point at a set of certificates that workwith that version? thanks!

[MiguelBarro]

We are trying to work this out but we are not yet able to reproduce the issue:

In an ubuntu disco:

    Distributor ID: Ubuntu
    Description:    Ubuntu 19.04
    Release:        19.04
    Codename:       disco

We have replaced the OpenSSL distro version:

/usr/local/bin/openssl
OpenSSL 1.1.1b  26 Feb 2019 (Library: OpenSSL 1.1.1d  10 Sep 2019)

with the OpenSSL 1.1.1d (https://github.com/openssl/openssl/archive/OpenSSL_1_1_1d.tar.gz).

We rebuild Fast-RTPS assuring CMake finds the new OpenSSL version by hinting him (-DOPENSSL_ROOT_DIR=/usr/local/ssl/).
And afterward run all tests and the SecureHelloWorldExample which unfortunately worked flawlessly for master & 1.9.x branches.

We are now going to try to reproduce the ros2/sros2#180 scenario on a Ubuntu Focal docker. We'll keep you update asap.

[kyrofa]

Thank you @MiguelBarro! I look forward to hearing more.

[mikaelarguedas]

And afterward run all tests and the SecureHelloWorldExample which unfortunately worked flawlessly for master & 1.9.x branches.

That's promising ! Looking forward to hearing your results with a Focal based system

[MiguelBarro]

A little update on the current status of our works. We have generated several dockerfiles in order to account for platform, OpenSSL source (direct build from 1.1.1d or libsll-dev distro package) and build config.

DockerFiles.zip

The dockerfiles:

• install Fast-RTPS with security, examples and unittests enabled. The dockerfiles CMD makes them run the security tests by default, that is:

     > docker build -t <image_tag> -f <file_name> .        
     > docker run -it <image_tag> 

will build the image and run the security tests.

• within each image a example.sh script is provided that allows easily testing the SecureHelloWorldExample using two containers:

    1> docker run -it <image_tag> ./example.sh publisher
    2> docker run -it <image_tag> ./example.sh subscriber 

We can summarize the results as:

DockerFile	Tests	SecureHelloWorldExample
DockerFile_Bionic	OK	Works
DockerFile_Bionic_Debug	OK	Works
DockerFile_Bionic_OpenSSL	OK	Works
DockerFile_Bionic_OpenSSL_Debug	OK	Works
DockerFile_Focal	Error	Error
DockerFile_Focal_Debug	Error	Error
DockerFile_Focal_OpenSSL	Timeout	Timeout
DockerFile_Focal_OpenSSL_Debug	OK	Works

These results are puzzling, basically OpenSSL 1.1.1d:

• works fine on Bionic.
• works different on Focal depending whether we build the sources or install the distro binaries.
  libssl-dev binaries don't work at all despite of been 1.1.1d also.
• works on Focal for debug and not for release. On release it doesn't arise errors like distro binaries.

Right now we are managing several hypothesis and debugging on focal is in progress...

Feel free to comment 😃. Hope we can update you soon.

[kyrofa]

So it appears that the libssl in Focal is, to quote the Ubuntu security team, "almost 1.1.1e." You might try with that as far as the from-source experiments are concerned, should be a closer match.

[mikaelarguedas]

Great minds :)

I was thinking the same thing so I launched a job based on ros2/ci#421 to see if fastrtps was passing wth it. It's still queued so we don't know yet.

However I tested a Dockerfile with openssl 1.1.1e from source and it fails in a similar way at with openssl from the Focal repos.
So It looks like that's the good place to explore @MiguelBarro to pin down the issue.

[kyrofa]

it fails in a similar way

Now that's what I like to hear! Well... kinda. You know what I mean.

[mikaelarguedas]

also OpenSSL 1.1.1f has been released a couple days ago and is already the one shipped Windows and homebrew, so we'll need to make sure to test against that one too

[MiguelBarro]

Your insight serves you well...

DockerFile	Tests	SecureHelloWorldExample
DockerFile_Bionic	OK	Works
DockerFile_Bionic_Debug	OK	Works
DockerFile_Bionic_OpenSSL	OK	Works
DockerFile_Bionic_OpenSSL_Debug	OK	Works
DockerFile_Focal	Error	Error
DockerFile_Focal_Debug	Error	Error
DockerFile_Focal_OpenSSL	Timeout	Timeout
DockerFile_Focal_OpenSSL_Debug	OK	Works
DockerFile_Focal_OpenSSLe	Error	Error
DockerFile_Focal_OpenSSLe_Debug	Error	Error

OpenSSL 1.1.1e mimics libssl-dev behaviour 👍.

[clalancette]

@MiguelBarro Any news on this? Security is going to be completely broken on ROS 2 Foxy unless this gets fixed in Fast-RTPS.

[nuclearsandwich]

When there's a PR resolving this issue if someone would be kind enough to ping the thread here ros2/system_tests#415 we'll be able to run the ROS 2 security tests over it. Thanks!

[MiguelCompany]

We (@MiguelBarro and myself) have been looking into this. As the problems seem to be on the key agreement, and we are defaulting the algorithm to DH+MODP-2048-256, I thought of a possible workaround for the issue, which is to use the other possible algorithm stated on the DDS-SEC 1.1 standard (i.e. ECDH+prime256v1-CEUM)

The proposed changes are on 23e40d3

On my VirtualBox with Focal, I did see an improvement on SecureHelloWorldExample. No error logs are generated, but Publisher and Subscriber do not match. Wireshark inspection shows that the PKI phase completes, and a lot of SRTPS messages are exchanged, but it seems that the key exchange phase is not working correctly.

@MiguelBarro will check my changes tomorrow on his dockers, but it would be great if someone could also test with my changes. I don't rely much on my Virtual Box.

[nuclearsandwich]

I merged https://github.com/eProsima/Fast-RTPS/compare/bugfix/pki-selection/1.10.x at 340bdf42f3bc1051d3a1c6a3ebb31e68abf548ca into 1.10.x locally.

I think I'm reproducing @MiguelCompany's scenario but I am out of time to wire up packet captures. Running the security tests last week I was seen log messages about being unable to create secure participants. Today I see the following:

test 6
      Start  6: SimpleCommunicationSecureBestEffort

6: Test command: /usr/bin/python3 "/home/steven/osrf/Fast-RTPS/ws/build/fastrtps/test/communication/simple_communication.py"
6: Environment variables:
6:  SIMPLE_COMMUNICATION_PUBLISHER_BIN=/home/steven/osrf/Fast-RTPS/ws/build/fastrtps/test/communication/SimpleCommunicationPublisher
6:  SIMPLE_COMMUNICATION_SUBSCRIBER_BIN=/home/steven/osrf/Fast-RTPS/ws/build/fastrtps/test/communication/SimpleCommunicationSubscriber
6:  XML_FILE_PUB=simple_secure_besteffort_pub.xml
6:  XML_FILE_SUB=simple_secure_besteffort_sub.xml
6:  EXTRA_PUB_ARG=--wait 2
6: Test timeout computed to be: 1500
6: Publisher participant  discovered participant a6.b1.a0.7e.9c.49.8d.98.d4.e1.25.ba|0.0.1.c1
6: Subscriber participant  discovered participant a6.b1.a0.7e.9c.49.5d.1d.ee.5b.13.d2|0.0.1.c1
6: Publisher participant  discovered participant a6.b1.a0.7e.9c.49.5d.1d.ee.5b.13.d2|0.0.1.c1
6: Subscriber participant  discovered participant a6.b1.a0.7e.9c.49.8d.98.d4.e1.25.ba|0.0.1.c1
6: Subscriber participant  discovered participant eb.ba.3f.10.a7.26.7f.51.ab.41.8d.5d|0.0.1.c1
6: Subscriber participant  authorized participant a6.b1.a0.7e.9c.49.8d.98.d4.e1.25.ba|0.0.1.c1
6: Subscriber participant  discovered participant eb.ba.3f.10.a7.26.7f.51.ab.41.8d.5d|0.0.1.c1
6: Subscriber participant  authorized participant eb.ba.3f.10.a7.26.7f.51.ab.41.8d.5d|0.0.1.c1
6: Publisher participant eb.ba.3f.10.a7.26.7f.51.ab.41.8d.5d|0.0.1.c1 authorized participant a6.b1.a0.7e.9c.49.8d.98.d4.e1.25.ba|0.0.1.c1
6: Subscriber participant  authorized participant a6.b1.a0.7e.9c.49.5d.1d.ee.5b.13.d2|0.0.1.c1
6: Subscriber participant  authorized participant eb.ba.3f.10.a7.26.7f.51.ab.41.8d.5d|0.0.1.c1
6: Publisher participant eb.ba.3f.10.a7.26.7f.51.ab.41.8d.5d|0.0.1.c1 authorized participant a6.b1.a0.7e.9c.49.5d.1d.ee.5b.13.d2|0.0.1.c1

Let me know if there's anything else I can extract and I'll try again tomorrow.

[MiguelCompany]

@mikaelarguedas @nuclearsandwich Could you please check with PR #1163, which should fix this on branch 1.10.x?

[mikaelarguedas]

I didnt test this on the ROS side but I confirmed the Fast-RTPS examples were fixed with that PR even when using OpenSSL 1.1.1f (the currently available version on WIndows and homebrew).

The following should tell us if it fixed it on the ROS side as well

• Linux
• Linux-aarch64
• macOS
• Windows

[MiguelCompany]

#1163 has been merged on 1.10.x, and the same changes have been applied on master.

@mikaelarguedas Close this when you think everything is fixed

[mikaelarguedas]

It seems to fail the ROS tests on MacOS.

Does it wok on Mac with openssl 1.1.1d for you ?
Edit: looks like it did pass your CI in #1163

[mikaelarguedas]

Closing as related ROS2 tests are now reenabled and passing, thanks @MiguelBarro @MiguelCompany @richiware for the fixes!