How to write configuration file to reduce CPU usage while multi participants are used.

[Barry-Xu-2018]

Is there an already existing issue for this?

• I have searched the existing issues

Expected behavior

Low CPU usage while no message is sent.

Current behavior

On my environment, while the number of participant is more than 50, CPU usage is close to 60%.

Steps to reproduce

Example codes:
Repository: https://github.com/Barry-Xu-2018/dds_multi_pub_multi_sub.git
Branch: multi-participant

Each participant create one publisher or subscriber, and each publisher/subscriber create 10 datawriters/datareaders.

After all connection are established (No message will be sent), get CPU usage.
Publisher are on Host. 10 means how many participants will be created.

$ test_multi_pub 10

Subscriber are on RasPi 4 board. 10 means how many participants will be created.

$ test_multi_sub 10

After all connection are established, you will see +++ All subscriptions connect publishers ! +++.
And check CPU usage on RasPi 4 board as the below

Participant Num	CPU usage (%)
10	2.94
20	11.04
30	23.35
40	39.38
50	59.55
60	83.58
70	110.95

(Note: there are 4 cores in RasPi 4. So the maximum CPU usage is 400%)

Fast DDS version/commit

v2.9.0

Platform/Architecture

Ubuntu Focal 20.04 arm64

Transport layer

Default configuration, UDPv4 & SHM

Additional context

I use perf to analyze CPU usage.

According to perf result, CPU consumption is at 2 points.

• Receive UDP message
• Send period message for discovering

XML configuration file

No response

Relevant log output

No response

Network traffic capture

No response

[Barry-Xu-2018]

I think multicast may lead to CPU consumed for receiving UDP message.
So I write a configuration file to disable all multicast traffic according to https://fast-dds.docs.eprosima.com/en/latest/fastdds/transport/disabling_multicast.html#disabling-all-multicast-traffic

<?xml version="1.0" encoding="UTF-8" ?>
<profiles xmlns="http://www.eprosima.com/XMLSchemas/fastRTPS_Profiles">
    <participant profile_name="name_is_mandatory" is_default_profile="true">
        <rtps>                                                                                                                                                                                             
            <builtin>
                <metatrafficUnicastLocatorList>
                    <locator/>
                </metatrafficUnicastLocatorList>
                <initialPeersList>
                    <locator>
                        <udpv4>
                            <address>192.168.0.122</address>
                        </udpv4>
                    </locator>
                </initialPeersList>
            </builtin>
        </rtps>
    </participant>
</profiles>

IP address will be changed to IP address of host and RasPi 4 board.
After setting, I find only 30 endpoints can be connected.

If I not use built-in transport like the below, I can add maxInitialPeersRange to resolve this problem.

<?xml version="1.0" encoding="UTF-8" ?>
<profiles xmlns="http://www.eprosima.com/XMLSchemas/fastRTPS_Profiles">
    <transport_descriptors>
        <transport_descriptor>
            <transport_id>udp_transport</transport_id>
            <type>UDPv4</type>
            <sendBufferSize>9216</sendBufferSize>
            <receiveBufferSize>9216</receiveBufferSize>
            <maxMessageSize>9216</maxMessageSize>
            <non_blocking_send>true</non_blocking_send>
            <maxInitialPeersRange>100</maxInitialPeersRange>
        </transport_descriptor>
    </transport_descriptors>

    <participant profile_name="name_is_mandatory" is_default_profile="true">
        <rtps>
            <!--
            <userTransports>
                <transport_id>udp_transport</transport_id>
            </userTransports>
            <useBuiltinTransports>false</useBuiltinTransports>
            -->                                                                                                                                                                                              
            <builtin>
                <metatrafficUnicastLocatorList>
                    <locator/>
                </metatrafficUnicastLocatorList>
                <initialPeersList>
                    <locator>
                        <udpv4>
                            <address>192.168.0.122</address>
                        </udpv4>
                    </locator>
                </initialPeersList>
            </builtin>
        </rtps>
    </participant>
</profiles>

But if I use built-in transport, how to set maxInitialPeersRange ?

[Mario-DL]

Hi @Barry-Xu-2018,

Thanks for the detailed description.
There are some parameters to adjust so that participants in steady state achieve lower CPU consumption (basically due to discovery traffic and builtin events). A good starting point would be to increase lease_duration or announcement_period and disabling multicast traffic as you proposed. More discovery settings can be found here
Anyway, I am moving this thread to a discussion since it is not an issue.

[fujitatomoya]

@Mario-DL thanks for iterating.

But if I use built-in transport, how to set maxInitialPeersRange ?

https://fast-dds.docs.eprosima.com/en/latest/fastdds/xml_configuration/transports.html does this work?

Yes. I want to set these items step by step to check the effect of CPU usage.

👍 totally agree, so that we can find the most effective setting.

[Barry-Xu-2018]

https://fast-dds.docs.eprosima.com/en/latest/fastdds/xml_configuration/transports.html does this work?

It is for transport descriptors.
If I use a self-defined transport, I can set maxInitialPeersRange.
But for built-in transport, I not find where to set it.

<?xml version="1.0" encoding="UTF-8" ?>
<profiles xmlns="http://www.eprosima.com/XMLSchemas/fastRTPS_Profiles">
    <transport_descriptors>
        <transport_descriptor>
            <transport_id>udp_transport</transport_id>
            <type>UDPv4</type>
            <sendBufferSize>9216</sendBufferSize>
            <receiveBufferSize>9216</receiveBufferSize>
            <maxMessageSize>9216</maxMessageSize>
            <non_blocking_send>true</non_blocking_send>
            <maxInitialPeersRange>100</maxInitialPeersRange> 
        </transport_descriptor>
    </transport_descriptors>
<participant profile_name="name_is_mandatory" is_default_profile="true">
        <rtps>
            <userTransports>
                <transport_id>udp_transport</transport_id>
            </userTransports>
...

[JLBuenoLopez]

Built-in transports cannot be configured and that is the reason why they are built-in. But they can be substituted and disabled in order to have your own custom transport configured with what you need for your application.

[Barry-Xu-2018]

@JLBuenoLopez-eProsima

Built-in transports cannot be configured and that is the reason why they are built-in. But they can be substituted and disabled in order to have your own custom transport configured with what you need for your application.

Thanks.
About the problem #3287 (comment)
If I disable all multicast traffic, there is a limitation of the number of endpoints for built-in transport ?
So I have to use my transport instead of built-in transport ?

[Barry-Xu-2018]

I changed lease_duration and announcement_period to test.
The default value:
lease_duration is 3 seconds.
announcement_period is 20 seconds.

Now change them to
lease_duration is 6 seconds.
announcement_period is INFINITY.

Used configuration file as below

<?xml version="1.0" encoding="UTF-8" ?>
<profiles xmlns="http://www.eprosima.com/XMLSchemas/fastRTPS_Profiles">
    <participant profile_name="name_is_mandatory" is_default_profile="true">
        <rtps>
            <builtin>
            <discovery_config>
                 <leaseDuration>
                     <sec>DURATION_INFINITY</sec>
                 </leaseDuration>

                 <leaseAnnouncement>
                     <sec>6</sec>
                     <nanosec>0</nanosec>
                 </leaseAnnouncement>
            </discovery_config>
            </builtin>
        </rtps>
    </participant>
</profiles>

And check CPU usage on RasPi 4 board and put result to column New CPU usage

Participant Num	CPU usage (%)	New CPU usage (%)
50	59.55	30.4
60	83.58	40.9
70	110.95	53.9

The connection process between pub and sub seems to become slow.

You will find CPU usage has significantly decreased.