SIGSEGV in removeRTPSReader on application exit [13602]

[StefanFabian]

I have some very simple tests for a ros2 package I am working on.
This involves publishing and subscribing messages in the same process.
After all tests passed, the executable crashes with a segmentation fault when the node is destructed.

System information

• Fast-RTPS version: 1.2.4
• OS: Ubuntu 20.04
• ROS2: Foxy

Additional resources

• Stack Trace:

eprosima::fastrtps::rtps::RTPSDomain::removeRTPSReader(eprosima::fastrtps::rtps::RTPSReader*) 0x00007ffff6c9e8ff
eprosima::fastrtps::SubscriberImpl::~SubscriberImpl() 0x00007ffff6cc5b06
eprosima::fastrtps::SubscriberImpl::~SubscriberImpl() 0x00007ffff6cc5dcd
eprosima::fastrtps::ParticipantImpl::removeSubscriber(eprosima::fastrtps::Subscriber*) 0x00007ffff6cab4a4
eprosima::fastrtps::Domain::removeSubscriber(eprosima::fastrtps::Subscriber*) 0x00007ffff6ca216b
rmw_fastrtps_shared_cpp::destroy_subscription(char const*, CustomParticipantInfo*, rmw_subscription_t*) 0x00007ffff7239683
rmw_fastrtps_shared_cpp::__rmw_destroy_subscription(char const*, rmw_node_t const*, rmw_subscription_t*) 0x00007ffff7237052
rcl_subscription_fini 0x00007ffff7d4db53
<unknown> 0x00007ffff7f3f005
<unknown> 0x00007ffff7f3f7d5
rclcpp::SubscriptionBase::~SubscriptionBase() 0x00007ffff7f3dcb2
std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release shared_ptr_base.h:155
rclcpp::TimeSource::detachNode() 0x00007ffff7f443ed
rclcpp::TimeSource::~TimeSource() 0x00007ffff7f44531
rclcpp::node_interfaces::NodeTimeSource::~NodeTimeSource() 0x00007ffff7f02543
rclcpp::node_interfaces::NodeTimeSource::~NodeTimeSource() 0x00007ffff7f0298d
rclcpp::Node::~Node() 0x00007ffff7eda420
__gnu_cxx::new_allocator<rclcpp::Node>::destroy<rclcpp::Node> new_allocator.h:153
std::allocator_traits<std::allocator<rclcpp::Node> >::destroy<rclcpp::Node> alloc_traits.h:497
std::_Sp_counted_ptr_inplace<rclcpp::Node, std::allocator<rclcpp::Node>, (__gnu_cxx::_Lock_policy)2>::_M_dispose shared_ptr_base.h:557
std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release shared_ptr_base.h:155
std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count shared_ptr_base.h:730
std::__shared_ptr<rclcpp::Node, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr shared_ptr_base.h:1169
std::shared_ptr<rclcpp::Node>::~shared_ptr shared_ptr.h:103
__run_exit_handlers 0x00007ffff7640a27
__GI_exit 0x00007ffff7640be0
__libc_start_main 0x00007ffff761e0ba
_start 0x000055555559fdee

• Valgrind:

=== Invalid free
operator delete(void*)
eprosima::fastrtps::rtps::RTPSDomain::removeRTPSParticipant(eprosima::fastrtps::rtps::RTPSParticipant*)
eprosima::fastrtps::ParticipantImpl::~ParticipantImpl()
eprosima::fastrtps::ParticipantImpl::~ParticipantImpl()
eprosima::fastrtps::Domain::removeParticipant(eprosima::fastrtps::Participant*)
rmw_fastrtps_shared_cpp::destroy_participant(CustomParticipantInfo*)
rmw_fastrtps_shared_cpp::decrement_context_impl_ref_count(rmw_context_t*)
rcl_node_fini
/opt/ros/foxy/lib/librclcpp.so
/opt/ros/foxy/lib/librclcpp.so
rclcpp::node_interfaces::NodeBase::~NodeBase()
rclcpp::node_interfaces::NodeBase::~NodeBase()
Address 0x5647b40 is 0 bytes inside a block of size 40 free'd
operator delete(void*)
std::_Rb_tree<unsigned int, unsigned int, std::_Identity, std::less, std::allocator>::_M_erase(std::_Rb_tree_node<unsigned int>*)
__run_exit_handlers
exit
(below main)
Block was alloc'd at
operator new(unsigned long)
std::pair<std::_Rb_tree_iterator, bool> std::_Rb_tree<unsigned int, unsigned int, std::_Identity, std::less, std::allocator>::_M_insert_unique<unsigned int const&>(unsigned int const&)
eprosima::fastrtps::rtps::RTPSDomain::createParticipant(unsigned int, bool, eprosima::fastrtps::rtps::RTPSParticipantAttributes const&, eprosima::fastrtps::rtps::RTPSParticipantListener*)
eprosima::fastrtps::Domain::createParticipant(eprosima::fastrtps::ParticipantAttributes const&, eprosima::fastrtps::ParticipantListener*)
rmw_fastrtps_shared_cpp::create_participant(char const*, unsigned long, rmw_security_options_t const*, bool, char const*, rmw_dds_common::Context*)
/opt/ros/foxy/lib/librmw_fastrtps_cpp.so
/opt/ros/foxy/lib/librmw_fastrtps_cpp.so
rmw_create_node
rcl_node_init
rclcpp::node_interfaces::NodeBase::NodeBase(std::__cxx11::basic_string<char, std::char_traits, std::allocator> const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator> const&, std::shared_ptr<rclcpp::Context>, rcl_node_options_t const&, bool, bool)
rclcpp::Node::Node(std::__cxx11::basic_string<char, std::char_traits, std::allocator> const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator> const&, rclcpp::NodeOptions const&)
rclcpp::Node::Node(std::__cxx11::basic_string<char, std::char_traits, std::allocator> const&, rclcpp::NodeOptions const&)

// And perhaps relevant to that, the following happens 4 times.
=== Invalid write of size 8
std::vector<std::shared_ptr, std::allocator>::_M_erase(__gnu_cxx::__normal_iterator<std::shared_ptr*, std::vector>)
eprosima::fastdds::rtps::SharedMemGlobal::Port::~Port()
std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release()
eprosima::fastdds::rtps::SharedMemManager::Listener::~Listener()
std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release()
eprosima::fastdds::rtps::SharedMemTransport::delete_input_channel(eprosima::fastdds::rtps::SharedMemChannelResource*)
eprosima::fastdds::rtps::SharedMemTransport::CloseInputChannel(eprosima::fastrtps::rtps::Locator_t const&)
eprosima::fastrtps::rtps::RTPSParticipantImpl::disable()
eprosima::fastrtps::rtps::RTPSDomain::removeRTPSParticipant(eprosima::fastrtps::rtps::RTPSParticipant*)
eprosima::fastrtps::ParticipantImpl::~ParticipantImpl()
eprosima::fastrtps::ParticipantImpl::~ParticipantImpl()
eprosima::fastrtps::Domain::removeParticipant(eprosima::fastrtps::Participant*)
Address 0x56a6630 is 0 bytes inside a block of size 32 free'd
operator delete(void*)
eprosima::fastdds::rtps::SharedMemGlobal::Port::WatchTask::~WatchTask()
__run_exit_handlers
exit
(below main)
Block was alloc'd at
operator new(unsigned long)
void std::vector<std::shared_ptr, std::allocator>::_M_realloc_insert<std::shared_ptr const&>(__gnu_cxx::__normal_iterator<std::shared_ptr*, std::vector>, std::shared_ptr<eprosima::fastdds::rtps::SharedMemGlobal::Port::WatchTask::PortContext> const&)
eprosima::fastdds::rtps::SharedMemGlobal::Port::Port(std::shared_ptr<eprosima::fastdds::rtps::SharedMemSegment>&&, eprosima::fastdds::rtps::SharedMemGlobal::PortNode*, std::unique_ptr<eprosima::fastdds::rtps::RobustExclusiveLock, std::default_delete>&&)
eprosima::fastdds::rtps::SharedMemGlobal::init_port(unsigned int, std::unique_ptr<eprosima::fastdds::rtps::SharedMemSegment, std::default_delete>&, unsigned int, eprosima::fastdds::rtps::SharedMemGlobal::Port::OpenMode, unsigned int)
eprosima::fastdds::rtps::SharedMemGlobal::open_port_internal(unsigned int, unsigned int, unsigned int, eprosima::fastdds::rtps::SharedMemGlobal::Port::OpenMode, std::shared_ptr<eprosima::fastdds::rtps::SharedMemGlobal::Port>)
eprosima::fastdds::rtps::SharedMemTransport::CreateInputChannelResource(eprosima::fastrtps::rtps::Locator_t const&, unsigned int, eprosima::fastdds::rtps::TransportReceiverInterface*)
eprosima::fastdds::rtps::SharedMemTransport::OpenInputChannel(eprosima::fastrtps::rtps::Locator_t const&, eprosima::fastdds::rtps::TransportReceiverInterface*, unsigned int)
eprosima::fastrtps::rtps::ReceiverResource::ReceiverResource(eprosima::fastdds::rtps::TransportInterface&, eprosima::fastrtps::rtps::Locator_t const&, unsigned int)
eprosima::fastrtps::rtps::NetworkFactory::BuildReceiverResources(eprosima::fastrtps::rtps::Locator_t&, std::vector<std::shared_ptr, std::allocator>&, unsigned int)
eprosima::fastrtps::rtps::RTPSParticipantImpl::createReceiverResources(eprosima::fastrtps::rtps::LocatorList_t&, bool, bool)
eprosima::fastrtps::rtps::RTPSParticipantImpl::RTPSParticipantImpl(unsigned int, eprosima::fastrtps::rtps::RTPSParticipantAttributes const&, eprosima::fastrtps::rtps::GuidPrefix_t const&, eprosima::fastrtps::rtps::GuidPrefix_t const&, eprosima::fastrtps::rtps::RTPSParticipant*, eprosima::fastrtps::rtps::RTPSParticipantListener*)
eprosima::fastrtps::rtps::RTPSParticipantImpl::RTPSParticipantImpl(unsigned int, eprosima::fastrtps::rtps::RTPSParticipantAttributes const&, eprosima::fastrtps::rtps::GuidPrefix_t const&, eprosima::fastrtps::rtps::RTPSParticipant*, eprosima::fastrtps::rtps::RTPSParticipantListener*)

[StefanFabian]

Another observation:
This only happens if the Node shared ptr is in static storage and destructed at application exit.
If the node is destroyed manually at the end of main using reset(), the process finishes cleanly.

[JLBuenoLopez]

Hi @StefanFabian,

First, I apologize for the late reply!

I would like to clarify the version you are using of Fast DDS. ROS 2 Foxy is using our 2.1.x branch which latest release is 2.1.1 from March 2021. But you are saying you are using version 1.2.4 (which I think it is a typo).

Nevertheless, I think your issue might be related with #937. Fast DDS uses a singleton factory to create the DDS entities (Domain class in Fast RTPS deprecated API and DomainParticipantFactory in Fast DDS API). When using static storage, the destruction order is really important, and if some resource is trying to access the singleton after it has been destroyed, what you mention in your issue is expected.

[JLBuenoLopez]

I am going to close it due to inactivity. Also, the singletons have been refactored in #2894 and #3185 so this is probably fixed. Feel free to reopen if it is still happening.