Bug Report: buffer overflow in ROS2 Gem

[pawelbudziszewski]

Describe the bug
If I add ROS2 Gem to the project and enter/exit game mode a few times, a crash occurs:

*** buffer overflow detected ***: terminated
[1]    25838 IOT instruction (core dumped)  ./build/linux/bin/profile/Editor

Full debug bt below.

Assets required
n/a

Steps to reproduce
Steps to reproduce the behavior:

1. Create a new project using default template
2. Add ROS2 to project.json
3. In the project.json change PhysX to PhysX5
4. Build
5. Open DefaultLevel
6. For a few times enter and exit game mode (Ctrl+G, Esc)

Expected behavior
No crash

Actual behavior
Crash

Screenshots/Video
n/a

Found in Branch
o3de: main
o3de-extras: main

Commit ID from o3de/o3de and o3de/o3de-extras repositories
o3de: e8570f9d635c0abbf3d733d0524a3c0ae75d726c
o3de-extras: f71979b

Desktop/Device (please complete the following information):

• Ubuntu Linux

Additional context
Debug bt:

<12:51:24> Exited game mode
*** buffer overflow detected ***: terminated
Process 55154 stopped
* thread #1, name = 'Editor', stop reason = signal SIGABRT
    frame #0: 0x00007ffff70969fc libc.so.6`__GI___pthread_kill at pthread_kill.c:44:76
(lldb) bt
* thread #1, name = 'Editor', stop reason = signal SIGABRT
  * frame #0: 0x00007ffff70969fc libc.so.6`__GI___pthread_kill at pthread_kill.c:44:76
    frame #1: 0x00007ffff70969b0 libc.so.6`__GI___pthread_kill [inlined] __pthread_kill_internal(signo=6, threadid=140737353133312) at pthread_kill.c:78:10
    frame #2: 0x00007ffff70969b0 libc.so.6`__GI___pthread_kill(threadid=140737353133312, signo=6) at pthread_kill.c:89:10
    frame #3: 0x00007ffff7042476 libc.so.6`__GI_raise(sig=6) at raise.c:26:13
    frame #4: 0x00007ffff70287f3 libc.so.6`__GI_abort at abort.c:79:7
    frame #5: 0x00007ffff7089676 libc.so.6`__libc_message(action=do_abort, fmt="\a") at libc_fatal.c:155:5
    frame #6: 0x00007ffff713659a libc.so.6`__GI___fortify_fail(msg="buffer overflow detected") at fortify_fail.c:26:5
    frame #7: 0x00007ffff7134f16 libc.so.6`__GI___chk_fail at chk_fail.c:28:3
    frame #8: 0x00007ffff71364db libc.so.6`__fdelt_chk(d=<unavailable>) at fdelt_chk.c:25:5
    frame #9: 0x00007fffd6b63e8f libddsc.so.0`___lldb_unnamed_symbol3689 + 3455
    frame #10: 0x00007fffd6b65d80 libddsc.so.0`dds_create_domain + 96
    frame #11: 0x00007fffd747b797 librmw_cyclonedds_cpp.so`rmw_create_node + 5975
    frame #12: 0x00007fffdc1782ea librcl.so`rcl_node_init + 1130
    frame #13: 0x00007fffd7f11cad librclcpp.so`rclcpp::node_interfaces::NodeBase::NodeBase(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<rclcpp::Context>, rcl_node_options_s const&, bool, bool) + 509
    frame #14: 0x00007fffd7f09bf7 librclcpp.so`rclcpp::Node::Node(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rclcpp::NodeOptions const&) + 215
    frame #15: 0x00007fffd7f0ae28 librclcpp.so`rclcpp::Node::Node(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rclcpp::NodeOptions const&) + 88
    frame #16: 0x00007ffed423a317 libROS2.Editor.so`void std::_Construct<rclcpp::Node, char const (&) [15]>(rclcpp::Node*, char const (&) [15]) + 247
    frame #17: 0x00007ffed421d4fe libROS2.Editor.so`ROS2::ROS2SystemComponent::Activate() + 110
    frame #18: 0x00007ffed298157a libROS2.Editor.so`non-virtual thunk to ROS2::ROS2EditorSystemComponent::OnStartPlayInEditorBegin() + 26
    frame #19: 0x00007ffff2f0dcd2 libEditorLib.so`void AZ::Internal::EBusContainer<AzToolsFramework::EditorEntityContextNotification, AzToolsFramework::EditorEntityContextNotification, (AZ::EBusAddressPolicy)0, (AZ::EBusHandlerPolicy)1>::Dispatcher<AZ::EBus<AzToolsFramework::EditorEntityContextNotification, AzToolsFramework::EditorEntityContextNotification> >::Broadcast<void (AzToolsFramework::EditorEntityContextNotification::*)()>(void (AzToolsFramework::EditorEntityContextNotification::*&&)()) + 418
    frame #20: 0x00007ffff2efe22f libEditorLib.so`AzToolsFramework::EditorEntityContextComponent::StartPlayInEditor() + 111
    frame #21: 0x00007ffff2efe35a libEditorLib.so`non-virtual thunk to AzToolsFramework::EditorEntityContextComponent::StartPlayInEditor() + 26
    frame #22: 0x00007ffff17752de libEditorLib.so`void AZ::Internal::EBusContainer<AzToolsFramework::EditorEntityContextRequests, AzToolsFramework::EditorEntityContextRequests, (AZ::EBusAddressPolicy)0, (AZ::EBusHandlerPolicy)0>::Dispatcher<AZ::EBus<AzToolsFramework::EditorEntityContextRequests, AzToolsFramework::EditorEntityContextRequests> >::Broadcast<void (AzToolsFramework::EditorEntityContextRequests::*)()>(void (AzToolsFramework::EditorEntityContextRequests::*&&)()) + 286
    frame #23: 0x00007ffff176e6a9 libEditorLib.so`CGameEngine::SwitchToInGame() + 425
    frame #24: 0x00007ffff176eb64 libEditorLib.so`CGameEngine::SetGameMode(bool) + 196
    frame #25: 0x00007ffff176f04e libEditorLib.so`CGameEngine::Update() + 190
    frame #26: 0x00007ffff19119b0 libEditorLib.so`CCryEditApp::IdleProcessing(bool) + 624
    frame #27: 0x00007ffff1911724 libEditorLib.so`CCryEditApp::OnIdle(int) + 52
    frame #28: 0x00007ffff189eb34 libEditorLib.so`Editor::EditorQtApplication::maybeProcessIdle() + 52
    frame #29: 0x00007ffff1903103 libEditorLib.so`QtPrivate::QSlotObject<void (Editor::EditorQtApplication::*)(), QtPrivate::List<>, void>::impl(int, QtPrivate::QSlotObjectBase*, QObject*, void**, bool*) + 83
    frame #30: 0x00007ffff7b0d596 libQt5Core.so.5`QSingleShotTimer::timerEvent(QTimerEvent*) [inlined] QtPrivate::QSlotObjectBase::call(a=<unavailable>, r=<unavailable>, this=<unavailable>) at qobjectdefs_impl.h:398:57
    frame #31: 0x00007ffff7b0d588 libQt5Core.so.5`QSingleShotTimer::timerEvent(this=0x000055556043e530, (null)=<unavailable>) at qtimer.cpp:320:26

[adamdbrw]

Traces down to cycloneDDS libraries. Is the simulation built and ran with the same ROS environment? Does it occur on Jazzy or just Humble?

[pawelbudziszewski]

@adamdbrw:

• It occurs on ROS2 Humble
• The simulation is built and run in the same environment
• Crash doesn't occur if I switch to rmw_fastrtps_cpp (when running O3DE)

[adamdbrw]

eclipse-cyclonedds/cyclonedds#2043 - could you check if this is relevant? Do you have tracing enabled? Did you run update for ros packages?

[michalpelka]

@pawelbudziszewski do you observe still those errors with the newest ros-humble-cyclone-dds packages?
I can experience it with version 0.10.5.2-jammy.202
after this update :

Unpacking ros-humble-cyclonedds (0.10.5-2jammy.20241127.232046) over (0.10.4-1jammy.20240516.153454) 
Unpacking ros-humble-rmw-cyclonedds-cpp (1.3.4-1jammy.20241128.004150) over (1.3.4-1jammy.20240517.155556)

[pawelbudziszewski]

@michalpelka yes, I'm still able to reproduce this crash. I'm using the same version of ros-humble-cyclonedds (0.10.5-2jammy.20241127.232046)