Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ros2 Service Send Request Does Not Send #625

Closed
borgmanJeremy opened this issue Jan 30, 2019 · 4 comments
Closed

Ros2 Service Send Request Does Not Send #625

borgmanJeremy opened this issue Jan 30, 2019 · 4 comments
Labels
duplicate This issue or pull request already exists more-information-needed Further information is required

Comments

@borgmanJeremy
Copy link

Bug report

Required Info:

  • Operating System:
    • Ubuntu 16.04
    • Docker instance based on 18.04
  • Installation type:
    • source
  • Version or commit hash:
    ros2.repos:
repositories:
  ament/ament_cmake:
    type: git
    url: https://github.com/ament/ament_cmake.git
    version: 0.6.0
  ament/ament_index:
    type: git
    url: https://github.com/ament/ament_index.git
    version: 0.5.1
  ament/ament_lint:
    type: git
    url: https://github.com/ament/ament_lint.git
    version: 0.6.2
  ament/ament_package:
    type: git
    url: https://github.com/ament/ament_package.git
    version: 0.6.0
  ament/googletest:
    type: git
    url: https://github.com/ament/googletest.git
    version: c8b6f2be853002b9a0abfb30ea537402383bf1f4
  ament/osrf_pycommon:
    type: git
    url: https://github.com/osrf/osrf_pycommon.git
    version: 0.1.6
  ament/uncrustify_vendor:
    type: git
    url: https://github.com/ament/uncrustify_vendor.git
    version: 1.0.0
  eProsima/Fast-CDR:
    type: git
    url: https://github.com/eProsima/Fast-CDR.git
    version: v1.0.8
  eProsima/Fast-RTPS:
    type: git
    url: https://github.com/eProsima/Fast-RTPS.git
    version: b48ce9d2fba6fc94e756da01c58b72f2ad848238
  osrf/osrf_testing_tools_cpp:
    type: git
    url: https://github.com/osrf/osrf_testing_tools_cpp.git
    version: 1.1.0
  ros/class_loader:
    type: git
    url: https://github.com/ros/class_loader.git
    version: 1.2.0
  ros/pluginlib:
    type: git
    url: https://github.com/ros/pluginlib.git
    version: 2.2.1
  ros/resource_retriever:
    type: git
    url: https://github.com/ros/resource_retriever.git
    version: 2.1.0
  ros/ros_environment:
    type: git
    url: https://github.com/ros/ros_environment.git
    version: 2.2.0
  ros/urdfdom_headers:
    type: git
    url: https://github.com/ros/urdfdom_headers.git
    version: 1.0.2
  ros-perception/laser_geometry:
    type: git
    url: https://github.com/ros-perception/laser_geometry.git
    version: 2.0.0
  ros-planning/navigation_msgs:
    type: git
    url: https://github.com/ros-planning/navigation_msgs.git
    version: 2.0.0
  ros-visualization/python_qt_binding:
    type: git
    url: https://github.com/ros-visualization/python_qt_binding.git
    version: 1.0.1
  ros-visualization/qt_gui_core:
    type: git
    url: https://github.com/ros-visualization/qt_gui_core.git
    version: 1.0.3
  ros-visualization/rqt:
    type: git
    url: https://github.com/ros-visualization/rqt.git
    version: 1.0.1
  ros-visualization/rqt_console:
    type: git
    url: https://github.com/ros-visualization/rqt_console.git
    version: 1.0.1
  ros-visualization/rqt_msg:
    type: git
    url: https://github.com/ros-visualization/rqt_msg.git
    version: 1.0.1
  ros-visualization/rqt_plot:
    type: git
    url: https://github.com/ros-visualization/rqt_plot.git
    version: 1.0.2
  ros-visualization/rqt_publisher:
    type: git
    url: https://github.com/ros-visualization/rqt_publisher.git
    version: 1.0.3
  ros-visualization/rqt_py_console:
    type: git
    url: https://github.com/ros-visualization/rqt_py_console.git
    version: 1.0.0
  ros-visualization/rqt_service_caller:
    type: git
    url: https://github.com/ros-visualization/rqt_service_caller.git
    version: 1.0.1
  ros-visualization/rqt_shell:
    type: git
    url: https://github.com/ros-visualization/rqt_shell.git
    version: 1.0.0
  ros-visualization/rqt_srv:
    type: git
    url: https://github.com/ros-visualization/rqt_srv.git
    version: 1.0.1
  ros-visualization/rqt_top:
    type: git
    url: https://github.com/ros-visualization/rqt_top.git
    version: 1.0.0
  ros2/ament_cmake_ros:
    type: git
    url: https://github.com/ros2/ament_cmake_ros.git
    version: 0.5.0
  ros2/common_interfaces:
    type: git
    url: https://github.com/ros2/common_interfaces.git
    version: 0.6.1
  ros2/console_bridge_vendor:
    type: git
    url: https://github.com/ros2/console_bridge_vendor.git
    version: 1.1.0
  ros2/demos:
    type: git
    url: https://github.com/ros2/demos.git
    version: 0.6.1
  ros2/examples:
    type: git
    url: https://github.com/ros2/examples.git
    version: 0.6.1
  ros2/example_interfaces:
    type: git
    url: https://github.com/ros2/example_interfaces.git
    version: 0.6.1
  ros2/geometry2:
    type: git
    url: https://github.com/ros2/geometry2.git
    version: 0.10.1
  ros2/kdl_parser:
    type: git
    url: https://github.com/ros2/kdl_parser.git
    version: 2.2.0
  ros2/launch:
    type: git
    url: https://github.com/ros2/launch.git
    version: 0.7.3
  ros2/libyaml_vendor:
    type: git
    url: https://github.com/ros2/libyaml_vendor.git
    version: 1.0.0
  ros2/message_filters:
    type: git
    url: https://github.com/ros2/message_filters.git
    version: 3.0.0
  ros2/orocos_kinematics_dynamics:
    type: git
    url: https://github.com/ros2/orocos_kinematics_dynamics.git
    version: 3.1.0
  ros2/poco_vendor:
    type: git
    url: https://github.com/ros2/poco_vendor.git
    version: 1.1.1
  ros2/rcl_logging:
    type: git
    url: https://github.com/ros2/rcl_logging.git
    version: 0.1.0
  ros2/rcl:
    type: git
    url: https://github.com/ros2/rcl.git
    version: 0.6.3
  ros2/rcl_interfaces:
    type: git
    url: https://github.com/ros2/rcl_interfaces.git
    version: 0.6.1
#  ros2/rclc:
#    type: git
#    url: https://github.com/ros2/rclc.git
#    version: master
  ros2/rclcpp:
    type: git
    url: https://github.com/ros2/rclcpp.git
    version: 0.6.2
  ros2/rclpy:
    type: git
    url: https://github.com/ros2/rclpy.git
    version: 0.6.1
  ros2/rcutils:
    type: git
    url: https://github.com/ros2/rcutils.git
    version: 0.6.1
  ros2/realtime_support:
    type: git
    url: https://github.com/ros2/realtime_support.git
    version: 0.6.0
  ros2/rmw:
    type: git
    url: https://github.com/ros2/rmw.git
    version: 0.6.1
  ros2/rmw_connext:
    type: git
    url: https://github.com/ros2/rmw_connext.git
    version: 0.6.1
  ros2/rmw_fastrtps:
    type: git
    url: https://github.com/ros2/rmw_fastrtps.git
    version: 0.6.1
  ros2/rmw_implementation:
    type: git
    url: https://github.com/ros2/rmw_implementation.git
    version: 0.6.1
  ros2/rmw_opensplice:
    type: git
    url: https://github.com/ros2/rmw_opensplice.git
    version: 0.6.2
  ros2/robot_state_publisher:
    type: git
    url: https://github.com/ros2/robot_state_publisher.git
    version: 2.1.0
  ros2/ros1_bridge:
    type: git
    url: https://github.com/ros2/ros1_bridge.git
    version: 0.6.1
  ros2/ros2cli:
    type: git
    url: https://github.com/ros2/ros2cli.git
    version: 0.6.2
  ros2/rosidl:
    type: git
    url: https://github.com/ros2/rosidl.git
    version: 0.6.2
  ros2/rosidl_dds:
    type: git
    url: https://github.com/ros2/rosidl_dds.git
    version: 0.6.0
  ros2/rosidl_defaults:
    type: git
    url: https://github.com/ros2/rosidl_defaults.git
    version: 0.6.0
  ros2/rosidl_python:
    type: git
    url: https://github.com/ros2/rosidl_python.git
    version: 0.6.1
  ros2/rosidl_typesupport:
    type: git
    url: https://github.com/ros2/rosidl_typesupport.git
    version: 0.6.1
  ros2/rosidl_typesupport_connext:
    type: git
    url: https://github.com/ros2/rosidl_typesupport_connext.git
    version: 0.6.3
  ros2/rosidl_typesupport_fastrtps:
    type: git
    url: https://github.com/ros2/rosidl_typesupport_fastrtps.git
    version: 0.6.0
  ros2/rosidl_typesupport_opensplice:
    type: git
    url: https://github.com/ros2/rosidl_typesupport_opensplice.git
    version: 0.6.1
  ros2/rviz:
    type: git
    url: https://github.com/ros2/rviz.git
    version: 5.0.0
  ros2/sros2:
    type: git
    url: https://github.com/ros2/sros2.git
    version: 0.6.0
  ros2/system_tests:
    type: git
    url: https://github.com/ros2/system_tests.git
    version: 0.6.0
  ros2/tinyxml_vendor:
    type: git
    url: https://github.com/ros2/tinyxml_vendor.git
    version: 0.5.0
  ros2/tinyxml2_vendor:
    type: git
    url: https://github.com/ros2/tinyxml2_vendor.git
    version: 0.6.0
  ros2/tlsf:
    type: git
    url: https://github.com/ros2/tlsf.git
    version: 0.5.0
#  ros2/tutorials:
#    type: git
#    url: https://github.com/ros2/tutorials.git
#    version: master
  ros2/unique_identifier_msgs:
    type: git
    url: https://github.com/ros2/unique_identifier_msgs.git
    version: 2.0.0
  ros2/urdf:
    type: git
    url: https://github.com/ros2/urdf.git
    version: 2.2.0
  ros2/urdfdom:
    type: git
    url: https://github.com/ros2/urdfdom.git
    version: 2.1.0
  ros2/yaml_cpp_vendor:
    type: git
    url: https://github.com/ros2/yaml_cpp_vendor.git
    version: 5.0.0
  • DDS implementation:
    -Fast-RTPS
  • Client library (if applicable):
    -rclcpp

Steps to reproduce issue

I initially posted this on ROS answers: https://answers.ros.org/question/313378/ros2-service-send-request-does-not-send/

But after digging into it some more I have become fairly convinced this is a bug.

This example uses ros2 services. The server manages a vector of ints. Clients can "register" new values, and the server will append these values to the vector. Clients can also query a position in the vector, and if the position exists, the server will return the value in that position.

The issue I have is that when starting with my launch file (ros2 launch ros_service_mwe mwe_service.launch.py) sometimes the client that is waiting for a value at particular position gets stuck in spin_until_future_complete(). I did some debugging with WireShark and it looks like the service request never makes it to the wire. This is tricky to troubleshoot as it fails ~1 in 10 times on my particular machine, but the failure rate changes depending on the machine.

The easiest way to detect this condition is to look at the terminal output and wait for an instance where the [getNode] does not print: [INFO] [getNode]: Value of Pos: 7

I can solve this issue by adding a timeout / retry strategy to the spin_until_future_complete() call, but I really don't understand the root cause of the issue and am afraid I will mask an issue that will manifest in the future.

I started this minimal example with the add_two_ints example, so the print_usage() function still matches that example.

Here is a link to the MWE: https://github.com/borgmanJeremy/ros_service_mwe

Expected behavior

The expected behavior is that spin_until_future_complete() will return once the server is available to handle the callback.

Actual behavior

The actual behavior is spin_until_future_complete() never returns

Additional Info:

This might be related to this question? #455
@DensoADAS
Copy link
Contributor

We encountered the same problem. We'd be glad about a solution.

@dschugk
Copy link

dschugk commented Feb 6, 2019

We encountered a similar problem. A solution would be great.

@dirk-thomas
Copy link
Member

The problem comes down to a race condition between (A) when it is detected that the server is running and (B) when the topics between the server and client have actually been matched. In the Crystal release the client->wait_for_service() call returns after (A). Hence your first service call races with the event (B). If the call happens before (B) it won't arrive at the server and therefore the getter node won't be able to retrieve the value (ever).

Since the Crystal release this patch has been merged which makes the wait_for_service call only return true after (B) has happened. That should avoid the race you are seeing.

Please consider trying to build the latest state from the master branch to check if it resolves your problem.

@dirk-thomas dirk-thomas added duplicate This issue or pull request already exists more-information-needed Further information is required labels Feb 6, 2019
@borgmanJeremy
Copy link
Author

Thanks - I can confirm that this patch on master fixes my issue.

nnmm pushed a commit to ApexAI/rclcpp that referenced this issue Jul 9, 2022
@sloretz
Doc rcl_pub/etc_fini() must come before rcl_node_fini() (ros2#625)
40f6a63 
Signed-off-by: Shane Loretz <sloretz@osrfoundation.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
duplicate This issue or pull request already exists more-information-needed Further information is required
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@dirk-thomas @dschugk @borgmanJeremy @DensoADAS