Migrate to IDL

[dirk-thomas]

This is a meta ticket based on #298 which has since been split up into multiple PRs.

The goal is to use .idl files (using a subset of OMG IDL 4.2) to specify messages, services, actions and potentially more in the future. See https://github.com/ros2/ros2/blob/idl/README.rst#goals for a collection of bullets and references.

List of tasks and features (to be collect here from various places):

• fix problems extracting verbatim annotations from AST into object representation
• split action definitions into user provided types (goal, result, feedback) and wrapping types (send_goal service, get_result service, feedback_message)
• investigate extreme runtime for parsing some .idl files
• change uuid type to external message (see update tests to use separated action types rcl#340 (comment))
• get all tests to pass again...
• ensure keys are passed through to the vendor specific tools
• resolve action API problems
• readd target dependency on absolute path of .idl files

The latest set of patches using the branch name idl-stage-7:

• change generators to IDL-based pipeline, change char type, separate action types #334
• change generators to IDL-based pipeline rosidl_python#24
• change generators to IDL-based pipeline rosidl_typesupport#39
• change generators to IDL-based pipeline rosidl_typesupport_connext#15
• change generators to IDL-based pipeline rosidl_typesupport_fastrtps#14
• change generators to IDL-based pipeline rosidl_typesupport_opensplice#11
• change generator to IDL-based pipeline rosidl_dds#47
• update test fixtures for char type rcl_interfaces#64
• update tests to use separated action types rcl#340
• update to use separated action types rclcpp#601
• update to use separated action types rclpy#274
• update to use separated action types examples#227
• update char type mapping, update to use separated action types system_tests#315
• update test to accept old as well as new type name ros-visualization/rqt#191

[dirk-thomas]

• Linux FastRTPS only
• Linux
• Linux aarch64
• macOS
• Windows (same test failing in the nightly jobs)

[dirk-thomas]

• Linux
• Linux-aarch64 (known flaky test test_params_yaml)
• macOS
• Windows (known flaky test test_params_yaml)

[dirk-thomas]

Including OpenSplice:

• Linux
• Linux-aarch64
• macOS
• Windows

[dirk-thomas]

Hopefully the last round before merging - again including OpenSplice:

• Linux (known: test_params_yaml)
• Linux-aarch64 (known: test_params_yaml, test_pendulum with OpenSplice)
• macOS (various tests around parameters with OpenSplice - not failing with master...)
• Windows (known: test_params_yaml, test_graph with OpenSplice)

[dirk-thomas]

All RMW:

• Linux (test_params_yaml)
• Linux-aarch64 (test_params_yaml, test_pendulum with OpenSplice)
• macOS (various parameter tests with OpenSplice)
• Windows (test_params_yaml)

Only FastRTPS:

• Linux
• Linux-aarch64 (test_params_yaml)
• macOS
• Windows (test_params_yaml)

Only Connext:

• Linux (same test failure as on master, e.g. https://ci.ros2.org/job/ci_linux/6366/)
• Linux-aarch64
• macOS
• Windows (build failure due to lxml, ...)

Only OpenSplice:

• Linux
• Linux-aarch64 (test_pendulum with OpenSplice)
• macOS (same 6 test failures as on master, rclcpp_action/test_client)
• Windows (test_timer, test_two_timers)

[dirk-thomas]

All the above referenced PRs have been merged. Before closing this ticket I will create a few tickets with follow up work...

[dirk-thomas]

The existing design documents have been updated and published in: ros2/design#215

A note has been added to the Dashing page about the need to update message generators: ros2/ros2_documentation#139

And with the current status being captures in https://github.com/ros2/ros2/blob/idl/README.rst I am closing this ticket.