Graph archetype

[vmayoral]

Most of the computations in robotics are expressed in the form of computational graphs. Whether you use ROS, ROS 2 or alternative frameworks, this seems to be the general consensus.

(an arbitrary ROS graph)

Besides their use for dev-purposes, these graphs are a fundamental tool when visualizing a robot's architecture. Most ROS-related graph representations are lacking. There's an interesting opportunity to get this right within rerun. That would make it much more useful for roboticists.

[nikolausWest]

Thanks @vmayoral!

There's an interesting opportunity to get this right within rerun.

could you expand on this? Sounds very interesting!

[vmayoral]

Hey @nikolausWest,

I'll try to expand on both the motivation and on the technical approach, sure:

• Motivation-wise: (As I believe you folks know already) Robots are networks of networks, with sensors capturing data, passing to compute technologies, and then on to actuators and back again in a deterministic manner. These networks can be understood as the nervous system of the robot, passing across compute Nodes, that represent neurons. Like the human nervous systems, real-time information across all these computational Nodes is fundamental for the robot to behave coherently. "Robot brains" are built with this same philosophy. Behaviors take the form of computational graphs, with data flowing between Nodes, across physical networks (communication buses) and while mapping to underlying sensors, compute technologies and actuators. Frameworks like ROS enables you to build these computational graphs and create robot behaviors by providing libraries, a communication infrastructure, drivers and tools to put it all together. Alternatives to ROS also bet into this graph-like concepts. In a way, robot behaviors derive from the dataflow across these graphs, thereby getting a good representation of such (computational) graphs is fundamental for robot visualization.
• Technically (getting this right¹): Let me start by telling you what works with limitations and what doesn't, and then throw a few suggestions.
  • Most of us still rely on rqt_graph for (robot) graph visualization. This tool hasn't evolved alongside modern software concepts in robotics (aspects such as ROS Components as opposed to Nodes (or Nodelets) aren't depicted). There're plenty of limitations on this regard. You still can't easily properly represent other-than-pub/sub relationships into these graphs, for example client-server interactions such as ROS services.
  • Web-based alternatives to rqt_graph that I know are very lacking. For example Foxglove's Topic Graph is useless, probably resulting from a lack of understanding of the criticality of these graphs for demonstrating/showcasing/building/repairing/improving/debuging robots. There're other web-based robot graph visualizations that have been prototyped, but they simply lack the simplicity, (programming) usability, easy of customization and responsiveness that you folks at rerun seem to be taking at your core.
  • Suggestions:
    • Graphs should be simple to visualize, responsive and easy to custom programatically.
    • It should be possible to represent basic pub/sub interactions between nodes in the graph as well as other communication paradigms, capturing abstractions like ROS services or actions.
    • Ease of graph customization is key for succeeding in my opinion. Modern/custom aspects such as depicting subgroups within the graph to capture Components vs Nodes, or inter-network vs intra-network communications within a robotic system come to mind as relevantly unmet.
    • There's a huge opportunity in rerun to make use of theSelectionPanel to further augment information by selecting a node in the graph. This information could/should bet set programatically within each graph node, and could be extracted/relied from data-frameworks like ROS. Things like frequency of interactions, bandwidth, latencies (max/mean/min), etc would be tremendously useful if used together with your already existing ploting capabilities (e.g. TimeSeriesView).

Footnotes

1. I can only throw a few initial ideas on the getting right topic, as "getting it right" would likely require iterations and feedback for its achievement ↩

[nikolausWest]

@vmayoral: we're considering giving this a go soon. Would you be willing to feedback on some early designs / requirements to help us make sure we actually get it right?

[grtlr]

I've written down a proposal for extending the data model to handle graphs here: #7431. This could be a first step to implement the use cases that @vmayoral describes here.

[grtlr]

Quick Update: With #7500 the graph primitives have landed in Rerun. The next steps are working laying out these graphs, so that they can also be used to visualize ROS node/topic graphs.