TurtleSim ChatGPT

This demo demonstrates how ChatGPT can be used to call into ROS services, specifically services in turtlesim.

ROS is interfaced via WebSockets through rosbridge_suite. ChatGPT:

• Calls into WebSockets
• Does not execute any code on your machine
• Is given knowledge about the API via turtlesim_msgs/srv

Prompt:

Move turtle1 left by 2, then rotate 180 degrees, and move back to (5, 5). Finally, spawn a turtle named turtle2 at (10, 10) and remove turtle1.

Result:

Installation

• Have a running ROS distribution, e.g. Humble
• Clone this repo git clone https://github.com/ros-ai/chatgpt_turtlesim.git
• Install Python dependencies: pip install -r requirements.txt
• Register an account at OpenAI and get a key Where do I find my Secret API Key?

Run the Demo

1. Run turtlesim

source /opt/ros/humble/setup.bash # source your ROS distribution
ros2 run turtlesim turtlesim_node

2. Run rosbridge_server

source /opt/ros/humble/setup.bash # source your ROS distribution
ros2 launch rosbridge_server rosbridge_websocket_launch.xml

3. Prompt OpenAI's GPT models

python main.py --key your_key

You will we asked to enter a prompt. Try out

Move turtle1 left by 2, then rotate 180 degrees, and move back to (5, 5). Finally, spawn a turtle named turtle2 at (10, 10) and remove turtle1.

You will be presented with the generated API calls and will have to accept each one prior to execution.

Why not try others:

Add 3 more turtles with names inspired by artists at random positions

Move the turtles with artist's names forward by 2

Maybe you can find new ones! You might find that the format is not always adhered to!

Notes

This repository is just a proof of concept and it is rather slow. It, however, demonstrates how such a system could work more broadly across ROS.

What are limitations?

ChatGPT does not always generate answers in the expected JSON format. This is something that requires improvements.

Why roslibpy and rosbridge_suite? Why not just execute Python code?

Operating ROS services through WebSockets limits GPT's access to your system. It further relieves users from ROS dependencies. In the future, someone might find executing Python code more powerful!

Where to go from here?

Play around and have ChatGPT call into any desired services.

Why not action clients instead of services?

By the time of this writing, action clients are not supported via rosbridge_suite yet, refer to RobotWebTools/rosbridge_suite#697.