A set of python processing tools for ROS2 task experiments

---

This repository contains a set of tools designed to read bag files generated by ROS2 and high-level from experiment recordings. Bag files, the ROS2 standard for saving data, can be saved as '.bag' or '.db3' files and can be converted in a .mat/.txt/.csv file for further analysis. It expands upon the capabilities of the nml_bag package from Andrew Whitmore for reading bag files created from ROS2.

(Note: Tested in Ubuntu 20.04-22.04, and Windows 10-11 with ROS2 Galactic-Iron)

Prerequisites

---

• ROS2
• Python 3.8, with packages pandas, numpy, openpyxl (Use the package manager pip to install).

Installation

---

Start by installing the minimum library dependencies then cloning the repository:

pip install pandas numpy openpyxl bag
git clone https://github.com/Neuro-Mechatronics-Interfaces/ROS2_Data_Agent.git

The nml_bag package can be cloned inside the ROS2_Data_Agent repo and run the local 'setup.py' installation file.

cd ROS2_Data_Agent
git clone https://github.com/ricmua/nml_bag.git
cd nml_bag
pip install .

Quickstart/Usage

---

Basic usage of the package follows this pattern:

1. Import package.

   import data_agent as da

2. Create the DataAgent object. A full list of potential initialization parameters (like the subject name, data search path, etc.) can be found in its documentation.

   agent = da.DataAgent()

3. Read the bag file

   agent.read_bag('/path/to/bagfile.mcap')

   If there are multiple bag files to be processed, we can search for bag files in the specified folder and return a Python dict datatype containing a key files with a list of the bag file directories.

   file_list = agent.search_for_files('/path/to/bag/files', file_type='.mcap')
   agent.read_bag(file_list[0]['files'])

   If only bag files created on a specific date are to be read, the date can be passed as a string for the date_tag parameter. The date string should be in the format DD/MM/YYYY, and it assumes that the matching date is located in either the filename or the folder.

   agent.search_path = '/path/to/bag/files' # Setting the search path avoids passing it as a parameter
   file_list = agent.search_for_files(file_type='.mcap', date_tag=date)
   agent.read_bag(file_list[0]['files'])

   We can also set the combine parameter to True to combine all the topic data from the bag files into continuous datasets (useful for bag files from segmented recordings for maintaining small files sizes).

   file_list = agent.search_for_files(file_type='.mcap', date_tag=date)
   agent.read_bag(['/path/to/bag_1-of-2.mcap','/path/to/bag_2-of-2.mcap'], combine=True)

4. Extract topic data from the bag file and return a DataFrame object

   topic_df = agent.get_topic_data('/topic_name')

5. Save the bag file as a .csv file for future analysis. The .h5, .hdf5, .txt, and .mat file formats are also supported

   agent.save_data('/path/to/save/csv_file', file_type='.csv')

TO-DO Notes:

• An ArgParser object is being developed to allow for command-line input for the tools. Future work will include expanding its documentation and usage
• Saving task metrics and evaluating pulse timing are being developed and will be added to the documentation soon.
• Data saving directly to a csv is supported, and documentation will be updated to include this feature soon.

Examples

---

A handful of example scripts are provided to demonstrate the use of the tools:

1. example_rosbag_to_mat.py Simple example showing how to convert a bag file and save it to a local directory as a .mat file (for MATLAB use)
2. rosbag_to_mat.py Converts '.bag' files to '.mat' files with terminal input support (useful for batch processing).
3. example_save_task_metrics.py Run file with string containing root directory as 1st argument, file type string as the second argument, and data field string as the third
4. example_evaluate_pulse_timing.py Grabs sync trigger and robot position data to show sampling consistency
5. example_read_config.py - Example of reading parameters from a configuration file