Improving Pedestrian Prediction Models with Self-Supervised Continual Learning

This repository contains the code for the paper:

Improving Pedestrian Prediction Models with Self-Supervised Continual Learning
Luzia Knoedler, Chadi Salmi, Hai Zhu, Bruno Brito, Javier Alonso-Mora
Accepted for publication in [RA-L + ICRA 2022].
Video presenting simulation results and results from real-world experiments

Autonomous mobile robots require accurate human motion predictions to safely and efficiently navigate among pedestrians, whose behavior may adapt to environmental changes. This paper introduces a self-supervised continual learning framework to improve data-driven pedestrian prediction models online across various scenarios continuously. In particular, we exploit online streams of pedestrian data, commonly available from the robot's detection and tracking pipelines, to refine the prediction model and its performance in unseen scenarios. To avoid the forgetting of previously learned concepts, a problem known as catastrophic forgetting, our framework includes a regularization loss to penalize changes of model parameters that are important for previous scenarios and retrains on a set of previous examples to retain past knowledge. Experimental results on real and simulation data show that our approach can improve prediction performance in unseen scenarios while retaining knowledge from seen scenarios when compared to naively training the prediction model online.

Description

The focus of this repo is to test online learning methods that train trajectory prediction model architectures in simulation as well as in the real world. Rosinterfaces are provided to easily couple the predictors with existing robots running ROS.

Installation Instructions

First, install the package (it is advised to create a python virtual environment):

# clone project   
git clone https://github.com/tud-amr/scl
cd scl

# (advised) Create virtual env
mkdir venv && cd venv
python3 -m venv scl
source scl/bin/activate
cd ..
pip install --upgrade pip

# install project   
pip install -r requirements.txt
pip install -e .   

Run the code

First, start the simulation environment:

roslaunch pedsim_simulator pedsim.launch scenario:=corridor

Models and other scripts in this repository should run from the base directory. This makes resolving paths easier and more consistent.

Example of a command to run inference on a saved prediction model using the rosinterface.predictor_node:

python -m project.rosinterfaces.predictor_node \
 --model EthPredictor \
 --save_name eth-ucy_pos \
 --frequency 20

Example of a command to train an EthPredictor model on eth and ucy datasets for 50 epochs.

# run module (example: train the ewc_eth_predictor)   
python -m project.training_routines.ewc \
 --model EthPredictor \
 --datasets eth ucy \
 --tbptt 15 \
 --max_epochs 50 \
 --save_name my_model