This repository contains the code for the innovation award solution of the nuPlan Planning Challenge at the CVPR'23 End-to-End Autonomous Driving Workshop.
GameFormer Planner: A Learning-enabled Interactive Prediction and Planning Framework for Autonomous Vehicles
Zhiyu Huang, Haochen Liu, Xiaoyu Mo, Chen Lv
AutoMan Research Lab, Nanyang Technological University
[Report] [GameFormer Paper] [Project Website] [Presentation]
This is an extension of GameFormer, focusing on a comprehensive planning framework for autonomous driving. The framework consists of four key steps: feature processing, path planning, model query, and trajectory refinement. Comprehensive evaluations conducted on the nuPlan benchmark demonstrate the competitive performance of the proposed planning framework, validating its effectiveness in both open-loop and closed-loop tests.
To begin, please follow these steps:
- Download the nuPlan dataset and set it up as described here.
- Install the nuPlan devkit here (version tested: v1.2.2).
- Clone this repository and navigate into the folder:
git clone https://github.com/MCZhi/GameFormer-Planner.git && cd GameFormer-Planner
- Activate the environment created when installing the nuPlan-devkit:
conda activate nuplan
- Install the required packages:
pip install -r requirements.txt
- Add the following environment variable to your
~/.bashrc(you can customize it):
export NUPLAN_EXP_ROOT="$HOME/nuplan/exp"
Before training the GameFormer model, you need to preprocess the raw data using:
python data_process.py \
--data_path nuplan/dataset/nuplan-v1.1/splits/mini \
--map_path nuplan/dataset/maps \
--save_path nuplan/processed_data
Three arguments are necessary: --data_path to specify the path to the stored nuPlan dataset, --map_path to specify the path to the nuPlan map data, and --save_path to specify the path to save the processed data.
Optional arguments like --scenarios_per_type and --total_scenarios can also be used to specify the amount of data to process.
To train the GameFormer model, run:
python train_predictor.py \
--train_set nuplan/processed_data/train \
--valid_set nuplan/processed_data/valid
Two arguments are necessary: --train_set to specify the path to the processed training data and --valid_set to specify the path to the processed validation data.
Optional model arguments: --encoder_layers for the number of encoding layers, --decoder_layers for the number of interaction decoding layers, and --num_neighbors for the number of neighboring agents to predict (max number is 20).
Optional training parameters:--train_epochs, --batch_size, and --learning_rate.
To test the planning framework in nuPlan simulation scenarios, use:
python run_nuplan_test.py \
--experiment_name open_loop_boxes \
--data_path nuplan/dataset/nuplan-v1.1/splits/mini \
--map_path nuplan/dataset/maps \
--model_path training_log/your/model
Choose one of the three options ('open_loop_boxes', 'closed_loop_nonreactive_agents', 'closed_loop_reactive_agents') for --experiment_name, and specify the --model_path, which points to your trained model. Ensure to provide --data_path and --map_path arguments as done in the data process step.
Adjust the --scenarios_per_type and --total_scenarios arguments to control the number of scenarios tested.
Make sure the model parameters in planner.py in _initialize_model match those used in training.
If you have any questions or suggestions, please feel free to open an issue or contact us (zhiyu001@e.ntu.edu.sg).
If you find this repository useful for your research, please consider giving us a star 🌟 and citing our paper.
@InProceedings{Huang_2023_ICCV,
author = {Huang, Zhiyu and Liu, Haochen and Lv, Chen},
title = {GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Driving},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2023},
pages = {3903-3913}
}