Skip to content

MIT-REALM/LLM_gcbfplus

Repository files navigation

LLM-GCBF+ for deadlock resolution

Jax Official Implementation of Paper: K Garg*, J Arkin, S Zhang, N Roy, C Fan: "Large Language Models to the Rescue: Deadlock Resolution in Multi-Robot Systems".

Dependencies

We recommend to use CONDA to install the requirements:

conda create -n gcbfplus python=3.10
conda activate gcbfplus
cd gcbfplus

Then install jax following the official instructions, and then install the rest of the dependencies:

pip install -r requirements.txt

Installation

Install GCBF:

pip install -e .

Run

High-level planner for deadlock resolution

To run the high-level planner for deadlock resolution, use:

python -u  test_with_LLM.py --path logs/SingleIntegrator/gcbf+/model_with_traj/seed0_20240227110346 -n 10 --epi 20 --obs 25 --max-step 2500 --area-size 4  --keep-mode 20 --nojit-rollout --num-incontext-prompts 0 --leader_model 'gpt3.5' --num_LLM_calls 1

where the flags are:

  • -n: number of agents
  • --obs: number of obstacles
  • --area-size: side length of the environment
  • --max-step: maximum number of steps for each episode, increase this if you have a large environment
  • --path: path to the log folder
  • --keep-mode: keep mode for the high-level planner
  • --num-incontext-prompts: number of in-context examples
  • --leader_model: leader model for the high-level planner including
    • 'gpt3.5'
    • 'gpt4'
    • 'hand' for hand-designed heuristic leader-assignment
    • 'fixed' for fixed leader-assignment
    • 'random' for random leader-assignment
    • 'none' for no leader-assignment
  • --num_LLM_calls: number of LLM calls for "Ensemble" implementation of the high-level planner

For testing on "Randomized room" environment, use:

python -u  test_with_LLM.py --path logs/SingleIntegrator/gcbf+/model_with_traj/seed0_20240227110346/ -n 1 --epi 20 --obs 1 --preset_reset --preset_scene 'rand box' --max-step 2500 --area-size 1 --keep-mode 20 --nojit-rollout --num-incontext-prompts 0 --leader_model 'gpt3.5' --num_LLM_calls 1 

where ---preset_reset is used to reset the environment to a fixed initial state from - 'rand box' for a random room environment - 'original box' for a fixed room environment - 'box' for room-like environment with more obstacles.

GCBF+ low-level controller for safe multi-agent navigation

Environments

We provide 3 2D environments including SingleIntegrator, DoubleIntegrator, and DubinsCar, and 2 3D environments including LinearDrone and CrazyFlie.

Algorithms

We provide algorithms including GCBF+ (gcbf+), GCBF (gcbf), centralized CBF-QP (centralized_cbf), and decentralized CBF-QP (dec_share_cbf). Use --algo to specify the algorithm.

Hyper-parameters

To reproduce the results shown in our paper, one can refer to settings.yaml.

Train

To train the model (only GCBF+ and GCBF need training), use:

python train.py --algo gcbf+ --env DoubleIntegrator -n 8 --area-size 4 --loss-action-coef 1e-4 --n-env-train 16 --lr-actor: 1e-5 --lr-cbf: 1e-5 --horizon: 32

In our paper, we use 8 agents with 1000 training steps. The training logs will be saved in folder ./logs/<env>/<algo>/seed<seed>_<training-start-time>. We also provide the following flags:

  • -n: number of agents
  • --env: environment, including SingleIntegrator, DoubleIntegrator, DubinsCar, LinearDrone, and CrazyFlie
  • --algo: algorithm, including gcbf, gcbf+
  • --seed: random seed
  • --steps: number of training steps
  • --name: name of the experiment
  • --debug: debug mode: no recording, no saving
  • --obs: number of obstacles
  • --n-rays: number of LiDAR rays
  • --area-size: side length of the environment
  • --n-env-train: number of environments for training
  • --n-env-test: number of environments for testing
  • --log-dir: path to save the training logs
  • --eval-interval: interval of evaluation
  • --eval-epi: number of episodes for evaluation
  • --save-interval: interval of saving the model

In addition, use the following flags to specify the hyper-parameters:

  • --alpha: GCBF alpha
  • --horizon: GCBF+ look forward horizon
  • --lr-actor: learning rate of the actor
  • --lr-cbf: learning rate of the CBF
  • --loss-action-coef: coefficient of the action loss
  • --loss-h-dot-coef: coefficient of the h_dot loss
  • --loss-safe-coef: coefficient of the safe loss
  • --loss-unsafe-coef: coefficient of the unsafe loss
  • --buffer-size: size of the replay buffer

Test

To test the learned model, use:

python test.py --path <path-to-log> --epi 5 --area-size 4 -n 16 --obs 0

This should report the safety rate, goal reaching rate, and success rate of the learned model, and generate videos of the learned model in <path-to-log>/videos. Use the following flags to customize the test:

  • -n: number of agents
  • --obs: number of obstacles
  • --area-size: side length of the environment
  • --max-step: maximum number of steps for each episode, increase this if you have a large environment
  • --path: path to the log folder
  • --n-rays: number of LiDAR rays
  • --alpha: CBF alpha, used in centralized CBF-QP and decentralized CBF-QP
  • --max-travel: maximum travel distance of agents
  • --cbf: plot the CBF contour of this agent, only support 2D environments
  • --seed: random seed
  • --debug: debug mode
  • --cpu: use CPU
  • --u-ref: test the nominal controller
  • --env: test environment (not needed if the log folder is specified)
  • --algo: test algorithm (not needed if the log folder is specified)
  • --step: test step (not needed if testing the last saved model)
  • --epi: number of episodes to test
  • --offset: offset of the random seeds
  • --no-video: do not generate videos
  • --log: log the results to a file
  • --dpi: dpi of the video
  • --nojit-rollout: do not use jit to speed up the rollout, used for large-scale tests

To test the nominal controller, use:

python test.py --env SingleIntegrator -n 16 --u-ref --epi 1 --area-size 4 --obs 0

To test the CBF-QPs, use:

python test.py --env SingleIntegrator -n 16 --algo dec_share_cbf --epi 1 --area-size 4 --obs 0 --alpha 1

Pre-trained models

We provide the pre-trained models in the folder logs.