Timothy Chen1 · Aiden Swann1 · Javier Yu1 · Ola Shorinwa1 · Riku Murai2 · Monroe Kennedy III1 · Mac Schwager1
1Stanford University
2Imperial College London
Project Page | Paper | arXiv | Data
SAFER-Splat (Simultaneous Action Filtering and Environment Reconstruction) is a real-time, scalable, and minimally invasive action filter, based on control barrier functions, for safe robotic navigation in a detailed map constructed at runtime using Gaussian Splatting.
We propose a novel Control Barrier Function (CBF) that not only induces safety with respect to all Gaussian primitives in the scene, but when synthesized into a controller, is capable of processing hundreds of thousands of Gaussians while maintaining a minimal memory footprint and operating at 15 Hz during online Splat training. Of the total compute time, a small fraction of it consumes GPU resources, enabling uninterrupted training. The safety layer is minimally invasive, correcting robot actions only when they are unsafe. To showcase the safety filter, we also introduce SplatBridge, an open-source software package built with ROS for real-time GSplat mapping for robots. We demonstrate the safety and robustness of our pipeline first in simulation, where our method is 20-50x faster, safer, and less conservative than competing methods based on neural radiance fields. Further, we demonstrate simultaneous GSplat mapping and safety filtering on a drone hardware platform using only on-board perception. We verify that under teleoperation a human pilot cannot invoke a collision.
- Upload SplatBridge and ROS nodes.
- Upload visualizer of trajectories in Nerfstudio viewer.
- Provide interactive Colab notebook that allows users to interface with the safety layer on a trained Splat.
This repository is built off of Nerfstudio. Please first follow the installation instructions there before installing any of the dependencies specifically for this repository. Once you have Nerfstudio installed in your Conda environment, install the following dependencies in that environment.
- CLARABEL. This library is for solving the quadratic program.
Our datasets are hosted on a Google Drive. The scenes used in the paper are flightgate, statues, stonehenge, adirondacks. The training data is in the data folder, while the model parameters are in outputs. You can drag and drop these folders into your working directory.
Here's an example:
SAFER-Splat
├── data
│ └── flight
│ └── images
│ └── transforms.json
│
├──outputs
│ └── flight
│ └── splatfacto
│ └── 2024-09-12_172434
│ └── nerfstudio_models
| └── config.yml
| └── dataparser_transforms.json # This file contains the transform that transforms from "Data" frame to "Nerfstudio" frame (which is typically a unit box)
├── run.py
After the dependencies and the data is setup, run
python run.py
The most important thing is to ensure that the path in NeRFWrapper is pointing to the right model location.
The variants for the distance can be changed.
Under construction...
To use your own datasets, simply train a Nerfstudio splatfacto model and follow the folder structure as illustrated above. If your images contain pixels that are transparent like in the case of synthetic scenes (i.e. the alpha value is 0), it is recommended to use the instant-ngp-data flag (e.g. ns-train splatfacto --data {DATA} instant-ngp-data) rather than blender-data or the default.
If you found SAFER-Splat useful, consider citing us! Thanks!
@misc{chen2024safersplatcontrolbarrierfunction,
title={SAFER-Splat: A Control Barrier Function for Safe Navigation with Online Gaussian Splatting Maps},
author={Timothy Chen and Aiden Swann and Javier Yu and Ola Shorinwa and Riku Murai and Monroe Kennedy III au2 and Mac Schwager},
year={2024},
eprint={2409.09868},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2409.09868},
}