UniMODE & MM-UniMODE

UniMODE: Unified Monocular 3D Object Detection

Towards Unified 3D Object Detection via Algorithm and Data Unification

Zhuoling Li, Xiaogang Xu, Ser-Nam Lim, Hengshuang Zhao

[Project Page] [UniMODE Paper] [MM-UniMODE Paper]

This is the official implementation of the paper "UniMODE: Unified Monocular 3D Object Detection" (published in CVPR2024) and "Towards Unified 3D Object Detection via Algorithm and Data Unification".

Contributions:

• We propose UniMODE, a monocular 3D object detector unifying diverse indoor and outdoor scenarios.

• We propose MM-UniMODE, a multi-modal 3D object detector unifying diverse indoor and outdoor scenarios.

• We release the first large-scale multi-modal 3D object detection dataset, MM-Omni3D.

Table of Contents:

1. Installation
2. MM-Omni3D Data
3. Training
4. Evaluation
5. License
6. Citing

Installation

We provide the script file install_env.sh to install all the dependencies. You can use the following command or run each command line in the script file step by step (recommended).

bash install_env.sh

MM-Omni3D Data

Please download all the data in Data Link and unzip them in $Root\datasets. The unzipped data folder should look like:

datasets/

├── ARKitScenes/

├── KITTI_object/

├── MM-Omni3D/

├── SUNRGBD/

├── hypersim/

├── nuScenes/

└── objection/

Training

For training UniMODE, you can follow the script command template as follows:

python tools/train_net.py \
  --config-file configs/UniMODE.yaml \
  --num-gpus 16 \
  --num-machines 1 \
  --machine-rank 0 \
  --dist-url tcp://127.0.0.1:12345 \
  OUTPUT_DIR output/UniMODE

For training MM-UniMODE, you can follow the script command template as follows:

python tools/train_net.py \
  --config-file configs/MM_UniMODE.yaml \
  --num-gpus 16 \
  --num-machines 1 \
  --machine-rank 0 \
  --dist-url tcp://127.0.0.1:12345 \
  OUTPUT_DIR output/MM_UniMODE

Evaluation

For evaluating UniMODE, you can follow the script command template as follows:

python tools/train_net.py \
  --eval-only \
  --config-file configs/UniMODE.yaml \
  OUTPUT_DIR output/UniMODE \
  MODEL.WEIGHTS output/UniMODE/model_recent.pth

For evaluating MM-UniMODE, you can follow the script command template as follows:

python tools/train_net.py \
  --eval-only \
  --config-file configs/MM_UniMODE.yaml \
  OUTPUT_DIR output/UniMODE \
  MODEL.WEIGHTS output/MM_UniMODE/model_recent.pth

License

This project is released under CC-BY-NC 4.0.

Citing

Please use the following BibTeX entry if you use UniMODE, MM-UniMODE, or MM-Omni3D in your research or refer to our results.

@inproceedings{li2024unimode,
  title={UniMODE: Unified Monocular 3D Object Detection},
  author={Li, Zhuoling and Xu, Xiaogang and Lim, SerNam and Zhao, Hengshuang},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={16561--16570},
  year={2024}
}

@article{li2024towards,
  title={Towards Unified 3D Object Detection via Algorithm and Data Unification},
  author={Li, Zhuoling and Xu, Xiaogang and Lim, SerNam and Zhao, Hengshuang},
  journal={arXiv:2402.18573},
  year={2024}
}