COARSE3D: Class-Prototypes for Contrastive Learning in Weakly-Supervised 3D Point Cloud Segmentation (BMVC 2022)
Rong Li 1,
Anh-Quan Cao 2,
Raoul de Charette 2
1SCUT, China. 2Inria, France
If you find this work or code useful, please cite our paper and give this repo a star:
@inproceedings{coarse3d,
title={COARSE3D: Class-Prototypes for Contrastive Learning in Weakly-Supervised 3D Point Cloud Segmentation},
author={Rong Li, Anh-Quan Cao and Raoul de Charette},
booktitle={BMVC},
year={2022},
}
- Create environment.
conda create -n coarse3d python==3.8 conda activate coarse3d
- Install dependencies.
conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=10.2 -c pytorch pip install -r requirements.txt
- Git clone and enter repo.
git clone https://github.com/cv-rits/COARSE3D cd COARSE3D
-
Download SemanticKITTI dataset:
- Download KITTI Odometry Benchmark Velodyne point clouds (80 GB)
- Download SemanticKITTI label data (179 MB)
-
Enter subdir.
cd tasks/prepare_data/
-
Set params in the
gen_sem_weak_label_rand_grid.py
, more details provided in the help comments of the parser. -
Train
python gen_sem_weak_label_rand_grid.py
Then, the dataset strcuture should be like:
└── semantic-kitti-coarse3d └── sequences ├── 00 │ └── xxx # weakly-sup label ├── 01 ... ...
-
Download SemanticPOSS dataset:
- LiDAR point cloud and label (2.2G)
-
Enter subdir.
cd tasks/prepare_data/
-
Set params in the
gen_sem_weak_label_rand_grid.py
, more details provided in the help comments of the parser. -
Generate weak label.
python gen_sem_weak_label_rand_grid.py
then, the data structure of weak label should be like:
└── semantic-poss-coarse3d └── sequences ├── 00 │ └── xxx # weakly-sup label ├── 01 ... ...
-
Register and download nuScenes-lidarseg dataset.
-
Enter subdir.
cd tasks/prepare_data/
-
Set params in the
gen_nuscenes_weak_label_rand_grid.py
, more details provided in the help comments of the parser. -
Generate weak label.
python gen_nuscenes_weak_label_rand_grid.py
Then, the data structure should be like:
└── nuscenes-coarse3d └── v1.0-trainval ├── 6d164dadab534fb087413b807bcf1de3_xxx.npy # weakly-sup label ├── 6d0eb63329684744b3a6ffd4fbab830c_xxx.npy # weakly-sup label ... ...
-
Download ImageNet pretrained model.
-
Enter sub dir.
cd tasks/weak_segmentation
-
Adapt
config_dataset.yaml
file (e.g.config_semantic_kitti.yaml
), details are provided in the comments. -
Adapt
run.sh
file. Replace theconfig_dataset.yaml
with the dataset you are interested. -
Train.
bash run.sh
Dataset | Anno. (%) | Val mIoU (%) | Trained Model |
---|---|---|---|
SemanticKITTI | 0.01 | 47.35 | link |
0.1 | 57.57 | link | |
SemanticPOSS | 0.01 | 31.10 | link |
0.1 | 43.00 | link | |
nuScenes | 0.01 | 42.90 | link |
0.1 | 58.70 | link |
Rong Li was supported by the SMIL lab of South China University of Technolog, received support and advices from Prof. Mingkui Tan, Prof. Caixia Li and Zhuangwei Zhuang. Inria members were partly funded by French project SIGHT (ANR-20-CE23-0016). This work was performed using HPC resources from GENCI–IDRIS (Grant 2021-AD011012808 and 2022-AD011012808R1). We would also like to thank Ivan Lopes for the video voice-over.
COARSE3D is released under the Apache 2.0 license.