Skip to content

[ICCV 2023] OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perception

License

Notifications You must be signed in to change notification settings

JeffWang987/OpenOccupancy

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

35 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perception

Abstract

Semantic occupancy perception is essential for autonomous driving, as automated vehicles require a fine-grained perception of the 3D urban structures. However, existing relevant benchmarks lack diversity in urban scenes, and they only evaluate front-view predictions. Towards a comprehensive benchmarking of surrounding perception algorithms, we propose OpenOccupancy, which is the first surrounding semantic occupancy perception benchmark. In the OpenOccupancy benchmark, we extend the large-scale nuScenes dataset with dense semantic occupancy annotations. Previous annotations rely on LiDAR points superimposition, where some occupancy labels are missed due to sparse LiDAR channels. To mitigate the problem, we introduce the Augmenting And Purifying (AAP) pipeline to ~2x densify the annotations, where ~4000 human hours are involved in the labeling process. Besides, camera-based, LiDAR-based and multi-modal baselines are established for the OpenOccupancy benchmark. Furthermore, considering the complexity of surrounding occupancy perception lies in the computational burden of high-resolution 3D predictions, we propose the Cascade Occupancy Network (CONet) to refine the coarse prediction, which relatively enhances the performance by ~30% than the baseline. We hope the OpenOccupancy benchmark will boost the development of surrounding occupancy perception algorithms.

arXiv

News

  • [2023/03/22] The pretrained models and logs are available (see Assets).
  • [2023/03/20] We have released annotation v0.1 (more lightweght and less artifacts). Check Prepare Dataset for more details.
  • [2023/03/19] We have updated model configs (using syncbn enhances the performance).

Getting Started

Demo

Semantic Occupancy Annotations

gt.mp4

Visualization of different baselines

baseline_comparison.mp4

Visualization of CONet

conet_vis.mp4

Baseline framwork occupancy-baselines

Cascade-Occupancy-Network framwork occupancy-conet

Bibtex

If this work is helpful for your research, please consider citing the following BibTeX entry.

@article{wang2023openoccupancy,
  title={Openoccupancy: A large scale benchmark for surrounding semantic occupancy perception},
  author={Wang, Xiaofeng and Zhu, Zheng and Xu, Wenbo and Zhang, Yunpeng and Wei, Yi and Chi, Xu and Ye, Yun and Du, Dalong and Lu, Jiwen and Wang, Xingang},
  journal={arXiv preprint arXiv:2303.03991},
  year={2023}
}

Acknowledgement

Many thanks to these excellent projects:

Related Work in Occupancy Perception:

About

[ICCV 2023] OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perception

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages