This repository provides the official PyTorch implementation of the following paper:
Standardized Max Logits: A Simple yet Effective Approach for Identifying Unexpected Road Obstacles in Urban-Scene Segmentation
Sanghun Jung* (KAIST AI), Jungsoo Lee* (KAIST AI), Daehoon Gwak (KAIST AI)
Sungha Choi (LG AI Research), and Jaegul Choo (KAIST AI) (*: equal contribution)
ICCV 2021 (Oral)
Paper: arxiv
Youtube Video (English): Youtube
Abstract: Identifying unexpected objects on roads in semantic segmentation (e.g., identifying dogs on roads) is crucial in safety-critical applications. Existing approaches use images of unexpected objects from external datasets or require additional training (e.g., retraining segmentation networks or training an extra network), which necessitate a non-trivial amount of labor intensity or lengthy inference time. One possible alternative is to use prediction scores of a pre-trained network such as the max logits (i.e., maximum values among classes before the final softmax layer) for detecting such objects. However, the distribution of max logits of each predicted class is significantly different from each other, which degrades the performance of identifying unexpected objects in urban-scene segmentation. To address this issue, we propose a simple yet effective approach that standardizes the max logits in order to align the different distributions and reflect the relative meanings of max logits within each predicted class. Moreover, we consider the local regions from two different perspectives based on the intuition that neighboring pixels share similar semantic information. In contrast to previous approaches, our method does not utilize any external datasets or require additional training, which makes our method widely applicable to existing pre-trained segmentation models. Such a straightforward approach achieves a new state-of-the-art performance on the publicly available Fishyscapes Lost & Found leaderboard with a large margin.
Sanghun Jung [Website] [LinkedIn] [Google Scholar] (KAIST AI)
Jungsoo Lee [Website] [LinkedIn] [Google Scholar] (KAIST AI)
Click the figure to watch the youtube video of our paper!
Clone this repository.
git clone https://github.com/shjung13/Standardized-max-logits.git
cd Standardized-max-logits
pip install -r requirements.txt
cityscapes
└ leftImg8bit_trainvaltest
└ leftImg8bit
└ train
└ val
└ test
└ gtFine_trainvaltest
└ gtFine
└ train
└ val
└ test
Fishyscapes (OoD Dataset)
└ leftImg8bit_trainvaltest
└ leftImg8bit
└ val
└ gtFine_trainvaltest
└ gtFine
└ val
Please download resnet101 network pretrained on ImageNet dataset from this link and place it in pretrained/resnet101-imagenet.pth path.
Then run the below command.
CUDA_VISIBLE_DEVICES=0,1 ./scripts/train_r101_os8.sh
CUDA_VISIBLE_DEVICES=0 ./scripts/calc_stat_r101_os8.sh
Download the pretrained model here and after creating "<Directory Home>/pretrained", place it under the folder.
CUDA_VISIBLE_DEVICES=0 python eval.py --ood_dataset_path <path_to_OoD_dataset>
Our result is also available at fishyscapes.com.
@InProceedings{Jung_2021_ICCV,
author = {Jung, Sanghun and Lee, Jungsoo and Gwak, Daehoon and Choi, Sungha and Choo, Jaegul},
title = {Standardized Max Logits: A Simple yet Effective Approach for Identifying Unexpected Road Obstacles in Urban-Scene Segmentation},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2021},
pages = {15425-15434}
}
We deeply appreciate Hermann Blum and FishyScapes team for their sincere help in providing the baseline performances and helping our team to update our model on the FishyScapes Leaderboard. Our pytorch implementation is heavily derived from NVIDIA segmentation and RobustNet. Thanks to the NVIDIA implementations.