README.md

EBLNet

This repo contains the the implementation of Our ICCV-2021 work: Enhanced Boundary Learning for Glass-like Object Segmentation.

This is a join work of Institute of Automation, Chinese Academy of Sciences, Peking University, and Sensetime Research. Much thanks for Sensetime's GPU clusters.

paper

Any Suggestions/Questions/Pull Requests are welcome.

The master branch works with PyTorch 1.5 and python 3.7.6

DataSet preparation

Dataloaders for Trans10k, MSD, and GDD are available in datasets. Details of preparing each dataset can be found at PREPARE_DATASETS.md

Model Checkpoint

Pretrained Models

Baidu Pan Link: https://pan.baidu.com/s/15-ldYHk-__VI0ZhLMUFPzA (bpkw)

Google Drive Link: https://drive.google.com/drive/folders/1S6Mv_559_0u851OOXbVyGXgvFCNoMknj?usp=sharing

After downloading the pretrained backbones, you can either change the path in network/resnet_d.py (for ResNet50 and ResNet101) and network/resnext.py (for ResNeXt101), or do soft link according to the default path in network/resnet_d.py and network/resnext.py.

For example, suppose you store the pretrained model at ~/user_name/pretrained_models, you can ln -s ~/user_name/pretrained_models/* ~/user_name/codes/EBLNet/pretrained_models

or you can update the model path in resnet_d.py and resnext.py, like ResNet50 in Line281 of network/resnet_d.py,

model.load_state_dict(torch.load("~/user_name/pretrained_models/resnet50-deep.pth", map_location='cpu'))

Trained Models and CKPTs

Here are the trained models reported in our paper, you can use them to evaluate.

Dataset	Backbone	mIoU	Model
Trans10k	ResNet50 (os16)	89.58	Google Drive | Baidu Pan(7eg1)
Trans10k	ResNet50 (os8)	90.28	Google Drive | Baidu Pan(5aaf)
GDD	ResNet101	88.16	Google Drive | Baidu Pan(fq6a)
GDD	ResNeXt101	88.72	Google Drive | Baidu Pan(fi6v)
MSD	ResNet101	78.84	Google Drive | Baidu Pan(t9pr)
MSD	ResNeXt101	80.33	Google Drive | Baidu Pan(pvs2)

Evaluation

After downloading the trained models, you can evaluate the models with the evaluation scripts, for example, when evaluating the EBLNet (ResNet50 as backbone) on test set of Trans10k dataset:

sh scripts/test/test_Trans10k_R50_EBLNet.sh path_to_checkpoint path_to_save_results

After running this script, you can get the iou of things and stuff are 92.73 and 87.82, respectively. Thus, the mIoU is 90.28 as reported in the last line of out paper Table. 2. Note that, when computing the mean IoU, we do not include the background.

During evaluation, if you want to save images during evaluating for visualization, all you need to do is add args: dump_images in the test scripts. Note that, saving images will take more time.

Training

To be note that, all our models are trained on 8 V-100 GPUs with 32G memory. It is hard to reproduce the results if you do not have such resources. For example, when training EBLNet with ResNet50 (os8) as backbone on the Trans10k dataset:

sh scripts/train/train_Trans10k_R50_EBLNet.sh

Citation

If you find this repo is helpful to your research. Please consider cite our work.

@InProceedings{He_2021_ICCV,
    author    = {He, Hao and Li, Xiangtai and Cheng, Guangliang and Shi, Jianping and Tong, Yunhai and Meng, Gaofeng and Prinet, Véronique and Weng, LuBin},
    title     = {Enhanced Boundary Learning for Glass-Like Object Segmentation},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2021},
    pages     = {15859-15868}
}

Acknowledgement

This repo is based on NVIDIA segmentation repo. We fully thank their open-sourced code.