News: Check our latest UniMatch V2 framework!
This codebase contains a strong re-implementation of FixMatch in the field of semi-supervised semantic segmentation, as well as the official PyTorch implementation of our UniMatch in the natural, remote sensing, and medical scenarios.
Revisiting Weak-to-Strong Consistency in Semi-Supervised Semantic Segmentation
Lihe Yang, Lei Qi, Litong Feng, Wayne Zhang, Yinghuan Shi
In Conference on Computer Vision and Pattern Recognition (CVPR), 2023
We provide a list of Awesome Semi-Supervised Semantic Segmentation works.
You can check our training logs for convenient comparisons during reproducing.
Note: we have added and updated some results in our camera-ready version. Please refer to our latest version.
Labeled images are sampled from the original high-quality training set. Results are obtained by DeepLabv3+ based on ResNet-101 with training size 321.
| Method | 1/16 (92) | 1/8 (183) | 1/4 (366) | 1/2 (732) | Full (1464) |
|---|---|---|---|---|---|
| SupBaseline | 45.1 | 55.3 | 64.8 | 69.7 | 73.5 |
| U2PL | 68.0 | 69.2 | 73.7 | 76.2 | 79.5 |
| ST++ | 65.2 | 71.0 | 74.6 | 77.3 | 79.1 |
| PS-MT | 65.8 | 69.6 | 76.6 | 78.4 | 80.0 |
| UniMatch (Ours) | 75.2 | 77.2 | 78.8 | 79.9 | 81.2 |
Results are obtained by DeepLabv3+ based on ResNet-50/101. We reproduce U2PL results on ResNet-50.
Note: the results differ from our arXiv-V1 because we change the confidence threshold from 0.95 to 0, and change the ResNet output stride from 8 to 16. Therefore, it is currently more efficient to run.
You can click on the numbers to be directed to corresponding checkpoints.
| ResNet-50 | 1/16 | 1/8 | 1/4 | 1/2 | ResNet-101 | 1/16 | 1/8 | 1/4 | 1/2 |
|---|---|---|---|---|---|---|---|---|---|
| SupBaseline | 63.3 | 70.2 | 73.1 | 76.6 | SupBaseline | 66.3 | 72.8 | 75.0 | 78.0 |
| U2PL | 70.6 | 73.0 | 76.3 | 77.2 | U2PL | 74.9 | 76.5 | 78.5 | 79.1 |
| UniMatch (Ours) | 75.0 | 76.8 | 77.5 | 78.6 | UniMatch (Ours) | 76.6 | 77.9 | 79.2 | 79.5 |
Results are obtained by DeepLabv3+ based on Xception-65.
You can click on the numbers to be directed to corresponding checkpoints.
| Method | 1/512 (232) | 1/256 (463) | 1/128 (925) | 1/64 (1849) | 1/32 (3697) |
|---|---|---|---|---|---|
| SupBaseline | 22.9 | 28.0 | 33.6 | 37.8 | 42.2 |
| PseudoSeg | 29.8 | 37.1 | 39.1 | 41.8 | 43.6 |
| PC2Seg | 29.9 | 37.5 | 40.1 | 43.7 | 46.1 |
| UniMatch (Ours) | 31.9 | 38.9 | 44.4 | 48.2 | 49.8 |
We also apply our UniMatch in the scenarios of semi-supervised remote sensing change detection and medical image segmentation, achieving tremendous improvements over previous methods:
cd UniMatch
conda create -n unimatch python=3.10.4
conda activate unimatch
pip install -r requirements.txt
pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 -f https://download.pytorch.org/whl/torch_stable.htmlResNet-50 | ResNet-101 | Xception-65
├── ./pretrained
├── resnet50.pth
├── resnet101.pth
└── xception.pth
- Pascal: JPEGImages | SegmentationClass
- Cityscapes: leftImg8bit | gtFine
- COCO: train2017 | val2017 | masks
Please modify your dataset path in configuration files.
The groundtruth masks have already been pre-processed by us. You can use them directly.
├── [Your Pascal Path]
├── JPEGImages
└── SegmentationClass
├── [Your Cityscapes Path]
├── leftImg8bit
└── gtFine
├── [Your COCO Path]
├── train2017
├── val2017
└── masks
# use torch.distributed.launch
sh scripts/train.sh <num_gpu> <port>
# to fully reproduce our results, the <num_gpu> should be set as 4 on all three datasets
# otherwise, you need to adjust the learning rate accordingly
# or use slurm
# sh scripts/slurm_train.sh <num_gpu> <port> <partition>To train on other datasets or splits, please modify
dataset and split in train.sh.
Modify the method from 'unimatch' to 'fixmatch' in train.sh.
Modify the method from 'unimatch' to 'supervised' in train.sh, and double the batch_size in configuration file if you use the same number of GPUs as semi-supervised setting (no need to change lr).
If you find this project useful, please consider citing:
@inproceedings{unimatch,
title={Revisiting Weak-to-Strong Consistency in Semi-Supervised Semantic Segmentation},
author={Yang, Lihe and Qi, Lei and Feng, Litong and Zhang, Wayne and Shi, Yinghuan},
booktitle={CVPR},
year={2023}
}We have some other works on semi-supervised semantic segmentation: