[ECCV 2024] Context-Guided Spatial Feature Reconstruction for Efficient Semantic Segmentation

Zhenliang Ni, Xinghao Chen, Yingjie Zhai, Yehui Tang, and Yunhe Wang

🔥 Updates

• 2024/10/08: The training code is released and fixed bugs in the issue.
• 2024/07/01: The paper of CGRSeg is accepted by ECCV 2024.
• 2024/05/10: Codes of CGRSeg are released in Pytorch and paper in [arXiv].

📸 Overview

The overall architecture of CGRSeg. The Rectangular Self-Calibration Module (RCM) is designed for spatial feature reconstruction and pyramid context extraction. The rectangular self-calibration attention (RCA) explicitly models the rectangular region and calibrates the attention shape. The Dynamic Prototype Guided (DPG) head is proposed to improve the classification of the foreground objects via explicit class embedding.

1️⃣ Results

ADE20K

COCO-Stuff-10k

Pascal Context

2️⃣ Requirements

• conda create --name ssa python=3.8 -y
  conda activate ssa
  pip install torch==1.8.2+cu102 torchvision==0.9.2+cu102 torchaudio==0.8.2
  pip install timm==0.6.13
  pip install mmcv-full==1.6.1
  pip install opencv-python==4.1.2.30
  pip install "mmsegmentation==0.27.0"

  CGRSeg is built based on mmsegmentation-0.27.0, which can be referenced for data preparation.

3️⃣ Training & Testing

• Train

  # Single-gpu training
  python train.py local_configs/cgrseg/cgrseg-t_ade20k_160k.py
  
  # Multi-gpu (4-gpu) training
  sh tools/dist_train.sh local_configs/cgrseg/cgrseg-t_ade20k_160k.py 4

• Test

  # Single-gpu testing
  python test.py local_configs/cgrseg/cgrseg-t_ade20k_160k.py ${CHECKPOINT_FILE} --eval mIoU
  
  # Multi-gpu (4-gpu) testing
  sh tools/dist_test.sh local_configs/cgrseg/cgrseg-t_ade20k_160k.py ${CHECKPOINT_FILE} 4 --eval mIoU

✏️ Reference

If you find CGRSeg useful in your research or applications, please consider giving a star ⭐ and citing using the following BibTeX:

@article{ni2024context,
  title={Context-Guided Spatial Feature Reconstruction for Efficient Semantic Segmentation},
  author={Ni, Zhenliang and Chen, Xinghao and Zhai, Yingjie and Tang, Yehui and Wang, Yunhe},
  journal={arXiv preprint arXiv:2405.06228},
  year={2024}
}