Updates | Introduction | Usage | Results & Models | Talk | Statement |
Scene Recognition: Please see Remote Sensing Pretraining for Scene Recognition;
Sementic Segmentation: Please see Remote Sensing Pretraining for Semantic Segmentation;
Object Detection: Please see Remote Sensing Pretraining for Object Detection;
Change Detection: Please see Remote Sensing Pretraining for Change Detection;
ViTAE: Please see ViTAE-Transformer;
Matting: Please see ViTAE-Transformer for matting;
018/10/2023
RSP won the highly cited paper!
027/05/2022
The early access is available!
020/05/2022
The paper has been accepted by IEEE TGRS.
011/04/2022
The baiduyun links of pretrained models are provided.
07/04/2022
The paper is post on arxiv!
06/04/2022
The pretrained models for ResNet-50, Swin-T and ViTAEv2-S are released. The code for pretraining and downstream tasks are also provided for reference.
This repository contains codes, models and test results for the paper "An Empirical Study of Remote Sensing Pretraining".
The aerial images are usually obtained by a camera in a birdview perspective lying on the planes or satellites, perceiving a large scope of land uses and land covers, whose scene is usually difficult to be interpreted since the interference of the scene-irrelevant regions and the complicated spatial distribution of land objects. Although deep learning has largely reshaped remote sensing research for aerial image understanding and made a great success. However, most of existing deep models are initialized with ImageNet pretrained weights, where the natural images inevitably presents a large domain gap relative to the aerial images, probably limiting the finetuning performance on downstream aerial scene tasks. This issue motivates us to conduct an empirical study of remote sensing pretraining. To this end, we train different networks from scratch with the help of the largest remote sensing scene recognition dataset up to now-MillionAID, to obtain the remote sensing pretrained backbones, including both convolutional neural networks (CNN) and vision transformers such as Swin and ViTAE, which have shown promising performance on computer vision tasks. Then, we investigate the impact of ImageNet pretraining (IMP) and RSP on a series of downstream tasks including scene recognition, semantic segmentation, object detection, and change detection using the CNN and vision transformers backbones.
NOTE: please extract model weight from these checkpoints for finetuning.
| Backbone | Input size | Acc@1 | Acc@5 | Param(M) | Pretrained model |
|---|---|---|---|---|---|
| RSP-ResNet-50-E300 | 224 × 224 | 98.99 | 99.82 | 23.6 | google & baidu |
| RSP-Swin-T-E300 | 224 × 224 | 98.59 | 99.88 | 27.6 | google & baidu |
| RSP-ViTAEv2-S-E100 | 224 × 224 | 98.97 | 99.88 | 18.8 | google & baidu |
Please refer to Readme.md for installation, dataset preparation, training and inference.
If this repo is useful for your research, please consider citation
@ARTICLE{rsp,
author={Wang, Di and Zhang, Jing and Du, Bo and Xia, Gui-Song and Tao, Dacheng},
journal={IEEE Transactions on Geoscience and Remote Sensing},
title={An Empirical Study of Remote Sensing Pretraining},
year={2023},
volume={61},
number={},
pages={1-20},
doi={10.1109/TGRS.2022.3176603}
}
A video talk about this study (In Chinese)
This project is under MIT licence. For any other questions please contact di.wang at gmail.com or di_wang at whu.edu.cn.
[1] Advancing Plain Vision Transformer Towards Remote Sensing Foundation Model, IEEE TGRS, 2022 | Paper | Github
Di Wang∗, Qiming Zhang∗, Yufei Xu∗, Jing Zhang, Bo Du, Dacheng Tao and Liangpei Zhang