Skip to content
This repository has been archived by the owner on Oct 21, 2023. It is now read-only.

Latest commit

 

History

History
93 lines (72 loc) · 4.66 KB

README.md

File metadata and controls

93 lines (72 loc) · 4.66 KB
Raw

MobileViTv3 : Mobile-Friendly Vision Transformer with Simple and Effective Fusion of Local, Global and Input Features [arXiv]

This repository contains MobileViTv3's source code for training and evaluation. It uses the CVNets library and is inspired by MobileViT (paper, code).

Installation and Training Models:

We recommend to use Python 3.8+ and PyTorch (version >= v1.8.0) with conda environment. For setting-up the python environment with conda, see here.

MobileViTv3-S,XS,XXS

Download MobileViTv1 and replace the files provided in MobileViTv3-v1. Conda environment used for training: environment_cvnet.yml. Then install according to instructions provided in the downloaded repository. For training, use training-and-evaluation readme provided in the downloaded repository.

MobileViTv3-1.0,0.75,0.5

Download MobileViTv2 and replace the files provided in MobileViTv3-v2. Conda environment used for training: environment_mbvt2.yml Then install according to instructions provided in the downloaded repository. For training, use training-and-evaluation readme provided in the downloaded repository.

Trained models:

Download the trained MobileViTv3 models from here. checkpoint_ema_best.pt files inside the model folder is used to generated the accuracy of models. Low-latency models are build by reducing the number of MobileViTv3-blocks in 'layer4' from 4 to 2. Please refer to the paper for more details. Note that for the segmentation and detection, only the backbone architecture parameters are listed.

Classification

ImageNet-1K:

Model name Accuracy (%) Parameters (Million) FLOPs (Million) Foldername
MobileViTv3-S 79.3 5.8 1841 mobilevitv3_S_e300_7930
MobileViTv3-XS 76.7 2.5 927 mobilevitv3_XS_e300_7671
MobileViTv3-XXS 70.98 1.2 289 mobilevitv3_XXS_e300_7098
MobileViTv3-1.0 78.64 5.1 1876 mobilevitv3_1_0_0
MobileViTv3-0.75 76.55 3.0 1064 mobilevitv3_0_7_5
MobileViTv3-0.5 72.33 1.4 481 mobilevitv3_0_5_0

ImageNet-1K using low-latency models:

Model name Accuracy (%) Parameters (Million) FLOPs (Million) Foldername
MobileViTv3-S-L2 79.06 5.2 1651 mobilevitv3_S_L2_e300_7906
MobileViTv3-XS-L2 76.10 2.3 853 mobilevitv3_XS_L2_e300_7610
MobileViTv3-XXS-L2 70.23 1.1 256 mobilevitv3_XXS_L2_e300_7023

Segmentation

PASCAL VOC 2012:

Model name mIoU Parameters (Million) Foldername
MobileViTv3-S 79.59 7.2 mobilevitv3_S_voc_e50_7959
MobileViTv3-XS 78.77 3.3 mobilevitv3_XS_voc_e50_7877
MobileViTv3-XXS 74.04 2.0 mobilevitv3_XXS_voc_e50_7404
MobileViTv3-1.0 80.04 13.6 mobilevitv3_voc_1_0_0
MobileViTv3-0.5 76.48 6.3 mobilevitv3_voc_0_5_0

ADE20K:

Model name mIoU Parameters (Million) Foldername
MobileViTv3-1.0 39.13 13.6 mobilevitv3_ade20k_1_0_0
MobileViTv3-0.75 36.43 9.7 mobilevitv3_ade20k_0_7_5
MobileViTv3-0.5 33.57 6.4 mobilevitv3_ade20k_0_5_0

Detection MS-COCO:

Model name mAP Parameters (Million) Foldername
MobileViTv3-S 27.3 5.5 mobilevitv3_S_coco_e200_2730
MobileViTv3-XS 25.6 2.7 mobilevitv3_XS_coco_e200_2560
MobileViTv3-XXS 19.3 1.5 mobilevitv3_XXS_coco_e200_1930
MobileViTv3-1.0 27.0 5.8 mobilevitv3_coco_1_0_0
MobileViTv3-0.75 25.0 3.7 mobilevitv3_coco_0_7_5
MobileViTv3-0.5 21.8 2.0 mobilevitv3_coco_0_5_0

Citation

If you find this repository useful, please consider giving a star ⭐ and citation 📣:

@inproceedings{wadekar2022mobilevitv3,
  title = {MobileViTv3: Mobile-Friendly Vision Transformer with Simple and Effective Fusion of Local, Global and Input Features},
  author = {Wadekar, Shakti N. and Chaurasia, Abhishek},
  doi = {10.48550/ARXIV.2209.15159},
  year = {2022}
}