Install the mmdetection library
pip install mmcv-full==1.3.0 mmdet==2.11.0
For mixed precision training , please install apex
git clone https://github.com/NVIDIA/apex
cd apex
pip install -v --disable-pip-version-check --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./
Please follow the dataset guide of mmdet to prepare the MS-COCO dataset.
| Backbone | patch size | bbox mAP | mask mAP | Config | Weights |
|---|---|---|---|---|---|
| XCiT-Tiny 12 | 16x16 | 42.7 | 38.5 | config | download |
| XCiT-Tiny 12 | 8x8 | 44.5 | 40.3 | config | download |
| XCiT-Small 12 | 16x16 | 45.3 | 40.8 | config | download |
| XCiT-Small 12 | 8x8 | 47.0 | 42.3 | config | download |
| XCiT-Small 24 | 16x16 | 46.5 | 41.8 | config | download |
| XCiT-Small 24 | 8x8 | 48.1 | 43.0 | config | download |
| XCiT-Medium 24 | 16x16 | 46.7 | 42.0 | config | download |
| XCiT-Medium 24 | 8x8 | 48.5 | 43.7 | config | download |
tools/dist_train.sh <CONFIG_PATH> <NUM_GPUS> --work-dir <SAVE_PATH> --seed 0 --deterministic --cfg-options model.pretrained=<IMAGENET_CHECKPOINT_PATH/URL>
For example, using an XCiT-S12/16 backbone
tools/dist_train.sh configs/xcit/mask_rcnn_xcit_small_12_p16_3x_coco.py 8 --work-dir /path/to/save --seed 0 --deterministic --cfg-options model.pretrained=https://dl.fbaipublicfiles.com/xcit/xcit_small_12_p16_384_dist.pth
tools/dist_test.sh <CONFIG_PATH> <CHECKPOINT_PATH> <NUM_GPUS> --eval bbox segm
For example, using an XCiT-S12/16 backbone
tools/dist_test.sh configs/xcit/mask_rcnn_xcit_small_12_p16_3x_coco.py https://dl.fbaipublicfiles.com/xcit/coco/maskrcnn_xcit_small_12_p16.pth 1 --eval bbox segm
This code is built using the mmdetection library. The optimization hyperparameters we use are adopted from Swin Transformer repository.