ObjectSeeker: Certifiably Robust Object Detection against Patch Hiding Attacks via Patch-agnostic Masking

By Chong Xiang, Alexander Valtchanov, Saeed Mahloujifar, Prateek Mittal

Code for "ObjectSeeker: Certifiably Robust Object Detection against Patch Hiding Attacks via Patch-agnostic Masking".

Check out this paper list for adversarial patch research for fun!

Files

├── README.md                        #this file 
├── requirement.txt                  #required packages
├── example_cmd.sh                   #example commands to run the code
| 
├── main_yolor.py                    #run ObjectSeeker with yolor 
├── main_mmdet.py                    #run ObjectSeeker with swin (mmdet)
├── clean_eval.py                    #scripts for clean AP evaluation; can be imported into main_yolor.py or main_mmdet.py
| 
├── objseeker                        #utils of objectseeker
|   ├── defense.py                   #implement the objectseeker defense
|   ├── coco_eval.py                 #utils for coco clean evaluation
|   └── voc_eval.py                  #utils for voc clean evaluation
|
├── yolor                            #scripts from the original yolor implementation
|   ├── data                        
|   ├── ....
|
├── mmdet                            #configs for mmdet/swin
|   ├── data                        
|   └── configs
|
└── checkpoints                      #directory for checkpoints
    ├── README.md                    #details of checkpoints
    └── ...                          #model checkpoints

Datasets

YOLOR is based on YOLOv5, and thus needs pre-processed data from ultralytics. mmdet (swin) uses official datasets directly.

• VOC
  • (mmdet.swin) The official dataset can be downloaded using torchvision.datasets.VOCDetection. we use test2007 for our evaluation.
    • specify the data directory in mmdet/configs/_base_/datasets/voc0712.py.
  • (yolor) see this script to download and generate the ultralytics-YOLO style VOC dataset.
    • specify the data directory in yolor/data/voc.yaml.
• COCO
  • (mmdet.swin) Download the official dataset (both data and annotations). we use val2007 for evaluation.
    • specify the data directory in mmdet/configs/_base_/datasets/coco_detection.py.
  • (yolor) see this script for downloading ultralytics-YOLO style COCO dataset.
    • specify the data directory in yolor/data/coco.yaml.

Dependency

• PyTorch
  • follow the official website to install PyTorch.
• mmdet
  • follow this tutorial to install mmdet (and mmcv).
• Others
  • see requirement.txt; install via pip install -r requirement.txt.

Usage

• See Files for details of each file.
• Setup the datasets and dependency as discussed above.
• Download checkpoints from this Google Drive link and move them to checkpoints/.
• See example_cmds.sh for example commands for running the code.

If anything is unclear, please open an issue or contact Chong Xiang (cxiang@princeton.edu).

Citations

If you find our work useful in your research, please consider citing:

@article{xiang2022objectseeker,
  title={ObjectSeeker: Certifiably Robust Object Detection against Patch Hiding Attacks via Patch-agnostic Masking},
  author={Xiang, Chong and Valtchanov, Alexander and Mahloujifar, Saeed and Mittal, Prateek},
  journal={arXiv preprint arXiv:2202.01811},
  year={2022}
}