HND & GHND for Object Detectors

Head Network Distillation (HND) and Generalized HND for Faster, Mask, and Keypoint R-CNNs

• "Neural Compression and Filtering for Edge-assisted Real-time Object Detection in Challenged Networks," ICPR 2020
  [PDF] [Supp] [Preprint]
• "Split Computing for Complex Object Detectors: Challenges and Preliminary Results," ACM MobiCom 2020 Workshop EMDL '20
  [PDF (Open Access)] [Preprint]

Looking for image classification tasks? Take a look at https://github.com/yoshitomo-matsubara/head-network-distillation as well

Citations

@inproceedings{matsubara2021neural,
  title={Neural Compression and Filtering for Edge-assisted Real-time Object Detection in Challenged Networks},
  author={Yoshitomo Matsubara and Marco Levorato},
  booktitle={2020 25th International Conference on Pattern Recognition (ICPR)},
  pages={2272--2279},
  year={2021}
}

@inproceedings{matsubara2020split,
  title={Split Computing for Complex Object Detectors: Challenges and Preliminary Results},
  author={Matsubara, Yoshitomo and Levorato, Marco},
  booktitle={Proceedings of the 4th International Workshop on Embedded and Mobile Deep Learning},
  pages={7--12},
  year={2020}
}

Requirements

• Python 3.6
• pipenv
• myutils

How to clone

git clone https://github.com/yoshitomo-matsubara/hnd-ghnd-object-detectors.git
cd hnd-ghnd-object-detectors/
git submodule init
git submodule update --recursive --remote
pipenv install

It is not necessary to use pipenv, and you can instead manually install the required packages listed in Pipfile, using pip3.

COCO 2017 Dataset

mkdir -p ./resource/dataset/coco2017
cd ./resource/dataset/coco2017
wget http://images.cocodataset.org/zips/train2017.zip
wget http://images.cocodataset.org/zips/val2017.zip
wget http://images.cocodataset.org/annotations/annotations_trainval2017.zip
unzip -q train2017.zip
unzip -q val2017.zip
unzip -q annotations_trainval2017.zip

Checkpoints with trained model weights

1. Download emdl2020.zip
2. Unzip emdl2020.zip at the root directory of this repository so that you can use the checkpoints with yaml config files under config/hnd/
3. Download icpr2020-faster_rcnn.zip, icpr2020-mask_rcnn.zip, and icpr2020-keypoint_rcnn.zip
4. Unzip the three zip files at the root directory of this repository so that you can use the checkpoints with yaml config files under config/hnd/ and config/ghnd/
5. Test the trained models using the checkpoints and yaml config files
   e.g., Faster R-CNN with 3 output channels for bottleneck

pipenv run python src/mimic_runner.py --config config/hnd/faster_rcnn-backbone_resnet50-b3ch.yaml
pipenv run python src/mimic_runner.py --config config/ghnd/faster_rcnn-backbone_resnet50-b3ch.yaml

Distilling head portion of R-CNNs

If you have already downloaded our trained model weights above, you should move the ckpt files in resource/ckpt/ to somewhere else or change ckpt file path (ckpt under student_model) in config files.

Bottleneck-injected Faster R-CNN with ResNet-50 and FPN

e.g., Bottleneck with 3 output channels

# HND
pipenv run python src/mimic_runner.py --config config/hnd/faster_rcnn-backbone_resnet50-b3ch.yaml -distill

# GHND
pipenv run python src/mimic_runner.py --config config/ghnd/faster_rcnn-backbone_resnet50-b3ch.yaml -distill

Bottleneck-injected Mask R-CNN with ResNet-50 and FPN

e.g., Bottleneck with 3 output channels

# HND
pipenv run python src/mimic_runner.py --config config/hnd/mask_rcnn-backbone_resnet50-b3ch.yaml -distill

# GHND
pipenv run python src/mimic_runner.py --config config/ghnd/mask_rcnn-backbone_resnet50-b3ch.yaml -distill

Bottleneck-injected Keypoint R-CNN with ResNet-50 and FPN

e.g., Bottleneck with 3 output channels

# HND
pipenv run python src/mimic_runner.py --config config/hnd/keypoint_rcnn-backbone_resnet50-b3ch.yaml -distill

# GHND
pipenv run python src/mimic_runner.py --config config/ghnd/keypoint_rcnn-backbone_resnet50-b3ch.yaml -distill

Training a neural filter on top of our trained, bottleneck-injected Keypoint R-CNN

pipenv run python src/ext_runner.py --config config/ext/keypoint_rcnn-backbone_ext_resnet50-b3ch.yaml -train

References

• pytorch/vision/references/detection/
• code for visualization in the object detection tutorial