Instance Enhancement Batch Normalization: an Adaptive Regulator of Batch Noise

By Senwei Liang*, Zhongzhan Huang* (* contribute equally), Mingfu Liang and Haizhao Yang.

This repository is the implementation of "Instance Enhancement Batch Normalization: an Adaptive Regulator of Batch Noise" [paper] on CIFAR-100 dataset. Our paper has been accepted for presentation at the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20). You can also check with the AAAI proceeding version.

Introduction

Instance Enhancement Batch Normalization (IEBN) is an attention-based version of BN which recalibrates channel information of BN by a simple linear transformation.

Requirement

• Python 3.6 and PyTorch 1.0

Usage

python cifar.py -a iebn_resnet --dataset cifar100 --block-name bottleneck --depth 164 --epochs 164 --schedule 81 122 --gamma 0.1 --wd 1e-4 --checkpoint checkpoints/cifar100/resnet-164-iebn

Results

	original	IEBN
ResNet164	74.29	77.09

Notes:

• Training on 2 GPUs

Citing IEBN

@inproceedings{liang2020instance,
  title={Instance Enhancement Batch Normalization: An Adaptive Regulator of Batch Noise.},
  author={Liang, Senwei and Huang, Zhongzhan and Liang, Mingfu and Yang, Haizhao},
  booktitle={AAAI},
  pages={4819--4827},
  year={2020}
}

Acknowledgments

Many thanks to bearpaw for his simple and clean Pytorch framework for image classification task.