This is a repository about using keras to do emotion recognition on FERPlus dataset
The FERPlus (FER+) annotations provide a set of new labels for the standard Emotion FER dataset, which is thought to be more accuract than the original FER2013 annotations.
The new label file is named fer2013new.csv and contains the same number of rows as the original fer2013.csv label file with the same order, so that you infer which emotion tag belongs to which image.
The main purpose of this repository is to replicate the pretrained CNN models' performances in this blog, which has the model weights in Pytorch and MatConvNet, but not in tensorflow or keras.
- Tensorflow 1.14.0 or higher
- keras 2.2.4 or higher
- keras_contrib
- keras_vggface
- albumentations
The training dataset is FERPlus. Data augmentations, such as random cropping and horizontal flipping are applied. The class weights are set to [2.74398525, 3.56296941, 7.9199129, 8.44722581, 12.27334083, 215.34868421, 50.51388889, 230.51408451] to match their distribution. Dropout is applied on the last dense layer (dropout ratio=0.3). Regularization is not applied.
To train the ResNet50 model:
python train.py --init_lr 0.001 --lr_policy plateau --model_name ResNet50 --dataset_name FERPlus --name Experiment --freeze_before -51 --batch_size 64
Notes: --freeze_before -51 means to freeze the parameters before the -51 th layer (check the resnet50_layers.txt, which turns out to be the best choice.
To train the SeNet50 model:
python train.py --init_lr 0.001 --lr_policy plateau --model_name SeNet50 --dataset_name FERPlus --name Experiment --freeze_before -39 --batch_size 64
Notes: --freeze_before -39 means to freeze the parameters before the -39 th layer (check the senet50_layers.txt, which turns out to be the best choice.
| Model | Pretrained | Training | FERplus Val | FERplus Test |
|---|---|---|---|---|
| ResNet50 | VGGFace2 | FERPlus | 88.0 | 86.5 |
| SeNet50 | VGGFace2 | FERPlus | 87.3 | 85.8 |
- ResNet50
- SeNet50
- Improve the performance
To evaluate the model performance on the validation set and the test set, use
python test.py --checkpoint_path path-to-the-checkpoint --model_name Model-Name --dataset_name FERPlus --name experiment-directory --batch_size 64