简体中文 | English
- 1. Introduction
- 2. Model Zoo
- 3. Installation
- 4. Data Pipline
- 5. Configuration File
- 6. Training and Inference
- 7. Citations
Arcface-Paddle is an open source deep face detection and recognition toolkit, powered by PaddlePaddle. Arcface-Paddle provides three related pretrained models now, include BlazeFace for face detection, ArcFace and MobileFace for face recognition.
- This tutorial is mainly about face detection based on
PaddleDetection. - For face recognition task, please refer to: Face recognition tuturial.
- For Whl package inference using PaddleInference, please refer to whl package inference.
| Model | input size | images/GPU | epochs | Easy/Medium/Hard Set | CPU time cost | GPU time cost | Model Size(MB) | Pretrained model | Inference model | Config |
|---|---|---|---|---|---|---|---|---|---|---|
| BlazeFace-FPN-SSH | 640×640 | 8 | 1000 | 0.9187 / 0.8979 / 0.8168 | 31.7ms | 5.6ms | 0.646 | download link | download link | config |
| RetinaFace | 480x640 | - | - | - / - / 0.8250 | 182.0ms | 17.4ms | 1.680 | - | - | - |
NOTE:
- Get mAP in
Easy/Medium/Hard Setby multi-scale evaluation. For details can refer to Evaluation. - Measuring the speed, we use the resolution of
640×640, in Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz environment, cpu-threads are set as 5. For more details, you can refer to Improvement of inference speed. - Benchmark code for
RetinaFaceis from: ../retinaface/README.md. - The benchmark environment is
- CPU: Intel(R) Xeon(R) Gold 6184 CPU @ 2.40GHz
- GPU: a single NVIDIA Tesla V100
Please refer to installation tutorial to install PaddlePaddle and PaddleDetection.
We use the WIDER FACE dataset to carry out the training and testing of the model, the official website gives detailed data introduction.
-
WIDER Face data source:
Loadswider_facetype dataset with directory structures like this:dataset/wider_face/ ├── wider_face_split │ ├── wider_face_train_bbx_gt.txt │ ├── wider_face_val_bbx_gt.txt ├── WIDER_train │ ├── images │ │ ├── 0--Parade │ │ │ ├── 0_Parade_marchingband_1_100.jpg │ │ │ ├── 0_Parade_marchingband_1_381.jpg │ │ │ │ ... │ │ ├── 10--People_Marching │ │ │ ... ├── WIDER_val │ ├── images │ │ ├── 0--Parade │ │ │ ├── 0_Parade_marchingband_1_1004.jpg │ │ │ ├── 0_Parade_marchingband_1_1045.jpg │ │ │ │ ... │ │ ├── 10--People_Marching │ │ │ ... -
Download dataset manually:
To download the WIDER FACE dataset, run the following commands:
cd dataset/wider_face && ./download_wider_face.sh
We use the configs/face_detection/blazeface_fpn_ssh_1000e.yml configuration for training. The summary of the configuration file is as follows:
_BASE_: [
'../datasets/wider_face.yml',
'../runtime.yml',
'_base_/optimizer_1000e.yml',
'_base_/blazeface_fpn.yml',
'_base_/face_reader.yml',
]
weights: output/blazeface_fpn_ssh_1000e/model_final
multi_scale_eval: Trueblazeface_fpn_ssh_1000e.yml The configuration needs to rely on other configuration files, in this example it needs to rely on:
wider_face.yml：Mainly explains the path of training data and verification data
runtime.yml：Mainly describes the common operating parameters, such as whether to use GPU, how many epochs to store checkpoints, etc.
optimizer_1000e.yml：Mainly explains the configuration of learning rate and optimizer
blazeface_fpn.yml：Mainly explain the situation of the model and the backbone network
face_reader.yml：It mainly describes the configuration of the data reader, such as batch size, the number of concurrent loading subprocesses, etc., and also includes post-reading preprocessing operations, such as resize, data enhancement, etc.
According to the actual situation, modify the above files, such as the data set path, batch size, etc.
For the configuration of the base model, please refer to configs/face_detection/_base_/blazeface.yml.
The improved model adds the neck structure of FPN and SSH. For the configuration file, please refer to configs/face_detection/_base_/blazeface_fpn.yml. You can configure FPN and SSH if needed, which is as follows:
BlazeNet:
blaze_filters: [[24, 24], [24, 24], [24, 48, 2], [48, 48], [48, 48]]
double_blaze_filters: [[48, 24, 96, 2], [96, 24, 96], [96, 24, 96],
[96, 24, 96, 2], [96, 24, 96], [96, 24, 96]]
act: hard_swish # Configure the activation function of BlazeBlock in backbone, the basic model is relu, hard_swish is required when adding FPN and SSH
BlazeNeck:
neck_type : fpn_ssh # Optional only_fpn, only_ssh and fpn_ssh
in_channel: [96,96]Firstly, download the pretrained model.
wget https://paddledet.bj.bcebos.com/models/pretrained/blazenet_pretrain.pdparamsPaddleDetection provides a single-GPU/multi-GPU training mode to meet the various training needs of users.
- single-GPU training
export CUDA_VISIBLE_DEVICES=0 # Do not need to execute this command under windows and Mac
python tools/train.py -c configs/face_detection/blazeface_fpn_ssh_1000e.yml -o pretrain_weight=blazenet_pretrain- multi-GPU training
export CUDA_VISIBLE_DEVICES=0,1,2,3 # Do not need to execute this command under windows and Mac
python -m paddle.distributed.launch --gpus 0,1,2,3 tools/train.py -c configs/face_detection/blazeface_fpn_ssh_1000e.yml -o pretrain_weight=blazenet_pretrain-
Resume training from Checkpoint
In the daily training process, if the training was be interrupted, using the -r command to resume training:
export CUDA_VISIBLE_DEVICES=0 # Do not need to execute this command under windows and Mac
python tools/train.py -c configs/face_detection/blazeface_fpn_ssh_1000e.yml -r output/blazeface_fan_ssh_1000e/100- Training hyperparameters
BlazeFace training is based on each GPU batch_size=32 training on 4 GPUs (total batch_size is 128), the learning rate is 0.002, and the total training epoch is set as 1000.
NOTE: Not support evaluation during train.
- Evaluate and generate results files:
python -u tools/eval.py -c configs/face_detection/blazeface_fpn_ssh_1000e.yml \
-o weights=output/blazeface_fpn_ssh_1000e/model_final \
multi_scale_eval=True BBoxPostProcess.nms.score_threshold=0.1Set multi_scale_eval=True for multi-scale evaluation，after the evaluation is completed, the test result in txt format will be generated in output/pred.
- Download the official evaluation script to evaluate the AP metrics:
wget http://mmlab.ie.cuhk.edu.hk/projects/WIDERFace/support/eval_script/eval_tools.zip
unzip eval_tools.zip && rm -f eval_tools.zip- Start evaluation:
Method One: Python evaluation:
git clone https://github.com/wondervictor/WiderFace-Evaluation.git
cd WiderFace-Evaluation
# Compile
python3 setup.py build_ext --inplace
# Start evaluation
python3 evaluation.py -p /path/to/PaddleDetection/output/pred -g /path/to/eval_tools/ground_truthMethod Two: MatLab evaluation:
# Modify the result path and the name of the curve to be drawn in `eval_tools/wider_eval.m`:
pred_dir = './pred';
legend_name = 'Paddle-BlazeFace';
`wider_eval.m` is the main execution program of the evaluation module. The run command is as follows:
matlab -nodesktop -nosplash -nojvm -r "run wider_eval.m;quit;"The model file saved in the model training process includes forward prediction and back propagation. In actual industrial deployment, back propagation is not required. Therefore, the model needs to be exported into the model format required for deployment.
The tools/export_model.py script is provided in PaddleDetection to export the model:
python tools/export_model.py -c configs/face_detection/blazeface_fpn_ssh_1000e.yml --output_dir=./inference_model \
-o weights=output/blazeface_fpn_ssh_1000e/best_model BBoxPostProcess.nms.score_threshold=0.1The inference model will be exported to the inference_model/blazeface_fpn_ssh_1000e directory, which are infer_cfg.yml, model.pdiparams, model.pdiparams.info, model.pdmodel If no folder is specified, the model will be exported In output_inference.
score_thresholdfor nms is modified as 0.1 for inference, because it takes great speed performance improvement while has little effect on mAP. For more documentation about model export, please refer to: export doc
PaddleDetection provides multiple deployment forms of PaddleInference, PaddleServing, and PaddleLite, supports multiple platforms such as server, mobile, and embedded, and provides a complete deployment plan for Python and C++.
- Here, we take Python as an example to illustrate how to use PaddleInference for model deployment:
python deploy/python/infer.py --model_dir=./inference_model/blazeface_fpn_ssh_1000e --image_file=demo/road554.png --use_gpu=True-
infer.pyprovides a rich interface for users to access video files and cameras for prediction. For more information, please refer to: Python deployment. -
For more documentation on deployment, please refer to: deploy doc.
If you want to reproduce our speed indicators, you need to modify the input size of inference model in the ./inference_model/blazeface_fpn_ssh_1000e/infer_cfg.yml configuration file. As follows:
mode: fluid
draw_threshold: 0.5
metric: WiderFace
arch: Face
min_subgraph_size: 3
Preprocess:
- is_scale: false
mean:
- 123
- 117
- 104
std:
- 127.502231
- 127.502231
- 127.502231
type: NormalizeImage
- interp: 1
keep_ratio: false
target_size:
- 640
- 640
type: Resize
- type: Permute
label_list:
- faceIf you want the model to be inferred faster in the CPU environment, install paddlepaddle_gpu-0.0.0 (dependency of mkldnn) and enable_mkldnn is set to True, when predicting acceleration.
# use GPU:
python deploy/python/infer.py --model_dir=./inference_model/blazeface_fpn_ssh_1000e --image_dir=./path/images --run_benchmark=True --use_gpu=True
# inference with mkldnn use CPU
# downdoad whl package
wget https://paddle-wheel.bj.bcebos.com/develop-cpu-mkl/paddlepaddle-0.0.0-cp37-cp37m-linux_x86_64.whl
#install paddlepaddle_gpu-0.0.0
pip install paddlepaddle-0.0.0-cp37-cp37m-linux_x86_64.whl
python deploy/python/infer.py --model_dir=./inference_model/blazeface_fpn_ssh_1000e --image_dir=./path/images --enable_mkldnn=True --run_benchmark=True --cpu_threads=5This part talks about how to detect faces using BlazeFace model.
Firstly, use the following commands to download the demo image and font file for visualization.
# Demo image
wget https://raw.githubusercontent.com/littletomatodonkey/insight-face-paddle/main/demo/friends/query/friends1.jpg
# Font file for visualization
wget https://raw.githubusercontent.com/littletomatodonkey/insight-face-paddle/main/SourceHanSansCN-Medium.otfThe demo image is shown as follows.
Use the following command to run the face detection process.
python3.7 test_blazeface.py --input=friends1.jpg --output="./output"The final result is save in folder output/, which is shown as follows.
For more details about parameter explanations, face recognition, index gallery construction and whl package inference, please refer to Whl package inference tutorial.
@misc{long2020ppyolo,
title={PP-YOLO: An Effective and Efficient Implementation of Object Detector},
author={Xiang Long and Kaipeng Deng and Guanzhong Wang and Yang Zhang and Qingqing Dang and Yuan Gao and Hui Shen and Jianguo Ren and Shumin Han and Errui Ding and Shilei Wen},
year={2020},
eprint={2007.12099},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
@misc{ppdet2019,
title={PaddleDetection, Object detection and instance segmentation toolkit based on PaddlePaddle.},
author={PaddlePaddle Authors},
howpublished = {\url{https://github.com/PaddlePaddle/PaddleDetection}},
year={2019}
}
@article{bazarevsky2019blazeface,
title={BlazeFace: Sub-millisecond Neural Face Detection on Mobile GPUs},
author={Valentin Bazarevsky and Yury Kartynnik and Andrey Vakunov and Karthik Raveendran and Matthias Grundmann},
year={2019},
eprint={1907.05047},
archivePrefix={arXiv}
}