Face recognizer

Test task for Summer Internship 2020 in VTF Solutions.

Face recognition Flask application that performs face detection, extraction of face embeddings from each face using deep learning, training a face recognition model on the embeddings, and then finally recognizes faces on video streams from web camera with OpenCV

About

Dataset

The dataset that is used to train system contains three people:

• Arnold Schwarzenegger
• George W. Bush
• Unknown, which is used to represent faces of people system do not know and wish to label as such

Each class contains a total of 40 images.

Project structure

Project has four directories in the root folder:

• dataset contains face images organized into subfolders by name
• facerecognizer contains files for training, detecting and recognizing faces
• pretrained_models contains a pre-trained Caffe deep learning model provided by OpenCV to detect faces and a Torch deep learning model which produces the 128-D facial embeddings
• templates contains html templates for Flask

Pipeline

Face recognition pipeline:

• Pre-trained OpenCV Caffe-based face detector to apply face detection, which detects the presence and location of a face in an image, but does not identify it. It is quite simple to use and efficient

• Pre-trained DL PyTorch-based model from OpenFace project to extract the 128-d feature vectors (embeddings) that quantify each face in an image. It is implementation of FaceNet: A Unified Embedding for Face Recognition and Clustering Main benefit is representational efficiency: can achieve state-of-the-art performance
  (record 99.63% accuracy on LFW, 95.12% on Youtube Faces DB) using only 128-bytes per face.

• Scikit-learn SVM model to actually recognize a person trained on embeddings. It has good performance and simple to train

Example of work

Database structure

In order to store information about persons and embeddings of faces a simple and easy Sqlite was chosen and SqlAlchemy library for simplification of work with database

Database scheme:

• Face

  • embedding - face embeddings
  • person_id

• Person

  • name

• Presence - model to track presence of a person in front of a camera

  • person_id
  • timestamp_first
  • timestamp_last

How to run

Prerequisites

• Python 3.6 and higher
• Git

Clone project and setup

• Clone repository

   git clone https://github.com/Alexeyzhu/face-detection.git

• Install requirements.txt

   pip install -r requirements.txt

You can face problems with installing dlib on Windows. In this case, try to install Cmake firstly

Run program

You can run Flask application with default IP and port

   python webstreaming.py

or specify IP and port that differ from default

   python webstreaming.py --ip "desired ip" --port port_number

Open user interface

Open in browser http://127.0.0.1:5000/ to access real-time face recognition

(Optional) Add your data

You can add more classes to the recognizer. Create a folder with the name of a class and add photos to it. Then open in browser http://127.0.0.1:5000/retrain to retrain the system