note The LFW benchmark is a face verification benchmark.

Face Recognition Workflow

note1 Please refer to "dlib/examples/dnn_face_recognition_ex.cpp".

note2 Please refer to "dlib/examples/dnn_metric_learning_ex.cpp", and "dlib/examples/dnn_metric_learning_on_images_ex.cpp" for details about model training.

note3 Face recognition accuracy can be improved if jittering is used when creating face descriptors. In particular, to get 99.38% on LFW you need to use the "jitter_image()" routine to compute the descriptors. More accurate, but slower.

Face Detection

We need a face detector. The face detector we use is made using the classic Histogram of Oriented Gradients feature combined with a linear classifier, an image pyramid and sliding window detection scheme.

The introduction might be in "dlib/image_processing/frontal_face_detector.h"
dlib::get_frontal_face_detector()

Face Landmarks

We also need a face landmarking model to align faces to a standard pose.

see "dlib/examples/face_landmark_detection_ex.cpp" for introduction

Embedding with DNN

Refer to lines 115 - 125 in "dlib/examples/dnn_face_recognition_ex.cpp".

Pre-processing

For each detected face, a copy is extracted that has been normalized to 150*150 (or something else) pixels in size and appropriately rotated and centered.

note We need to look into "extract_image_chip" adn "get_face_chip_details" routine.

WTF are face_chip, extract_image_chip...

Jittering

Face recognition accuracy can be improved if jittering is used when creating face embeddings.

For example, jitter the original image 100 times, and generate its embedding as the average of those 100 images.

What jittering does is to zoom, rotate and translate a little bit differently.