Face comparison (Not recognition or detection) using OpenCV and Keras?
Question:
First of all here is my github link for the question.
And here is my question:
I would like to do a face comparison function using Python. And I can successfully(?) recognize faces using OpenCV. Now, how do I do the comparison thing?
What I understand is this:
In general Machine learning approach, I need to gather lots of data about that particular person and finalize it using a CNN.
However, I just got 2 images, how do I do the comparison? Should I think it in terms of classification or clustering (Using KNN)?
Thank you very much in advance for all your help.
Answers:
You need to learn similarity metric for faces. It will allow to extract features good to distinguish different persons. Then you’ll be able to find dissimilarity (distance) between them. You can read in more detail here for instance. kNN and such things are useful to find groups of similar faces, but it need to use features, extracted before.
You can use the idea of face-embeddings, which for example is proposed in the highly-cited paper FaceNet and implemented in OpenFace (which also comes pre-trained).
The general idea: take some preprocessed face (frontal, cropped, …) and embedd it to some lower dimension with the characteristic, that similar faces in input should have low euclidean-distance in the output.
So in your case: use the embedding-CNN to map your faces to the reduced space (usually a vector of size 128) and calculate the distance as in the euclidean-space. Of course you also cluster faces then, but that’s not your task.
The good thing here besides the general idea: openface is a nice implementation ready to use and it’s homepage also explains the idea:
Use a deep neural network to represent (or embed) the face on a 128-dimensional unit hypersphere.
The embedding is a generic representation for anybody’s face. Unlike other face representations, this embedding has the nice property that a larger distance between two face embeddings means that the faces are likely not of the same person.
This property makes clustering, similarity detection, and classification tasks easier than other face recognition techniques where the Euclidean distance between features is not meaningful.
They even have a comparison-demo here.
Use face_recognition library (compare faces feature) . It will compare encoding of face features and give you boolean in return.
I used DeepFace.
It has a face comparator out of the box:
from deepface import DeepFace
f1 = "/Users/face/path/face1.jpg"
f2 = "/Users/face/path/face2.jpg"
backends = ['opencv', 'ssd', 'dlib', 'mtcnn', 'retinaface', 'mediapipe']
result = DeepFace.verify(img1_path=f1, img2_path=f2, detector_backend=backends[1])
The output would be:
{'verified': True,
'distance': 0.2304540972887159,
'threshold': 0.4,
'model': 'VGG-Face',
'detector_backend': 'ssd',
'similarity_metric': 'cosine'}
And False if there were two different persons in the two photos.
Note ssd and retinaface are more accurate but slower, opencv is super fast but misses.
First of all here is my github link for the question.
And here is my question:
I would like to do a face comparison function using Python. And I can successfully(?) recognize faces using OpenCV. Now, how do I do the comparison thing?
What I understand is this:
In general Machine learning approach, I need to gather lots of data about that particular person and finalize it using a CNN.
However, I just got 2 images, how do I do the comparison? Should I think it in terms of classification or clustering (Using KNN)?
Thank you very much in advance for all your help.
You need to learn similarity metric for faces. It will allow to extract features good to distinguish different persons. Then you’ll be able to find dissimilarity (distance) between them. You can read in more detail here for instance. kNN and such things are useful to find groups of similar faces, but it need to use features, extracted before.
You can use the idea of face-embeddings, which for example is proposed in the highly-cited paper FaceNet and implemented in OpenFace (which also comes pre-trained).
The general idea: take some preprocessed face (frontal, cropped, …) and embedd it to some lower dimension with the characteristic, that similar faces in input should have low euclidean-distance in the output.
So in your case: use the embedding-CNN to map your faces to the reduced space (usually a vector of size 128) and calculate the distance as in the euclidean-space. Of course you also cluster faces then, but that’s not your task.
The good thing here besides the general idea: openface is a nice implementation ready to use and it’s homepage also explains the idea:
Use a deep neural network to represent (or embed) the face on a 128-dimensional unit hypersphere.
The embedding is a generic representation for anybody’s face. Unlike other face representations, this embedding has the nice property that a larger distance between two face embeddings means that the faces are likely not of the same person.
This property makes clustering, similarity detection, and classification tasks easier than other face recognition techniques where the Euclidean distance between features is not meaningful.
They even have a comparison-demo here.
Use face_recognition library (compare faces feature) . It will compare encoding of face features and give you boolean in return.
I used DeepFace.
It has a face comparator out of the box:
from deepface import DeepFace
f1 = "/Users/face/path/face1.jpg"
f2 = "/Users/face/path/face2.jpg"
backends = ['opencv', 'ssd', 'dlib', 'mtcnn', 'retinaface', 'mediapipe']
result = DeepFace.verify(img1_path=f1, img2_path=f2, detector_backend=backends[1])
The output would be:
{'verified': True,
'distance': 0.2304540972887159,
'threshold': 0.4,
'model': 'VGG-Face',
'detector_backend': 'ssd',
'similarity_metric': 'cosine'}
And False if there were two different persons in the two photos.
Note ssd and retinaface are more accurate but slower, opencv is super fast but misses.