Sightcorp logo

How Facial Recognition Works

Everything about How Facial Recognition Works

     

How Facial Recognition works

Facial recognition uses artificial intelligence (AI) to compare facial images and determine whether there is a match. It can determine whether two facial images are of the same person, and it can also determine whether a facial image matches any of the facial images contained in a particular database. These comparisons are referred to as 1:1 and 1:N matching respectively.

A brief background
While face recognition may seem like a new technology, it’s actually been in development since the 1960s. It is only in recent years, however, that it started to emerge as a viable technology that we can use at scale. The speed of the recent progress is due to various factors, including more powerful hardware (such as GPUs) and the advancements that have been made in machine learning and deep learning. Keep reading below to find out more!

There are three general applications for facial recognition software. 

 

  • Basic face identification model

 

Usually used in applications such as Instagram. The software uses a cellphone camera in order to identify facial features, specifically, eyes, a nose, and a mouth position. After the features have been defined, the application utilizes algorithms to focus on the face and determine whether the mouth or eyes is open or which direction the person is facing. It is important to note that these actions aren’t face identification, as they are plain programmatic actions that are used by the software that is looking for the faces through the camera.

 

  • Basic Face Identification matching model

 

The software of this programmatic principle measures the distance between your facial features characteristics by taking a picture of your face. The measured features are the same as in the previous method, the eyes, nose, mouth, and other possible options depending on the preferences of the software creator. This principle is used every time when you are required to prove your identity. The potential example can be Apple Face ID. 

 

  • Database Match Model

 

This method is used when a company or individual wishes to identify faces for security reasons, for promotional purposes, or any other reason where face recognition technology can be used. Database matching differs from the previous methods but is also a bit similar to face identification models that are used within cell phones. The process of database matching can be described as comparing a face towards a database that has been predetermined by the operator. The person approaches the location that is covered by the camera, the camera records the facial features and then matches those features towards a database. In case there is a match, a predetermined action takes place, such as for example, a door opens. In theory, any database can be used as well as any action can take place based on the requirements and wishes of the operator, the core principle remains the same.

The principle of 2D face recognition

Most face recognition software solutions are functioning by using only 2d images. This is done due to the limiting factor of most cameras, as most of them take pictures without any depth and most of the pictures that are used for face verification are available in 2D resolution. 2D face recognition may be convenient, but it also limited in many different ways. The first and most obvious is a lack of depth. The program can measure the distance between your facial features but still, it is unable to measure the length of your nose or any other given feature. This results in situations where extra verification characteristics aren’t being used and therefore 2D verification can be described as less effective when compared to 3D counterpart. Moreover, 2D facial recognition relies on a visible light spectrum, meaning that it is unable to function in complete darkness and may be prone to reliability issues in areas with poor light conditions. Most of the 2D limiting factors are fixed when the software uses 3D imagery instead of 2D.

You may wonder how can we battle the lightning problem and start identifying faces in the dark? 

There is way to solve this issue, by using thermal imaging solutions. It works a bit similar to the 3D method, but instead of sending IR light towards the object and measuring the response, alternately this technology detects the IR light that is emitting from the objects. The warmer the object, the more IR light it emits, unlike the cold one. Depending on the camera characteristics and price, the thermal cameras are able to detect small temperature differences across any given area that is covered by a camera, meaning that this kind of solution can also be used for face recognition.

It also has to be mentioned that thermal light detection also has significant disadvantages such as its cost and low efficiency during the daytime, as a result, it is rarely used outside of the military. There are many different techniques that can be used in order to detect and identify a face using thermal cameras. Some of these methods are complicated and technology-dependent, whereas some are not, but all of them share the same basic principles that can be described as follows.

  1. A Dataset is created for identification purposes. The software is using multiple IR images in order to create a dataset. The created dataset is used when it is compared to a given database in order to identify a subject that has been detected by the camera.
  2. Dependency on multiple pictures. The thermal camera needs multiple photos that are taken in different light perspectives in order to recognize a face. It has to measure different spectrums of IR light, such as long, short, and medium. In general, the longwave spectrum is used when the software needs the maximum number of facial details. 

What hides behind 3D Face Recognition

While most solutions utilize the 2D method, some use 3D. The 3D face identification is a bit different from 2D as instead if measuring the distance between facial features it utilizes a technique that is called lidar. Lidar is a method that is very similar to sonar. It functions by using a matrix, that is pushed over your face and then the reflection is recorded by the IR camera. 

The measurement takes place when the IR camera measures how long does it take for each particle of IR light to reach your face and the return to the input source. The principle is the following: the light that has been reflected from your ears normally has a longer journey to the source (camera) when compared to the light that has been reflected from your nose, which on the other hand has a shorter journey of IR particles towards the camera. The camera uses the gained data in order to create a depth map of your face. This method drastically increases the accuracy of face recognition as it simply utilizes more facial characteristics when compared to a 2D counterpart. The typical example of 3D face identification can be seen in various cell phone programs such as Face ID, that is used in order to authenticate a phone user.

Steps in the facial recognition process

While there are different AI algorithms [link to Face Matching Algorithms article] that can be used for facial recognition, they all follow more or less the same steps:

  1. Detect a face or faces in an image or video.
  2. Extract the facial features to create a faceprint or face embedding.
  3. Compare faceprints or face embeddings to determine whether there is a match. How similar the embeddings need to be for a match to be established will depend on the chosen confidence threshold. Comparisons can either be 1:1 (one-to-one) or 1:N (one-to-many).
  4. Return the result of the comparison or search.

 

Why is facial recognition important?

Facial recognition is transforming many aspects of daily life. Already, many of us are used to unlocking our smartphones or computers using Face ID. This has paved the way for facial recognition to be adopted for additional purposes, such as authenticating payments and enhancing access control.

Along with other forms of biometric identification and verification, facial recognition is transforming the landscape when it comes to applications ranging from access control and payment verification to retail and security. For this reason, it is important to understand how it works and how it could transform your own industry.

Click here to learn more about the different ways in which facial recognition technology can be used.

What are some of the common problems with face recognition, and how can we overcome them?

Even though there are multiple use cases of Facial Recognition, every technology has its disadvantages and limitations as in example with thermal cameras. Some problems can be resolved, whereas others stay fixed. Face recognition isn’t perfect. While there has been huge progress, there is still room for error. Take a look at this example:

How facial recognition works

On a more serious note, here are some of the challenges with facial recognition:

  • Variations in images compared: When there are variations in the position of the head, lighting conditions, and facial expressions in the two images that you want to compare, it can result in lower accuracy. The same is true when it comes to make-up, glasses, hats, and anything that partially obscures the face. Luckily, there are a few ways to deal with these challenges. The first is to use a large and diverse training set. Second, deep learning techniques are making it easier to correct for these variations. 
  • Less than 100% accuracy: No face recognition algorithm is 100% accurate. Even with the most sophisticated software, you still have a chance that an authorized person will be denied access in an access control scenario or vice versa. In these types of situations, human intervention will still be required. However, with accuracy rates improving and with the technology achieving 98% accuracy and higher, it is clear that the technology is useful in all but the highest-risk scenarios.
  • Privacy: One of the biggest concerns surrounding the use of facial recognition technology is privacy. This is a valid concern, and therefore there are various measures that you can take to reduce the privacy risks when implementing facial recognition technology. These include: storing only faceprints/face embeddings, and not the facial images themselves; not storing facial image data at all (if your use case allows for this); storing and processing data locally, so that you have full control over the data and so that third parties do not have access; and purging data regularly.
  • Spoofing: In many instances, there are concerns that facial recognition software can be fooled by masks or copies of photos presented to the camera. To prevent this, many facial recognition programs incorporate liveness detection features, which ensure that the submitted video is of a live person, and that it is captured in real time.
  • Light. Lightning conditions may drastically affect the performance and accuracy of face recognition technology. All of the facial recognition solutions are affected by this problem, as all of them work based on light or rely on some other way on lighting sources. 
  • Database necessity. The efficiency of face recognition very tightly depends on the database that is used by the system. It is close to impossible to identify and verify a face that hasn’t been correctly identified in the past.
  •  Data analysis limitation. Depending on the load, some solutions will require more time to process all given data.

 

Learn More About FaceMatch

How Sightcorp’s facial recognition technology works

Sightcorp’s facial recognition technology is 100% deep learning-based [link to Face Recognition Using Deep Learning article]. The technology uses Convolutional Neural Networks (CNNs) to detect faces, extract facial features, generate a faceprint, and compare the generated faceprint to one or more other faceprints to determine whether or not there is a match.

FaceMatch performs with state-of-the-art accuracy. It is also customizable, allowing you to determine your own confidence thresholds and to retrain the models on your own or your customers’ data.

With FaceMatch, you can:

  • Compare two facial images to determine whether they match (face comparison)
  • Search for a matching facial image within a database (face search)
  • Cluster similar faces together (face grouping)

 

Learn More About FaceMatch

Here are other articles you might find interesting:

 

   

Technical Specifications

The table below shows how FaceMatch SDK performs on the Labelled Faces in the Wild (LFW) dataset:

FPRTPRThreshold (Inverse of distance)
0.10.99900 ±0.002130.55448
0.010.99667 ±0.005370.59791
0.0010.99367 ±0.006050.62989

FPR = False Positive Rate
TPR = True Positive Rate

These results are an indication only and are based on the specific dataset Labelled Faces in the Wild. Customers can expect similar performance, with possible variations due to hardware and the availability of annotated data.