Everything About

How Facial Recognition Works

How Facial Recognition works

Facial recognition uses artificial intelligence (AI) to compare facial images and determine whether there is a match. It can determine whether two facial images are of the same person, and it can also determine whether a facial image matches any of the facial images contained in a particular database. These comparisons are referred to as 1:1 and 1:N matching respectively.


Applications for Facial Recognition software

Basic face identification model

Usually used in applications such as Instagram. The software uses a cellphone camera in order to identify facial features, specifically, eyes, a nose, and a mouth position. After the features have been defined, the application utilizes algorithms to focus on the face and determine whether the mouth or eyes is open or which direction the person is facing. It is important to note that these actions aren’t face identification, as they are plain programmatic actions that are used by the software that is looking for the faces through the camera.

Basic face identification matching model

The software of this programmatic principle measures the distance between your facial features characteristics by taking a picture of your face. The measured features are the same as in the previous method, the eyes, nose, mouth, and other possible options depending on the preferences of the software creator. This principle is used every time when you are required to prove your identity. The potential example can be Apple Face ID. 

Database match model

This method is used when a company or individual wishes to identify faces for security reasons, for promotional purposes, or any other reason where face recognition technology can be used. Database matching differs from the previous methods but is also a bit similar to face identification models that are used within cell phones. The process of database matching can be described as comparing a face towards a database that has been predetermined by the operator. The person approaches the location that is covered by the camera, the camera records the facial features and then matches those features towards a database. In case there is a match, a predetermined action takes place, such as for example, a door opens. In theory, any database can be used as well as any action can take place based on the requirements and wishes of the operator, the core principle remains the same.


The principle of 2D Face Recognition

Most face recognition software solutions are functioning by using only 2d images. This is done due to the limiting factor of most cameras, as most of them take pictures without any depth and most of the pictures that are used for face verification are available in 2D resolution. 2D face recognition may be convenient, but it also limited in many different ways. The first and most obvious is a lack of depth. The program can measure the distance between your facial features but still, it is unable to measure the length of your nose or any other given feature. This results in situations where extra verification characteristics aren’t being used and therefore 2D verification can be described as less effective when compared to 3D counterpart. Moreover, 2D facial recognition relies on a visible light spectrum, meaning that it is unable to function in complete darkness and may be prone to reliability issues in areas with poor light conditions. Most of the 2D limiting factors are fixed when the software uses 3D imagery instead of 2D.

You may wonder how can we battle the lightning problem and start identifying faces in the dark? 

There is way to solve this issue, by using thermal imaging solutions. It works a bit similar to the 3D method, but instead of sending IR light towards the object and measuring the response, alternately this technology detects the IR light that is emitting from the objects. The warmer the object, the more IR light it emits, unlike the cold one. Depending on the camera characteristics and price, the thermal cameras are able to detect small temperature differences across any given area that is covered by a camera, meaning that this kind of solution can also be used for face recognition.

It also has to be mentioned that thermal light detection also has significant disadvantages such as its cost and low efficiency during the daytime, as a result, it is rarely used outside of the military. There are many different techniques that can be used in order to detect and identify a face using thermal cameras. Some of these methods are complicated and technology-dependent, whereas some are not, but all of them share the same basic principles that can be described as follows.

  1. A Dataset is created for identification purposes. The software is using multiple IR images in order to create a dataset. The created dataset is used when it is compared to a given database in order to identify a subject that has been detected by the camera.
  2. Dependency on multiple pictures. The thermal camera needs multiple photos that are taken in different light perspectives in order to recognize a face. It has to measure different spectrums of IR light, such as long, short, and medium. In general, the longwave spectrum is used when the software needs the maximum number of facial details. 

What hides behind 3D Face Recognition

While most solutions utilize the 2D method, some use 3D. The 3D face identification is a bit different from 2D as instead of measuring the distance between facial features it utilizes a technique that is called lidar. Lidar is a method that is very similar to sonar. It functions by using a matrix, that is pushed over your face and then the reflection is recorded by the IR camera. 

The measurement takes place when the IR camera measures how long does it take for each particle of IR light to reach your face and the return to the input source. The principle is the following: the light that has been reflected from your ears normally has a longer journey to the source (camera) when compared to the light that has been reflected from your nose, which on the other hand has a shorter journey of IR particles towards the camera. The camera uses the gained data in order to create a depth map of your face. This method drastically increases the accuracy of face recognition as it simply utilizes more facial characteristics when compared to a 2D counterpart. The typical example of 3D face identification can be seen in various cell phone programs such as Face ID, that is used in order to authenticate a phone user. 


A brief background

While face recognition may seem like new technology, it’s actually been in development since the 1960s. It is only in recent years, however, that it started to emerge as a viable technology that we can use at scale. The speed of the recent progress is due to various factors, including more powerful hardware (such as GPUs) and the advancements that have been made in machine learning and deep learning. Keep reading below to find out more!


Steps in the Facial Recognition process

While there are different AI algorithms [link to Face Matching Algorithms article] that can be used for facial recognition, they all follow more or less the same steps:

  1. Detect a face or faces in an image or video.
  2. Extract the facial features to create a faceprint or face embedding.
  3. Compare faceprints or face embeddings to determine whether there is a match. How similar the embeddings need to be for a match to be established will depend on the chosen confidence threshold. Comparisons can either be 1:1 (one-to-one) or 1:N (one-to-many).
  4. Return the result of the comparison or search.

Why is Facial Recognition important?

Facial recognition is transforming many aspects of daily life. Already, many of us are used to unlocking our smartphones or computers using Face ID. This has paved the way for facial recognition to be adopted for additional purposes, such as authenticating payments and enhancing access control.

Along with other forms of biometric identification and verification, facial recognition is transforming the landscape when it comes to applications ranging from access control and payment verification to retail and security. For this reason, it is important to understand how it works and how it could transform your own industry.


Face Recognition application field

Even though face recognition may appear as something futuristic and made the future rather than for today, surprisingly it is currently being used in many ways in various industries across the globe. There are many use cases and application fields. Moreover, the tech is highly adaptive and can be customized for clients personal goals and needs, as all it takes is a spark of imagination and creativity.

Payments and Authentication

Face Recognition can be used in order to facilitate payments and make them more convenient for both, the business and the customer. Instead of paying in cash or with credit cards, in some countries like China consumers are able to pay with their face. Moreover, Face Recognition can also be used in order to verify identity during the payment process. Different business may utilize face recognition in order to check if the customer is really who he says he is. This feature can be applied in many use cases such as mobile banking or in government related institutions. 

Access Control and Security

Apart from being used as a verification measure, face recognition can also be integrated with various physical devices and objects. Face Recognition can be used in order to access different fields of interest, such phone for example. Apple is utilizing FaceTech in order to allow the users to unlock their phone with their face. Moreover, face recognition can be used in order to ensure that security measures are implemented to their maximum potential and that there is no room for fraud or error within a given system or business facility. Furthermore, various Facial Recognition software pieces contain a liveness check that helps prevent hackers from using a picture of the customer for impersonation purposes. The liveness detection asks users to perform a random sequence of movements (e.g. with their head) to make sure they are a real person. Only after the software verifies that the sequence was performed correctly, the user is certified as “alive”.

Attendance Monitoring

With face recognition software, schools, universities and other institutions are able to easily track the attendance of their students, employees and other visitors as well avoid any suspicious activity. 

Airport services

Various airports employ Face Recognition technology in order to facilitate the onboarding process as well as to minimize risks and improve security. Read mode about usage of Face Recognition in Airports here. 

Age control

Some businesses utilize face recognition as a measure to check ones age. The possible application can be a grocery store using face recognition to check age of their customers who wish to purchase age restricted goods such as alcohol. The same can be applied for age restricted areas such as bars or adult shops, where face recognition can be used on pair with camera and human monitoring in order to make sure that the area visitors are age compliant. 


Concerns regarding Face Recognition

Even though face recognition may seem as a perfect feature to solve many problems, everything comes with a price and many companies and enterprises are still hesitant to accept the advantages that face recognition has to offer. Face recognition has been criticized for its accuracy, privacy concerns and misuse of private data.

Accuracy Issues

Some might say that face recognition is not always accurate in matching faces with the ones within the given database. It should be noted that errors occur due to various reasons that are usually associated with the quality of the input image. Poor lightning, low image quality or inappropriate face angles contribute towards decreasing the efficiency of face recognition of a given face. Moreover, it is also important to keep an eye on the database itself that is used for verification purposes, as in case it lacks necessary information, it would be very challenging to verify a human face with obstructed data.

Privacy concerns

One of the main concerns that are related to Face Recognition are the privacy issues. As some people may find it worrying that the given technology can track you and record your data. Most face recognition software pieces do not store your data as they discard the gathered data automatically after use, but not all of the system do it, as a result creating a possibility of the data breach which are relatively uncommon nowadays. The result of a potential data breach can include personal information becoming public or accessible by unauthorized personnel. 

On the other hand data breaches highlight the flaws and problems within the security systems and as a result this alone contributes towards improving the security of the given systems. As a conclusion it can be said that privacy issues are mainly depend on the software itself rather than general face recognition in a nutshell, as there exist numerous solutions that act and work differently from one another, and it all comes down towards the individual settings of every solution.

Data Misusage 

Most of the consumers trust big companies or government institutions that use face recognition technology, whereas others are not so confident when it comes down to personal data handling and ethics. Some use cases of face recognition are seen as acceptable in public eyes, such as payment authentication and other convenience features. On the other hand features like attendance tracking or CCTV monitoring can be seen in negative field. 

Moreover, the distrust can also come in a form of a fear from the fact that data holders can misuse the data that they possess. This directly leads to a fact that not only private organizations have access to face recognition data, as most of the databases is public, which means that anyone can find anyone within those databases and use the gathered data in their own intent.


What are some of the common problems with Face Recognition, and how can we overcome them?

Even though there are multiple use cases of Facial Recognition, every technology has its disadvantages and limitations as in example with thermal cameras. Some problems can be resolved, whereas others stay fixed. Face recognition isn’t perfect. While there has been huge progress, there is still room for error. Take a look at this example:

On a more serious note, here are some of the challenges with facial recognition:
  • Variations in images compared: The system works best when the face that is scanned is presented upfront. Any tilt of the head or turn of the body can make the process more difficult. Even a simple smile, or any other emotion will make the computer work harder in order to recognize who is presented in front of the camera. When there are variations in the position of the head, lighting conditions, and facial expressions in the two images that you want to compare, it can result in lower accuracy. The same is true when it comes to make-up, glasses, hats, and anything that partially obscures the face. Luckily, there are a few ways to deal with these challenges. The first is to use a large and diverse training set. Second, deep learning techniques are making it easier to correct for these variations. 
  • Less than 100% accuracy: No face recognition algorithm is 100% accurate. Even with the most sophisticated software, you still have a chance that an authorized person will be denied access in an access control scenario or vice versa. In these types of situations, human intervention will still be required. However, with accuracy rates improving and with the technology achieving 98% accuracy and higher, it is clear that the technology is useful in all but the highest-risk scenarios.
  • Privacy: One of the biggest concerns surrounding the use of facial recognition technology is privacy. This is a valid concern, and therefore there are various measures that you can take to reduce the privacy risks when implementing facial recognition technology. These include: storing only faceprints/face embeddings, and not the facial images themselves; not storing facial image data at all (if your use case allows for this); storing and processing data locally, so that you have full control over the data and so that third parties do not have access; and purging data regularly.
  • Spoofing: In many instances, there are concerns that facial recognition software can be fooled by masks or copies of photos presented to the camera. To prevent this, many facial recognition programs incorporate liveness detection features, which ensure that the submitted video is of a live person, and that it is captured in real time.
  • Light. Lightning conditions may drastically affect the performance and accuracy of face recognition technology. All of the facial recognition solutions are affected by this problem, as all of them work based on light or rely on some other way on lighting sources. 
  • Database necessity. The efficiency of face recognition very tightly depends on the database that is used by the system. It is close to impossible to identify and verify a face that hasn’t been correctly identified in the past.
  • Data analysis limitation. Depending on the load, some solutions will require more time to process all given data.