How Does Facial Recognition Work?

You may have unlocked your phone with your face multiple times today, but do you understand how the underlying biometric technology, facial recognition, works? In this article you’ll learn how using an individual’s physical characteristics proves a better choice than traditional security methods such as easily lost, stolen or forgotten ID cards, passwords/PINs, or keys.

While smartphones offer one of the most ubiquitous uses for facial recognition-based security, the possibilities extend far beyond personal mobile devices - with biometrics adding significant benefits to safety, security, and efficiency across a variety of industries. As a compelling, comprehensive, and rewarding biometric technology, it is vital to understand how facial recognition works, what sets it apart from other biometric modalities, how it can be deployed and optimized, the technical considerations and specifications, varied use cases, and its future potential.

1. What is facial recognition?

Facial recognition technology identifies facial vectors and features then matches them with pre-enrolled individuals.

AI algorithms run mathematical equations to create a unique template of an individual by measuring their facial variables. Measurements such as nose depth and width, forehead length, and eye shape are processed by the AI. Facial recognition technology then compares this newly generated template with existing templates in a database. If there is a match, it can confirm an individual’s identity.

CyberLink's FaceMe® is an AI-based facial recognition engine that pushes the boundaries and is leveraging the latest in AI, such as deep learning and neural networks. The result is one of the world’s most accurate, secure, and flexible edge-based solutions.

What are the different types of biometric technologies?

Three of the main biometric technologies in use today are:

1. Fingerprint Recognition: Unique fingerprint patterns are analyzed on a touch or optical sensor, and matched to an individual
2. Iris Recognition: The unique patterns and colors in a person’s iris are scanned and used to verify identity
3. Facial Recognition: A mix of unique facial vectors and features are used to match and identify an individual

The chart below compares the strengths and weaknesses of each biometric solution.

While it can be fast and highly accurate, Iris recognition requires specialized and expensive imaging equipment. The downside to fingerprint verification is the hygiene concerns associated with touch verification — in addition to dirt or oil from fingers interfering with the efficacy and integrity of these sensors. Facial recognition is recognized as a superior method of biometric security technology because it is affordable, hygienic, fast and flexible.

Facial recognition is the best choice

Facial recognition can be leveraged to perform tasks far beyond face detection and recognition – making it one of the most powerful and relevant AI enhanced biometric technologies. Moreover, thanks to the use of robust and feature-forward facial recognition platforms, such as FaceMe®, the benefits for users continue to grow.

How does facial recognition work?

A facial recognition engine performs several steps, as outlined below:

Step 1 Face detection

Face detection is the first step in any facial recognition process. The technology scans an area for any full or partial human faces within camera view. After faces are confirmed to be present, the technology can begin the face match using its AI-powered recognition processes. FaceMe® can detect multiple faces at a time, and count how many faces are simultaneously present, making it a suitable engine for busy areas.

Step 2 Face feature extraction

Feature extraction is the next step. After a face has been detected, a template is generated. The engine extracts an n-dimensional vector set - a template - from the detected facial image. Achieving a precise template requires a high “n” value.

Face attribute detection is simply facial analysis for the purpose of identifying and analyzing customer characteristics such as age, gender, and mood, as well as head orientation or movements (e.g., nodding, shaking). Face analysis is an important feature in smart retail and digital signage, which allows for customized ads and messaging while collecting detailed visitor statistics.

Step 3 Face match and face search

1:1 Face match

If the goal is to verify a person’s identity, the engine will then perform a face match. A match uses a 1:1 method. The newly extracted facial template is either matched with a facial template in the pre-enrolled database or cross-referenced with a piece of identification to perform a match. Some of the most common uses of face matching occur on mobile devices, such as Apple’s Face ID on iPhones, or in mobile banking apps where you log in to your financial service portal by authenticating with your face.

1:N Face search

For this function, the question progresses from “Is this person who they say they are?” to “Who is this person?” FaceMe® can answer this question using a face search (or face image search). This time the AI-powered engine completes a 1:N search, comparing an individual’s template against pre-enrolled faces in the database to find the best match and confirm the person’s identity. To ensure privacy, no actual images of faces are stored on the platform — FaceMe® only stores encrypted vector data. The most common use of face search is for security and surveillance camera systems, to verify that people on premises belong to a particular group or organization. In this example, if an individual's face has been pre-enrolled in the company’s database, face search will will look for a template that corresponds to the live capture and will grant access if there is a match.

For additional information about the key qualities required by a facial recognition system, please check out an executive summary of the technology, written for the US Department of Homeland Security.

Which FaceMe® solution works best for you?

Are you a developer who just needs a facial recognition SDK or API to start developing your own applications? Or are you a business owner or decision-maker that wants an out-of-the-box facial recognition solution?

For developers: SDK or API

FaceMe® has two different types of solutions for developers, depending on the development and coding requirements of their facial recognition application.

• FaceMe® SDK : FaceMe® SDK is a top-ranking, cross-platform engine that integrates edge-based AI facial recognition in many IoT and AIoT devices. It is compatible with a wide variety of hardware configurations and operating systems. Face recognition terminals, smart robotics (example: Lovot), and medical cabinet access control are just a few examples.
• FaceMe® Platform : This web-based service includes facial recognition APIs, a console, testing tools, and system services so that developers can flexibly deploy facial recognition in virtually any connected scenario. It offers ensured high service availability with load balancing and failover functions. User examples for FaceMe® Platform could include company computers or university/school systems.

For business decision-makers: ready-to-deploy solutions

• FaceMe® Security : FaceMe® Security is an out-of-the-box security and access control solution. It enables identity verification, attendance management, and access control through facial recognition, with real-time monitoring and alerts. Additionally, it can identify and track a person’s movements using facial images or physical attributes. Integrated with major VMS offerings, FaceMe® Security enables easier and more effective smart security management.

  Related articles: Building a Security & Surveillance System with Facial Recognition

• FaceMe® eKYC & Fintech : This solution offers the banking, financial services, and insurance industry (BFSI) secure, real-time biometric verification that can be deployed at the edge or on the cloud. It enables companies in these industries with remote digital onboarding of new customers. Financial institutions can protect customers against biometric fraud by deploying eKYC (electronic Know Your Customer) solutions that confirm a person's photo to their ID.

  Related articles: With Facial Recognition for the BFSI Industry KYC Becomes eKYC

What’s the future of facial recognition?

Facial recognition has a significant role to play in the future of AI biometric sensing technology. In the past 5 years, the use of AI has brought levels of flexibility, affordability, and accuracy that were unthinkable until now and makes it a core lever to process and workflow improvements everywhere, from industrial plants and mines to customer experience and financial transactions. Businesses are now using facial recognition for use cases from automated door systems to control access. Retailers use it to enhance in-store customer experiences. Manufacturers are adding it to their processes for quality control. Banks and fintech organizations are introducing enhanced face matching, anti-spoofing authentication and cutting-edge security controls. These are just a few examples where facial recognition technology is gaining traction.

Without a doubt, the ethical deployment of facial recognition technology is on track to make the world a better place.

FaceMe® facial recognition FAQ