What is facial recognition and will it work when people are wearing a mask?

International Business Machines (IBM) Corp, earlier today, announced that it will no longer offer general-purpose facial recognition or analysis software for purposes of mass surveillance and racial profiling.

The tech giant's decision to get out of the facial recognition business, employed by multiple companies, comes as the United States grapples with nationwide protests over the death of George Floyd, under police custody in Minneapolis.

Facial recognition is a technology which is capable of identifying a person’s face from an image or a video. The tool requires a user’s face to be stored in its database for it to identify when detecting from a source. 

Facial recognition only requires the geometry of your face. The software first maps all the basic data like your eyes, nose, ears, lips, and then also marks the distance between each of these key factors. For example, the distance between the two eyes, the length of the nose, the distance between the chin and forehead, etc. The algorithm then uses these factor as mapping points to distinguish and verify you among the crowd.

Some advanced facial recognition systems also use artificial intelligence (AI) to enhance the system’s face reading capabilities.

Facial recognition tools have been available for a while for government sources as a surveillance tool. The tech traditionally used CCTV cameras and computers to detect faces. Facial recognition only requires a photo or video available on a public domain or any captured via security cameras.

However, the use case has expanded to smartphones and several other devices. The most popular facial recognition tool on a smartphone is Apple’s Face ID. The tech uses nearly 30,000 infrared dots to map a user’s face, creating a 3D map and storing it on the device for unlocking an iPhone or to authenticate purchases.

While smartphone manufacturers claim that the facial data remains stored on the device, there have been allegations that some companies store the data and use it to either target ads or share it with government bodies and/ or third-party.

This also raises questions about the security and privacy of users. Governments and law enforcement authorities in several countries claim to use facial recognition for security purposes. While they state the data remains secure, there have been instances or claims of these servers getting hacked, resulting in the breach of critical facial data that can be used for malicious purposes.

Several social media platforms like Facebook use your profile photo and other images to read your face data and target ads. Users, while registering, knowingly or unknowingly give up ownership of their data on such social media website, thus losing control over the practical usage of their face data.

The short and simple answer is yes. With the use of AI, it has become possible for governments and other law enforcement bodies to use advanced facial recognition to identify individuals even if a part of their faceprint is covered or layered with makeup. The algorithm is developed enough to match certain mapping points of an individual with the one stored in the database. This could take seconds or minutes, depending upon how advanced the system is and also how large the database is.

However, facial recognition does not work on smartphones when a user is wearing masks. We have tried and failed to unlock smartphones like the iPhone 11 Pro, which uses 3D mapping, and also many Android devices that use optical facial recognition.