Face Recognition & Lighting

We train computers to identify objects in a similar way to how we train humans. A child points at a cat and says “dog” and we correct her and eventually she learns the difference. For a fun exercise, try to write down explicit how you can tell a dog from a cat. It’s almost impossible.

We train computers to recognize faces in a similar way. We show it pictures of people and say “These two are the same, but these other two are different. Learn how to tell same from different.” Sounds simple, right? But the devil is in the details. A computer has no idea what a person is, let alone a face. All it has to go on is an array of colored squares (pixels). It will dutifully try to achieve the task we give it, but it is devoid of a lot of the context that humans use to get around the world.

In a recent Reddit post, user Sysaximus posted a fantastic visualization of an aspect of human faces called “Eigenfaces” which is an older technique used for computerized face recognition. Unlike modern methods using deep learning, it gives us a glimmer into what the computer is thinking. This visualization was generated from a dataset of 13,000 faces.

Eigenfaces from a 13,000 face dataset

Eigenfaces are an interesting concept. You can think of these as showing what parts of a human face are relevant for distinguishing different people in the dataset - the computer would create a unique weighted sum of these components to represent a particular face.

Notice there are many colored areas that are obviously not faces, and huge variations on the color depth of the faces themselves - this is the important part. The computer has wrongly decided that parts of the picture that aren’t faces are important for distinguishing between faces in the original dataset. Remember it is just looking at an array of pixels - it has no idea what a face is. A human would obviously ignore the non-face parts of the images because we do know what a face is.

What’s critical for us is that the computer is erroneously focusing (mostly) on variations in lighting, because those provide it with a good way to distinguish the pictures from each other in this particular dataset. It’s doing exactly what we told it to do, just not what we wanted it to do. This focus on lighting works with this dataset, but obviously won’t help the computer recognize faces with other types of lighting. Lighting is THE major challenge for face recognition system accuracy.

In the real world, ambient lighting can vary from pitch black to full sun, and everywhere in between. Furthermore, skin reflectance (color) varies enormously as well - a person with pale skin won’t look the same to a computer as a dark-skinned person. We want the algorithm to ignore ALL of that lighting variation because it isn’t important to the task.

When we started Blink Identity, we wanted to ensure our system was as accurate as we could possibly make it, in any environment, and we knew lighting would be the biggest challenge. We want the computer to focus on faces only. As an engineer, my favorite way to solve a problem is to eliminate it, because if that works you always know it is the right solution. So that’s what we did - we removed ambient lighting as a factor entirely.

Using some unique software and what has been called “un-natural” use of conventional technology, our system will always see you the same way no matter where you are, and we see all people of any color equally well. We can’t ever make a computer understand faces the way humans do, but our creative solutions allow us to limit the computer to just what we want it to see, and nothing else.