One of the first questions we get asked by clients is how to ‘defeat’ facial recognition systems. Everyone wants to know how you could break a face recognition system – common guesses include sunglasses, a change in facial hair, or large hats. (There is a really simple way to defeat most systems. I’ll tell you what it is at the end of this article.) Most methods to ‘break’ face recognition are based on obscuring the face, or visually breaking up the key features – eyes, nose, and mouth.
One of the more interesting examples of this is the CV Dazzle Look Book, which “uses avant-garde hairstyling and makeup designs to break apart the continuity of a face.” We frequently test our system against the silliest hats, or largest most reflective sunglasses that we can find, so I decided to take an afternoon and see how well these fun looks performed using our system.
Accuracy in biometrics is a complicated subject that involves conditional probability and advanced math. If you want to know more about that, our CTO wrote a blog article about it. Blink Identity has incredibly high standards for accuracy. However, face matching by computers, just like face matching by humans, is probabilistic, meaning no system can be 100% accurate. So when actively trying to fool the system, I was expecting some non-matches. There are two kinds of errors in this circumstance, either a false-match or a false non-match. The first is difficult to do intentionally – it’s when the system finds a face, but incorrectly identifies it. Most methods on defeating facial recognition systems are trying for the second kind of error, where the system is unable to find a face in a given image.
After talking the look over with one of our developers, Ben, we were both interested to see what effects the glitter and crystals would have, since they scatter so much light. We were also fairly certain that the looks that obscured more than 50% of the face would give all systems trouble. If the eyes are covered, it’s very difficult to find a face.
An important thing to note is that computers see images very differently than we do. Our CEO wrote about the difference between computer vision and human vision here but generally, humans are very good at detecting faces of people that they know, and not as good at recognizing the faces of strangers. So while I have no problem recognizing Ben, even with most of his face obscured, a computer would struggle. On the other hand, a computer can look at a million faces and compare them without getting tired.
These first six are attempted recreations of the CV Dazzle looks. Ben’s a good sport, and agreed to help model for these tests. It goes without saying I am not a professional artist of any kind nor are either of us models, and thus please be kind to my attempts to recreate these beautiful and fairly challenging looks.
The common features between them are that they obscure the face, either entirely or in part, focus on asymmetric designs, and use paint or gems to break up the standard image of a face. We saw the expected strong direct correlation between the amount of face obscured, and the number of non-matches. We were able to get a match off of all of them, but some took ~10 attempts and non-matches first, which counts in my book as defeating facial recognition. Photo number 2, 4 & 5 matched every time, and numbers 1 and 3 matched more than half the time. Only number 6 gave our system real trouble.
These last two are face paints you see now at sporting events that we thought followed the same ideas – breaking up facial features, glitter to scatter light, and obscure the face to some degree. While these look to us to significantly break up the lines of the face, and obscure the nose and lips especially, they were the only two that matched every time we tried, even though Ben’s face has a truly enormous amount of glitter on it.
Thanks to Ben for being a great sport/model, and helping me run all the tests. While these are fun looks, there are easier ways to beat face recognition – just look down. Most cameras are mounted near the tops of walls, and look down. If you look down, the camera will see the top of your head instead of your face and will see nothing to match.
If you want to read more about Blink Identity’s biometric identification service, Ben wrote a blog that digs into the details of how it works.