This blog post is meant for people who aren’t familiar with Face Recognition Technology (FRT) but would like to learn!

In our previous FRT for Dummies post, we covered the basics of face recognition technology. Hopefully the definitions provided were a great starting point. In this second post, we would like to go further and discuss a few more terms to help us understand how the technology works in more depth. This post will be more technical, but we’ll do our best to simplify the terms and define it as clearly as possible. To give a brief preview, I’ll be talking about the accuracy of face recognition systems and the different ways FRT can be broken. If you get confused with some of the terminologies, please feel free to refer back to the first FRT for Dummies post!

Accuracy & Thresholds

As a starting point, let’s talk about the concept of accept/reject rate in relation to face recognition. In our previous post, we discussed the complexity of the matching process, false positive, false negative, and how it’s a trade-off. False positives occur when the system incorrectly matches the probe with template - it makes a positive match but it’s incorrect. This is the same thing as False Accept. False negatives occur when the system incorrectly fails to match the probe with the template even though it’s the right probe. This is also called a False Reject. We talked about the repercussions of each error and how reducing one error increases the other.

Any identification system determines a match by comparing the two images and looking at the percent likelihood they are the same image. Comparing two identical images would give you a 100% match. Comparing the photos of two very different people would give a match score close to zero. Because identity isn’t 100%, no identification system can ever be 100%. The math just doesn’t work out that way. So each system sets a threshold used to determine which images match. You might expect that the threshold is always set very high - 99.99% or something like that. But it really depends on the use case. If you set the match threshold very high, you will increase the number of false rejects you get. That means people who should match will be stopped and have to try again. If you set the match threshold very low, you increase the change of a false accept - someone who isn’t in the database will be incorrectly matched. So different use cases will set different thresholds. In law enforcement applications, they typically will set the thresholds very low - 80% or even lower. This is because every match is verified by a person and they want to see (for example) the 20 top matches in the database (even if they aren’t likely to be a match) so a person can eliminate them from consideration. Because the stakes are high, they want a person, not a machine, to be making the decisions. In other applications, for example in the case of season ticket holders at a theme park, it makes sense to reduce the threshold because you are trading the very small risk of wrongly allowing a person in to the park with the cost of annoying a valued customer.

How to Avoid Being Recognized

Face recognition technology isn’t foolproof - there are many easy ways to trick any system. The most obvious way is to avoid letting the sensor see your face. No system can do face matching unless it can capture an image of the face. Because most sensors are mounted above eye level, simply looking down or sideways is often enough. Although this only works if you know where the sensors are located.

Another popular evasion method is to use makeup to disrupt the face but it’s easier said than done. Normal makeup won’t work so you need to create a makeup pattern that’ll confuse the system. Basically, you need to look like the lead singer from KISS or a Juggalo (the clown, not to be confused with a gigolo). While the makeup method could work, it’s not the most effective if you’re trying to be inconspicuous. There’s also a tech company, called Hyperface, that designs camouflage patterns on a scarf (or any piece of clothing) to confuse the face recognition algorithm. They’re exploiting the algorithm’s function to seek out faces by creating small patterns that resembles faces. As you can see, face recognition technology isn’t perfect, and people will continue to find new ways to disrupt the technology. With that said, the technology is still more accurate than humans.