Face and Voice

20.0 What this chapter gives you#

1. You will be able to explain why “does this face match this one record” and “does this face appear anywhere in this database” are two different machines with two different failure modes, and why calling both “face recognition” causes most of the confusion in the field.
2. You will be able to describe how a photograph becomes a short list of numbers, say roughly how many, name the papers that made that list work, and explain what the numbers are and are not.
3. You will be able to compare two of those lists by hand, using the same arithmetic a real matcher uses, and say where the number you compare against comes from and who chose it.
4. You will be able to read the current NIST face recognition tables, quote real error rates with the dates they were published, and say what changes if the picture is taken at a passport desk rather than by a camera on a ceiling.
5. You will be able to state the 2019 NISTIR 8280 findings precisely, without exaggerating or softening them, and say what the same measurement programme reports in the middle of 2026.
6. You will be able to calculate how many false alarms a watchlist will produce per day, given a gallery size, a threshold and a number of faces seen, and defend or refuse a deployment on that arithmetic.
7. You will be able to distinguish a presentation attack from an injection attack, say which international standard covers which, and name the defences that work against each.
8. You will be able to explain how voice biometrics differ from face, what text-dependent and text-independent mean, and why one major standards body now forbids voice comparison outright for a whole class of systems.
9. You will be able to name where face recognition is legally restricted, with jurisdictions and dates, and say what the restrictions actually prohibit rather than what headlines say they prohibit.

There is a sentence people say about face recognition, and it is always wrong. The sentence is “face recognition is now 99 point something per cent accurate”. It is wrong not because the number is too high or too low, but because the sentence has no meaning. Accurate at what? Matching a passport photograph to the person standing at the desk, or picking a face out of a crowd from a camera on a lamp post? Against a database of two hundred people or twelve million? At a setting that lets one stranger in ten thousand through, or one in a million? For a thirty-year-old man photographed in good light, or a seventy-year-old woman photographed at a bad angle in a corridor?

Each of those questions changes the answer by a factor of ten or a hundred. Some of them change it by a factor of a thousand. A single number cannot survive that, and anybody who gives you one has either not understood the technology or is selling it.

This chapter is built around one distinction, and everything else in it hangs from that distinction. There are two modes. In the first, the system already knows who you claim to be, and it asks one question: is this the same person as the one record I hold under that name. In the second, the system knows nothing, and it asks a completely different question: does this person appear anywhere in this collection of faces I hold. The first mode makes one comparison. The second makes as many comparisons as there are records. The mathematics of the two are not related by a small correction. They are related by multiplication, and multiplication by a large number is how a system that looks safe on a datasheet becomes a system that arrests the wrong person.

The chapter before this one, “Biometrics”, dealt with the general shape of the problem: any biometric is a measurement, measurements are noisy, comparison produces a score rather than a verdict, and somebody has to choose the threshold that turns the score into a decision. All of that applies here and we will not repeat it at length. What is specific to faces, and what earns this chapter its own place, is that faces are the one biometric routinely collected without the subject’s participation, at a distance, from cameras that were installed for something else, and searched against galleries of millions. Voice sits beside face in this chapter because it shares that property. You leave your face and your voice in public, constantly, for free.

The plain version#

Two questions that sound the same#

Imagine a small private club with a doorman called Ravi.

On Monday, Ravi works the members’ entrance. Every member has a card with their name and photograph on it. A woman hands him her card. Ravi looks at the photograph on the card, then at her face, then back at the card. He is answering one question, and only one: is this the same person. He is not thinking about anybody else in the world. If he decides yes, she goes in.

On Tuesday, Ravi is given a different job. The police have left him a folder of two hundred photographs of people who are not allowed in. Now nobody hands him anything. People simply walk past, and for each face that walks past, Ravi has to flick through all two hundred photographs and decide whether that face is any of them.

Same doorman. Same eyes. Same skill at recognizing faces. Two completely different jobs, and — this is the thing everybody misses — two completely different chances of getting it wrong.

On Monday, Ravi can only make a mistake about one person: the woman in front of him. On Tuesday, he gets two hundred chances to make a mistake for every single person who walks past. If he has a one in a thousand chance of wrongly thinking two photographs are the same person, then on Monday he is wrong about one visitor in a thousand. On Tuesday, with two hundred photographs to flick through, he will wrongly stop roughly one person in five.

Nothing about Ravi changed. The number of comparisons changed.

That is the entire idea of this chapter, and if you take nothing else from it, take that. Computers do exactly the same thing, at exactly the same disadvantage, only faster and against far larger folders.

The first job has a name and so does the second#

The first job, where somebody claims to be a particular person and the machine checks that one claim, is called verification, and people write it as one-to-one, or 1:1. Your phone unlocking when you look at it is 1:1. The passport gate that reads your passport chip and then photographs you is 1:1. A bank confirming a customer against the selfie they took when they opened the account is 1:1.

The second job, where nobody claims anything and the machine searches a collection, is called identification, and people write it as one-to-many, or 1:N, where N is however many records are in the collection. A police search of a mugshot database is 1:N. A shop watching for known shoplifters is 1:N. A government checking whether a new driving licence applicant already holds a licence under another name is 1:N.

Both get called “face recognition” in the newspaper. They are as different as weighing a parcel and searching a warehouse.

How a face turns into a list of numbers#

Here is the part that sounds like magic and is not.

The computer does not store your photograph and compare pictures. It stores a list of numbers.

Think of it like this. Suppose you had to describe a face over the telephone using only measurements. You might say: the distance between the eyes is 63 millimetres; the nose is 51 millimetres long; the mouth is 47 millimetres wide; the chin sticks out this far; the eyebrows sit this high. Twenty or thirty measurements and you have said something useful. Somebody at the other end could not draw the face, but they could probably pick it out of a small group.

Early face systems really did work like that, and they worked badly, because those measurements change when you smile, or turn your head, or get older, and because two different people quite often have very similar noses.

What the computer does now is the same idea with a crucial difference: nobody chose the measurements. The machine was shown millions of photographs and told, over and over, “these two are the same person, these two are different people”. Over time it invented its own set of measurements — around five hundred of them — that happen to be very good at telling those cases apart. Nobody can say what any single one means. There is no number in there labelled “nose length”. They are five hundred quantities that, taken together, are similar for two pictures of one person and different for two pictures of two people.

That list is called an embedding, or a template, or a face vector. When you enrol, the system computes one and keeps it. Your photograph is normally thrown away. The list stays.

Two things follow. The list is short: five hundred numbers of a few bytes each is roughly two kilobytes, which is why a database of twelve million faces is manageable and a database of twelve million photographs is not. And the list is not a picture, but it is not nothing either. People will tell you it cannot be turned back into a face. That is a comfortable thing to say and it is not a law of nature.

Comparing two lists is measuring an angle#

Now, how do you compare two lists of five hundred numbers?

Picture two arrows drawn from the same point. Each list of numbers describes the direction one arrow points. Two pictures of the same person produce two arrows pointing in almost the same direction — a small angle between them. Two pictures of different people produce arrows pointing off at a wide angle.

So the comparison is simply: what is the angle between these two arrows. A small angle means “probably the same person”. A wide angle means “probably not”.

The machine does not report an angle in degrees; it reports a number between minus one and plus one that goes up as the angle goes down. That number is called the cosine similarity, and you do not need the trigonometry to use it. Just remember: near one means the arrows point the same way, near zero means they point in unrelated directions.

Here is what that looks like with real-feeling values.

Two photographs of the same woman taken the same afternoon: 0.71. Two photographs of her twelve years apart: 0.48. A photograph of her and a photograph of a stranger: 0.08. A photograph of her and a photograph of her sister: 0.31.

You can see the problem already. The sister sits between the strangers and the twelve-years-apart pictures. Wherever you draw the line, something lands on the wrong side of it.

Somebody draws the line, and it is not the machine#

The system is given one number: the threshold. If the similarity is above it, the system says match. If below, no match.

Set the threshold at 0.55 and the same-day pictures match, the twelve-years-apart pictures do not, the sister does not, and the stranger does not. Very secure, and it turns away the woman herself whenever her photograph is old.

Set it at 0.25 and everything of hers matches, and so does her sister, and so, occasionally, does a stranger.

There is no setting that is simply correct. There is a dial. Turning it towards security means turning it away from the people who are entitled to get in, and turning it towards convenience means letting some strangers through. This is true of every biometric, which is why the previous chapter spent a long time on it. What is new here is what happens to that dial when you move from Monday’s job to Tuesday’s.

Why searching a database is a different animal#

Come back to Ravi and the folder of two hundred photographs.

Suppose the threshold is set so that any two random people have a one in a million chance of scoring above it. For Monday’s job, that is a wonderful number. A stranger who claims to be a member and presents a stolen card has a one in a million chance of being let in.

Now Tuesday. Every face that walks past gets compared with two hundred photographs. Each of those two hundred comparisons has a one in a million chance of a false hit, so each passing face has about two hundred chances in a million — roughly one in five thousand — of being wrongly flagged.

Two hundred people walk past in an hour. Four thousand a day. So about one wrong flag a day. Tolerable, perhaps.

Now make the folder bigger, because that is what always happens. Make it two hundred thousand photographs. Now each passing face has about two hundred thousand chances in a million, which is about one in five. One person in five walking past that door gets flagged as a wanted criminal.

The threshold did not change. The camera did not change. The algorithm did not change. The folder got bigger, and the system became useless.

This is the single most important practical fact about face recognition and it is arithmetic a child can do. The chance of a false alarm in a search goes up roughly in proportion to the number of records searched.

The photograph held up in front of the camera#

Everything so far assumed a real face in front of a real camera. Attackers do not oblige.

The simplest attack is to print a photograph of somebody and hold it up. It works more often than you would like against cheap systems, because a printed photograph, well lit, produces the same list of numbers as the face did. The next is to hold up a phone playing a video of the person, which defeats systems that look for blinking or small movements, because the video blinks and moves. The next is a mask — not a Halloween mask but a moulded, painted, three-dimensional copy of a face, which defeats systems that check for depth.

Defenders answer these with what is called liveness detection: is there a real, living, three-dimensional face here, or a flat thing, or a rubber thing. The defence and the attack chase each other, and neither wins permanently.

And then there is the attack that changed the field, and it is none of the above. Suppose the attacker never puts anything in front of the camera at all. Suppose instead they make the computer believe that a video file is the camera. The system asks for a live selfie; the software hands it a synthetic video of the victim’s face, moving and blinking and turning its head on command. There is nothing in front of the lens because the lens is not involved.

Every clever check about lighting, depth, blood flow under the skin and reflections in the eye is answered at once, because the attacker is not fooling the camera. They have replaced it.

Your voice is a face you make out of air#

Voice works on the same plan. A recording of you speaking is turned into a list of a few hundred numbers, and two lists are compared by angle, exactly as with faces.

There are two flavours. In the first, you must say a specific phrase — banks used “my voice is my password” for years — and the system checks both the voice and the phrase. In the second, you can say anything at all, and the system listens to the sound of your voice while you talk about something else. The first is used to log in. The second is used in call centres, to check who is speaking while they are speaking.

Voice has one property that faces do not, and it is a bad one. Making a convincing copy of somebody’s voice used to require a talented impressionist. It now requires a few seconds of recorded audio and a piece of software that anybody can obtain. The copy is good enough to fool the people who know the person, and, as we shall see with real cases and real dates, good enough to fool the banks.

What the plain version leaves you with#

Two modes, and the second is far more dangerous than the first. A face becomes a list of about five hundred numbers. Two lists are compared by the angle between them. Somebody chooses the line. In search, the chance of a wrong answer grows with the size of the database. And the attacker’s best move is not to fool the camera but to bypass it.

Hold on to those six things. Everything technical below is a more exact version of one of them.

Where the plain version stops being true#

“Five hundred measurements of your face” is a helpful lie#

The plain version said the machine invented its own measurements. That is close, and it is worth saying exactly how it is wrong.

The honest version: the numbers are not measurements of anything on your face. They are the output of a large stack of arithmetic operations trained so that the outputs for two pictures of one person end up close together. There is no correspondence between any individual number and any facial feature, and there is no guarantee that the thing the network learned to key on is a property of your face at all. Networks have been shown to key on backgrounds, lighting, camera characteristics and image compression. When a face system performs differently on two groups of people, the cause is often not the faces but the photographs of them.

That distinction matters when somebody tells you a system is “measuring your bone structure”. It is not. It is producing a number that has been optimized to separate identities in the data it was trained on.

The angle is not a probability, and the vendor’s percentage is worse#

The plain version said the machine reports a similarity between minus one and one. True. Many products then convert that number into something they display as “97% match”.

The honest version: that percentage is almost never a probability that the two images are the same person. It is usually a rescaling of the similarity so that it looks friendly on a screen. A “97% match” from one vendor and a “97% match” from another are not comparable, and neither tells you the chance the match is wrong. The chance the match is wrong depends on the threshold, on the quality of both images, on the demographic of the subject, and — in a search — on how many records were searched and how many of them could plausibly be the right answer. A number on the screen cannot contain any of that.

The correct way to talk about a face system’s confidence is to state the operating point: at this threshold, on this kind of imagery, the false match rate is this and the false non-match rate is that. Anything else is decoration.

Two modes, but not two accuracies#

The plain version implied that 1:1 has one accuracy and 1:N has another. That is far too tidy.

The honest version: each mode has a whole family of accuracies, and the family members differ by more than the two modes differ from each other. The same algorithm, at the same threshold, on the same day, gives an error rate that varies by a factor of more than twenty depending only on how the photographs were taken. We will put real numbers on that below, from the NIST tables published in 2026. A visa photograph compared with a border photograph and a border photograph compared with a face captured by a kiosk camera are the same task for the same software, and the second is roughly twenty-four times worse.

So “1:1 accuracy” is not a number. It is a table.

The gallery multiplies false positives, but not exactly by N#

The plain version said the false alarm chance grows in proportion to the number of records. That is the right instinct and a rough approximation, and the exactness matters when you are arguing with a supplier.

The honest version: the proportional rule assumes every comparison in the search is an independent coin flip with the same probability. It is not. Some faces have many lookalikes in a given population and some have few. A search for a person with an unusual face may produce no false candidates at a threshold where a search for a person with a very common facial type produces six. The proportional rule gives you the average, and averages are exactly the wrong summary when the harm is concentrated on specific people who are repeatedly and predictably flagged.

The vocabulary for this is old and comes from voice, not face. In 1998, George Doddington and colleagues, analysing that year’s NIST speaker recognition evaluation, divided subjects into sheep, goats, lambs and wolves: ordinary subjects, subjects who are hard to match to themselves, subjects who are easy for others to impersonate, and subjects who are good at impersonating others. Every biometric has this zoo. The published average error rate belongs to the sheep.

Liveness detection and injection detection are different problems#

The plain version treated the printed photograph, the replayed video, the mask and the injected deepfake as a list of attacks that liveness detection handles.

The honest version: they are two families with two defences, and buying the first does not give you the second. Attacks presented to the sensor — print, screen, mask, prosthetic — are called presentation attacks, and there is an international standard for testing defences against them, which the previous chapter describes. Attacks that bypass the sensor and insert data further down the pipeline are called injection attacks, and that standard does not cover them, because by definition nothing was ever presented. A product can hold a perfect certificate for presentation attack detection and be trivially defeated by a virtual camera. As of August 2026 this is the single most common gap between what a biometric product is certified for and what it is deployed against.

Bias in face recognition is a moving quantity with a date on it#

The plain version did not mention demographic differences at all. The public conversation mentions little else, and it is usually a decade out of date in one direction or the other.

The honest version: two things are true at once, and people usually want only one of them. The first is that the differences documented in 2019 were large, real, measured on operational government photographs rather than internet images, and reproducible. The second is that the most accurate algorithms have improved enormously since then, and the gap between the best and the worst algorithms is larger than the gap between demographic groups within the best algorithm.

But — and this is where optimists overreach — improvement is not parity. On the NIST tables current as of the middle of 2026, the most accurate 1:1 algorithm in the world, at a threshold set to give a false match rate of three in ten thousand overall, still gives a false match rate roughly two hundred times higher for its worst demographic group than for its best. We will state those figures exactly below. Anybody who tells you the problem is solved has a number they are not showing you, and anybody who tells you nothing has improved has not read the tables since 2019.

Voice is not simply a weaker face#

The plain version treated voice as face-with-sound. It is not.

The honest version: a face is a physical structure that changes slowly. A voice is a behaviour produced by a physical structure, and behaviours vary enormously within one person from hour to hour. You sound different with a cold, after running, when tired, when speaking to your mother, over a mobile connection with a codec that throws away most of the frequency range. The genuine-score distribution for voice is therefore much wider than for face, which forces thresholds down, which admits more impostors.

On top of that, voice is the modality where synthetic generation matured first and fastest, and where the attack channel — a telephone call — offers the defender almost no side information. There is no depth to check, no reflection in the eye, no infrared. This is why one major standards body now simply forbids it for a class of systems, which we come to below.

The technical version#

The vocabulary, stated exactly#

The terms below are defined in ISO/IEC 2382-37, the standard vocabulary for biometrics, and used consistently by NIST. Getting them right is not pedantry; the two modes use different terms and mixing them is how bad procurement decisions happen.

For verification, one-to-one:

False match rate (FMR) is the proportion of impostor comparisons that produce a score at or above the threshold. False non-match rate (FNMR) is the proportion of genuine comparisons that produce a score below the threshold. Both are properties of the comparison subsystem at a stated threshold, on a stated dataset.

For identification, one-to-many:

False positive identification rate (FPIR) is the proportion of non-mated searches — searches for someone who is genuinely not in the gallery — that return one or more candidates at or above threshold. False negative identification rate (FNIR) is the proportion of mated searches — searches for someone who is in the gallery — that fail to return the correct person above threshold.

Note the shape of the FPIR definition carefully. It is one or more. A single search returning eleven wrong candidates counts once. This is why FPIR is a kinder statistic than it first appears, and why the number of false candidates per search matters separately.

Two more, which apply to both and which quietly exclude people:

Failure to enrol (FTE) is the proportion of subjects for whom a usable template cannot be created at registration. Failure to acquire (FTA) is the proportion of attempts where no usable sample can be captured. A system with a beautiful FNMR and a five per cent FTE has simply moved its errors somewhere that does not appear on the datasheet.

One more distinction that vendors blur. A closed-set identification assumes the person is definitely in the gallery and asks only which record; its metric is rank-1 accuracy. An open-set identification, which is what every real deployment is, makes no such assumption, needs a threshold, and reports FPIR and FNIR. A supplier quoting rank-1 accuracy for an open-set problem is quoting a number from a different problem.

From pixels to a vector, and the papers that made it work#

Every modern face pipeline has four stages, in this order, and each has its own failure modes.

Stage one is detection: find the faces in the image and return bounding boxes. The classical method here is the boosted cascade of Haar-like features published by Paul Viola and Michael Jones at CVPR in 2001, which made real-time face detection possible on ordinary hardware and dominated for a decade. Modern systems use convolutional detectors, but the job is unchanged. A detector that misses a face causes a failure to acquire, and detectors have historically missed faces at extreme angles and in poor light.

Stage two is alignment: find landmarks — eye centres, nose tip, mouth corners — and warp the face to a canonical position and size, typically a square of 112 by 112 or 160 by 160 pixels with the eyes on a fixed line. Alignment is why a matcher can tolerate a tilted head. Bad alignment is a common and invisible cause of poor scores.

Stage three is embedding: run the aligned crop through a trained network and take the output vector. This is where the identity lives.

Stage four is comparison: reduce two vectors to one similarity score.

  image
    |
    v
  [ detect ]   -> box, or nothing (failure to acquire)
    |
    v
  [ align ]    -> 112x112 crop, eyes on a fixed line
    |
    v
  [ embed ]    -> vector of 512 floats, L2 norm 1.0
    |
    v
  [ compare ]  -> cosine similarity, one number
    |
    v
  [ threshold ] -> match / no match  (1:1)
       or
  [ search ]    -> ranked candidate list (1:N)

The idea of representing a face as a vector is old. Matthew Turk and Alex Pentland turned principal component analysis of faces into a working recognition system in “Eigenfaces for Recognition”, published in the Journal of Cognitive Neuroscience in 1991, representing a face as a short list of coefficients over a set of average face-shaped images. It worked in controlled conditions and collapsed outside them.

Three later papers moved the field to where it is now.

Yaniv Taigman, Ming Yang, Marc’Aurelio Ranzato and Lior Wolf published “DeepFace” at CVPR in 2014, reporting 97.35 per cent accuracy on the Labeled Faces in the Wild benchmark — a set of internet photographs assembled by Gary Huang, Manu Ramesh, Tamara Berg and Erik Learned-Miller and released as University of Massachusetts Amherst technical report 07-49 in October 2007. DeepFace trained a deep network to classify identities and used a high-dimensional intermediate layer as the representation.

Florian Schroff, Dmitry Kalenichenko and James Philbin published “FaceNet” at CVPR in 2015. FaceNet dropped the classification step and trained directly on the geometry of the embedding space with a triplet loss: take an anchor image, a second image of the same person and an image of somebody else, and push the anchor closer to the first than to the second by a margin. The output was a 128-dimensional unit vector, and it reported 99.63 per cent on Labeled Faces in the Wild.

Jiankang Deng, Jia Guo, Niannan Xue and Stefanos Zafeiriou published “ArcFace” at CVPR in 2019, and most current systems descend from it. ArcFace normalizes both the embedding and the class weights to unit length, so all the information lives in direction rather than magnitude, then adds an angular margin to the correct class before the softmax. In the paper the embedding dimension is 512, the scale factor s is 64 and the angular margin m is 0.5. The effect is to force one identity’s embeddings into a tight cone on a hypersphere, with visible gaps between identities.

That is why the comparison is an angle. It is not an analogy. The training objective was literally angular.

Cosine similarity and where the threshold comes from#

Because the embedding is L2-normalized — scaled so its length is exactly one — the dot product of two embeddings is the cosine of the angle between them. That is the whole comparison.

e = enrolled template   (512 floats, |e| = 1.0)
p = probe template      (512 floats, |p| = 1.0)

cos_sim  = dot(e, p)          range -1.0 .. +1.0
cos_dist = 1 - cos_sim        range  0.0 ..  2.0
angle    = degrees(acos(cos_sim))

same person, same session : 0.712  ->  44.6 degrees
same person, 12 years on  : 0.478  ->  61.5 degrees
sibling                   : 0.310  ->  71.9 degrees
unrelated stranger        : 0.081  ->  85.4 degrees

Some libraries report cosine distance and some report Euclidean distance between the normalized vectors; for unit vectors the two are related by a fixed formula, so a threshold quoted in one converts to the other. What cannot be converted is a threshold quoted without saying which. A migration between two products that both default to “0.4”, meaning different things, is a recurring production incident.

The threshold itself is set empirically, never analytically. Assemble a calibration set representative of the deployment, with known identities; compute all genuine pairs and a very large number of impostor pairs; sort the impostor scores and take the threshold at the quantile matching your target false match rate; then read off the resulting FNMR from the genuine scores, because that is what your users will feel. Note the consequence: to calibrate for an FMR of one in a million you need well over a million impostor comparisons to see the tail at all, which is why a small pilot cannot calibrate a strict threshold.

The threshold is a convention inside each product, not a standard. No specification says a face match must be above 0.4. What specifications constrain is the resulting error rates.

NIST Special Publication 800-63B revision 4, the United States federal digital identity guidelines, published in July 2025 and superseding revision 3 as of 1 August 2025, states in section 3.2.3 that a biometric system used for authentication “SHALL operate with an FMR of one in 10000 or better for all demographic groups”, that demographic categories considered “SHALL include sex and skin tone when these factors affect biometric performance”, that the system “SHOULD demonstrate a false non-match rate of less than 5 %”, and — this is the part practitioners miss — that “the biometric system SHALL be configured with a fixed threshold; it is not feasible to change the threshold for each demographic”. That last clause is a rule of law inside a rule of engineering: you may not fix the demographic gap by giving different groups different thresholds. You must fix the algorithm.

What NIST actually measures, and what the tables say now#

The programme most people mean when they say “the NIST test” began in February 2017 as an ongoing rather than periodic evaluation, and has since been renamed. What was Face Recognition Vendor Test, FRVT, is now split into FRTE, Face Recognition Technology Evaluation, covering 1:1 verification, 1:N identification, twins and video, and FATE, Face Analysis Technology Evaluation, covering morphing, image quality, presentation attack detection and age estimation.

Developers submit compiled implementations against a published C++ API; NIST runs them on its own hardware against sequestered operational imagery — visa applications, border crossings, mugshots, immigration kiosks — supplied by United States government agencies. Nothing is scraped from the web. Developers may resubmit no more often than every four calendar months. As of the participation table last updated on 31 July 2026, FRTE had evaluated 1,441 verification algorithms from 439 unique developers since 2017, and, as of the 1:N table of 4 August 2026, 681 identification algorithms from 213 developers since 2018. This is the only large, adversarial, independent, continuously updated measurement of this technology in existence, and reading it is a skill worth having.

Here is the current 1:1 headline, from the performance summary last updated on 30 July 2026, following a report published on 8 May 2026. Each row is the lowest false non-match rate achieved by any developer’s leading algorithm on that image pairing, at the false match rate NIST fixes for that column.

Read the first and last rows together and you have the point of this section. On a visa photograph compared with a border photograph, the best algorithm misses about 14 genuine people in ten thousand, at a threshold strict enough that only one impostor pair in a million gets through. On a border photograph compared with an image from an immigration kiosk — a person walking up to a machine, not posing — the same class of algorithm misses about 334 in ten thousand, and that is at a threshold ten times looser. The imagery changed. The error rate changed by roughly a factor of twenty-four.

The row marked “turned” is NIST’s yaw-limited subset, where the face is rotated 45 degrees or more from frontal. It kills the intuition that a face system either works or does not: turn the head and the same software degrades by about a factor of two, even on high-quality government imagery.

One more FRTE track settles a common argument. FRTE 1:1 includes a twins measurement, at a threshold set for a false match rate of one in ten thousand on mugshots, using images collected at the Twins Days festival. Sorting that table by lowest false match rate on identical twins produces a list of algorithms that also have a failure-to-enrol rate at or near 1.0, or a mugshot false non-match rate at or near 1.0 — algorithms that score well on twins because they do not work at all. As of the leaderboard of 30 July 2026 no algorithm on that list both works and separates identical twins. Face recognition does not distinguish identical twins. That is an established fact, not a temporary limitation, and any system whose security model requires it is broken.

The 1:N tables, and why false positives scale with N#

The FRTE 1:N tables, last updated on 4 August 2026, report false negative identification rate at a threshold chosen to hold false positive identification rate at 0.003 — three false-positive searches in a thousand non-mated searches. The galleries are not toys. The mugshot-to-mugshot column enrols 12,000,000 identities; most other columns enrol 1,600,000; the long-time-gap mugshot column enrols 3,000,000. Each person is enrolled with one image only.

The best figures on that table, for the leading algorithm from each developer, look like this.

Two things in that table deserve comment. The first is that searching twelve million mugshots for a mugshot misses the right person five times in ten thousand, which is genuinely remarkable and is why deduplication of large identity registers works. The second is the same lesson as before in a new place: change the probe from a mugshot to a kiosk capture or a profile view and the miss rate rises by a factor of about eighty, on the same gallery, with the same false positive budget. The gallery is not what breaks these systems. The probe image is.

Now the arithmetic that matters most, done properly.

Take a system in which the per-comparison false match rate at the chosen threshold is p. A search compares the probe against N enrolled templates. If those comparisons were independent, the probability that at least one of them falsely exceeds the threshold is:

FPIR = 1 - (1 - p)^N

and for small p*N this is very close to:

FPIR ~ p * N

With p fixed at one in a million, the table below is what happens as N grows. These are exact evaluations of the formula, not estimates.

At a gallery of twelve million, holding the threshold where it was, essentially every single search returns at least one false candidate. That is not a defect of a bad product. It is what the formula says must happen.

Which is why NIST fixes FPIR rather than the threshold. To hold FPIR at 0.003 while N grows from 1,600,000 to 12,000,000, an operator must raise the threshold by enough to cut p by roughly a factor of seven and a half, and raising the threshold raises FNIR. The reason the very best algorithms show only a small penalty going from 1.6 million to 12 million on mugshots — 0.0005 either way in the table above — is that their genuine and impostor score distributions are so far apart that a large threshold increase costs almost no genuine matches. That property does not survive contact with poor imagery, which is exactly where deployments live.

The honest version: the independence assumption behind the formula is false, and it is false in a direction that matters. Impostor scores are not identically distributed. A probe that resembles a common facial type draws false candidates in clusters; a probe with unusual features draws none. So the formula predicts the average correctly and predicts nobody’s individual experience correctly. The people who are repeatedly flagged are repeatedly flagged, and they are not a random selection of the population.

The worked example, carried through: a city transport watchlist#

Let us take this all the way through with one deployment and keep it.

A city transport authority runs a metro network. It proposes two face systems.

The first is at the staff depot: 4,000 employees, each with a card. Present card, look at camera, gate opens. This is 1:1. Each employee passes twice a day, so 8,000 comparisons a day, each against exactly one template. Set the threshold for an FMR of one in a million and, apart from the small number of employees who will be refused because of image quality, nothing dramatic happens. An impostor with a stolen card has a one in a million chance per attempt. This system is defensible.

The second is a watchlist on 60 platform cameras. The gallery is 900 people — a mix of persons barred from the network and persons wanted by police. The cameras see 40,000 usable faces a day.

Set the same threshold, p equal to one in a million. Then per search:

N        = 900
p        = 0.000001
FPIR     = 1 - (1 - p)^900 = 0.0008996

searches per day       = 40,000
false alerts per day   = 40,000 * 0.0008996 = 36

Thirty-six false alerts a day. Now the other side of the ledger. Suppose the 900 people on the list genuinely appear on the network twice a day between them. Platform camera imagery is worse than the kiosk imagery in the NIST tables, so take a false negative identification rate of 0.30, which is generous. Then real detections are 2 times 0.70, which is 1.4 a day.

true alerts  per day = 1.4
false alerts per day = 36.0
precision = 1.4 / (1.4 + 36.0) = 3.7 per cent

Ninety-six alerts in every hundred are wrong. Every one of them is a person stopped by staff or police on a platform.

Now tighten the threshold to bring false alerts down to about one a day. That needs p of about 2.5 in a hundred million:

p        = 0.000000025
FPIR     = 900 * p = 0.0000225
false alerts per day = 40,000 * 0.0000225 = 0.9

Better. But moving the threshold that far up the impostor tail moves it a long way up the genuine distribution too. Take FNIR rising to 0.55, which for that imagery is realistic. Detections fall to 0.9 a day.

The tight setting gives you a system that is right half the time and finds fewer than one wanted person a day. The loose setting gives you a system that is wrong ninety-six times in a hundred. There is no third column where the system is both accurate and useful, because the base rate is against you: the fraction of passers-by who are actually on the list is about one in twenty thousand, and no threshold repeals that.

This is the arithmetic that should be on the first page of every watchlist procurement document, and it almost never is.

Demographic differentials: 2019 stated exactly, 2026 measured#

On 19 December 2019, NIST published NISTIR 8280, “Face Recognition Vendor Test Part 3: Demographic Effects”, by Patrick Grother, Mei Ngan and Kayee Hanaoka. It evaluated 189 algorithms from 99 developers against four collections totalling 18.27 million images of 8.49 million people, drawn from operational databases held by the United States Department of State, Department of Homeland Security and Federal Bureau of Investigation. No web-scraped or surveillance images were used.

The findings, stated as the report and the accompanying NIST summary state them:

For one-to-one matching, false positive rates were higher for Asian and African American faces relative to Caucasian faces, with the differentials often ranging from a factor of 10 to a factor of 100, depending on the algorithm.

Among algorithms developed in the United States, there were similarly elevated false positive rates in one-to-one matching for Asian, African American and native groups, the latter including Native American, American Indian, Alaskan Indian and Pacific Islander subjects. The American Indian demographic showed the highest false positive rates.

A number of algorithms developed in Asian countries showed no such dramatic difference between Asian and Caucasian faces. NIST explicitly declined to claim causation and noted that the relationship between training data and outcome is an area for research, with Grother quoted as saying the results are “an encouraging sign that more diverse training data may produce more equitable outcomes”.

False positives were higher in women than in men across datasets and algorithms, and elevated in the elderly and in children, with the effects largest at the extremes of age.

For one-to-many matching, higher false positive rates were found for African American females. This finding used one FBI database of 1.6 million domestic mugshots. NIST singled it out because a false positive in a search can result in a false accusation, where a false negative in a verification is an inconvenience.

Not all algorithms showed the one-to-many differential, and those that were the most equitable also ranked among the most accurate.

That last sentence is the one that gets left out of both the alarmed and the dismissive summaries, and it is the most useful sentence in the report. The differential is a property of individual algorithms, not of the technology as a category.

NIST followed this in July 2022 with NISTIR 8429, “Summarizing Demographic Differentials”, which proposed summary measures for reporting the spread: the ratio of the maximum group error rate to the geometric mean across groups, the max-over-min ratio, and a Gini coefficient describing how concentrated the errors are. Those measures are now reported routinely on the FRTE pages.

Here is where the argument gets settled with current data rather than memory.

The FRTE 1:1 demographics table, last updated on 30 July 2026, sets each algorithm’s threshold to give an overall false match rate of 0.0003 — three in ten thousand — and then measures the false match rate within demographic cells defined by region of birth, sex and age group, comparing only high-quality frontal portraits of two people in the same cell. The figures below are read from that table.

For every row above except the last, the lowest cell is East European men aged 20 to 35 or 35 to 50, and the highest cell is West African women aged 65 to 99.

Take the first row, which belongs to the algorithm that also holds the top rank on the main 1:1 accuracy table as of that date. At a single fixed threshold: a false match rate of 0.00012 for young East European men, which is about one in 8,300; and a false match rate of 0.02510 for West African women over 65, which is about one in 40. That is a ratio of about 209 to one, in the best algorithm publicly measured, in 2026.

The same table reports NIST’s summary measures for that algorithm: a maximum-over-geometric-mean of 28.8 and a Gini coefficient of 0.66, where 1.0 would be parity for the first measure and 0 would be perfect equality for the second.

Two honest qualifications, both of which matter.

First, the last row is real. The stcon_009 algorithm reaches a worst-cell false match rate of 0.00120, about one in 830, roughly twenty times better in the worst cell than the top-ranked algorithm, while remaining near the top for overall accuracy. Parity and accuracy are not in opposition. Developers who work on it get it.

Second, the pattern is not universal. The paravision_020 entry on the same table has its highest cell at East European men aged 12 to 20, not West African women. The direction of the differential is a property of a specific trained model, which is precisely NIST’s point and precisely what gets lost when the finding is summarized as “face recognition is racist”. The accurate summary is: most algorithms exhibit large demographic differentials in false match rate, the direction and size vary by algorithm, the best available algorithms are far better than the average, and none of them has achieved parity.

Presentation attacks: print, replay, mask#

An attack presented to the sensor is a presentation attack, and the artefact used is a presentation attack instrument. ISO/IEC 30107-1:2023 gives the framework and ISO/IEC 30107-3 gives the testing methodology; the previous chapter covers what a laboratory certificate against that standard does and does not promise, and we will not repeat it.

What is specific to face is the ladder of instruments and what each defeats.

A printed photograph is defeated by almost any modern liveness check, because paper reflects light differently from skin and has no depth. A printed photograph with the eyes cut out, held over the attacker’s face, defeats eye-movement checks.

A screen replay — a phone or tablet playing a video of the victim — defeats challenge-response checks that ask for a blink or a smile, if the attacker has video of the victim blinking or smiling. It is often caught by moiré patterns from the screen’s pixel grid, by the screen’s refresh flicker, and by the colour gamut of a display differing from real skin under real light.

A 3D mask, whether a resin mask from a photograph or a silicone mask made from a cast, defeats depth-based checks entirely. The best current defences are material-based: skin scatters near-infrared light in a characteristic way that silicone does not, and blood flow produces a small periodic colour change in real skin that a mask cannot reproduce.

A partial prosthetic, altering only the parts of the face the algorithm weights most heavily, is the least studied and, for a determined attacker, the most practical.

Face has one more attack that other biometrics do not, and it belongs here. Morphing takes photographs of two people and blends them into a single image that matches both above threshold. Submit the morph as a passport photograph and two people share one document. NIST runs a dedicated track for this, FATE MORPH, precisely because it defeats a 1:1 system that is working perfectly: nothing is spoofed at the gate, because the enrolled reference itself is the attack.

Injection attacks: the camera that is not there#

Now the attack that has reshaped remote identity verification since 2023.

ISO/IEC 30107 concerns itself with presentation attacks — with what happens at the capture device. An injection attack does not happen at the capture device. The attacker substitutes the media stream somewhere between the sensor and the decision, and there is therefore nothing for a presentation attack detector to detect.

  real face
     |
     v
  [ lens and sensor ]  <- print, screen replay, mask
     |                    PRESENTATION attacks, in scope
     v                    of ISO/IEC 30107-3
  [ camera driver ]    <- virtual camera device
     |
     v
  [ app / capture SDK ] <- hooked library, emulator,
     |                     rooted or jailbroken OS
     v
  [ TLS to server ]    <- replayed or forged upload,
     |                    stolen session, scripted API
     v
  [ liveness + matcher ]

The three injection points in that diagram are three different engineering problems.

A virtual camera is a software device that registers with the operating system as a camera and returns frames from a file or a live face-swap. On desktop operating systems this requires no privilege at all; virtual camera software is ordinary, legitimate, widely installed video-conferencing tooling. On mobile operating systems it requires compromising the device first, which is why attackers use emulators pretending to be phones, and why the attack volume against mobile targets has climbed as emulator tooling has improved.

A hooked SDK replaces or instruments the vendor’s capture library inside the app, so that the app believes it captured a frame that in fact came from the attacker.

A replayed or forged upload skips the client entirely. If the verification server accepts an image or video over an API, an attacker who can produce a valid session can simply post a generated video.

Vendor threat reporting belongs in the marketing claim category and must be read as such, but the direction is consistent across sources. The iProov Threat Intelligence Report published in 2025, covering 2024, reported a 2,665 per cent year-on-year increase in what it calls native virtual camera attacks and a 300 per cent increase in face-swap attacks; its 2026 report described a 741 per cent annual increase in injection attacks, with a 1,151 per cent surge against iOS targets in the second half of 2025. Treat the percentages as vendor-measured and the trend as real.

What defeats injection is not better image analysis. It is evidence about the origin of the media. Four defences, in increasing order of strength:

1. Client and environment integrity. Attest that the app is unmodified and the operating system is not rooted or emulated, using platform attestation services, and refuse enrolment from environments that will not attest. This is the cheapest and it is bypassable by a sufficiently resourced attacker.
2. Server-issued, one-time, unpredictable challenge. The server generates a challenge the attacker cannot know in advance, and the response must be produced during the capture. A pre-rendered deepfake video cannot contain the answer to a question that had not been asked when it was rendered.
3. Active illumination. The strongest widely deployed form of the above: the server chooses a random sequence of colours, the client’s screen illuminates the subject’s face with that sequence, and the returned video must show that exact sequence reflecting off a three-dimensional face at the right times. Reproducing it requires rendering the deepfake in real time with correct scene lighting and returning it inside the round-trip window. iProov’s implementation of this is called Flashmark; other vendors have equivalents. It is an implementation detail of specific products, but the principle is general and it is the right principle.
4. Cryptographic binding of the capture to hardware. The strongest possible answer: the sensor or a secure element signs the frames, and the server verifies the signature. Almost nothing supports this today. It is the direction the field is moving and it is what “replace the camera as the unit of evidence with an issuer-signed attestation” means when you hear it at conferences.

The federal position has caught up. NIST Special Publication 800-63B revision 4 has a subsection headed “Injection Attack Detection” in section 3.2.3, and it requires that the verifier “SHOULD determine the performance and integrity of the sensor and its associated endpoint”, and imposes hard limits on consecutive failed biometric attempts — no more than five, or ten where conformant presentation attack detection is implemented, then a delay of at least 30 seconds before each further attempt, with an overall cap of 50 consecutive failures, or 100 with presentation attack detection, after which biometric authentication must be disabled and another factor offered.

Voice biometrics, and the modality NIST now forbids#

A voice biometric system embeds a recording as a vector, exactly as a face system embeds an image, and compares by cosine similarity. The vector is usually called a speaker embedding; the dominant family since about 2018 is the x-vector and its descendants, produced by a time-delay or convolutional network over spectral features with a pooling layer that turns a variable-length utterance into a fixed-length vector.

Two deployment modes:

Text-dependent verification requires a fixed phrase, agreed at enrolment. Banks used “my voice is my password” for many years. The advantage is that the acoustic content is controlled, so the comparison is easier and shorter recordings suffice — two or three seconds is enough. The disadvantage is fatal in an era of synthesis: the attacker knows exactly what sentence to generate.

Text-independent verification works on whatever the speaker happens to say, and is used for passive checking during a call. It needs far more audio — typically 10 to 30 seconds of speech — and it is harder, but it cannot be defeated by generating one known phrase.

The research community has measured the spoofing problem since 2015 through the ASVspoof challenge series, run by researchers from EURECOM, the University of Eastern Finland, the National Institute of Informatics in Japan, Carnegie Mellon University and others. Editions ran in 2015, 2017, 2019 and 2021, and ASVspoof 5 ran through 2023 and 2024, with evaluation scores submitted in July 2024 and results presented at the ASVspoof Workshop held alongside Interspeech on 31 August 2024. It had two tracks: standalone countermeasures, and spoofing-aware speaker verification, where the countermeasure and the verifier are assessed jointly. Assessing them jointly matters, because a countermeasure that is excellent alone can be useless when composed with a verifier whose errors are correlated with it.

The operational record is worse than the research record, and it is documented. In 2017 a BBC Click reporter’s non-identical twin defeated HSBC’s Voice ID by imitation, with no synthesis at all. In February 2023 the journalist Joseph Cox reported in Vice that he had entered his own Lloyds Bank account using an AI-generated clone of his voice. On 28 November 2024 the BBC’s You and Yours programme reported that its presenter Shari Vahl had cloned her own voice from a radio interview and used it to pass the voice authentication of Santander and Halifax, first through studio speakers and then through an ordinary tablet speaker at her kitchen table.

Alongside these sits the fraud record. On 30 August 2019 the Wall Street Journal reported what is widely treated as the first known case of AI voice cloning used for fraud: in March of that year the chief executive of a United Kingdom energy firm transferred 220,000 euros, about 243,000 US dollars, after a caller reproduced the voice, German accent and speech melody of the chief executive of the German parent company, an account given by Rüdiger Kirsch of the insurer Euler Hermes. Voice cloning is now part of ordinary business fraud: in February 2024 Hong Kong police described a case in which an employee of a multinational was persuaded, on a video conference where every other participant was synthetic, to transfer HK$200 million, about 25.6 million US dollars, across 15 transactions to five accounts. The company confirmed in May 2024 that it was the engineering firm Arup.

Which brings us to the sentence that ought to end most voice biometric procurement conversations. NIST Special Publication 800-63B revision 4, section 3.2.3, in the paragraph on presentation attack detection, states plainly: “Biometric comparison based on voice SHALL NOT be used.” Not “should be used carefully”. Not “requires additional controls”. A flat prohibition, in the current United States federal digital identity guidelines as of August 2026, for any system claiming conformance.

That is a standard, and it applies to systems that must conform to SP 800-63. It does not make voice biometrics illegal, and banks continue to use them, generally arguing — as Santander and Halifax both did to the BBC — that voice is one layer among several and that they have seen no fraud attributable to it. That argument is not absurd. It is also not measurable from outside, and the honest position is that experts disagree: one camp holds that voice adds real friction to bulk fraud at low cost, the other that a factor which a free tool defeats in minutes is not a factor at all.

Where it is restricted, and what happens where it is not#

Regulation of this technology is uneven, fast-moving and easy to misreport. What follows is accurate as of August 2026.

In the European Union, the AI Act, Regulation (EU) 2024/1689, entered into force on 1 August 2024. Its Article 5 prohibitions became applicable on 2 February 2025 and include four that bear directly on this chapter: Article 5(1)(e) prohibits creating or expanding facial recognition databases through untargeted scraping of facial images from the internet or CCTV footage; Article 5(1)(f) prohibits inferring emotions in the workplace and in education institutions, except for medical or safety reasons; Article 5(1)(g) prohibits biometric categorization systems that deduce race, political opinions, trade union membership, religious or philosophical beliefs, sex life or sexual orientation; and Article 5(1)(h) prohibits real-time remote biometric identification in publicly accessible spaces for law enforcement, subject to three narrow exceptions — searching for victims of trafficking or missing persons, preventing a specific and imminent threat to life or a terrorist attack, and locating suspects in serious offences — with prior judicial or independent administrative authorization required.

Those prohibitions have not moved. What did move is elsewhere in the Act. The Digital Omnibus on AI, Regulation (EU) 2026/1744, was published in the Official Journal on 24 July 2026 and entered into force on 27 July 2026, six days before the AI Act’s original high-risk deadline. It defers the obligations for Annex III high-risk systems to 2 December 2027 and for Annex I systems to 2 August 2028. It leaves the Article 5 prohibitions untouched, and adds a new Article 5 prohibition applying from 2 December 2026. If somebody tells you the EU has delayed its face recognition ban, they have confused the high-risk regime with the prohibition regime.

In the United States there is no federal statute, and the action is in state law, city ordinances and litigation. Illinois enacted the Biometric Information Privacy Act in 2008; it requires written notice and consent before collecting a biometric identifier, and it is one of very few such laws with a private right of action, which is why nearly all the case law is from Illinois. In “Rosenbach v. Six Flags Entertainment Corp.”, 2019 IL 123186, decided in January 2019, the Illinois Supreme Court held that a person need not plead actual injury beyond the statutory violation to sue. In “Cothron v. White Castle System, Inc.”, 2023 IL 128004, decided on 17 February 2023 and modified on denial of rehearing on 18 July 2023, the same court held that a claim accrues on each scan, not only the first — a holding that, with statutory damages of 1,000 dollars for negligent and 5,000 dollars for reckless or intentional violations, created the possibility of ruinous awards. The legislature answered with Senate Bill 2979, signed in August 2024, which makes repeated collection of the same identifier from the same person by the same entity using the same method a single violation; the Seventh Circuit held on 1 April 2026 that the amendment applies retroactively.

The settlements set the commercial stakes. Facebook settled the Illinois tag-suggestion class action for 650 million dollars, approved in February 2021. BNSF Railway settled for 75 million dollars after a jury verdict and a new trial on damages. Clearview AI, which built a face search engine from scraped web images, settled the consolidated federal litigation with an unusual instrument: rather than cash it does not have, the class received a 23 per cent equity stake in the company, valued at approximately 51.75 million dollars as of January 2024, with final approval granted on 20 March 2025 in the Northern District of Illinois. Texas brought its own actions under the Capture or Use of Biometric Identifier Act and settled with Meta for 1.4 billion dollars in July 2024, payable over five years.

Regulators elsewhere have gone after Clearview directly. The Dutch data protection authority fined it 30.5 million euros in September 2024. In the United Kingdom, the Information Commissioner’s Office fined Clearview 7.5 million pounds in May 2022; the First-tier Tribunal allowed Clearview’s appeal in October 2023, holding the company outside the ICO’s jurisdiction; and on 6 October 2025 the Upper Tribunal allowed the ICO’s appeal and remitted the case, holding that the processing falls within the scope of United Kingdom data protection law.

The Federal Trade Commission banned Rite Aid from using facial recognition for surveillance for five years, in an order announced on 19 December 2023, after finding that the retailer’s system, run from 2012 to 2020, generated thousands of false positives disproportionately affecting people of colour. In Australia, the Office of the Australian Information Commissioner determined in November 2024 that the retailer Bunnings had breached the Privacy Act by collecting sensitive biometric information without consent; Bunnings sought review, and in February 2026 the Administrative Review Tribunal affirmed aspects of the determination, after which the Privacy Commissioner published updated guidance on facial recognition in retail spaces.

Two American examples show the shape of what regulation actually does, as opposed to what it is reported to do. San Francisco’s Board of Supervisors voted on 14 May 2019 to bar city departments from obtaining, retaining, accessing or using face recognition technology — the first such ban by a United States city, and one that reaches only city agencies, not state or federal agencies operating in the city and not private companies. Virginia went the other way and then partly back: a 2021 statute barred local law enforcement from using the technology at all, and Senate Bill 741, enacted on 27 April 2022, replaced the ban with a list of authorized uses plus a rule with real teeth — a facial recognition match “shall not be included in an affidavit to establish probable cause for purposes of issuance of a search warrant or an arrest warrant but shall be admissible as exculpatory evidence”.

That Virginia clause is the most quietly important sentence in this section, because it encodes the arithmetic of the previous pages into law. A 1:N hit is a lead, not evidence. It may exonerate; it may not incriminate.

The case law above is about privacy. The human record is about false positives in 1:N, and it is short enough to state.

As of an ACLU account published on 14 April 2026, at least fourteen people in the United States are publicly known to have been wrongfully arrested because police relied on an erroneous facial recognition result. Named cases include Nijeer Parks, arrested in Woodbridge, New Jersey in February 2019, who spent ten days in jail while police did not wait for fingerprint and DNA results that pointed elsewhere; Michael Oliver, arrested in Detroit in July 2019, who had full tattoo sleeves the suspect did not; Robert Williams, arrested in Detroit in January 2020, the first publicly reported case; Randal Quran Reid, arrested in Georgia in November 2022 on a Louisiana warrant for a crime in a state he had never visited; Porcha Woodruff, arrested in Detroit in February 2023 while eight months pregnant, for a carjacking whose perpetrator was not; and Trevis Williams, arrested in New York City in April 2025, eight inches taller and seventy pounds heavier than the suspect.

The mechanism is the same every time and it is not a mystery. A search returns a candidate list. A human looks at it and sees a face that resembles the suspect, because that is what the search was designed to produce. The candidate then goes into a photographic lineup surrounded by fillers who resemble the suspect less, and a witness picks the one that looks most like the person they remember. The lineup appears to corroborate the search, but it cannot: the lineup was built from the search’s own output. The ACLU counts at least seven wrongful arrests produced by that closed loop.

Robert Williams sued the City of Detroit with the ACLU in 2021. The case settled in June 2024, with 300,000 dollars and departmental restrictions widely described as the strongest in the United States: face recognition results may serve only as investigative leads; police may not go directly from a result to a witness lineup; arrest warrants may not rest on a result; and past cases from 2017 to 2023 had to be reviewed, with prosecutors notified where an arrest lacked independent evidence.

Every one of those rules is the arithmetic of the previous sections written as police procedure.

Building one honestly: the short list#

If you are responsible for deploying a face system, the following questions decide whether it is defensible. They are ordered by how often they are skipped.

1. Which mode is this, 1:1 or 1:N. If anybody in the room is unsure, stop.
2. If 1:N, what is N today, what will N be in three years, and what is the FPIR at your chosen threshold, computed rather than asserted.
3. What is the prior probability that a given probe is genuinely in the gallery. Multiply it out. If the resulting precision is under fifty per cent, you are building an accusation machine.
4. On what imagery was the accuracy figure measured, and does it resemble yours. A number from visa photographs does not transfer to a ceiling camera.
5. What is the false match rate for your worst demographic cell, not overall. Demand the demographic table. NIST SP 800-63B revision 4 requires one in ten thousand or better for all demographic groups, at a single fixed threshold.
6. What is the failure-to-enrol rate, and who is in it. That number is where exclusion hides.
7. Is presentation attack detection tested to ISO/IEC 30107-3, and separately, is there injection attack detection with a server-issued unpredictable challenge. Two questions, two answers.
8. Where is the template stored, who can read it, and what is the deletion policy. A face template is personal data under the GDPR, sensitive information under the Australian Privacy Act and a biometric identifier under BIPA.
9. What is the human procedure on a hit, and does it forbid going straight from a candidate list to a witness identification.
10. What is the appeal route for the person the system refuses, and how long does it take. If the answer is “come back tomorrow”, the system has an exclusion rate nobody is measuring.

The next chapter, “The Key Pair”, introduces the one mechanism in this book that has none of these problems: a secret that never leaves your possession, that a stranger can verify without ever holding it, and that you can revoke and replace on a Tuesday afternoon. After twenty chapters of measurements and thresholds, the relief of that idea is considerable.

20.98 Common wrong ideas#

Wrong: Face recognition is 99.9 per cent accurate. Right: There is no single accuracy figure. On the NIST FRTE tables current in mid-2026, the same class of leading algorithm has a false non-match rate of 0.0014 on visa-to-border comparisons at a false match rate of one in a million, and 0.0334 on border-to-kiosk comparisons at a false match rate ten times looser — about twenty-four times worse, for the same software, purely because the photographs are taken differently. Accuracy is a table indexed by imagery, threshold, mode, gallery size and demographic group.

Wrong: 1:1 and 1:N are the same technology used two ways, so the accuracy carries over. Right: They share an embedding model and nothing else that matters. A 1:1 comparison makes one comparison; a 1:N search makes N. At a per-comparison false match rate of one in a million, a search of 12 million records produces at least one false candidate on essentially every search. The metrics are different too: FMR and FNMR for 1:1, FPIR and FNIR for 1:N, and quoting one where the other belongs is the most common error in procurement.

Wrong: A 97 per cent match score means there is a 97 per cent chance it is the same person. Right: The score is a rescaled cosine similarity between two unit vectors, not a probability of identity. Whether a match is correct depends on the threshold, the image quality, the demographic group, the gallery size and the prior probability that the person is in the gallery at all. In this chapter’s worked watchlist, alerts at a strict threshold were correct 3.7 per cent of the time.

Wrong: The 2019 NIST demographic findings have been fixed by better training data. Right: They have been substantially reduced and not eliminated. On the FRTE 1:1 demographics table of 30 July 2026, the top-ranked algorithm, at one fixed threshold giving three false matches in ten thousand overall, gives about one in 8,300 for East European men aged 20 to 35 and about one in 40 for West African women aged 65 to 99 — a ratio of about 209. A different algorithm on the same table reaches a worst cell of about one in 830, so parity is achievable; it has not been achieved.

Wrong: Our liveness detection is certified to ISO/IEC 30107-3, so deepfakes cannot get through. Right: That standard covers presentation attacks — artefacts presented to the sensor. An injection attack replaces the media stream behind the sensor, through a virtual camera, a hooked capture library or a forged API upload, and nothing is presented at all. A perfect presentation attack detection certificate says nothing about injection resistance, which requires client attestation, server-issued unpredictable challenges, active illumination or hardware-signed capture.

Wrong: A face check proves that the right person was physically present. Right: Morphing attacks blend two people’s photographs into one image that matches both above threshold, so the enrolled reference itself can be the attack and the gate can be working perfectly at the moment of the fraud; NIST runs a dedicated FATE MORPH track for this. Separately, no algorithm on NIST’s twins table both works and distinguishes identical twins.

Wrong: Voice biometrics are a reasonable second factor because a fraudster cannot copy your voice. Right: A usable clone can be built from a few seconds of recorded speech. A BBC reporter’s twin defeated HSBC’s system by imitation in 2017; a journalist entered his Lloyds Bank account with a synthetic clone in February 2023; a BBC reporter passed both Santander’s and Halifax’s voice authentication with a clone played through a tablet speaker, as reported on 28 November 2024. NIST SP 800-63B revision 4, section 3.2.3, states that biometric comparison based on voice SHALL NOT be used.

Wrong: The EU AI Act’s ban on real-time face recognition in public was delayed by the 2026 omnibus. Right: The Article 5 prohibitions, including untargeted scraping of facial images, emotion inference at work and in education, biometric categorization by protected characteristic, and real-time remote biometric identification for law enforcement, have applied since 2 February 2025 and were not changed. The Digital Omnibus on AI, Regulation (EU) 2026/1744, in force from 27 July 2026, deferred the high-risk obligations to December 2027 and August 2028. Two different regimes.

Wrong: A facial recognition match is evidence that police can act on. Right: It is an investigative lead with a false positive rate that scales with the database. Virginia’s Senate Bill 741, enacted on 27 April 2022, provides that a facial recognition match shall not be included in an affidavit to establish probable cause for a warrant but shall be admissible as exculpatory evidence. As of an ACLU account of 14 April 2026, at least fourteen people in the United States are publicly known to have been wrongfully arrested on such matches, in at least seven cases via a lineup constructed from the search’s own output.

Wrong: Storing an embedding rather than a photograph means the data is not really biometric data. Right: It is a biometric identifier under Illinois BIPA, personal data and a special category under the GDPR, and sensitive information under the Australian Privacy Act. Regulators have fined on that basis: 30.5 million euros against Clearview AI by the Dutch authority in September 2024, and a 1.4 billion dollar settlement between Texas and Meta in July 2024. The storage format changes the engineering, not the legal character.

20.99 Chapter summary in 20 lines#

1. Face recognition is not one technology and does not have one accuracy, and the difference between its two modes is the most important thing about it.
2. Verification, written 1:1, checks a claimed identity with a single comparison, and is what a phone unlock or a passport gate does.
3. Identification, written 1:N, searches a gallery of N records with no claim made, and is what a watchlist, a mugshot search or a deduplication run does.
4. A face is converted into an embedding: a vector of a few hundred numbers, commonly 512, normalized to unit length, whose individual components mean nothing on their own.
5. Two embeddings are compared by cosine similarity, which is the cosine of the angle between them, and a threshold chosen by a human turns that number into a decision.
6. The lineage of the modern embedding runs from Eigenfaces in 1991, through DeepFace in 2014 and FaceNet’s 128-dimensional triplet-loss embedding in 2015, to ArcFace’s 512-dimensional angular-margin embedding in 2019.
7. NIST measures this technology continuously through FRTE and FATE, having started an ongoing evaluation in February 2017, and had evaluated 1,441 verification algorithms from 439 developers by the participation table of 31 July 2026.
8. On the FRTE 1:1 summary of 30 July 2026, the best false non-match rate is 0.0014 for visa-to-border images at a false match rate of one in a million, and 0.0334 for border-to-kiosk images at one in a hundred thousand.
9. On the FRTE 1:N tables of 4 August 2026, the best false negative identification rate searching a gallery of 12,000,000 mugshots is 0.0005, at a fixed false positive identification rate of 0.003.
10. In a 1:N search the chance of at least one false candidate is one minus the quantity one minus p raised to the power N, which for small values is approximately p times N, so false positives scale with gallery size.
11. At a per-comparison false match rate of one in a million, a gallery of 1,000 gives a 0.001 chance of a false candidate per search, a gallery of 100,000 gives 0.095, and a gallery of 12,000,000 gives 0.999994.
12. The worked watchlist example in this chapter, with 900 gallery entries and 40,000 faces a day, produces 36 false alerts a day against 1.4 real detections, a precision of 3.7 per cent, and tightening the threshold to one false alert a day halves the detections.
13. NISTIR 8280, published on 19 December 2019, evaluated 189 algorithms from 99 developers on 18.27 million images of 8.49 million people from operational United States government databases.
14. It found one-to-one false positives 10 to 100 times higher for Asian and African American faces than Caucasian faces on most algorithms, higher rates for women, elevated rates at the extremes of age, no such gap for several algorithms developed in Asian countries, and, for one-to-many search against 1.6 million FBI mugshots, elevated false positives for African American females.
15. On the FRTE demographics table of 30 July 2026, the top-ranked algorithm at a fixed overall false match rate of 0.0003 gives 0.00012 for East European men aged 20 to 35 and 0.02510 for West African women aged 65 to 99, a ratio of about 209, while another algorithm on the same table reaches a worst cell of 0.00120.
16. Presentation attacks put an artefact in front of the sensor — print, screen replay, silicone mask, partial prosthetic, or a morph enrolled as the reference — and are covered by ISO/IEC 30107-1:2023 and 30107-3.
17. Injection attacks bypass the sensor entirely through a virtual camera, a hooked capture library or a forged upload, are outside the scope of ISO/IEC 30107, and are defeated only by client attestation, server-issued unpredictable challenges, active illumination or hardware-signed capture.
18. Voice biometrics come in text-dependent and text-independent forms, have been measured against synthesis by the ASVspoof challenges since 2015 through ASVspoof 5 in 2024, and have been defeated in public by journalists against Lloyds Bank in February 2023 and against Santander and Halifax as reported on 28 November 2024.
19. NIST SP 800-63B revision 4, published in July 2025 and effective from 1 August 2025, requires a false match rate of one in 10,000 or better for all demographic groups at a single fixed threshold, mandates presentation attack detection for facial recognition, and states that biometric comparison based on voice SHALL NOT be used.
20. Legally, EU AI Act Article 5 has banned untargeted facial image scraping, workplace emotion inference, protected-characteristic biometric categorization and most real-time remote biometric identification since 2 February 2025 and was untouched by the Digital Omnibus of 27 July 2026, while Illinois BIPA, Virginia’s rule that a match is a lead and not probable cause, the FTC’s five-year Rite Aid ban of December 2023 and Australia’s Bunnings determination of November 2024 mark the shape of restriction elsewhere.

Chapter sources: NISTIR 8280, “Face Recognition Vendor Test (FRVT) Part 3: Demographic Effects”, Grother, Ngan and Hanaoka, 19 December 2019, with the NIST news release of the same date; NISTIR 8429, “Summarizing Demographic Differentials”, July 2022; the NIST FRTE 1:1 Verification accuracy, demographic variation and twins tables last updated 30 July 2026, participation statistics of 31 July 2026 and the FRTE 1:1 report of 8 May 2026; the FRTE 1:N Identification tables last updated 4 August 2026; the FRTE and FATE programme pages covering the 1:1, 1:N, Twins, FIVE, MORPH, Quality, PAD and Age Estimation tracks; NIST Special Publication 800-63B revision 4 section 3.2.3 “Use of Biometrics” and its Biometric Accuracy, Presentation Attack Detection and Injection Attack Detection subsections, published July 2025 and superseding revision 3 on 1 August 2025; ISO/IEC 2382-37 for vocabulary and ISO/IEC 30107-1:2023 and 30107-3 for presentation attack detection; Turk and Pentland, “Eigenfaces for Recognition”, Journal of Cognitive Neuroscience, 1991; Viola and Jones, “Rapid Object Detection using a Boosted Cascade of Simple Features”, CVPR 2001; Huang, Ramesh, Berg and Learned-Miller, “Labeled Faces in the Wild”, University of Massachusetts Amherst technical report 07-49, October 2007; Taigman, Yang, Ranzato and Wolf, “DeepFace”, CVPR 2014; Schroff, Kalenichenko and Philbin, “FaceNet”, CVPR 2015; Deng, Guo, Xue and Zafeiriou, “ArcFace”, CVPR 2019; Doddington and colleagues, “Sheep, Goats, Lambs and Wolves”, NIST 1998 speaker recognition evaluation; the ASVspoof evaluation plans and the ASVspoof 5 schedule ending at the workshop held with Interspeech on 31 August 2024; Regulation (EU) 2024/1689, the AI Act, Article 5(1)(e) to 5(1)(h), applicable from 2 February 2025, and Regulation (EU) 2026/1744, the Digital Omnibus on AI, in the Official Journal of 24 July 2026 and in force from 27 July 2026; the Illinois Biometric Information Privacy Act of 2008, “Rosenbach v. Six Flags Entertainment Corp.”, 2019 IL 123186, “Cothron v. White Castle System, Inc.”, 2023 IL 128004, Illinois Senate Bill 2979 of August 2024 and the Seventh Circuit retroactivity opinion of 1 April 2026; the Clearview AI settlement approved in the Northern District of Illinois on 20 March 2025, the Dutch authority’s 30.5 million euro Clearview decision of September 2024 and the United Kingdom Upper Tribunal judgment of 6 October 2025; the Texas Attorney General’s 1.4 billion dollar Meta settlement of July 2024 under the Capture or Use of Biometric Identifier Act; the Federal Trade Commission’s Rite Aid order of 19 December 2023; the Office of the Australian Information Commissioner’s Bunnings determination of November 2024 and the Administrative Review Tribunal decision of February 2026; Virginia Code section 15.2-1723.2 as enacted by Senate Bill 741 on 27 April 2022; San Francisco’s Stop Secret Surveillance ordinance of 14 May 2019; the ACLU wrongful-arrest account of 14 April 2026 and the June 2024 settlement in Williams v. City of Detroit; the Wall Street Journal report of 30 August 2019; Vice’s report of February 2023 on Lloyds Bank Voice ID; the BBC You and Yours report of 28 November 2024; Hong Kong Police statements of February 2024 with Arup’s confirmation of May 2024; and the iProov Threat Intelligence Reports of 2025 and 2026, whose figures are vendor-measured and cited as such.