An Introduction to Evaluating Biometric Systems
How and where biometric systems are deployed will depend on their performance. Knowing what to ask and how to decipher the answers can help you evaluate the performance of these emerging technologies.
P. Jonathon Phillips Alvin Martin C.L. Wilson Mark Przybocki National Institute of Standards and Technology
n the basis of media hype alone, you might conclude that biometric passwords will soon replace their alphanumeric counterparts with versions that cannot be stolen, forgotten, lost, or given to another person. But what if the performance estimates of these systems are far more impressive than their actual performance? To measure the real-life performance of biometric systems—and to understand their strengths and weaknesses better—we must understand the elements that comprise an ideal biometric system. In an ideal system • all members of the population possess the characteristic that the biometric identiﬁes, like irises or ﬁngerprints; • each biometric signature differs from all others in the controlled population; • the biometric signatures don’t vary under the conditions in which they are collected; and • the system resists countermeasures. Biometric-system evaluation quantiﬁes how well biometric systems accommodate these properties. Typically, biometric evaluations require that an independent party design the evaluation, collect the test data, administer the test, and analyze the results. We designed this article to provide you with sufﬁcient information to know what questions to ask when evaluating a biometric system, and to assist you in determining if performance levels meet the requirements of your application. For example, if you plan to use a biometric to reduce—as opposed to eliminate— fraud, then a low-performance biometric system may be sufﬁcient. On the other hand, completely replacing
an existing security system with a biometric-based one may require a high-performance biometric system, or the required performance may be beyond what current technology can provide. Here we focus on biometric applications that give the user some control over data acquisition. These applications recognize subjects from mug shots, passport photos, and scanned ﬁngerprints. Examples not covered include recognition from surveillance photos or from latent ﬁngerprints left at a crime scene. Of the biometrics that meet these constraints, voice, face, and ﬁngerprint systems have undergone the most study and testing—and therefore occupy the bulk of our discussion. While iris recognition has received much attention in the media lately, few independent evaluations of its effectiveness have been published.
There are two kinds of biometric systems: identiﬁcation and veriﬁcation. In identiﬁcation systems, a biometric signature of an unknown person is presented to a system. The system compares the new biometric signature with a database of biometric signatures of known individuals. On the basis of the comparison, the system reports (or estimates) the identity of the unknown person from this database. Systems that rely on identiﬁcation include those that the police use to identify people from ﬁngerprints and mug shots. Civilian applications include those that check for multiple applications by the same person for welfare beneﬁts and driver’s licenses. In veriﬁcation systems, a user presents a biometric signature and a claim that a particular identity belongs to the biometric signature. The algorithm either accepts 0018-9162/00/$10.00 © 2000 IEEE
or rejects the claim. Alternatively, the algorithm can return a conﬁdence measurement of the claim’s validity. Veriﬁcation applications include those that authenticate identity during point-of-sale transactions or that control access to computers or secure buildings. Performance statistics for veriﬁcation applications differ substantially from those for identiﬁcation systems. The main...
References: 1. J.P. Egan, Signal Detection Theory and ROC Analysis, Academic Press, New York, 1975. 2. A. Martin et al., “The DET Curve Assessment of Detection Task Performance,” Proc. EuroSpeech 97, IEEE CS
Press, Los Alamitos, Calif., 1997, pp
Please join StudyMode to read the full document