The Facial Recognition Technology (FERET) Database

I. Introduction

The FERET program ran from 1993 through 1997. Sponsored by the Department of Defense's Counterdrug Technology Development Program through the Defense Advanced Research Products Agency (DARPA), its primary mission was to develop automatic face recognition capabilities that could be employed to assist security, intelligence and law enforcement personnel in the performance of their duties.

The FERET image corpus was assembled to support government monitored testing and evaluation of face recognition algorithms using standardized tests and procedures. The final corpus, presented here, consists of 14051 eight-bit grayscale images of human heads with views ranging from frontal to left and right profiles.

The CDROM distribution that accompanies this document constitutes the first and final release of all the FERET imagery. A previous collection provided to academic and industrial concerns from 1998 through to February 2001 held just 3737 images. Results from the FERET evaluations have been published; see the bibliography below.

II. Nomenclature

The naming convention for the FERET imagery in this distribution is of the form nnnnnxxfffq_yymmdd.ext where:

1. nnnnn is a five digit integer that uniquely identifies the subject

2. xx is a two lowercase character string that indicates the kind of imagery:

Two letter code	Pose Angle (degrees)	Description	Number in Database	Number of Subjects
Fa	0 = frontal	Regular facial expression	1762	1010
Fb	0	Alternative facial expression	1518	1009
ba	0	Frontal "b" series	200	200
bj	0	Alternative expression to ba	200	200
bk	0	Different illumination to ba	200	200
bb	+60	Subject faces to his left which is the photographer's right	200	200
bc	+40		200	200
bd	+25		200	200
be	+15		200	200
bf	-15	Subject faces to his right which is the photographer's left	200	200
bg	-25		200	200
Bh	-40		200	200
bi	-60		200	200
ql	-22.5	Quarter left and right	763	508
qr	+22.5		763	508
hl	-67.5	Half left and right	1246	904
hr	+67.5		1298	939
pl	-90	Profile left and right	1318	974
pr	+90		1342	980
Ra	+45	Random images. See note below. Positive angles indicate subject faces to photographer's right	322	264
Rb	+10		322	264
Rc	-10		613	429
Rd	-45		292	238
Re	-80		292	238

Notes:

1. fa indicates a regular frontal image

2. fb indicates an alternative frontal image, taken seconds after the corresponding fa

3. ba is a frontal images which is entirely analogous to the fa series

4. bj is an alternative frontal image, corresponding to a ba image, and analogous to the fb image

5. bk is also a frontal image corresponding to ba, but taken under different lighting

6. bb through bi is a series of images taken with the express intention of investigating pose angle effects (see below). Specfically, bf - bi are symmetric analogues of bb - be.

7. ra through re are "random" orientations. Their precise angle is unknown. It appears that the pose angles are random but consistent. The pose angles in the table were derived by manual measurement of inter-eye distances in the image, and in their corresponding frontal image.

3. fff is a set of three binary (zero or one) single character flags. In order these denote:
1. Indicates whether the image is releasable for publication. The flag has fallen into disuse: All images are available via this CDROM distribution, but still none may be published without the explicit written permission of the government.
2. Image is histogram adjusted if this flag is 1
3. Indicates whether the image was captured using ASA 200 or 400 film, 0 implies 200.

4. q is a modifier that is not always present. When it is, the meanings are as follows:
1. Glasses worn. Note that this flag is a sufficient condition only, images of subjects wearing glasses do not necessarily carry this flag. Some retroactive re-truthing of such images to fix this problem is warranted. See also "c" below.
2. Duplicate with different hair length.
3. Glasses worn and different hair length
4. Electronically scaled (resized) and histogram adjusted.
5. Clothing has been electronically retouched.
6. Image brightness has been reduced by 40%
7. Image brightness has been reduced by 80%
8. Image size has been reduced by 10%, with white border replacement
9. Image size has been reduced by 20%, with white border replacement
10. Image size has been reduced by 30%, with white border replacement

Note that the modifications d through j are the result of applying various off-line operations to real images in the database; the "parent" image is that image without the "q" modifier present at all.

5. The three fields are the date that the picture was taken in year, month, day format.

6. The filename extension is .tif. The images on the CDROMs carry an additional .bz2 suffix that indicates that the files have been losslessly compressed using the free bzip2 compressor, supplied with the database in misc/bzip2/.

III. Standard Testing Subsets

Empirical testing of pattern recognition algorithms is predicated on standardized data sets. Previously publications using FERET images have reported performance on the images described below. The concepts of gallery and probe sets apply: Each probe image[1] is matched against those in a gallery, and the ranked matches can be analyzed to produce recognition performance measures such a cumulative match score for identification, and receiver operating characteristic for verification applications.

Users are strongly encouraged to utilize the specific gallery and probe sets included in this FERET distribution. The reports detailed in the previous footnotes quote performance measures for contemporary algorithms on exactly these standardized test sets. These partitions of the corpus defined by the *.names files in the directories underneath the to-level folder partitions/.

The standard subsets have two distinct provenances. The first was employed originally in the FERET tests themselves, and were used again in the Face Recognition Vendor Test 2000 (FRVT2000, see footnote 5). The second series were designed to evaluate the sensitivity of algorithms to head direction (pose angle) as part of the FRVT 2000 tests.

In the tables below the parenthesized numbers indicate the number of images in the respective sets.

1. FERET Tests September 1996

These tests employed frontal images gathered between 1993 and 1996. The image sets are in the partitions/by_previously_reported/feret/ directory.

Evaluation Task	Recognized Names	Gallery (1196)	Probe Set
Aging of subjects	Duplicate I or T1	gallery.names	probe_dup_1_*.names (722)
Aging of subjects	Duplicate II or T2	gallery.names	probe_dup_2_*.names (234)
Facial Expression	fafb	gallery.names	probe_fafb_*.names (1195)
Illumination	fafc	gallery.names	probe_fafc_*.names (194)

Note that all the above tests used a single gallery containing 1196 images[2]. The Duplicate I probe images were obtained anywhere between one minute and 1031 days after[3] their respective gallery matches. The harder Duplicate II probe images re a strict subset of the Duplicate I images; they are those taken only at least 18 months[4] after their gallery entries. For assessment of the effect of facial expression two probe sets have been used. There is usually only a few seconds between the capture of the gallery-probe pairs.

2. FRVT 2000 Tests May 2000

The following four tests were conduced as part of the Facial Recognition Vendor Test 2000[5] and used a single frontal gallery, and four non-frontal probe sets taken at increasing azimuthal[6] angles. The intention of these tests was to quantify the effect of non-frontal image capture on recognition performance. The images are named in the partitions/by_previously_reported/frvt2000/ directory.

Evaluation Task	Recognized Names	Gallery (200)	Probe Set
Pose Azimuth 15 deg	P1	gallery.names	P1_probe.names (400)
Pose Azimuth 25 deg	P2	gallery.names	P2_probe.names (400)
Pose Azimuth 40 deg	P3	gallery.names	P3_probe.names (400)
Pose Azimuth 60 deg	P4	gallery.names	P4_probe.names (400)

The FRVT 2000 evaluation repeated the tests using the FERET Duplicate I and II imagery, used the working names T1 and T2 (see FRVT 2000 report in footnote 5).

The table and the accompanying figure show the relationships between the sets defined in the preceeding sections.

FERET Subset 1		Feret Subset 2	Cardinality
FERET Gallery	Ç	Training CD	270
Duplicate I	Ç	Training CD	184
Duplicate II	Ç	Training CD	0
Probe set fafb	Ç	Training CD	270
Probe set fafc	Ç	Training CD	0
Probe set fafb	Ç	Probe set fafc	0
Probe set fafb	Ç	Duplicate I	0
Duplicate II	Ì	Duplicate I	234

IV. Ground Truth Information

The previous section describes the naming of the FERET image files. Those file names encode the entire known ground truth information for each image. We have also included on the CDROM two alternative ground truth formats that may assist the user in selection and inspection of FERET imagery. The directories data/cd_*/ground_truths/ contain those two alternative markups, namely name_value and xml. Those directories contain precisely one file for each image in the images directories.

· The name_value tree holds *.gnd files which contain ground truth entries of the form "name = value". For example "date_taken = 8 May 1982".

· The XML files are similar but are constrained syntactically by a Document Type Definition (DTD) which resides on the FERET website. Similarly linked is a stylesheet that controls browser function.

V. Contents of the CDROMs

Each CDROM contains nine top-level directories, and two files, each of which is described below. The content of all the top-level directories is identical for each CDROM with the obvious exception that the data directories contain disjoint subsets of the FERET imagery. In detail:

· Makefile top-level recursive Makefile for eigenface implementation

· bin various executables for eigenface recognition

· data main repository of images and associated ground truth files

· doc documentation, FERET related publications and this file

· include top level C/C++ include files for eigenface implementation

· lib library files for eigenface executables

· misc collection of files used in assembly and final preparation of this CDROM; eye-mouth coordinate information for some images; the bzip2 decompressor

· partitions lists of image files relevant to certain tasks: by pose, expression, and subject; probe and gallery sets suitable for assessing recognition performance

· src C/C++ source code hierarchy for the eigenface implementation

· test directory containing scripts and debugging code for eigenface-based recognition system

VI. Contact Information

To obtain a copy of the FERET database the user should see the FERET homepage. Beyond that enquires should be directed to feret@nist.gov.

VII. Bibliography

The following publications describe the construction and use of the FERET database. These publications. With the exception of the PAMI paper all are on the web or in the doc/feret directory on the CDROMs

1. P. J. Phillips, H. Moon, P. J. Rauss, and S. Rizvi, "The FERET evaluation methodology for face recognition algorithms", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, No. 10, October 2000.

2. P. J. Phillips, P. J. Rauss, and S. Z. Der, " FERET (Face Recognition Technology) Recognition Algorithm Development and Test Results", October 1996. Army Research Lab technical report 995.

3. P. J. Phillips, H. Moon, P. J. Rauss, and S. Rizvi, " The FERET Evaluation Methodology for Face Recognition Algorithms".

4. S. Rizvi, P. J. Phillips and H. Moon, " The FERET Verification Testing Protocol for Face Recognition Algorithms".

---

[1] In the FERET protocol a probe set contains one or more images from a set of individuals. Each person will have exactly one match in the gallery. The gallery may contain images from other individuals who are not in the probe population.

[2] The tables in the published FERET reports in the previous footnote indicate that the gallery for the Duplicate II test held 864 images. This is erroneous; the correct number is 1196.

[3] The Duplicate I probe set holds 722 images whose matches were taken between 0 and 1031 days after the match. The median is 72 days and the mean is 251 days.

[4] The Duplicate II probe set contains 234 images from subjects whose gallery match was taken between 540 and 1031 days beforehand. The median is 569 and the mean is 627 days. Thus the Duplicate II probe images were taken at least 18 months after their gallery match.

[5] The FRVT 2000 tests evaluated leading commercial face recognition vendors. They were sponsored by the DoD Counterdrug Technology Development Program Office, the National Institute of Justice, and the Defense Advanced Research Projects Agency, and were administered between May and June 2000. For the final report see http://www.dodcounterdrug.com/facialrecognition.

[6] To be explicit, non-frontal and azimuthal, mean here that the subject rotated his head and body about a vertical axis facing in a direction other than toward the camera with eyes a horizontal plane. The other degrees of freedom in which the head rotates about horizontal axes: pitch (looking up or down) and roll (tipping head on it's side) are not represented in the FERET database.