GAO-08-992, Aviation Security: TSA Is Enhancing Its Oversight of Air Carrier Efforts to Identify Passengers on the No Fly and Selectee Lists, but Expects Ultimate Solution to Be Implementation of Secure Flight


This is the accessible text file for GAO report number GAO-08-992 
entitled 'Aviation Security: TSA Is Enhancing Its Oversight of Air 
Carrier Efforts to Identify Passengers on the No Fly and Selectee 
Lists, but Expects Ultimate Solution to Be Implementation of Secure 
Flight' which was released on September 10, 2008.

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States Government Accountability Office: 
GAO: 

Report to Congressional Committees: 

September 2008: 

Aviation Security: 

TSA Is Enhancing Its Oversight of Air Carrier Efforts to Identify 
Passengers on the No Fly and Selectee Lists, but Expects Ultimate 
Solution to Be Implementation of Secure Flight: 

Aviation Security and Watch List Matching: 

GAO-08-992: 

GAO Highlights: 

Highlights of GAO-08-992, a report to congressional committees. 

Why GAO Did This Study: 

Air carriers remain a front-line defense against acts of terrorism that 
target the nation’s civil aviation system. A key responsibility of air 
carriers is to check passengers’ names against terrorist watch-list 
records to identify persons who should be prevented from boarding (the 
No Fly List) or who should undergo additional security scrutiny (the 
Selectee List). Eventually, the Transportation Security Administration 
(TSA) is to assume this responsibility through its Secure Flight 
program. However, due to program delays, air carriers retain this role. 
You asked GAO to review domestic air carriers’ watch-list-matching 
processes. GAO examined (1) the watch-list-matching requirements air 
carriers must follow that have been established by TSA, and (2) the 
extent to which TSA has assessed air carriers’ compliance with these 
requirements. GAO reviewed TSA’s security directives, internal guidance 
used by TSA’s inspectors to assess air carriers’ compliance with 
requirements, and inspection results, as well as interviewed staff from 
14 of 95 domestic air carriers (selected to reflect a range in 
operational sizes). This report is the public version of a restricted 
report (GAO-08-453SU) issued in July 2008. 

What GAO Found: 

TSA’s requirements for domestic air carriers to conduct watch-list 
matching include a requirement to identify passengers whose names are 
either identical or similar to those on the No Fly and Selectee lists. 
Similar-name matching is important because individuals on the watch 
list may try to avoid detection by making travel reservations using 
name variations. According to TSA’s Office of Intelligence, there have 
been incidents of air carriers failing to identify potential matches by 
not successfully conducting similar-name matching. However, until 
revisions were initiated in April 2008, TSA’s security directives did 
not specify what types of similar-name variations were to be considered 
by air carriers. Thus, in interviews with 14 air carriers GAO found 
inconsistent approaches to conducting similar-name matching. Due to 
such inconsistency, a passenger could be identified as a match by one 
air carrier and not by another. In addition, not every air carrier 
reported conducting similar name comparisons. Further, in January 2008, 
TSA conducted an evaluation of air carriers and found deficiencies in 
their capability to conduct similar-name matching. Shortly thereafter, 
in April 2008, TSA revised the No Fly List security directive to 
specify a baseline capability for conducting watch-list matching, and 
TSA reported that it planned to similarly revise the Selectee List 
security directive. Because the baseline capability requires that air 
carriers compare only the types of name variations specified in the 
directive, TSA recognizes that the new baseline capability will not 
address all vulnerabilities. However, TSA emphasized that establishing 
the baseline capability should improve air carriers’ performance of 
watch-list matching and, in TSA’s view, is the best interim solution 
pending the implementation of Secure Flight. 

TSA has undertaken various efforts to assess domestic air carriers’ 
compliance with watch-list matching requirements; however, until 2008, 
TSA had conducted limited testing of air carriers’ similar-name-
matching capability. In 2005, for instance, TSA conducted an evaluation 
to determine whether air carriers had the capability to identify names 
that were identical—but not similar—to those on the No Fly List. Also, 
regarding regularly conducted inspections, TSA’s guidance did not 
specifically direct inspectors to test air carriers’ similar-name-
matching capability, nor did the guidance specify the number or types 
of name variations to be assessed. Records in TSA’s database for 
regular inspections conducted during 2007 made reference to name-match 
testing in 61 of the 1,145 watch-list-related inspections that GAO 
reviewed. Without criteria or standards for air carriers to follow in 
comparing name variations, TSA did not have a uniform basis for 
assessing compliance and addressing deficiencies. However, during the 
course of GAO’s review and prompted by findings of the evaluation 
conducted in January 2008, TSA reported that its guidance for 
inspectors would be revised to help ensure air carriers’ compliance 
with security directives. Although TSA has plans to strengthen its 
oversight of air carriers’ compliance with the revised security 
directives, it is too early to assess the extent of such oversight 
since TSA’s efforts are ongoing and not completed. 

What GAO Recommends: 

GAO is not making any recommendations because TSA initiated actions in 
April 2008 to strengthen watch-list-matching requirements and its 
oversight of air carriers’ implementation of these requirements. 

To view the full product, including the scope and methodology, click on 
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-992]. For more 
information, contact Cathleen A. Berrick at (202) 512-3404 or 
berrickc@gao.gov. 

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

TSA Took Action in 2008 to Enhance Watch-List Matching Conducted by Air 
Carriers but Believes the Ultimate Solution Will Be Implementation of 
Secure Flight: 

Until a 2008 Special Emphasis Inspection, TSA Had Conducted Limited 
Testing of Air Carriers' Capability to Perform Similar-Name Matching: 

Concluding Observations: 

Agency Comments: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: Overview of Selected Domestic Air Carriers' Watch-List- 
Matching Processes: 

Appendix III: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: TSA Watch-List-Matching Requirements Prior to the April 2008 
Revision to the No Fly List Security Directive: 

Table 2: Requirements for Matching Passenger Data to No Fly and 
Selectee Lists and Inspection Guidelines Used to Assess Compliance with 
the Requirements: 

Table 3: Watch-List-Matching Requirements and the Related Inspection 
Guidelines (Fiscal Year 2007): 

Figure: 

Figure 1: Overview of the Current Passenger Watch-List-Matching 
Process: 

Abbreviations: 

CAPPS: Computer Assisted Passenger Prescreening System: 

DHS: Department of Homeland Security: 

FBI: Federal Bureau of Investigation: 

PARIS: Performance and Results Information System: 

PNR: passenger name record: 

TRIP: Traveler Redress Inquiry Program: 

TSA: Transportation Security Administration: 

[End of section] 

United States Government Accountability Office:
Washington, DC 20548: 

September 9, 2008: 

Congressional Committees: 

Currently, more than 6 years after the terrorist attacks on September 
11, 2001, air carriers remain a front-line defense against acts of 
terrorism that target the nation's civil aviation system. A key aspect 
of air carriers' security responsibilities is to conduct preboarding 
checks of all passengers' personal information against terrorist watch- 
list records that contain information on thousands of individuals with 
known or potential links to terrorism. This process, referred to 
hereafter as watch-list matching, involves comparing passenger data-- 
most prominently name and date of birth--against the No Fly List to 
identify individuals who should be prevented from boarding an aircraft, 
and against the Selectee List to identify individuals who must undergo 
enhanced screening at the checkpoint prior to boarding.[Footnote 1] 

The Transportation Security Administration (TSA) requires that domestic 
air carriers operating to, from, and within the United States conduct 
watch-list matching.[Footnote 2] Data compiled by TSA's Office of 
Intelligence indicate that, at times, these air carriers have failed to 
identify individuals who are on the No Fly List. For instance, for the 
3-year period from January 2005 through December 2007, TSA documented 
several known incidents involving individuals on the No Fly List who, 
because of failures of domestic air carriers' watch-list-matching 
processes, were allowed to board international flights traveling to or 
from the United States.[Footnote 3] Data for these types of incidents, 
referred to as false negative watch-list-matching results, generally 
are not available for domestic flights--that is, domestic air carrier 
operations between two points within the United States or its 
territories.[Footnote 4] Nevertheless, because the requirements for air 
carriers to conduct watch-list matching are generally the same 
irrespective of the departure or arrival location, false negative 
incidents may be occurring on domestic flights if watch-listed 
individuals attempt to fly domestically. 

At present, domestic air carriers generally conduct watch-list matching 
in accordance with requirements that TSA sets forth in security 
directives--a regulatory tool through which TSA may impose security 
measures on a regulated entity, in this case air carriers, generally in 
response to an immediate or imminent threat.[Footnote 5] For example, 
security directives require that air carriers execute comparisons of 
passenger information with No Fly and Selectee list information within 
24 hours of a flight's scheduled departure. TSA also has responsibility 
for overseeing how air carriers implement the requirements set forth in 
security directives. Critical to this effort are the agency's aviation 
security inspectors, who oversee air carrier efforts at air carriers' 
corporate security offices (principal security inspectors) and at 
airport locations (transportation security inspectors). 

As required by law, TSA is to take over from air carriers the function 
of matching passenger information to the No Fly and Selectee lists for 
domestic flights.[Footnote 6] Since 2003, we have been assessing TSA's 
efforts to develop such a watch-list-matching program, currently known 
as Secure Flight, and have reported that significant challenges, 
including the need to follow a more structured systems development 
approach and to fully address how the program would protect passengers' 
privacy rights, have delayed its implementation.[Footnote 7] In April 
2008, we reported that TSA has made significant progress in developing 
Secure Flight, but that challenges remained in a number of areas, 
including the need to develop more robust cost and schedule estimates. 
[Footnote 8] We are continuing to review TSA's development and 
implementation of Secure Flight in response to requests from the U.S. 
Senate (Committee on Commerce, Science, and Transportation, and its 
Subcommittee on Aviation Operations, Safety, and Security; Committee on 
Appropriations, Subcommittee on Homeland Security; Committee on 
Homeland Security and Governmental Affairs; and Committee on the 
Judiciary) and the U.S. House of Representatives (Committee on 
Transportation and Infrastructure, Committee on Homeland Security, and 
the Committee on Oversight and Government Reform). In addition, the 
Consolidated Appropriations Act, 2008, requires that we report to the 
Committees on Appropriations of the Senate and House of Representatives 
on the Department of Homeland Security's (DHS) certification of 10 
conditions outlined in section 522(a) of the Department of Homeland 
Security Appropriations Act, 2005, related to the development and 
implementation of the Secure Flight program.[Footnote 9] The report is 
to be submitted 90 days after the DHS's Secretary certifies that all 10 
conditions have been successfully met. 

Pending Secure Flight's implementation, air carriers will continue to 
have primary responsibility for the watch-list-matching function. In 
conjunction with our ongoing evaluation of Secure Flight, we testified 
in June 2006 that due to delays and uncertainty surrounding Secure 
Flight's implementation, some air carriers were enhancing their watch- 
list-matching processes. We further identified that these improvements, 
though beneficial to the respective air carrier's operations, could 
further exacerbate differences that currently exist among the various 
air carriers, and could result in varying levels of effectiveness 
across air carriers in matching passenger information to the No Fly and 
Selectee lists.[Footnote 10] 

Due to the importance of identifying passengers who may pose a threat 
to commercial aviation, we were asked to review the current processes 
that domestic air carriers use to conduct watch-list matching for 
domestic flights.[Footnote 11] Accordingly, this report addresses the 
following questions: 

* What are TSA's requirements for domestic air carriers to conduct 
watch-list matching for domestic flights? 

* To what extent has TSA assessed domestic air carriers' compliance 
with watch-list-matching requirements? 

This report is a public version of the restricted report (GAO-08-453SU) 
that we provided to you on July 10, 2008. DHS and TSA deemed some of 
the information in the restricted report as Sensitive Security 
Information, which must be protected from public disclosure. Therefore, 
this report omits this information, such as the specific details 
associated with the current processes that domestic air carriers use to 
conduct watch-list matching. Although the information provided in this 
report is more limited in scope, it addresses the same principal 
questions as the restricted report. Also, the overall methodology used 
for both reports is generally the same. 

To determine TSA's requirements for matching passenger information 
against the No Fly and Selectee lists for domestic flights, we reviewed 
TSA's security directives, policies, and other guidance applicable to 
watch-list matching. We also interviewed officials at TSA's Office of 
Transportation Sector Network Management, Office of Security 
Operations, Office of Intelligence, and Office of Chief Counsel. We 
also reviewed key policy documents for Secure Flight, as well as our 
most recent reports and testimonies on the program to determine the 
planned matching process. In addition, to identify the composition and 
use of the No Fly and Selectee lists, we interviewed officials with the 
Department of Justice, Federal Bureau of Investigation's (FBI) 
Terrorist Screening Center, which has responsibility for managing the 
use of terrorist information in screening processes.[Footnote 12] We 
also contacted officials from a federally sponsored working group on 
identity matching to discuss the challenges associated with name-based 
matching. Moreover, to understand how air carriers have responded to 
watch-list-matching requirements, we conducted telephone interviews 
with officials from 14 domestic air carriers.[Footnote 13] Our 
selection of air carriers was based, in part, on operational size with 
the goal of obtaining a range of sizes based on operating revenue. For 
example, the Department of Transportation classifies eight of the air 
carriers in our review as major air carriers that provide service to 
locations across the nation and, with the exception of one air carrier, 
around the world.[Footnote 14] The remaining six air carriers had 
comparatively smaller business operations that generally provided 
service covering a geographical area, such as the Pacific Northwest, or 
commuter service.[Footnote 15] Although the 14 air carriers we spoke 
with represent a range in the types of air carriers that conduct watch- 
list matching, and, according to our calculations, accounted for 
approximately 70 percent of all passengers that boarded domestic 
flights in 2005, the results of our telephone interviews are not 
generalizable to the domestic operations of all domestic air carriers. 
However, our selection allowed us to understand how watch-list matching 
was performed for the majority of passengers flying domestically in 
2005. In addition, although our work summarizes the 14 air carriers' 
watch-list-matching capabilities as described to us in interviews, we 
did not independently verify each air carrier's reported method of 
implementation to determine the reliability of the data. 

To determine the extent to which TSA has assessed domestic air 
carriers' compliance with watch-list-matching requirements in the No 
Fly and Selectee list security directives,[Footnote 16] we first 
assessed TSA's inspection process, including the focus of inspections 
and inspection methods. We also examined TSA's national inspection 
plans and related guidance and policy documents. Further, at TSA 
headquarters, we interviewed officials responsible for developing and 
implementing inspection guidance and compiling and analyzing inspection 
results. Specifically, we interviewed representatives from the Office 
of Security Operations and the Office of Transportation Sector Network 
Management. We analyzed the results of both regular inspections (i.e., 
inspections conducted in conjunction with annual inspection plans) and 
nonroutine watch-list-related inspections that TSA conducted. For 
instance, we analyzed regular watch-list-related inspections that TSA 
conducted during fiscal year 2007 to ensure that air carriers were in 
compliance with applicable requirements. Although we concluded that 
these regular inspection data were sufficiently reliable for the 
purposes of this report, we have concerns about the potential for error 
based on TSA's process for querying its inspection database (we discuss 
these concerns in more detail in app. I). To assess data reliability, 
we performed electronic testing, discussed the data system and any data 
inconsistencies we found with knowledgeable TSA officials, and reviewed 
existing information about the data system. We also reviewed results 
from a special emphasis assessment that TSA conducted in 2005, and a 
special emphasis inspection it conducted in January 2008, both of which 
addressed air carriers' capability to conduct watch-list matching. 
[Footnote 17] We determined that the sampling and related procedures 
used for the special emphasis assessment were insufficient for 
providing a reliable estimate of the success rate of all attempted 
matches by air carriers. We did not assess the initial data TSA 
provided in February 2008 for the special emphasis inspection it 
conducted the previous month.[Footnote 18] 

We conducted this performance audit from July 2006 to September 2008 in 
accordance with generally accepted government auditing standards. Those 
standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on the audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on the audit objectives. More details about the scope 
and methodology of our work are presented in appendix I. 

Results in Brief: 

TSA has issued two security directives (one for the No Fly List and 
another for the Selectee List) that delineate requirements related to 
air carrier watch-list matching, including the identification of 
passengers with names similar to those on the lists. Identifying 
passengers with names similar to those on the No Fly and Selectee 
lists--a process TSA refers to as similar-name matching--is a critical 
component of watch-list matching because individuals may travel using 
abbreviated name forms or other variations of their names. Therefore, 
searching for only an exact match of the passenger's name may not 
result in identifying all watch-listed individuals. There have been 
incidents, according to TSA's Office of Intelligence, of air carriers 
failing to identify potential matches by not successfully conducting 
similar-name matching. Before revisions to the security directives were 
initiated in 2008, TSA expected air carriers to find similar names but 
provided no specificity on the extent to which air carriers should make 
these comparisons. The 14 air carriers we interviewed reported 
implementing varied approaches to similar-name matching. Because air 
carriers used different approaches, a passenger could be identified as 
a match to a watch-list record by one carrier and not by another 
carrier, which results in uneven effectiveness of watch-list matching. 
Generally, TSA had been aware that air carriers were not using 
equivalent processes to compare passenger names with names on the No 
Fly and Selectee lists. However, in early 2008 the significance of such 
differences was crystallized during the course of our review and 
following TSA's special emphasis inspection of air carriers' name- 
matching capability. On the basis of these inspection results, TSA 
issued a revised security directive governing the use of the No Fly 
List in April 2008 to establish a baseline capability for similar-name 
matching to which all air carriers must conform. Also, TSA announced 
that it plans to revise the Selectee List security directive to 
similarly require the new baseline capability.[Footnote 19] According 
to TSA officials, the new baseline capability is intended to improve 
the effectiveness of watch-list matching, particularly for those air 
carriers that did not compare the types of name variations specified by 
the new baseline capability or that compared none at all. However, TSA 
officials noted that the new baseline is not intended to address all 
possible types of name variations and the related security 
vulnerabilities. Agency officials explained that based on their 
analysis of the No Fly and Selectee lists and interviews with 
intelligence community officials, the newly established baseline covers 
the most critical types of name variations. TSA officials further 
stated that this is an interim solution that will strengthen security 
while not requiring air carriers to invest in significant modifications 
to their watch-list-matching processes, given TSA's expected 
implementation of Secure Flight beginning in 2009. These officials 
added that when implemented, Secure Flight will be better able to use 
passenger names and other identifying information to more accurately 
match passengers to the subjects of watch-list records. 

TSA has undertaken various efforts to assess domestic air carriers' 
compliance with watch-list-matching requirements in the No Fly and 
Selectee list security directives; however, until 2008, TSA had 
conducted limited testing of air carriers' similar-name-matching 
capability. In 2005, for instance, TSA conducted a special emphasis 
assessment that focused on air carriers' capability to prescreen 
passengers for exact-name matches with the No Fly List, but did not 
address the air carriers' capability to conduct similar-name 
comparisons. Regarding inspections conducted as part of regular 
inspection cycles, TSA's guidance establishes that regulatory 
requirements encompassing critical layers of security need intensive 
oversight, and that testing is the preferred method for validating 
compliance. However, before being revised in 2008, TSA's inspection 
guidelines (called PARIS prompts)[Footnote 20] for watch-list-related 
inspections were broadly stated and did not specifically direct 
inspectors to test air carriers' similar-name-matching capability. 
Moreover, TSA's guidance provided no baseline criteria or standards 
regarding the number or types of such variations that must be assessed. 
In response to our inquiry, 6 of TSA's 9 principal security inspectors 
told us that their assessments during annual inspection cycles have not 
included examining air carriers' capability to conduct certain basic 
types of similar-name comparisons. Also, in reviewing documentation of 
the results of the most recent inspection cycle (fiscal year 2007), we 
found that available records in TSA's database made references to name- 
matching tests in 6 of the 36 watch-list-related inspections that 
principal security inspectors conducted, and in 55 of the 1,109 
inspections that transportation security inspectors conducted.[Footnote 
21] Without baseline criteria or standards for air carriers to follow 
in conducting similar-name comparisons, TSA has not had a uniform basis 
for assessing compliance. Further, without routinely and uniformly 
testing how effectively air carriers are conducting similar-name 
matching, TSA may not have had an accurate understanding of the quality 
of air carriers' watch-list-matching processes. However, TSA began 
taking corrective actions during the course of our review and after it 
found deficiencies in the capability of air carriers to conduct similar-
name matching during a January 2008 special emphasis inspection. 
[Footnote 22] More specifically, following the January 2008 inspection, 
TSA officials reported that TSA immediately began working with 
individual air carriers to address deficiencies. Also, officials 
reported that, following the issuance of TSA's revised No Fly List 
security directive in April 2008, the agency had plans to assess air 
carriers' progress in meeting the baseline capability specified in the 
new security directive after 30 days, and that the annual inspection 
plan for transportation security inspectors would be revised to help 
ensure compliance by air carriers with requirements in the new security 
directive. In September 2008, TSA provided us with results from a May 
2008 special emphasis assessment of seven air carriers' compliance with 
the revised No Fly List security directive. Although the details of 
this special emphasis assessment are classified, TSA generally 
characterized the results as positive. Further, the TSA officials noted 
that the agency's internal handbook-- which provides guidance to 
transportation security inspectors on how to inspect air carriers' 
performance of various requirements, including watch-list-matching 
requirements--was being revised and was expected to be released later 
this year. Thus, the TSA officials stated that the new inspection 
guidance would be used in conjunction with the nationwide regulatory 
activities plan for fiscal year 2009. While these actions and plans are 
positive developments, it is too early to determine the extent to which 
TSA will assess air carriers' compliance with watch-list-matching 
requirements moving forward since these efforts are still underway. 

We provided a draft of our restricted report to DHS and the Department 
of Justice for review and comment. DHS had no comments. The Department 
of Justice provided technical comments to the restricted version of 
this report, which we incorporated where appropriate. 

Background: 

TSA uses a layered system of defense to secure civil aviation whereby 
additional layers provide security when any one security measure may 
fail. Watch-list matching is one such layer of defense. Air carriers 
began checking passenger names against government-supplied terrorist 
watch lists (compiled by the FBI and distributed by the Federal 
Aviation Administration) in the early 1990s. After the attacks of 
September 11, 2001, and the subsequent establishment of TSA during the 
same year, primary responsibility for civil aviation security, 
including overseeing the watch-list-matching process, fell to TSA. 
[Footnote 23] The Aviation and Transportation Security Act, enacted in 
November 2001, requires that a system be used to evaluate all 
passengers before they board an aircraft and ensure that selected 
individuals and their carry-on and checked baggage are adequately 
screened.[Footnote 24] TSA fulfilled this mandate by continuing to 
require and oversee air carrier operation of the Computer Assisted 
Passenger Prescreening System (CAPPS)--an electronic application that 
selects individuals for enhanced screening at the passenger checkpoint 
based on certain travel characteristics identified by TSA as indicating 
potential risk--and by issuing security directives in April 2002 that 
continued and amended the requirements that domestic air carriers match 
passenger information against the No Fly and Selectee lists. These 
security directives are the No Fly List Procedures security directive, 
requiring domestic air carriers to conduct checks of passenger 
information against the No Fly List to identify individuals who should 
be precluded from boarding flights, and the Selectee List Procedures 
security directive, directing domestic air carriers to conduct checks 
of passenger information against the Selectee List to identify 
individuals who should receive enhanced screening (e.g., additional 
physical screening or a hand-search of carry-on baggage) before 
proceeding through the security checkpoint.[Footnote 25] Since 2002, 
TSA has issued numerous revisions to the No Fly and Selectee list 
security directives to strengthen and clarify requirements, and has 
issued guidance to assist air carriers in implementing their watch- 
list-matching processes.[Footnote 26] 

So that they may carry out watch-list-matching requirements, TSA 
provides air carriers with access to the No Fly and Selectee lists-- 
subsets of the terrorist screening database managed by the FBI's 
Terrorist Screening Center. The terrorist screening database is 
composed of records that contain identifying information (e.g., name 
and date of birth) on both foreign and U.S. citizens with known or 
appropriately suspected links to terrorism. Only those nominations in 
the terrorist screening database submitted by elements within the 
intelligence community, including the FBI, that meet criteria specified 
by the Homeland Security Council[Footnote 27] relating to the threat 
that an individual poses to civil aviation are exported as records to 
be included on the No Fly or Selectee lists.[Footnote 28] At present, 
the Terrorist Screening Center forwards the No Fly and Selectee lists 
to TSA's Office of Intelligence, which generally posts new lists daily 
to a secure Web board that air carriers may access to retrieve the 
lists.[Footnote 29] The Terrorist Screening Center provides TSA's 
Office of Intelligence with new No Fly and Selectee lists on a daily 
basis as well as any time a nominating entity submits additions and 
deletions that require immediate notification to the aviation 
community. 

TSA's Regulatory Inspection Framework: 

TSA is responsible for ensuring air carriers' compliance with 
regulatory requirements, including requirements reflected in TSA 
security directives and TSA-approved security programs. According to 
TSA inspection guidance, compliance with regulatory requirements may be 
validated in various ways, depending on the risk associated with the 
requirements. For example, when regulatory requirements are largely 
administrative and encompass the least critical layers of security, 
compliance may be validated largely through inspections based on 
documentation reviews. However, when regulatory requirements encompass 
more critical layers of security, more intensive oversight is needed, 
and compliance typically is to be validated through testing, 
inspections, surveillance, special emphasis assessments, and special 
emphasis inspections. 

TSA conducts inspections of air carriers throughout the year as part of 
regular inspection cycles based on annual inspection plans. These 
inspections are based on inspection guidelines known as PARIS prompts, 
which address a broad range of regulatory requirements (including 
airport perimeter security and cargo security, as well as screening of 
employees, baggage, and passengers). With respect to watch-list 
matching, an inspection guideline (PARIS prompt) instructs inspectors 
to determine, for example, whether the air carrier is comparing the 
names of all passengers against names on the most current No Fly and 
Selectee lists in accordance with the procedures outlined in TSA's 
security directives. 

TSA conducts watch-list-related inspections at air carriers' corporate 
security offices (where policies and procedures are established on how 
watch-list matching is to be performed) and at airports (where policies 
and procedures for responding to a potential match are implemented). 
TSA's principal security inspectors are responsible for conducting 
inspections at domestic air carriers' corporate headquarters. These 
inspectors assess air carriers' compliance with security requirements 
and provide direct oversight of air carriers' implementation of and 
compliance with TSA-approved security programs. TSA considers principal 
security inspectors to be subject-matter experts for the air carrier 
community concerning implementation of and compliance with security 
programs and other requirements. As of January 2008, nine principal 
security inspectors were responsible for assessing the compliance of 
domestic air carriers with requirements in the No Fly and Selectee list 
security directives (as well as with other regulatory requirements 
pertaining to commercial aviation). Each of these inspectors has 
responsibility for one or more domestic air carriers. For fiscal year 
2007, there were 72 domestic air carriers to which the No Fly and 
Selectee list security directives applied. 

Field inspectors--known as transportation security inspectors--conduct 
watch-list-related inspections at airports. They are responsible for a 
multitude of TSA-related activities, including conducting inspections 
and investigations of airports and air carriers, monitoring compliance 
with applicable civil aviation security policies and regulations, 
resolving routine situations that may be encountered in the assessment 
of airport security, participating in testing of security systems in 
connection with compliance inspections, identifying when enforcement 
actions should be initiated, and providing input on the type of action 
and level of penalty commensurate with the nature and severity of a 
violation that is ultimately recommended to TSA's Office of Chief 
Counsel. As of June 2008, there were 681 transportation security 
inspectors responsible for 459 commercial airports across the United 
States. 

Secure Flight: Development of a Government-Run Watch-List-Matching 
Process: 

TSA began developing a program to take over watch-list-matching 
capability from air carriers in March 2003.[Footnote 30] TSA cancelled 
this earlier effort, known as CAPPS II, due to development challenges 
and privacy concerns. In July 2004, the National Commission on 
Terrorist Attacks Upon the United States (the 9/11 Commission) 
recommended that the federal government take over the watch-list- 
matching function from air carriers.[Footnote 31] Subsequently, the 
Intelligence Reform and Terrorism Prevention Act of 2004 required that 
TSA develop such a watch-list-matching capability.[Footnote 32] Shortly 
after suspending work on the CAPPS II program in August 2004, TSA 
initiated development of Secure Flight, a program that the agency 
expects will allow the federal government to perform watch-list 
matching for passengers on all flights within the United States and 
ultimately for international flights with departures from or arrivals 
in the United States. 

In February 2006, we testified that although some progress had been 
made in developing Secure Flight, long-standing issues related to 
systems development and testing, program management, privacy 
protections, and redress remained.[Footnote 33] We reported in 
testimony that as a result of these deficiencies the program was at 
risk of failure. Following our February 2006 testimony, TSA announced a 
temporary suspension of Secure Flight's development to reassess program 
goals and capabilities. TSA completed this reassessment in January 
2007, moved forward to complete its concept-of-operations plan for the 
Secure Flight program and strengthen systems development efforts, and, 
in August 2007, issued a notice of proposed rulemaking describing the 
requirements TSA will expect air carriers to implement to facilitate 
the government-run prescreening process.[Footnote 34] TSA expects that, 
beginning in early calendar year 2009, the Secure Flight program will 
begin assuming from air carriers the watch-list-matching responsibility 
for domestic flights. At some point following this assumption for 
domestic flights, TSA plans to assume from U.S. Customs and Border 
Protection this watch-list-matching function for international flights 
that depart from or arrive in the United States. However, we testified 
in February 2008 that despite significant progress in the development 
of Secure Flight, TSA did not fully follow best practices for 
developing Secure Flight's life-cycle cost and schedule estimates, and 
that failure to do so put the program at risk of cost overruns, missed 
deadlines, and performance shortfalls, among other issues.[Footnote 35] 

TSA Took Action in 2008 to Enhance Watch-List Matching Conducted by Air 
Carriers but Believes the Ultimate Solution Will Be Implementation of 
Secure Flight: 

Through its security directives, TSA has issued requirements for watch- 
list matching, which include identifying passengers with names similar 
to those on the No Fly and Selectee lists--a process TSA refers to as 
similar-name matching. Before undertaking revisions of the relevant 
security directives in 2008, TSA expected air carriers to conduct 
similar-name matching but TSA's security directives did not specify how 
many and what types of such name variations air carriers should 
compare. Consequently, some of the 14 air carriers we interviewed 
reported that they compared more name variations than others. Air 
carriers that do not conduct similar-name comparisons and carriers that 
conduct relatively limited comparisons are less effective in 
identifying watch-listed individuals who travel under name variations. 
Also, due to inconsistent air carrier processes, a passenger could be 
identified as a match by one carrier and not by another. In April 2008, 
during the course of our review, TSA revised and issued the No Fly List 
security directive to specify a baseline capability for similar-name 
matching to which all air carriers must conform. Also, in April 2008, 
TSA officials reported that the agency had plans to similarly revise 
the Selectee List security directive to require the same baseline 
capability.[Footnote 36] TSA officials acknowledged that the new 
baseline capability will not address all vulnerabilities identified by 
TSA. However, the officials stated that the new baseline capability was 
their best interim approach for improving air carriers' matching 
efforts because, among other reasons, it will strengthen watch-list 
matching without requiring considerable investment in a solution that 
will be replaced when Secure Flight is implemented. TSA officials 
further stated that the longer term solution for watch-list matching is 
Secure Flight, which will have the capability to undertake more 
advanced searches for individuals on the No Fly and Selectee lists. 

Prior to April 2008, TSA Watch-List-Matching Requirements Were Broad 
and Allowed Air Carriers to Implement Less Effective Processes: 

Prior to a revision of the No Fly List security directive in April 
2008--and a similar revision planned for the Selectee List security 
directive--TSA's watch-list-matching requirements for domestic flights 
(summarized in table 1) addressed five key processes: (1) retrieval of 
the No Fly and Selectee lists, (2) the matching of passenger and list 
information, (3) the use of TSA's Cleared List,[Footnote 37] (4) 
notification procedures, and (5) record-keeping activities.[Footnote 
38] In April 2008, TSA revised the No Fly List security directive for 
watch-list matching and also reported plans for similarly revising the 
Selectee List security directive. The security directive revisions-- 
discussed later in this section--still address the five key process 
areas, but provide greater specificity on TSA's requirements for 
matching passenger and watch-list information (the second key process 
shown in table 1).[Footnote 39] Prior to the April 2008 revision of the 
No Fly List security directive, TSA's requirements in this area lacked 
specificity for purposes of implementation, although the then-current 
security directives addressed the need for air carriers to identify 
passengers with names that are either identical or similar to those on 
the No Fly List or the Selectee List. To identify passengers with 
similar names--an activity known as similar-name matching--air 
carriers' automated programs or manual reviews were expected to capture 
No Fly and Selectee list names that are variations of the name on the 
passenger's reservation. 

Table 1: TSA Watch-List-Matching Requirements Prior to the April 2008 
Revision to the No Fly List Security Directive: 

Requirements (key processes): 
(1) Retrieving the No Fly and Selectee lists; 
Discussion: 
* Air carriers must monitor the TSA Web board throughout the day for 
the most recent postings of the No Fly and Selectee lists. 

Requirements (key processes): 
(2) Matching passenger data to No Fly and Selectee lists; 
Discussion: 
* Within 24 hours of scheduled flight departure time, but no later than 
passenger check-in, air carriers are to compare records from the most 
recently issued No Fly and Selectee lists with identifying information 
on passengers found in the respective air carrier's reservation system 
and offered by passengers at the time of check-in; 
* When comparing data, air carriers must identify name matches to the 
No Fly and Selectee lists. To identify similar-name matches, automated 
and manual processes are expected to have the capability to compare 
name variations; 
* To determine which passengers are matches, a passenger's name and one 
piece of identifying information (found either within the air carrier's 
reservation system or supplied by the passenger at check-in) must match 
with corresponding information provided on the No Fly or Selectee 
lists. 

Requirements (key processes): 
(3) Using the TSA Cleared List[A]; 
Discussion: 
* When making determinations on matches, air carriers must use the TSA 
Cleared List, which is composed of names and other personal-identifying 
information on individuals whom the Department of Homeland Security has 
reviewed and determined are not individuals on the No Fly or Selectee 
lists. Individuals determined to be on the TSA Cleared List should be 
accepted for travel and not be subject to further procedures for 
handling matches to No Fly or Selectee lists identified in the security 
directives. 

Requirements (key processes): 
(4) Notifying authorities; Discussion: 
* Upon identifying a passenger whose information matches with the No 
Fly or Selectee lists and who is not on the TSA Cleared List, air 
carriers must follow certain notification procedures, such as to 
contact the federal security director and the appropriate local law 
enforcement officer (for matches to the No Fly List) or to designate 
the passenger as a selectee for enhanced checkpoint screening 
procedures (for matches to the Selectee List). 

Requirements (key processes): 
(5) Keeping records; Discussion: 
* Air carriers must keep records on the results of watch-list matching 
for specified time periods--for example, air carriers must keep a 
record of all flights operated with passengers designated as selectees 
for 7 calendar days from the date of the flight's departure. 

Sources: GAO analysis of TSA's No Fly List Procedures security 
directive (SD 1544-01-20 series) and Selectee List Procedures security 
directive (SD 1544-01-21 series), versions dated July 8, 2004, and 
March 8, 2007. 

[A] Security directives in effect prior to the April 2008 revision of 
the No Fly List Procedures security directive referenced a "cleared 
column," a format for clearing passengers. TSA eventually replaced this 
format with the Cleared List, and revised language for the April 2008 
No Fly List security directive. 

[End of table] 

Air carriers must conduct similar-name matching because watch-listed 
individuals may travel using variations of the names attributed to them 
on the No Fly or Selectee lists and, thus, would not be identified if 
air carriers searched only for an exact-name match. At present, TSA 
does not require that air carriers collect the full name from 
passengers making travel reservations, thus, passengers may travel 
using variations of their legally documented names; for example, 
abbreviated name forms or portions of their names. Such name variations 
may arise due to unintentional errors--for example, a travel agent 
mistakenly books travel for "Jon" when the name spelling is actually 
"John," or the agent accidentally transposes a passenger's first and 
middle names for a flight reservation. Traveling under a name variation 
could also represent a watch-listed individual's intentional effort to 
evade detection. For example, an individual identified as John Robert 
Smith on his driver's license may make a travel reservation using a 
common name variation--such as using his middle and last names (Robert 
Smith) or his initials and last name (J.R. Smith). If the John Robert 
Smith in this example were a name on the No Fly List, an exact, letter- 
for-letter comparison of the passenger's reservation name (either 
Robert Smith or J.R. Smith) with the No Fly List would fail to identify 
the watch-listed individual. However, a comparison of possible 
variations of the watch-list name (John Robert Smith) could identify 
either Robert Smith or J.R. Smith as a potential match--that is, an 
individual who is a possible match to the No Fly List or Selectee List 
and whose personal identifying information requires further review 
before a match can be determined. 

Before 2008, TSA's Security Directives Allowed Air Carriers More 
Discretion in Comparing Name Variations: 

Regarding similar-name matching, before 2008, TSA's security directives 
had broad requirements that allowed air carriers discretion in 
determining the extent to which they compared name variations. For 
instance, to identify watch-listed individuals who travel using 
variations of their name, TSA's security directives did not specify how 
many possible combinations of name elements should be compared. TSA 
officials explained that the agency initially issued broad security 
directives to allow air carriers flexibility in implementing 
requirements and--until the April 2008 revision of the No Fly List 
security directive--left the directives relatively unchanged because 
the agency was developing a government-run capability to take over this 
function. The operations of those air carriers that are subject to the 
watch-list-matching requirements of TSA's security directives range 
from commuter providers to international-service providers. According 
to TSA officials, broad security directive requirements permit air 
carriers with such diverse operations to implement processes that best 
meet their operational needs and technological capabilities. 

Officials further explained that TSA's focus has been on developing its 
own watch-list-matching capability (now Secure Flight) since 2003. TSA 
officials noted that, though not an impetus for making requirements 
broad when first articulated in 2002, this focus on developing a 
government-run watch-list-matching program is one reason why these 
requirements remained relatively unchanged until April 2008. 

Failure to Conduct Similar-Name Matching or Comparing Name Variations 
to a Lesser Extent Reduces the Effectiveness of Watch-List Matching: 

The 14 air carriers we interviewed reported adopting different 
approaches to name matching. Although each of the 14 air carriers we 
spoke with during our review reported conducting comparisons to 
identify exact-name matches of passengers and names on the No Fly List 
or the Selectee List, not every air carrier reported conducting similar-
name comparisons.[Footnote 40] Those air carriers that conducted 
similar-name comparisons reported using various approaches, some of 
which compared more name variations than others. 

According to air carriers, a critical factor affecting their 
implementation of similar-name-matching requirements was their 
observation that conducting more comparisons for variations results in 
longer lines at ticket counters and passenger inconvenience. 
Specifically, 10 air carriers commented that conducting similar-name 
comparisons resulted in more passengers being identified as potential 
matches. At the time of check-in, air carriers must perform additional 
checks at the ticket counter of each potentially matched passenger's 
government-issued identification against data on the No Fly and 
Selectee lists. Therefore, according to 12 of the 14 air carriers we 
spoke with, a large number of potential matches can lead to congestion 
at the ticket counter and longer wait times for all passengers. 

Inconsistent approaches to conducting similar-name matching could lead 
a passenger to be identified as a match by one air carrier and not by 
another. Further, not conducting similar-name matching--or conducting 
such matching to only a very limited extent--compromises the usefulness 
of the No Fly List and Selectee List. There have been incidents, 
according to TSA's Office of Intelligence, of air carriers failing to 
identify potential matches by not effectively conducting similar-name 
matching. In these incidents, the air carriers' processes led to false 
negative watch-list-matching results--that is, individuals who were on 
the No Fly List and were not identified by the respective air carrier's 
watch-list-matching process. In some of these incidents, the 
individual's flight reservation contained a name that varied somewhat 
from the name on the No Fly List, and the air carrier's watch-list- 
matching process did not identify the name as a possible match. 

In most of these cases, the failures of the air carriers to identify 
the potential matches were discovered as a result of the U.S. Customs 
and Border Protection's comparison of passenger and watch-list data for 
international flights. Specifically, TSA learned of the failures 
through U.S. Customs and Border Protection, which identified the No Fly 
listed individual when conducting its own comparison of passenger 
information against the No Fly and Selectee lists for international 
flights.[Footnote 41] These comparisons, performed as part of U.S. 
Customs and Border Protection's border security mission, took place 
after the air carriers completed their comparisons, in effect 
constituting a second check of passenger and watch-list information. 
U.S. Customs and Border Protection does not screen passengers on 
domestic flights; thus, there is no opportunity for a second comparison 
of passenger information against the No Fly and Selectee lists for 
domestic flights. Therefore, it is difficult to determine the extent to 
which domestic air carriers may be failing to identify watch-listed 
individuals who are able to board domestic flights. 

In October 2007, we reported that of the known cases in which 
individuals on the No Fly List flew on international flights bound to 
or from the United States, some were allowed to fly because the 
respective air carrier's process failed to identify the passenger's 
name as a match.[Footnote 42] Although these individuals were 
subsequently identified in-flight by other means, the onboard security 
threats required an immediate counterterrorism response, which in some 
instances resulted in diverting the aircraft to a location other than 
its original destination.[Footnote 43] According to TSA's Office of 
Intelligence, some of these incidents may be attributed to air 
carriers' inability to identify similar-name matches when passengers 
travel using variations of their name. 

TSA had been aware that air carriers were not using equivalent 
processes to compare passenger names with names on the No Fly and 
Selectee lists. For instance, in June 2006, we reported that the 
improvements air carriers were making to their individual watch-list- 
matching processes, though beneficial to the respective air carrier's 
operations, could further exacerbate differences that currently exist 
among the various air carriers and could result in varying levels of 
effectiveness across air carriers in matching passenger information to 
the No Fly and Selectee lists.[Footnote 44] Furthermore, TSA's March 
2007 Secure Flight Program Baseline explained "because each aircraft 
operator conducts its own matching process, the ability to conduct 
watch-list matching and coordinate law enforcement responses is not 
consistent across the aviation industry."[Footnote 45] Moreover, in 
several interviews over the course of our work, TSA officials 
acknowledged that in general, some air carriers were performing more 
similar-name comparisons than other air carriers. TSA's understanding 
of the significance of these differences was crystallized in January 
2008, when results of a special emphasis inspection identified 
deficiencies in air carriers' similar-name-matching capability. 

To Address Deficiencies in Air Carriers' Similar-Name-Matching 
Capability, TSA Issued a Revised No Fly List Security Directive in 
April 2008 to Provide More Specific Requirements: 

During the course of our work and in response to findings of the 
January 2008 special emphasis inspection that identified deficiencies 
in air carriers' similar-name-matching capability, TSA officials 
reported that the agency immediately began to assess options for 
corrective actions to implement across the aviation industry. In doing 
so, officials noted that they consulted with representatives from the 
intelligence community, the Secure Flight program, and the aviation 
industry. On the basis of its assessment, TSA revised the No Fly List 
security directive in April 2008 to establish a specific baseline 
capability for air carriers in conducting similar-name matching. Also, 
in April 2008, TSA officials reported that the agency had plans to 
similarly revise the Selectee List security directive to require the 
same baseline capability.[Footnote 46] 

TSA officials acknowledged that the new baseline capability will not 
address all vulnerabilities identified by TSA. However, TSA officials 
explained that they expect the new similar-name matching baseline 
capability to strengthen the watch-list matching currently performed by 
air carriers. In particular, the officials expect the newly established 
baseline capability to improve the matching processes of those air 
carriers that do not compare the kinds of variations required by the 
new baseline or that compare none at all. Furthermore, according to 
agency officials, the variations specified by the new baseline address 
the types of situations air carriers will encounter due to passengers 
making their own reservations. Accordingly, TSA concluded that 
requiring air carriers to conduct similar-name comparisons beyond the 
baseline capability specified in the revised No Fly List security 
directive was not warranted for the interim period pending the 
implementation of Secure Flight. TSA was not able to provide us with 
data or analysis to support this assertion, and we did not undertake an 
independent analysis to determine the sufficiency of the newly 
established baseline. 

TSA officials also explained they determined that revising the security 
directives to be the most feasible approach for strengthening the 
current watch-list-matching process over other options because it was 
expedient and would have the least negative impact on air carriers' 
operations. Specifically, TSA officials determined that upon issuing 
the revised No Fly List security directive, air carriers would need 
only 2 to 4 weeks to implement new requirements. When considering how 
this option would affect air carrier operations, TSA officials 
explained that they considered the number of potential matches that 
likely would be generated by the new baseline capability. As previously 
discussed, air carriers reported that comparing more name variations 
results in more passengers being identified as potential matches, who 
then must go to the ticket counter to obtain their boarding passes. 
Thus, large numbers of potential matches could overwhelm air carriers' 
check-in operations. TSA officials explained that the industry 
officials with whom they consulted in developing the new baseline 
capability believed it would produce a manageable number of potential 
matches. 

In exploring actions to strengthen the watch-list-matching process, TSA 
considered two other options--one that would have required each air 
carrier to contract with third-party providers to develop customized 
watch-list-matching software, and another that involved the creation of 
an expanded version of the No Fly and Selectee lists to include name 
variations so that air carriers need only conduct comparisons to 
identify an identical match. TSA identified significant obstacles to 
implementing these options. Specifically, TSA determined that 
contracting with third-party vendors was impracticable due to 
availability and timing concerns. For instance, identifying appropriate 
vendors and implementing vendor-provided solutions could take almost 2 
years--an unrealistic time frame given that Secure Flight's 
implementation is scheduled to begin in 2009. In this regard, TSA 
officials also expressed reluctance to requiring air carriers to 
undertake the expense of contracting with third-party vendors for an 
interim approach, while at the same time requiring that air carriers 
invest in system changes for Secure Flight. With regard to the option 
of adding name variations to the No Fly and Selectee lists, according 
to TSA officials, creating these variations would have greatly expanded 
the total size of the No Fly List, which could overwhelm the name- 
matching capability of some air carriers and could potentially send an 
unmanageable number of potential matches to the ticket counters of air 
carriers. As previously discussed, in our air carrier interviews, 10 of 
the 14 air carriers reported that searching for more name variations 
leads to the identification of more potential matches. In this regard, 
there is some support for TSA's determination that expansion of the No 
Fly and Selectee lists could produce an unmanageable number of 
potential matches. However, we did not independently assess this issue. 

Although TSA officials characterized the new baseline capability as a 
good interim solution for strengthening watch-list matching--one that 
balances TSA's need to strengthen watch-list matching with the air 
carriers' need for efficient operations--they stressed that the Secure 
Flight program is ultimately the solution. For example, in its 
development of Secure Flight, TSA plans to develop a name-matching 
process that will have the capability to identify name variations 
beyond those specified by the new baseline. Further, according to TSA, 
Secure Flight will be better able to use passenger names and other 
identifying information (such as date of birth and gender) to more 
accurately match passengers to the subjects of watch-list records and, 
thereby, further reduce the risks of false negatives without 
unacceptably increasing the number of false positives (mistakenly 
identifying a passenger's name as a potential match with watch-list 
records). 

Until a 2008 Special Emphasis Inspection, TSA Had Conducted Limited 
Testing of Air Carriers' Capability to Perform Similar-Name Matching: 

Although TSA assessed air carriers' compliance with watch-list-matching 
requirements through a special emphasis assessment conducted in 2005 
and through planned inspections conducted in conjunction with annual 
inspection cycles, the agency had tested similar-name matching to a 
limited extent until 2008. For instance, the 2005 special emphasis 
assessment focused on air carriers' capability to identify passenger 
names that were exact matches with names on the No Fly List, but did 
not address the capability to conduct similar-name matching. Also, 
during the most recent annual inspection cycle (fiscal year 2007), 
although some TSA inspectors tested air carriers' effectiveness in 
conducting similar-name matching, the inspectors did so at their own 
discretion and without specific evaluation criteria. However, during a 
special emphasis inspection conducted in January 2008, TSA found 
deficiencies in the capability of air carriers to conduct similar-name 
matching.[Footnote 47] Thereafter, following TSA's revision of the No 
Fly List security directive in April 2008, officials planned to issue 
new guidance for inspectors to better ensure compliance by air carriers 
with requirements in the new security directive (e.g., by providing 
uniform evaluation criteria consistent with the new requirements). In 
response to our request for updated information on its oversight 
efforts, TSA provided us the results of a special emphasis assessment 
(conducted in May 2008) of seven air carriers' compliance with the 
revised No Fly List security directive. Although the details of this 
special emphasis assessment are classified, TSA officials generally 
characterized the results as positive. Further, TSA's noted that the 
agency's internal handbook--which provides guidance to transportation 
security inspectors on how to inspect air carriers' performance of 
various requirements, including watch-list-matching requirements--was 
being revised and was expected to be released later this year. Thus, 
TSA indicated that the new inspection guidance would be used in 
conjunction with the nationwide regulatory activities plan for fiscal 
year 2009. While these actions and plans are positive developments, it 
is too soon to determine the extent to which air carriers' compliance 
with watch-list-matching requirements will be assessed based on the new 
security directives since these efforts are still underway. 

TSA's Special Emphasis Assessment in 2005 Focused on Air Carriers' 
Exact-Name-Matching Capability: 

TSA conducted a special emphasis assessment in 2005 that tested the 
capability of domestic air carriers to find passenger names that were 
exact matches to names on the No Fly List. The 2005 special emphasis 
assessment was undertaken at the request of the TSA Administrator due 
to serious failures in air carriers' watch-list-matching processes, 
according to a senior TSA official. To conduct the assessment, TSA 
inspectors made flight reservations using the exact name of an 
individual who was on the No Fly List and not on the TSA Cleared List. 
If the air carrier identified the name on the reservation as a 
potential match to the individual on the No Fly List--and the check-in 
agent identified through the reservation system that further assistance 
was needed to finish the check-in process (e.g., to call security)--the 
test was considered to be successfully completed. According to TSA 
data: 

* air carriers passed a large majority of the initial tests conducted 
in June and July 2005, although several air carriers failed one or more 
tests and: 

* those air carriers that failed a test were retested in September 
2005, and a large majority of these air carriers passed the tests. 
[Footnote 48] 

Although TSA conducted a large number of tests, TSA officials stated-- 
and our own analyses confirmed--that results from this special emphasis 
assessment would not produce a reliable estimate of the success rate of 
all attempted matches by air carriers because TSA did not randomly 
select the air carriers, airports, or individual flights for review. As 
a result, the findings from this assessment cannot be used to infer 
overall or individual rates of success in identifying exact name 
matches in accordance with the No Fly and Selectee list security 
directives. That is, although the 2005 special emphasis assessment 
provided insight into air carriers' effectiveness in conducting a basic 
form of name matching, the picture provided was incomplete. Moreover, 
the air carriers' failure rates may have been considerably higher had 
the special emphasis assessment tested similar-name-matching 
capability, given that this capability involves more than finding a 
name that is a letter-for-letter match to another name. However, TSA 
officials told us that at the time of the special emphasis assessment 
in 2005, exact-name matching was the agency's focus. 

TSA Conducted Planned Watch-List-Related Inspections throughout the 
Year, but Inspectors Tested Air Carriers' Effectiveness at Similar-Name 
Matching at Their Own Discretion and without Baseline Evaluation 
Criteria: 

Since issuing the No Fly and Selectee list security directives in 2002, 
TSA has incorporated watch-list-related inspections into its regular 
inspection cycle, but inspectors tested air carriers' effectiveness in 
similar-name matching during these planned inspections to a limited 
extent and without specific evaluation criteria. In the most recent 
annual inspection cycle (fiscal year 2007), TSA conducted 1,145 
inspections of air carriers' compliance with watch-list-related 
requirements in the No Fly and Selectee security directives; 1,109 of 
these inspections were conducted at air carriers' airport locations by 
transportation security inspectors and 36 at air carriers' corporate 
security offices by principal security inspectors.[Footnote 49] The 
1,145 inspections covered 60 of the 72 domestic air carriers to which 
the security directives applied during fiscal year 2007, and most of 
the carriers were inspected multiple times that year.[Footnote 50] TSA 
found air carriers in compliance with required procedures in 1,133 (99 
percent) of the 1,145 inspections.[Footnote 51] 

These inspections were based on one or more inspection guidelines 
(called PARIS prompts) and were sometimes conducted in combination with 
inspections related to other regulatory requirements, such as 
performing criminal history record checks on employees or implementing 
CAPPS procedures. Table 2 presents the inspection guidelines TSA used 
to assess a key security directive requirement that we reviewed-- 
matching passenger names to the No Fly and Selectee lists.[Footnote 52] 
Additional guidelines used to assess other requirements in our review 
are presented in appendix I.[Footnote 53] 

Table 2: Requirements for Matching Passenger Data to No Fly and 
Selectee Lists and Inspection Guidelines Used to Assess Compliance with 
the Requirements: 

Requirements for matching passenger data to No Fly and Selectee lists: 
* Within 24 hours of scheduled flight departure time, air carriers are 
to compare records from the most recently issued No Fly and Selectee 
lists with identifying information on passengers found in the 
respective air carrier's reservation system and offered by passengers 
at the time of check-in; 
* When comparing data, air carriers must identify name matches 
(including similar-name matches) to the No Fly and Selectee lists; 
* To determine which passengers are matches, a passenger's name and one 
piece of identifying information (found either within the air carriers' 
reservation system or supplied by the passenger at check-in) must match 
with corresponding information provided on the No Fly or Selectee 
lists; 

Inspection guidelines: Transportation security inspectors: 
* All passenger names are compared to the most current No Fly and 
Selectee lists; 
* The aircraft operator is comparing all passenger names to the most 
current No Fly and Selectee lists in accordance with the procedures 
outlined in Security Directive 1544-01-20 series (No Fly) and Security 
Directive 1544-01-21 series (Selectee); 

Inspection guidelines: Principal security inspectors: 
* Procedures are in place to ensure the most recently issued No Fly 
List is utilized within 24 hours of receipt; 
* Procedures are in place to ensure the most recently issued Selectee 
List is utilized within 24 hours of receipt; 
* Procedures are in place to contact the Federal Security Director, 
local law enforcement, the FBI, and TSA Office of Intelligence for 
matches to the No Fly List; 
* Records are maintained of all flights operated with passengers who 
were determined by local law enforcement, U.S. legal attaché, or TSA 
Office of Intelligence not to be a match. 

Sources: GAO analysis of TSA's No Fly List Procedures security 
directive (SD 1544-01-20 series) and Selectee List Procedures security 
directive (SD 1544-01-21 series), versions dated July 8, 2004, and 
March 8, 2007, and inspection guidelines applicable during fiscal year 
2007. 

[End of table] 

The inspections conducted by transportation security inspectors at 
airports used the guidelines in table 2 to assess air carriers' 
compliance in matching passenger data to the No Fly and Selectee lists 
in fiscal year 2007. However, these inspectors tested exact-name and 
similar-name matching during these inspections at their own discretion; 
moreover, an official in TSA's Office of Security Operations, 
Compliance Division, stated that, generally, transportation security 
inspectors test exact-name-matching capability only. This inspection 
guideline is broadly written and does not specify the methods for 
validating compliance with the requirement to perform name comparisons. 
According to a TSA official in the Office of Security Operations, field 
inspectors may validate compliance by asking check-in agents to 
demonstrate that they have access to the current No Fly and Selectee 
lists and that any hard copies of the lists are properly protected; 
they may also interview check-in agents to ensure that they understand 
the security directive requirements, observe them as they process 
passengers who have been identified as Selectee or No Fly individuals, 
and/or test the air carriers' system by requesting a gate pass in the 
name of an individual on the watch list. We found evidence of field 
inspectors testing air carriers' name matching systems in 55 of the 
1,109 inspections they conducted in fiscal year 2007 (such tests may 
have been administered during the other inspections conducted in fiscal 
year 2007 but were not documented). 

For the 36 inspections conducted by principal security inspectors at 
air carriers' corporate security offices, we found 6 inspection records 
that referred to tests of exact-name and similar-name matching 
capability (they may have administered such tests during the other 
inspections they conducted that year but did not document the tests). 
Principal security inspectors did not have an inspection guideline 
directing them to assess exact-name and similar-name matching 
capability specifically--thus they tested this capability at their own 
initiative, and then reported their methods and results in conjunction 
with one of the four guidelines presented in table 2. Further, in 
response to our inquiry, 6 of TSA's 9 principal security inspectors 
told us that their assessments have not included examining air 
carriers' capability to conduct certain basic types of similar-name 
comparisons. 

TSA establishes in guidance for inspections (including watch-list- 
related inspections) that testing is the preferred method for assessing 
air carriers' compliance with regulations whenever possible and that it 
is only through testing that security can be assured.[Footnote 54] TSA 
further establishes in inspection guidance that when regulatory 
requirements encompass critical layers of security, more intensive 
oversight is needed, and compliance typically is to be validated 
through testing, inspections, surveillance, special emphasis 
assessments, and special emphasis inspections.[Footnote 55] Without 
routinely testing air carriers' compliance with the similar-name- 
matching requirement, TSA may not have reliable data on the 
effectiveness of air carriers' watch-list-matching processes and could 
be hindered in taking timely action to address any deficiencies. 

Inspectors who have tested air carriers' effectiveness in performing 
similar-name matching have done so without specific evaluation 
criteria. As discussed earlier, for any given name there are a number 
of possible name variations that could be used for travel, but TSA 
inspectors did not have baseline criteria on the number or types of 
such variations that must be evaluated. In the absence of specific 
standards for similar-name matching that all air carriers must follow, 
TSA has had no assurance that its inspections are based on uniform 
evaluation criteria. The inspections may not have been conducted 
uniformly and may have produced inconsistent results, given the absence 
of specific standards. In fall 2007, TSA began to review the adequacy 
of inspection guidance used by principal security inspectors, including 
guidance for watch-list-related inspections. As discussed in the 
following section, TSA expects to provide baseline criteria on the 
number and types of such variations inspectors must evaluate, but had 
not completed these efforts as of early September 2008. 

A Special Emphasis Inspection Conducted in 2008 Found Deficiencies in 
Air Carriers' Similar-Name-Matching Capabilities, and TSA Has Plans for 
Corrective Actions: 

During the course of our review and following TSA's discovery of a 
major air carrier's inability to effectively conduct both exact-name 
and similar-name-matching against the No Fly List, TSA initiated a 3- 
day, special emphasis inspection in January 2008 that tested the 
capability of 83 air carriers to conduct watch-list matching.[Footnote 
56] According to TSA officials, this inspection covered 52 domestic air 
carriers and 31 foreign air carriers. To implement the special emphasis 
inspection, TSA used 100 names on the No Fly List to test the 83 air 
carriers' capability to identify both exact-name and similar-name 
matches based on various types of possible name variations. On the 
basis of test results, a senior TSA official stated that the agency has 
confidence in air carriers' capability to identify exact-name matches. 
Regarding the capability to identify similar-name matches, TSA found 
that no air carrier was successful in identifying matches involving all 
types of name variations, although some carriers were more effective 
than others. 

On the basis of this inspection, TSA officials stated that they began 
to strengthen oversight of air carriers' similar-name-matching 
capability. For example, the TSA officials explained that--after a 30- 
day period following issuance of the revised No Fly List security 
directive in April 2008--the agency's inspectors would begin to 
evaluate air carriers' performance in complying with the new 
requirements. TSA officials explained that these initial inspections 
would be conducted at air carriers' corporate security offices and at 
airports. Officials further stated that after these initial 
inspections, others would be conducted periodically and, if applicable, 
TSA would impose progressively stronger enforcement actions against air 
carriers that are not successful in meeting the new standards. 

In September 2008, in response to our request for updated information 
on the status of its oversight efforts, TSA provided us the results of 
a special emphasis assessment (conducted during May 20-29, 2008) of 
seven air carriers' compliance with new requirements in the No Fly List 
security directive. Although the details of this special emphasis 
assessment are classified, TSA generally characterized the results as 
positive. Also, TSA plans to work with individual air carriers, as 
applicable, to analyze specific failures, improve system performance, 
and conduct follow-up testing as needed. 

In further reference to revision of the No Fly List security directive 
in April 2008, TSA officials stated that the agency's internal guidance 
is being updated to align inspection guidance with the revised 
directive. The officials elaborated that the new inspection guidance 
will place more emphasis on testing the effectiveness of security 
measures rather than using a checklist approach to determine whether an 
air carrier has a particular procedure in place. Regarding the emphasis 
on testing, our review noted that the draft guidance being developed 
for principal security inspectors included testing scenarios based on 
the types of name variations that air carriers must be capable of 
conducting in accordance with the revised watch-list-matching 
requirements. Also, according to TSA, guidance for transportation 
security inspectors is being developed (as part of the 2009 Regulatory 
Activities Plan) to provide more specific direction to inspectors for 
assessing name-matching capability. In September 2008, in response to 
our inquiry, TSA noted that the agency's internal handbook--which 
provides guidance to transportation security inspectors on how to 
inspect air carriers' performance of various requirements, including 
watch-list-matching requirements--was being revised and was expected to 
be released later this year. Thus, TSA indicated that the new 
inspection guidance would be used in conjunction with the nationwide 
regulatory activities plan for fiscal year 2009. Overall, the actions 
taken (and planned to be taken) by TSA are positive developments, 
although it is too soon to determine the extent to which TSA will 
assess air carriers' compliance with the revised watch-list-matching 
requirements. 

According to TSA officials, there were other benefits stemming from the 
January 2008 special emphasis inspection. For example, officials stated 
that in considering options for corrective actions, TSA consulted with 
representatives from the intelligence community, which is responsible 
for identifying names (and variations of names)[Footnote 57] for 
inclusion on the No Fly and Selectee lists. According to TSA, these 
discussions enhanced the intelligence community's understanding of how 
air carriers use the No Fly and Selectee lists, and as a result, the 
intelligence community is better positioned to carefully consider which 
name variations are appropriate for being added to the lists and 
whether these variations would be helpful for the purposes of watch- 
list matching. Further, TSA officials noted that such considerations, 
in turn, could benefit air carriers and the public by limiting the 
number of passengers who are misidentified as being potential matches 
with watch-list records. TSA officials added that insights regarding 
the extent to which name variations exist on the No Fly and Selectee 
lists also have benefited ongoing efforts to design and implement the 
Secure Flight program. Specifically, officials explained that TSA now 
has a fuller understanding of the types of name variations presently 
contained in watch-list records and, in turn, a fuller understanding of 
what types of comparisons Secure Flight should be capable of 
performing. 

Concluding Observations: 

Shortcomings that have national security implications exist in the 
watch-list-matching capability of domestic air carriers, as confirmed 
by the results of TSA's recent special emphasis inspection. 
Specifically, TSA found differences among air carriers in the 
thoroughness and effectiveness of their processes for comparing 
passengers' names with those on the No Fly List. A particular concern 
involves similar-name comparisons. However, TSA's April 2008 revision 
of the No Fly List security directive establishes a baseline name- 
matching capability by specifying the types of name variations that air 
carriers' processes must be capable of identifying. Effective 
implementation of the baseline capability should strengthen watch-list- 
matching processes, especially for those air carriers that had been 
using less thorough approaches for identifying similar-name matches. 
Concurrently, revised internal guidance for TSA's inspectors can help 
ensure that compliance decisions are based upon testing and that these 
tests are carried out regularly, using the standards specified within 
the security directives as evaluation criteria. Also, if properly 
documented in inspection reports, the results of these tests could give 
TSA management better information on the quality of watch-list matching 
being conducted by air carriers, thereby improving TSA's monitoring of 
the overall security posture of the aviation sector. At the time of our 
review, TSA's process for revising its guidance was in the initial 
stages; thus it is too early to determine the extent to which updated 
guidance for principal security inspectors and transportation security 
inspectors would strengthen oversight of air carriers' compliance with 
the security directive requirements. Given continued delays in the 
implementation of the Secure Flight program, TSA's oversight of air 
carriers' compliance with watch-list-matching requirements remains an 
important responsibility. TSA officials acknowledge that the baseline 
capability specified in the revised No Fly List security directive and 
the similar revision planned for the Selectee List security directive-
-while an improvement--does not address all vulnerabilities identified 
by TSA and does not provide the level of risk mitigation that is 
expected to be achieved from Secure Flight. Thus, TSA intends to deploy 
the Secure Flight program beginning in January 2009 so that it may 
implement this more robust matching capability. 

Agency Comments: 

We provided a draft of our restricted report (GAO-08-453SU) to the 
Department of Homeland Security and the Department of Justice for 
review and comment. The Department of Homeland Security had no 
comments. The Department of Justice provided technical comments on the 
restricted version of this report, which we incorporated where 
appropriate. 

We will send copies of this report to the appropriate congressional 
committees; the Secretary of Homeland Security; and the U.S. Attorney 
General. We will make copies available to others upon request. The 
report will also be available at no charge on our Web site at 
[hyperlink, http://www.gao.gov]. 

If you or your staff have any questions about this report or wish to 
discuss the matter further, please contact me at (202) 512-3404 or 
berrickc@gao.gov. 

Contact points for our Offices of Congressional Relations and Public 
Affairs may be found on the last page of this report. GAO staff who 
made major contributions to this report are listed in appendix III. 

Signed by: 

Cathleen A. Berrick: 
Director, Homeland Security and Justice Issues: 

List of Congressional Committees: 

The Honorable Robert C. Byrd:
Chairman:
The Honorable Thad Cochran:
Ranking Member:
Committee on Appropriations:
United States Senate: 

The Honorable Daniel K. Inouye:
Chairman:
The Honorable Kay Bailey Hutchison:
Ranking Member:
Committee on Commerce, Science, and Transportation:
United States Senate: 

The Honorable Joseph I. Lieberman:
Chairman:
The Honorable Susan M. Collins:
Ranking Member:
Committee on Homeland Security and Governmental Affairs:
United States Senate:
The Honorable Patrick J. Leahy: 

Chairman:
The Honorable Arlen Specter:
Ranking Member:
Committee on the Judiciary:
United States Senate: 

The Honorable John D. Rockefeller, IV:
Chairman:
The Honorable Kay Bailey Hutchison:
Ranking Member:
Subcommittee on Aviation Operations, Safety, and Security:
Committee on Commerce, Science, and Transportation:
United States Senate: 

The Honorable Dave Obey:
Chairman:
The Honorable Jerry Lewis:
Ranking Member:
Committee on Appropriations:
House of Representatives: 

The Honorable Bennie G. Thompson:
Chairman:
The Honorable Peter T. King:
Ranking Member:
Committee on Homeland Security:
House of Representatives: 

The Honorable Henry A. Waxman:
Chairman:
The Honorable Tom Davis:
Ranking Member:
Committee on Oversight and Government Reform:
House of Representatives: 

The Honorable James L. Oberstar:
Chairman:
The Honorable John L. Mica:
Ranking Republican Member:
Committee on Transportation and Infrastructure:
House of Representatives: 

The Honorable David E. Price:
Chairman:
The Honorable Harold Rogers:
Ranking Member:
Subcommittee on Homeland Security:
Committee on Appropriations: 

House of Representatives:
The Honorable Judd Gregg:
United States Senate:
The Honorable Don Young:
House of Representatives: 

[End of section] 

Appendix I: Objectives, Scope, and Methodology: 

Objectives: 

To examine the current processes that domestic air carriers use to 
conduct watch-list matching for domestic flights, we addressed the 
following questions: (1) What are TSA's requirements for domestic air 
carriers to conduct watch-list matching for domestic flights? (2) To 
what extent has TSA assessed domestic air carriers' compliance with 
watch-list-matching requirements? 

Scope and Methodology: 

In addressing the principal questions, we drew upon our previous work 
and reports on aviation security--specifically, reports covering TSA's 
inspection process, Secure Flight, and other passenger prescreening 
programs. We also consulted our most recent reports and testimonies on 
terrorist watch lists. In addition, we reviewed relevant studies 
conducted by other governmental agencies, including the Congressional 
Research Service and the Department of Justice's Office of Inspector 
General. This report is a public version of the restricted report that 
we provided to congressional committees in July 2008.[Footnote 58] 

More details about the scope and methodology of our work to address 
each of the principal questions are presented in the following 
sections, respectively. 

TSA's Requirements for Air Carriers to Conduct Watch-List Matching for 
Domestic Flights: 

To determine TSA's requirements for air carriers to match passenger 
information against the No Fly List and the Selectee List for domestic 
flights, we assessed two key TSA documents--the No Fly List Procedures 
security directive and the Selectee List Procedures security directive. 
[Footnote 59] We reviewed versions of these security directives--
including the revisions made in April 2008--to identify applicable 
requirements for watch-list matching. For the purposes of this report, 
we considered applicable requirements to be those that, according to 
TSA, would be assumed by the Secure Flight program, once operational, 
and those that TSA had itself identified for its oversight 
activities.[Footnote 60] Thus, we identified the following requirements 
(or key processes) as being within this scope (see table 1, which is 
presented earlier in this report): (1) the retrieval of the No Fly and 
Selectee lists, (2) the matching of passenger and watch-list 
information, (3) the use of the TSA Cleared List, (4) procedures for 
notifying authorities, and (5) keeping appropriate records.[Footnote 
61] 

To further our understanding of these requirements, we reviewed TSA 
policies and other guidance applicable to watch-list matching. We also 
interviewed officials from TSA's Office of Security Operations, which 
had primary responsibility for writing the security directives, and 
officials from two TSA offices that collaborated with the Office of 
Security Operations in crafting critical sections of the directives-- 
the Office of Transportation Sector Network Management and the Office 
of Intelligence. To better understand TSA's rationale for similar-name- 
matching requirements as well as the challenges associated with name- 
based matching, we attended meetings of the interagency Federal 
Identity Match Search Engine Performance Standards Working Group, which 
was organized by the Terrorist Screening Center to help ensure 
awareness of best practices with regard to identity matching among 
federal agencies, and spoke with one of the group's experts working in 
the field of name matching.[Footnote 62] To obtain information on the 
composition and use of the No Fly and Selectee lists, we spoke with 
officials from the Department of Justice's Terrorist Screening Center 
and TSA's Office of Intelligence. Further, to understand how TSA 
compiles and disseminates its Cleared List to air carriers, we spoke 
with officials from the Department of Homeland Security's Traveler 
Redress Inquiry Program (TRIP) and TSA's Office of Transportation 
Security Redress, which share responsibility for managing the TSA 
Cleared List for the current watch-list-matching process. Finally, to 
compare the current watch-list-matching process with that proposed once 
the federal government performs watch-list matching, we reviewed recent 
Secure Flight program documents.[Footnote 63] 

To generally understand how domestic air carriers have responded to 
TSA's requirements, we selected for interviews a nonprobability sample 
of 14 air carriers from a TSA-provided list of 95 air carriers that 
were subject to the watch-list-matching security directives for fiscal 
year 2005. To ensure that our sample of air carriers reflected a range 
of operational sizes, we based our selections partly on data from the 
U.S. Department of Transportation, which places air carriers in size 
categories based on operating revenue. Specifically, we selected 8 that 
were considered "major" air carriers, each having more than $1 billion 
in operating revenue in 2005; all but one of these 8 major air carriers 
flew internationally. In addition, we selected 3 air carriers the 
Department of Transportation identified as "national" air carriers, 
each having more $100 million to $1 billion in operating revenue in 
2005, and 1 air carrier the department identified as a "regional" air 
carrier, with $100 million or less in operating revenue. We also 
selected two air carriers from the list that were not included in the 
Department of Transportation's revenue groupings, given the small scale 
of their operations, but were identified by the department as air 
carriers that provide commuter service. National, regional, and 
commuter air carriers--which generally provided service covering a 
geographical area, such as the Pacific Northwest--had comparatively 
smaller business operations. 

In selecting the 14 air carriers, we also considered the number of 
passengers transported. To determine this number, we used the 
Department of Transportation's data for number of revenue passengers 
who enplaned (boarded) domestic air carriers during calendar year 2005-
-the most recent year for which data were available when making our 
selections in 2006.[Footnote 64] To the extent possible, we identified 
the number of domestic enplanements for those air carriers required to 
perform watch-list matching in 2005, identified within the previously 
cited TSA list. According to our calculations, the 14 air carriers in 
our study accounted for approximately 70 percent of all passengers who 
boarded domestic air carriers' flights during calendar year 2005, and 
thus, our selection allowed us to understand how watch-list matching 
was performed for the majority of passengers flying domestically in 
2005. Although the 14 domestic air carriers we selected represent a 
range in size of air carrier operations and transported a majority of 
passengers that boarded domestic flights in calendar year 2005, the 
results of our interviews are not generalizable to all domestic air 
carriers. 

To help ensure consistency in conducting our interviews with air 
carriers, we developed a data collection instrument with questions 
focusing on air carriers' implementation of certain requirements of the 
No Fly and Selectee list security directives. We conducted four of 
these interviews in person at the air carriers' headquarters and the 
rest via telephone. In addition, to clarify our understanding of air 
carriers' processes, we conducted follow-up phone interviews with four 
selected air carriers and received written answers to our follow-up 
questions from an additional four selected air carriers. The air 
carrier officials who answered our questions generally held positions 
in corporate security and regulatory affairs; however, half of the air 
carriers also had information technology systems specialists 
participate to answer technical questions related to automated name- 
matching systems. We did not audit or independently verify each air 
carrier's implementation of TSA's security directive requirements; 
rather, our work summarizes the capabilities as reported by officials 
at the 14 air carriers. 

Finally, to understand challenges air carriers have experienced in 
implementing watch-list-matching requirements, we examined TSA's case 
files on all regulatory violations of the No Fly List Procedures and 
the Selectee List Procedures security directives reported since the 
directives were first issued by TSA in 2002 to the time TSA provided us 
with the data in November 2007--a total of 32 cases.[Footnote 65] We 
reviewed these case files, which contained documentation and other 
legal analyses pertaining to TSA's inspection findings following the 
discovery of the violation, to determine the nature and causes (i.e., 
human or electronic) of the violations and to identify any patterns 
among the cases. Finally, to clarify the agency's process for 
investigating and adjudicating security directive violations, we spoke 
with officials from TSA's Office of Chief Counsel. 

Extent to Which TSA Has Assessed Domestic Air Carriers' Compliance with 
Watch-List-Matching Requirements for Prescreening Passengers: 

To address this objective, we first obtained an overview of TSA's plans 
and guidance for assessing air carriers' compliance with regulatory 
requirements. For instance, to understand the inspection process, the 
focus of inspections, and inspection methods, we reviewed TSA's 
National Inspection Manual, the Principal Security Inspector Handbook, 
and related implementing guidance and policy documents. Further, we 
interviewed or received written responses to our submitted questions 
from the general manager of TSA's Office of Transportation Sector 
Network Management, the two branch chiefs in the office's Commercial 
Aviation Sector, and all nine of the office's principal security 
inspectors. We particularly focused on contacting the principal 
security inspectors because they are responsible for conducting 
inspections at air carriers' corporate security offices (where watch- 
list-matching policies and procedures are formulated) that apply across 
an air carrier's operations. In addition, to obtain information on the 
creation of inspection plans and guidance and the compilation and 
analysis of inspection data, we spoke with individuals in the Office of 
Security Operations and the Office of Transportation Sector Network 
Management. Also, to obtain management's perspectives on inspections, 
we spoke with the assistant general managers of the Office of Security 
Operations' Compliance Division and its Procedures Division. We also 
interviewed two federal security directors[Footnote 66] and two 
transportation security inspectors, also within TSA's Office of 
Security Operations and who were located in the Washington, D.C., 
metropolitan area, on planning and conducting inspections. 

After obtaining an understanding of TSA's plans and guidance for 
assessing air carriers' compliance with regulatory requirements, we 
reviewed the results of TSA inspections that are scheduled on a regular 
basis in conjunction with annual inspection plans. In conducting 
inspections each year, TSA's inspectors use an extensive list of 
inspection guidelines (known as PARIS prompts)[Footnote 67] that cover 
a broad range of applicable topics--including topics outside the scope 
of our review, such as airport perimeter security and cargo security, 
as well as screening of employees and baggage.[Footnote 68] As 
presented in table 3, we determined that TSA used 11 inspection 
guidelines during fiscal year 2007 that were relevant to the objectives 
of our review.[Footnote 69] Of these, guidelines 1, 2, and 6 through 11 
were applicable to inspections conducted by principal security 
inspectors, while guidelines 3 through 5 were applicable to inspections 
conducted by transportation security inspectors. 

Table 3: Watch-List-Matching Requirements and the Related Inspection 
Guidelines (Fiscal Year 2007): 

Requirements (key processes): Retrieving the No Fly and Selectee lists; 
Inspection guidelines (prompts): 
1. Procedures are in place to ensure the most recently issued No Fly 
List is utilized within 24 hours of receipt; 
2. Procedures are in place to ensure the most recently issued Selectee 
List is utilized within 24 hours of receipt. 

Requirements (key processes): Matching passenger data to No Fly and 
Selectee lists; 
Inspection guidelines (prompts): 
3. All passenger names are compared to the most current No Fly and 
Selectee lists in accordance with the Private Charter Standard Security 
Program; 
4. The aircraft operator is comparing all passenger names to the most 
current No Fly and Selectee lists in accordance with the procedures 
outlined in Security Directive 1544-01-20 series (No Fly) and Security 
Directive 1544-01-21 series (Selectee). 

Requirements (key processes): Using the TSA Cleared List; 
Inspection guidelines (prompts): 
5. A passenger identified as a match on the Selectee List is cleared, 
along with his or her accessible property. 

Requirements (key processes): Notifying authorities; 
Inspection guidelines (prompts): 
6. Procedures are in place to contact the federal security director, 
local law enforcement, FBI, and TSA Office of Intelligence for matches 
to the No Fly List; 
7. Procedures are in place to contact the TSA Office of Intelligence 
for matches to the Selectee List. 

Requirements (key processes): Keeping records[A]; 
Inspection guidelines (prompts): 
8. Records are maintained of all flights operated with passengers who 
were determined by a local law enforcement, U.S. legal attaché, or TSA 
Office of Intelligence not to be a match; 
9. Records are maintained of every flight operated with passengers who 
are designated as selectees; 
10. Records are maintained of every flight with an individual who is 
cleared to fly utilizing data in the TSA Cleared List including the 
name of the cleared individual and the accepting aircraft operator 
representative[B] (No Fly List); 
11. Records are maintained of every flight with an individual who is 
cleared to fly utilizing data in the TSA Cleared List including the 
name of the cleared individual and the accepting aircraft operator 
representative[B] (Selectee List). 

Sources: GAO analysis of TSA's security directives and related 
guidance. 

[A] Maintaining accurate records, according to TSA officials, provides 
a starting point for an investigation in the event of a terrorist 
incident. 

[B] This inspection guideline reflects the current process, which is to 
use the TSA Cleared List. Security directives in effect prior to April 
2008 referenced a "cleared column," a format for clearing passengers. 
TSA eventually replaced this format with the Cleared List and updated 
language in the April 2008 revision of the No Fly List Procedures 
security directive to reflect the new process. 

[End of table] 

In reference to the 12 inspection guidelines--the 11 guidelines listed 
in table 3 and the 1 guideline discussed in footnote 12 of this 
appendix--TSA queried its PARIS database to identify all inspections of 
domestic air carriers conducted during fiscal year 2007 that used at 
least one of these guidelines. In addition to determining the number of 
inspections, we reviewed the fiscal year 2007 inspection data to 
calculate compliance rates.[Footnote 70] We did not evaluate the 
substantive basis for the inspectors' assessment decisions regarding 
compliance with requirements. 

To determine whether and to what extent TSA's inspectors tested the air 
carriers' capability to conduct exact-name and similar-name matching, 
we also reviewed documentation of testing in a data field (in the PARIS 
database) that allowed inspectors to enter narrative comments regarding 
similar-name matching, among other inspection activities. In doing so, 
we conducted a formal content analysis by having two analysts 
independently review comments in the data field and then resolve any 
inconsistencies between the two sets of analytical observations. 
Moreover, we submitted written questions to each of TSA's nine 
principal security inspectors asking them to describe their practices 
for testing air carriers' capability to identify similar-name 
variations. 

In contrast to these regular inspections, TSA also conducted a special 
emphasis assessment and a special emphasis inspection, nonroutine 
activities conducted at the direction of TSA headquarters. A special 
emphasis assessment addresses a vulnerability that generally is not 
tied to a regulation, while a special emphasis inspection is tied to a 
regulatory requirement. TSA provided us information on the scope, 
methodology, and results of a special emphasis assessment that TSA 
conducted during June, July, and September 2005. We reviewed the scope, 
methodology, and results of this assessment with our methodologists and 
with TSA officials. We determined that the sampling and related 
procedures used for the special emphasis assessment were insufficient 
for providing a reliable estimate of the success rate of all attempted 
matches by air carriers; thus, the results cannot be used to infer 
overall or individual rates of compliance with the name-matching 
requirements in TSA's security directives. 

In February 2008, TSA provided us a briefing on the scope and 
methodology of a special emphasis inspection conducted the month before 
in which the similar-name-matching capability of 52 domestic air 
carriers and 31 foreign air carriers was tested. The briefing also 
covered analyses of the results to date of the special emphasis 
inspection and a discussion of the corrective actions that TSA was 
planning to implement to address deficiencies. In April 2008, TSA 
provided us with an updated briefing on its plans for corrective 
actions. In September 2008, we requested information on TSA's progress 
with these corrective actions. In response, TSA provided us the results 
of a special emphasis assessment (conducted during May 20-29, 2008) of 
seven air carriers' compliance with requirements in the April 2008 No 
Fly List security directive. We did not assess the reliability of the 
data TSA collected during the January 2008 special emphasis inspection 
nor the May 2008 special emphasis assessment. 

Reliability of Fiscal Year 2007 Inspections Data: 

In assessing the reliability of the fiscal year 2007 data that TSA 
provided us for watch-list-related inspections based on annual 
inspection cycles, we performed electronic testing, discussed the data 
system and any data inconsistencies we found with knowledgeable TSA 
officials, and reviewed existing information about the system. Although 
we determined that the data were reliable for the purposes of this 
report, we have concerns about TSA's process for querying its 
inspection database, and the potential for faulty output. The process 
is cumbersome and prone to user error due, in part, to differences that 
occur in the verbiage of inspection guidelines and types of inspections 
as they are revised over time. 

We conducted this performance audit from July 2006 to September 2008 in 
accordance with generally accepted government auditing standards. Those 
standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on the audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on the audit objectives. 

[End of section] 

Appendix II: Overview of Selected Domestic Air Carriers' Watch-List- 
Matching Processes: 

TSA's watch-list-matching requirements for domestic flights address 
five key process areas: (1) retrieval of the No Fly and Selectee lists, 
(2) the matching of passenger and list information, (3) the use of 
TSA's Cleared List, (4) notification procedures, and (5) record-keeping 
activities (see table 1).[Footnote 71] 

To generally understand how TSA's requirements for watch-list matching 
were being implemented, we reviewed documents in which TSA provided 
general information on air carriers' processes. We also interviewed 14 
domestic air carriers with operations ranging in size from 
international to commuter service about their watch-list-matching 
processes. All 14 air carriers were subject to TSA's requirements for 
comparing passenger information with records on the No Fly and Selectee 
lists and the TSA Cleared List.[Footnote 72] We asked each of the 14 to 
describe their processes for meeting TSA's requirements.[Footnote 73] 
The air carriers' implementation of these requirements can be discussed 
in reference to three time periods--before passenger check-in, at 
passenger check-in, and after passenger check-in--as reflected in the 
following sections, respectively, and as illustrated in figure 1. 

Before Passenger Check-in: Retrieving the No Fly and Selectee Lists and 
Executing Name Comparisons: 

The 14 air carriers told us that they obtain new versions of the No Fly 
and Selectee lists through one or both of the following methods (1) 
assigning an employee to monitor TSA's Web board for new postings at 
certain intervals throughout the day, and (2) receiving an e-mail 
message from TSA to the respective air carrier's security staff 
informing them of new No Fly and Selectee lists. Also, all 14 air 
carriers reported using passenger name record (PNR) data--data 
collected from the passenger at the time a reservation is made--to make 
comparisons against the No Fly and Selectee lists. Specifically, the 
air carriers said that they have implemented procedures to execute 
comparisons of PNR and watch-list data prior to scheduled flight 
departure. Most of the air carriers told us they do this by using 
computerized matching programs that automatically execute comparisons. 

Because the 14 air carriers we interviewed did not collect date of 
birth (an identifying data element that air carriers receive on the No 
Fly and Selectee lists) within PNR data, this information generally was 
not available for matching purposes prior to check-in. However, as 
discussed later in this appendix, several air carriers reported 
developing systems capable of accessing passenger date-of-birth 
information collected and stored outside of PNR data for use in 
comparisons conducted prior to check-in, but this information was not 
available for all of their passengers. Thus, the 14 air carriers we 
spoke with were limited to performing name-only comparisons--that is, 
comparisons of passenger names with names on the No Fly and Selectee 
lists--prior to check-in for at least some, if not all, passengers. All 
14 air carriers we spoke with reported conducting comparisons to 
identify exact-name matches of passengers and watch-list names. 
However, not every air carrier reported conducting comparisons to 
identify similar-name matches. 

At Passenger Check-in: Completing Comparisons of Passenger and Watch- 
List Information and Using TSA's Cleared List: 

In accordance with TSA requirements, air carriers are to collect 
additional identifying information at check-in to assist in identifying 
passengers who are matches with information on the No Fly or Selectee 
lists. Air carriers collect additional identifying information at check-
in only for those passengers identified as potential matches to the No 
Fly or Selectee lists through the name-only comparisons they conduct 
prior to check-in. To prevent individuals who are potential matches 
from checking in by other means, such as using Internet or airport 
kiosk check-in, air carriers with automated systems place an automatic 
"lock" on boarding passes (see fig. 1).[Footnote 74] By doing so, the 
air carriers force all potentially matched passengers to check in at 
the ticket counter, where an agent is to collect a valid form of 
identification with date of birth (typically, a government-issued 
identification document such as a driver's license or passport) to 
complete the comparison of passenger and watch-list information. 

To check the potentially matched passenger's date of birth information 
against the No Fly and Selectee lists, most of the 14 air carriers we 
interviewed reported comparing the two dates manually, and the other 
air carriers reported keying the passenger's date of birth into a 
computer system that would automatically execute the comparison. 
[Footnote 75] The 14 air carriers reported that if they determine that 
the dates of birth do not match, they unlock the boarding pass without 
consulting TSA, in accordance with TSA requirements, thereby allowing 
the passenger to continue the boarding process (see fig. 1, post-check-
in number 1).[Footnote 76] However, if a passenger's date of birth 
matches with that of an individual on the No Fly or Selectee lists, the 
14 air carriers said that they consider the passenger to be a match and 
followed the procedures outlined in TSA's security directives for 
handling matches to the No Fly or Selectee lists (see fig. 1, post-
check-in numbers 2 and 3). 

Figure 1: Overview of the Current Passenger Watch-List-Matching 
Process: 

[See PDF for image] 

This figure is an illustration of the current passenger watch-list-
matching process, as follows: 

Precheck-in: 
* Passenger makes reservation; PNR is created; 
* TSA posts No Fly, Selectee, and TSA Cleared lists to secure Web 
board; 
* Air carrier retrieves list data; 
* Air carrier system compares PNR data to List data. 
* Potential match: Yes, proceed to check-in. 

Check-in: 
* Clearance process for locked PNRs (system match during comparison)? 
- Passengers present government ID at ticket counter; 
Agent compares passenger’s ID to No Fly, Selectee, and TSA Cleared list 
data. 
* Check-in process for Nonlocked PNRs (not a potential match in system 
comparison): Internet, kiosk or ticket counter. 

Post Check-in: 
* Matching results: 
1. Cleared: Passenger identified as not being on No Fly or Selectee 
list; Passenger identified as a match to cleared list: 
* PNR unlocked; Checkpoint screening; Passenger proceeds to flight. 

2. Selectee match: Passenger identified as a match to Selectee list: 
* PNR unlocked; Additional screening; Checkpoint screening; Passenger 
proceeds to flight. 

3. No Fly: Passenger identified as a match to No Fly list: 
* Air carrier contacts appropriate officials; 
- Not cleared; PNR locked; Boarding pass denied; Or: 
- PNR unlocked; Cleared; PNR unlocked; downgraded to selectee; 
Additional screening; Checkpoint screening; Passenger proceeds to 
flight; Or: 
- PNR unlocked; Cleared; Additional screening; Checkpoint screening; 
Passenger proceeds to flight. 

Source: GAO analysis. 

[End of figure] 

Also, 10 air carriers reported using the TSA Cleared List to identify 
and clear passengers misidentified as a match to the No Fly List or the 
Selectee List, generally at the time of check in. The other 4 air 
carriers reported not using the list--despite TSA's requirement that 
all air carriers do so. In addition, of the 10 air carriers that 
reported using the cleared list, 2 reported using the list in 
conjunction with their independently developed processes to "pre-clear" 
individuals (discussed below). Development of such processes was 
undertaken to allow air carriers to identify and clear misidentified 
passengers without requiring them to check in at the ticket counter. 
Specifically, 11 of the 14 air carriers we interviewed reported that 
individuals on the TSA Cleared List still must approach the ticket 
counter at check in.[Footnote 77] Consequently, 6 of the 14 air 
carriers that we interviewed reported developing alternative clearance 
processes to decrease the number of potentially matched individuals who 
are required to check in at the ticket counter. These 6 carriers 
explained that their internally developed clearance processes operate 
by using additional data sources, such as passenger information 
collected in frequent flier databases, to resolve potential matches 
prior to check in. For example, if an air carrier collected date of 
birth within its frequent flier database, its internal clearance system 
would compare the date of birth of a potentially matched passenger who 
had entered a frequent flier number when making a reservation with the 
date of birth of the respective individual on the No Fly List or the 
Selectee List.[Footnote 78] 

After Passenger Check in: Implementing the Notification and Record- 
Keeping Procedures Specified in TSA's No Fly and Selectee Security 
Directives: 

For match determinations made at the time of passenger check in, TSA's 
No Fly and Selectee list security directives require that air carriers 
follow certain notification and record-keeping procedures. With regard 
to notification procedures: 

* If the air carrier identifies a passenger as a potential match to the 
No Fly List, the air carrier must contact both the applicable federal 
security director and the appropriate law enforcement officer. Then, if 
the law enforcement officer confirms that the passenger is a match, the 
air carrier is to contact the local Federal Bureau of Investigation 
(FBI) field office and TSA's Office of Intelligence. 

* If the air carrier identifies a passenger as a potential match to the 
Selectee List, the air carrier must mark the passenger's boarding pass 
to indicate to checkpoint screeners that the passenger should be 
subject to enhanced checkpoint screening. Also, the air carrier must 
notify TSA's Office of Intelligence that the passenger has been matched 
with the Selectee List. 

With regard to record-keeping procedures, TSA's security directives 
require that air carriers maintain a record of (1) all passengers 
cleared using the TSA Cleared List, (2) all flights that had 
potentially matched passengers who were determined by local law 
enforcement not to be a match to the No Fly List, and (3) all 
passengers identified as matches with the Selectee List. 

Generally, the 14 air carriers told us that they followed the 
notification and record-keeping requirements specified in TSA's 
security directives, but reported having different procedures in place 
to implement these requirements. For example, upon identifying a 
potential match to the No Fly List, 5 air carriers reported requiring 
their ticket agents to notify their respective air carrier's ground 
security coordinator, who would then make the necessary calls to the 
applicable TSA federal security director and to local law enforcement. 
Three other air carriers reported requiring that ticket agents contact 
security staff at a centralized call center, and these staff would then 
make the necessary notifications.[Footnote 79] In addition, some of the 
carriers reported using some slight deviations from the stated 
requirements. For example, rather than notifying the local FBI field 
office and TSA's Office of Intelligence of a match only after a local 
law enforcement officer has confirmed the match, 8 air carriers 
reported contacting TSA's Office of Intelligence for every passenger 
whose information matched the No Fly List, regardless of the local law 
enforcement officer's input. [Footnote 80] 

[End of section] 

Appendix III: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

Cathleen A. Berrick, (202) 512-3404 or berrickc@gao.gov: 

Staff Acknowledgments: 

In addition to the contact named above, Danny Burton and Christine 
Fossett (Assistant Directors) and Mona Blake and Mike Bollinger 
(Analysts-in-Charge) managed this assignment. 

Suzanne Heimbach, Matt Mohning, Justin Monroe, Alison Sands, and Susan 
Woodward made significant contributions to the work. 

David Alexander, Michele Fejfar, and Rich Hung assisted with design, 
methodology, and data analysis. 

Tom Lombardi and David Plocher provided legal support. 

Richard Ascarate, Ryan Consaul, Kevin Copping, Kristen Jensen, Lara 
Kaskie, Maria Soriano, William D. Updegraff, and Margaret Vo provided 
assistance in report preparation. 

[End of section] 

Footnotes: 

[1] Watch-list matching is one of two TSA-mandated prescreening 
processes conducted by air carriers. The other mandated prescreening 
activity is the Computer Assisted Passenger Prescreening System, 
discussed later this report, which does not involve matching passenger 
information against the No Fly and Selectee lists. These lists contain 
applicable records from the Terrorist Screening Center's consolidated 
database of known or appropriately suspected terrorists. See GAO, 
Terrorist Watch List Screening: Recommendations to Promote a 
Comprehensive and Coordinated Approach to Terrorist-Related Screening, 
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-253T] (Washington, 
D.C.: Nov. 8, 2007). 

[2] The number of domestic air carriers has varied over time, for 
example, from 95 in 2005 to about 70 in 2007. For the purposes of this 
report, domestic air carriers are those with operations based in the 
United States that maintain full security programs in accordance with 
49 C.F.R. part 1544. Foreign air carriers--air carriers with operations 
based outside the United States--must also comply with U.S. security 
regulations, including applicable requirements for watch-list matching, 
when operating flights to or from the United States in accordance with 
49 C.F.R. part 1546. Both domestic and foreign air carriers may conduct 
international flights to and from the United States; however, these 
operations are outside the scope of this report. 

[3] See GAO, Terrorist Watch List Screening: Opportunities Exist to 
Enhance Management Oversight, Reduce Vulnerabilities in Agency 
Screening Processes, and Expand Use of the List, [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-08-110] (Washington, D.C.: Oct. 
11, 2007). We reported that TSA's Office of Intelligence documented 
various incidents (for the period January 1, 2005, through June 3, 
2007) in which air carriers--both domestic and foreign--allowed 
individuals on the No Fly List to board international flights traveling 
to or from the United States. Several of these incidents involved 
flights of domestic air carriers. We asked TSA's Office of Intelligence 
to identify any additional incidents in which a No Fly listed 
individual flew on a domestic air carrier for the period June 4, 2007, 
through December 31, 2007, and TSA identified no additional incidents 
occurring within this time period. 

[4] This issue of false negatives is addressed later in this report. 

[5] See, e.g., 49 C.F.R. § 1544.305. Although generally issued in 
response to an immediate or imminent threat, security directives may be 
effective for an indefinite duration if TSA determines that a 
continuing need for such measures exists. In some cases, aviation- 
related measures implemented through a security directive have been 
discontinued, amended, or incorporated into air carrier security 
programs. 

[6] See 49 U.S.C. 44903(j)(2)(C). 

[7] GAO, Aviation Security: Computer-Assisted Passenger Prescreening 
System Faces Significant Implementation Challenges, [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-04-385] (Washington, D.C.: Feb. 
13, 2004); Aviation Security: Management Challenges Remain for the 
Transportation Security Administration's Secure Flight Program, 
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-864T] (Washington, 
D.C.: June 14, 2006); and Aviation Security: Transportation Security 
Administration Has Strengthened Planning to Guide Investments in Key 
Aviation Security Programs, but More Work Remains, [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-08-456T] (Washington, D.C.: Feb. 
28, 2008). 

[8] GAO, Transportation Security: Efforts to Strengthen Aviation and 
Surface Transportation Security Continue to Progress, but More Work 
Remains, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-651T] 
(Washington, D.C.: Apr. 15, 2008). 

[9] See Pub. L. No. 110-161, Div. E, § 513, 121 Stat. 1844, 2072-73 
(2007). 

[10] GAO, Aviation Security: Management Challenges Remain for the 
Transportation Security Administration's Secure Flight Program, 
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-864T] (Washington, 
D.C.: June 14, 2006). 

[11] We are conducting this review in response to requests from the 
House of Representatives (Committee on Transportation and 
Infrastructure, Committee on Homeland Security, and Committee on 
Oversight and Government Reform). These requesters asked that we review 
the current passenger prescreening system in conjunction with our 
ongoing work related to TSA's progress with Secure Flight. In addition, 
we are reporting on this issue to the U.S. Senate requesters and the 
mandate committees associated with our Secure Flight work. 

[12] Pursuant to Homeland Security Presidential Directive 6, dated 
September 16, 2003, the Terrorist Screening Center--an entity that has 
been operational since December 2003 under the administration of the 
FBI--was established to develop and maintain the U.S. government's 
consolidated terrorist screening database (the watch list) and to 
provide for the use of watch-list records during security-related 
screening processes. 

[13] All 14 air carriers we interviewed operate under full security 
programs in accordance with 49 C.F.R. part 1544 and conduct watch-list 
matching in accordance with the No Fly and Selectee list security 
directives issued by TSA. 

[14] The Department of Transportation groups U.S.-based air carriers 
according to their operating revenue. In the 2005 groupings, each of 
the "major" air carriers had over $1 billion in operating revenue. 

[15] Of these six, the Department of Transportation's 2005 revenue 
groupings identified three as "national" air carriers, with each having 
over $100 million to $1 billion in operating revenue, and one as a 
"regional" air carrier, with $100 million or less in operating revenue. 
The other two air carriers were not included in the department's 
revenue groupings, given the small scale of operations, but were 
identified by the department as air carriers that provide commuter 
service. Major air carriers have over $1 billion in operating revenue. 

[16] The No Fly and Selectee list security directives also address the 
screening of air carrier employees against the No Fly and Selectee 
lists, but our scope was confined to the passenger-specific 
prescreening requirements in the security directives. 

[17] Special emphasis assessments and special emphasis inspections are 
nonroutine activities undertaken at the direction of TSA headquarters. 
According to TSA, a special emphasis assessment addresses a 
vulnerability that generally is not tied to a regulation, while a 
special emphasis inspection is tied to a regulatory requirement. 

[18] In September 2008, TSA provided us the results of a special 
emphasis assessment (conducted during May 2008) of seven air carriers' 
compliance with new requirements in the No Fly List security directive, 
which was revised in April 2008 to specify a baseline capability for 
conducting watch-list matching. This special emphasis assessment is 
discussed later in this report. 

[19] In September 2008, TSA informed us that the revised Selectee List 
security directive was still in the agency's internal clearance process 
but did not provide us a targeted issuance date. 

[20] PARIS is the acronym for the Performance and Results Information 
System, which is TSA's inspections database. This database assists TSA 
management by providing factual and analytical information on the 
compliance of TSA-regulated entities. There are approximately 1,700 
PARIS prompts, which serve as guidelines for TSA inspectors. 

[21] According to TSA data, these 1,145 watch-list-related inspections 
(36 plus 1,109) covered 60 domestic air carriers, and most of the air 
carriers were inspected multiple times. 

[22] TSA reported that the January 2008 special emphasis inspection 
covered 52 domestic air carriers and 31 foreign air carriers. 

[23] In accordance with 49 U.S.C. § 114(h), TSA adopted policies and 
procedures for ensuring that air carriers use information from 
government agencies to identify individuals on passenger lists who may 
be a threat to civil aviation or national security and, if such an 
individual is identified, notify appropriate law enforcement agencies, 
prevent the individual from boarding an aircraft, or take other 
appropriate action with respect to that individual. 

[24] Pub. L. No. 107-71, § 136, 115 Stat. 597, 637 (2001) (codified at 
49 U.S.C. § 44903(j)(2)(A)) (requiring use of the Computer Assisted 
Passenger Prescreening System or any successor system). 

[25] For the purposes of this report, we address policies and 
procedures applicable to air carriers regulated under 49 C.F.R. part 
1544 (U.S.-flagged air carriers), which we refer to as domestic air 
carriers. For these air carriers, we limit our discussion to the watch- 
list matching TSA requires to secure the aviation sector for domestic 
flights--air carrier operations between two points within the United 
States or its territories. TSA requirements also address the 
international operations of domestic air carriers, and the operations 
of foreign-flagged air carriers flying to and from destinations within 
the United States and its territories in accordance with 49 U.S.C. part 
1546; however, these requirements are outside the scope of our review. 

[26] The most recent version of the No Fly List Procedures security 
directive is SD 1544-01-20F, dated April 9, 2008, and the most recent 
version of the Selectee List Procedures security directive is SD 1544- 
01-21F, dated March 8, 2007. 

[27] On June 10, 2008, the Department of Justice provided us comments 
on a draft of the restricted version of this report (GAO-08-453SU) and 
noted that the Principals Committee, which is a senior interagency 
forum under the Homeland Security Council, had approved additional 
criteria that the Terrorist Screening Center would begin implementing 
on June 23, 2008. The Homeland Security Council was established to 
ensure coordination of all homeland-security-related activities among 
executive departments and agencies and promote the effective 
development and implementation of all homeland security policies. See 
The White House, Homeland Security Presidential Directive/HSPD-1, 
Organization and Operation of the Homeland Security Council 
(Washington, D.C.: Oct. 29, 2001). 

[28] Each watch-list record, however, does not necessarily indicate a 
separate individual on the list. Some listed individuals have multiple 
records attributed to them due to the inclusion of known aliases and 
name variations. 

[29] The lists may also be provided via password-protected e-mail. 

[30] TSA initiated this effort in response to the Aviation and 
Transportation Security Act, which requires that TSA ensure that a 
system is used to evaluate all passengers before they board an aircraft 
and ensure that selected individuals and their carry-on and checked 
baggage are adequately screened. See Pub. L. No. 107-71, § 136, 115 
Stat. at 637 (codified at 49 U.S.C. § 44903(j)(2)(A)). 

[31] The National Commission on Terrorist Attacks Upon the United 
States, The 9/11 Commission Report - Final Report of the National 
Commission on Terrorist Attacks Upon the United States (Washington, 
D.C.: 2004), p. 393. 

[32] Pub. L. No. 108-458, § 4012(a)(1), 118 Stat. 3638, 3714-17 (2004) 
(codified at 49 U.S.C. § 44903(j)(2)(C) (2004)). A separate provision 
enacted at section 4012(a)(2) addressed the predeparture screening of 
international passengers, with the Secretary of Homeland Security 
giving this responsibility to U.S. Customs and Border Protection. See 
49 U.S.C. § 44909(c)(6). 

[33] With regard to redress protections, DHS must have a process 
whereby aviation passengers determined to pose a threat to aviation 
security by Secure Flight may appeal that determination and correct 
erroneous information contained within the prescreening system. See 
GAO, Aviation Security: Significant Management Challenges May Adversely 
Affect Implementation of the Transportation Security Administration's 
Secure Flight Program, [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-06-374T] (Washington, D.C.: Feb. 9, 2006). 

[34] See 72 Fed. Reg. 48,356 (Aug. 23, 2007). Requirements described in 
the notice of proposed rulemaking are subject to revisions based on 
various considerations, including input that TSA received during the 
public comment period. As of the date of this report's issuance, DHS 
had not issued a final Secure Flight rule. 

[35] GAO, Aviation Security: Transportation Security Administration Has 
Strengthened Planning to Guide Investments in Key Aviations Security 
Programs, but More Work Remains, [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-08-465T] (Washington, D.C.: Feb. 28, 2008). 

[36] In September 2008, TSA informed us that the revised Selectee List 
security directive was still in the agency's internal clearance process 
but did not provide us a targeted issuance date. 

[37] When making determinations on matches, air carriers must use the 
TSA Cleared List, which is composed of names and other personal- 
identifying information on individuals whom the Department of Homeland 
Security has reviewed and determined are not individuals on the No Fly 
or Selectee lists. 

[38] Specifically, we reviewed and discussed the No Fly and Selectee 
list security directives and identified within each the key 
requirements pertaining to domestic flights. Although the same 
requirements generally apply to the international flights of both 
domestic and foreign air carriers, such operations fall outside the 
scope of our review. For more information on how we identified 
requirements for watch-list matching, see appendix I. 

[39] TSA's revised No Fly List Procedures security directive (SD 1544- 
01-20F) is dated April 9, 2008. Also, in April 2008, TSA reported that 
the current Selectee List Procedures security directive (SD 1544-01- 
21F) would be similarly revised. In September 2008, TSA informed us 
that the revised Selectee List security directive was still in the 
agency's internal clearance process but did not provide us a targeted 
issuance date. 

[40] We did not independently verify the air carriers' approaches to 
watch-list matching. Unless noted otherwise, our summary of the air 
carriers' approaches is based on system capabilities reported to us in 
14 separate interviews with the respective air carriers. Appendix II 
provides more detail on the 14 air carriers' reported approaches to 
watch-list matching. 

[41] Some of these flights involved passengers who flew from one 
domestic location to another domestic location, where they boarded an 
international flight. TSA learned that the individual on the No Fly 
List flew domestically after U.S. Customs and Border Protection 
identified the individual on the international leg. 

[42] GAO, Terrorist Watch List Screening: Opportunities Exist to 
Enhance Management Oversight, Reduce Vulnerabilities in Agency 
Screening Processes, and Expand the Use of the List, GAO-08-110 
(Washington, D.C.: Oct. 11, 2007). 

[43] [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-110]. 

[44] GAO, Aviation Security: Management Challenges Remain for the 
Transportation Security Administration's Secure Flight Program, 
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-864T] (Washington, 
D.C.: June 14, 2006). 

[45] Upon completing a reassessment of the Secure Flight program in 
February 2007, TSA produced this document to identify decisions made 
about Secure Flight's capabilities during the reassessment. See TSA, 
Secure Flight Program Baseline (Washington, D.C.: March 2007), p. 5. 

[46] As mentioned previously, in September 2008, TSA informed us that 
the revised Selectee List security directive was still in the agency's 
internal clearance process but did not provide us a targeted issuance 
date. 

[47] TSA reported that the January 2008 special emphasis inspection 
covered 52 domestic air carriers and 31 foreign air carriers. 

[48] According to TSA officials, the agency had planned to conduct 
tests of all 81 domestic air carriers that were subject to the No Fly 
List Procedures security directive at that time. However, the officials 
explained that due to limited resources, initial testing covered 63 air 
carriers (encompassing operations at 354 airports), and the retesting 
covered 36 air carriers (encompassing operations at 290 airports). 

[49] As noted earlier, we concluded that these inspection data were 
sufficiently reliable for the purposes of this report, but we have 
concerns about the potential for error based on TSA's process for 
querying its inspection database (we discuss these concerns in more 
detail in app. I). 

[50] Regarding the air carriers that did not receive a watch-list- 
related inspection during fiscal year 2007, TSA does not require 
inspectors to inspect each air carrier every year in terms of watch- 
list-related requirements. However, a senior TSA official in the 
compliance area who supervises inspectors stated that annually 
inspecting every air carrier is a goal, at least for principal security 
inspectors. 

[51] We did not evaluate the basis for the inspectors' assessment 
decisions regarding compliance with requirements. Although TSA's 
security directives require comparisons of passenger and employee names 
to the No Fly and Selectee lists, our review was confined to 
requirements related to passengers only. 

[52] To report their findings in TSA's automated database, inspectors 
select one of four options from a computer-generated list: not 
inspected, not applicable, not in compliance, and in compliance. If the 
inspectors wish to add narrative to describe their findings, they can 
do so in a data field reserved for comments. 

[53] In appendix I, see table 3. 

[54] TSA, National Inspection Manual, 2007. Inspections for all 
regulated areas (not just watch-list-related inspections) generally 
incorporate all of four methods--testing, document review, interviews, 
and surveillance. 

[55] TSA, Regulatory Activities Plan for Transportation Security 
Inspectors Fiscal Year 2008. 

[56] We briefed the TSA Administrator and other senior officials on the 
results of our work in November 2007. 

[57] As noted previously, each watch-list record does not necessarily 
indicate a separate individual on the list. Some listed individuals 
have multiple records attributed to them due to the inclusion of known 
aliases and name variations. 

[58] GAO, Aviation Security: Pending Implementation of Secure Flight, 
TSA Is Enhancing Its Oversight of Air Carrier Efforts to Identify 
Passengers on the No Fly and Selectee Lists, [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-08-453SU] (Washington, D.C.: July 
10, 2008). 

[59] These directives apply to domestic air carriers--that is, U.S. air 
carriers that maintain security programs in accordance with 49 C.F.R. 
part 1544. The directives govern watch-list matching for flights 
operating between two points within the United States or its 
territories. Although outside the scope of our review, the directives 
also apply to domestic air carriers' international operations. At the 
start of our review, we based our analysis on the No Fly List 
Procedures (1544-01-20D) security directive and the Selectee List 
Procedures (1544-01-21E) security directive, both dated July 8, 2004. 
Over the course of our review, TSA first issued revised security 
directives in 2007 and has undertaken to revise them again in April 
2008. The 2007 revisions of the No Fly and Selectee list security 
directives (SD 1544-01-20E and SD1544-01-21F, respectively) clarified 
certain elements of the directives but resulted in no substantive 
changes in the requirements. Generally, in this report, we focus on the 
changes in requirements resulting from revisions undertaken in April 
2008 (SD 1544-01-20F and anticipated SD 1544-01-21G (Selectee List), 
respectively). 

[60] We based our understanding of TSA's planned capabilities for 
Secure Flight on our February 2006 testimony before the Senate 
Committee on Commerce, Science, and Transportation, our most recent, 
comprehensive testimony on the program when we initiated our work in 
July 2006. See GAO, Aviation Security: Significant Management 
Challenges May Adversely Affect Implementation of the Transportation 
Security Administration's Secure Flight Program, [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO-06-374T] (Washington, D.C.: Feb. 
9, 2006). 

[61] Although addressed in the security directives, other requirements 
that we excluded from our scope involved, for example, procedures 
involving the screening of employees and procedures related to the 
international operations of domestic air carriers. We did not consider 
requirements for domestic air carriers' international flights as part 
of our review because at the time we were planning our review, TSA 
intended for Secure Flight to take over the watch-list-matching 
function for only domestic flights. U.S. Customs and Border Protection 
was expected to conduct the watch-list-matching function for flights 
arriving from or departing to locations outside the United States, not 
Secure Flight. However, in February 2008 we reported in testimony that, 
as agreed to by the respective agencies, TSA will also take over the 
matching of international passengers against the No Fly and Selectee 
lists from U.S. Customs and Border Protection. GAO, Aviation Security: 
Transportation Security Administration Has Strengthened Planning to 
Guide Investments in Key Aviation Security Programs, but More Work 
Remains, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-465T] 
(Washington, D.C.: Feb. 28, 2008). 

[62] One objective of the Federal Identity Match Search Engine 
Performance Standards Working Group is to provide guidance to improve 
the effectiveness of the automated search engines that federal agencies 
use for conducting identity matching. The group began meeting in 
December 2005. It included representatives from the departments of 
Homeland Security, State, and Defense; FBI; the intelligence community; 
and the National Institute of Standards and Technology. 

[63] Specifically, we reviewed the Secure Flight notice of proposed 
rulemaking (72 Fed. Reg. 48,356 (Aug. 23, 2007)) and final concept of 
operations for Secure Flight (dated Mar. 9, 2007). We also reviewed our 
most recent reports and testimonies on the program. 

[64] Specifically, the data reflect the number of domestic passengers 
who boarded (enplaned) at a flight's point of origin in calendar year 
2005. The data include only revenue passengers, or passengers from whom 
the air carrier received payment. As such, the data exclude passengers 
using frequent flier vouchers, infants, air carrier employees, etc. 

[65] The earliest case was dated December 3, 2003; the most recent was 
dated August 24, 2007. Because some domestic air carriers that are 
subject to security directives fly internationally, 7 of the 32 cases 
involved flights arriving from or departing to international locations. 
Although we excluded such flights from our review of watch-list- 
matching requirements, as mentioned previously, we retained these 7 
cases in our analysis of regulatory violations. We did so because (1) 
the requirements for air carriers to perform watch-list matching for 
flights involving an international location are, for the most part, the 
same as those for air carrier operations between two points within the 
United States or its territories, and (2) in August 2007, TSA announced 
that Secure Flight would eventually assume watch-list matching for 
passengers on flights arriving from or departing to locations outside 
the United States. 

[66] Federal security directors are responsible for leading and 
coordinating TSA security activities at airports across the nation. 

[67] The Performance and Results Information System (PARIS) is an 
inspections database that assists TSA management by providing factual 
and analytical information on the compliance of TSA-regulated entities. 

[68] As mentioned previously, the watch-list-matching requirements 
relevant to the objectives of our review are shown in table 1, which is 
presented earlier in this report. 

[69] TSA provided us with data for 12 inspection guidelines. These 12 
are the 11 guidelines shown in table 3--plus the following guideline, 
which was replaced in March 2007 with guideline 4 in table 3: "All 
passenger names are compared to the most current No Fly and Selectee 
Lists in accordance with the procedures outlined in Security Directive 
1544-01-20 series (No Fly) and Security Directive 1544-01-21 series 
(Selectee)." Because these two guidelines were used for the same 
purpose but at different times during fiscal year 2007, we combined the 
data associated with each one and treated them as one inspection 
guideline for the purposes of this report. 

[70] Our calculations were based only on the 12 inspection guidelines 
relevant to our review. 

[71] To identify these requirements, we reviewed the No Fly List 
Procedures and Selectee List Procedures security directives (series SD 
1544-01-20 and SD 1544-01-21, respectively). This report discusses only 
the requirements within the two security directives pertaining to 
domestic flights (defined as flights occurring between points within 
the United States and its territories), though these same requirements 
generally apply to the international flights of both domestic and 
foreign air carriers. For more information on how we identified 
requirements for watch-list matching, see appendix I. 

[72] For information on our methodology for selecting the 14 air 
carriers and conducting the interviews, see appendix I. 

[73] The implementation methods described in this appendix are based on 
descriptions obtained from the 14 air carriers. We did not undertake 
audits of the air carriers' processes to confirm that the processes 
functioned as described in the interviews. Specifically, we asked air 
carriers questions on methods for securing the most recent No Fly and 
Selectee lists, executing comparisons within required time frames, 
determining valid matches, and implementing required notification and 
reporting procedures. 

[74] The one air carrier in our review without an automated system 
reported requiring all passengers, regardless of whether they were a 
potential match, to check in at the ticket counter. To identify those 
passengers who should submit additional information for further 
comparison against the No Fly and Selectee lists at check-in, this air 
carrier reported having its employee in charge of watch-list matching 
make a written notation next to the name of all identified potential 
matches on a printed list of passengers with reservations. 

[75] In addition, to check potentially matched passenger information 
against the No Fly and Selectee lists, three air carriers reported that 
they had developed kiosks with capabilities to read electronic date of 
birth information from certain forms of identification that are machine 
readable. 

[76] After this point, the passenger generally experiences no further 
inconvenience due to watch-list matching. However, the passenger may be 
selected for enhanced checkpoint screening as a result of the Computer 
Assisted Passenger Prescreening System (CAPPS)--an electronic 
application that selects individuals for enhanced screening at the 
passenger checkpoint based on certain travel characteristics identified 
by TSA as indicating potential risk. 

[77] These individuals are required to check in at the ticket counter 
because the air carrier must confirm that the passenger is the cleared 
individual by comparing the passenger's legal identifying documentation 
with the TSA Cleared List. 

[78] Air carriers with frequent flier programs generally have the 
capability to collect a frequent flier number within the PNR; 
therefore, unlike date of birth information, frequent flier numbers are 
available to air carriers prior to a passenger's arrival at check-in 
and can be used to assist in the confirmation of a passenger's identity 
because of the presence of date of birth information in the passenger's 
frequent flier account. 

[79] Another air carrier reported requiring the ticket agent to make 
these notifications; the other five air carriers we interviewed did not 
discuss this aspect of the watch-list-matching process. 

[80] Two air carriers reported that (per the security directive 
requirement) they waited for local law enforcement officer confirmation 
before calling the FBI field office or TSA's Office of Intelligence. 
One air carrier reported that it could not answer the question; that 
is, having never identified an individual as a name and date of birth 
match to the No Fly List, the air carrier could not say what its 
actions would be. During our interviews, three air carriers did not 
discuss this aspect of the watch-list-matching process. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office: 
441 G Street NW, Room LM: 
Washington, D.C. 20548: 

To order by Phone: 
Voice: (202) 512-6000: 
TDD: (202) 512-2537: 
Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: