TSA Acquisitions:

TSA Needs to Establish Metrics and Evaluate Third Party Testing Outcomes for Screening Technologies

GAO-21-50: Published: Oct 29, 2020. Publicly Released: Oct 29, 2020.

Additional Materials:

Contact:

Marie A. Mak
(202) 512-4841
MakM@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

The machines that screen airline passengers and baggage must be tested before they can be purchased by the Transportation Security Administration. TSA does most of this testing itself. To make the process more efficient and get new technology into airports faster, TSA allows companies to have testing done by other testers in certain situations.

Companies have used this option just five times since 2013. TSA hopes more companies will take this route in the future, but does not know if the approach makes its process faster or more efficient.

Our 3 recommendations could help TSA determine whether third-party testing is helping it reach its goals.

airport security

Additional Materials:

Contact:

Marie A. Mak
(202) 512-4841
MakM@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

What GAO Found

In 2013, the Transportation Security Administration (TSA) introduced the concept of third party testing—having an independent testing entity verify that a security screening system meets certain requirements. The concept is that screening system vendors would take this additional step either prior to submitting their technologies to TSA or if their system failed TSA's test and evaluation process. The goal is for third party testing to reduce the time and resources that TSA spends on its own testing. However, since introduced, TSA has directed only three vendors that failed TSA tests to use third party testing, with varying outcomes. In two other cases, TSA supplemented its test capabilities by using third party testers to determine that systems installed at airports were working properly. TSA officials and industry representatives pointed to several reasons for third party testing's limited use since 2013, such as the cost to industry to use third party testers and TSA's reluctance to date to accept third party test data as an alternative to its own. Despite this, TSA officials told GAO they hope to use third party testing more in the future. For example, in recent announcements to evaluate and qualify new screening systems, TSA stated that it will require a system that fails TSA testing to go to a third party tester to address the identified issues (see figure).

Example of Use of Third Party Testing When a System Experiences a Failure in TSA's Testing

Example of Use of Third Party Testing When a System Experiences a Failure in TSA's Testing

TSA set a goal in 2013 to increase screening technology testing efficiency. In addition, TSA reported to Congress in January 2020 that third party testing is a part of its efforts to increase supplier diversity and innovation. However, TSA has not established metrics to determine third party testing's contribution toward the goal of increasing efficiency. Further, GAO found no link between third party testing and supplier diversity and innovation. Some TSA officials and industry representatives also questioned third party testing's relevance to these efforts. Without metrics to measure and assess the extent to which third party testing increases testing efficiency, TSA will be unable to determine the value of this concept. Similarly, without assessing whether third party testing contributes to supplier diversity and innovation, TSA cannot know if third party testing activities are contributing to these goals as planned.

Why GAO Did This Study

TSA relies on technologies like imaging systems and explosives detection systems to screen passengers and baggage to prevent prohibited items from getting on board commercial aircraft. As part of its process of acquiring these systems and deploying them to airports, TSA tests the systems to ensure they meet requirements.

The 2018 TSA Modernization Act contained a provision for GAO to review the third party testing program. GAO assessed the extent to which TSA (1) used third party testing, and (2) articulated its goals and developed metrics to measure the effects of third party testing.

GAO reviewed TSA's strategic plans, acquisition guidance, program documentation, and testing policies. GAO interviewed officials from TSA's Test and Evaluation Division and acquisition programs, as well as representatives of vendors producing security screening systems and companies providing third party testing services.

What GAO Recommends

GAO is recommending that TSA develop metrics to measure the effects of third party testing on efficiency, assess its effects on efficiency, and assess whether third party testing contributes to supplier diversity and innovation. DHS concurred with GAO's three recommendations and has actions planned to address them.

For more information, contact Marie A. Mak at (202) 512-4841 or MakM@gao.gov.

Recommendations for Executive Action

  1. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: The Administrator of TSA should establish metrics to measure the effects of third party testing on the efficiency of the test and evaluation process. (Recommendation 1)

    Agency Affected: Department of Homeland Security: Transportation Security Administration

  2. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: When performance metrics have been established, the Administrator of TSA should assess gains in efficiency resulting from third party testing. (Recommendation 2)

    Agency Affected: Department of Homeland Security: Transportation Security Administration

  3. Status: Open

    Comments: When we confirm what actions the agency has taken in response to this recommendation, we will provide updated information.

    Recommendation: The Administrator of TSA should assess whether third party testing contributes to its goals of increasing supplier diversity and innovation. (Recommendation 3)

    Agency Affected: Department of Homeland Security: Transportation Security Administration

 

Explore the full database of GAO's Open Recommendations »

Dec 16, 2020

Nov 23, 2020

Nov 12, 2020

Oct 20, 2020

Sep 30, 2020

Sep 29, 2020

Looking for more? Browse all our products here