Summary

The Department of Homeland Security's (DHS) Domestic Nuclear Detection Office (DNDO) is responsible for addressing the threat of nuclear smuggling. Radiation detection portal monitors are key elements in the nation's defenses against such threats. DHS has sponsored testing to develop new monitors, known as advanced spectroscopic portal (ASP) monitors, to replace radiation detection equipment being used at ports of entry. DNDO expects that ASPs may offer improvements over current-generation portal monitors, particularly the potential to identify as well as detect radioactive material and thereby to reduce both the risk of missed threats and the rate of innocent alarms, which DNDO considers to be key limitations of radiation detection equipment currently used by Customs and Border Protection (CBP) at U.S. ports of entry. However, ASPs cost significantly more than current generation portal monitors. Due to concerns about ASPs' cost and performance, Congress has required that the Secretary of Homeland Security certify that ASPs provide a significant increase in operational effectiveness before obligating funds for full-scale ASP procurement. This testimony addresses (1) GAO findings on DNDO's latest round of ASP testing, and (2) lessons from ASP testing that can be applied to other DHS technology investments. These findings are based on GAO's May 2009 report GAO-09-655 and other related reports.

GAO's report on the latest round of ASP testing found that DHS increased the rigor in comparison with previous tests and thereby added credibility to the test results. However, GAO's report also questioned whether the benefits of the ASPs justify the high cost. In particular, the DHS criteria for a significant increase in operational effectiveness require only a marginal improvement in the detection of certain weapons-usable nuclear materials, which DNDO considers a key limitation of current-generation portal monitors. The marginal improvement required of ASPs is particularly notable given that DNDO has not completed efforts to fine-tune current-generation equipment to provide greater sensitivity. Moreover, the preliminary test results show that ASPs performed better than current-generation portal monitors in detection of such materials concealed by light shielding approximating the threat guidance for setting detection thresholds, but that differences in sensitivity were less notable when shielding was slightly below or above that level. Finally, DNDO has not yet updated its cost-benefit analysis to take into account the results of the latest round of ASP testing and does not plan to complete computer simulations that could provide additional insight into ASP capabilities and limitations prior to certification even though test delays have allowed more time to conduct the simulations. DNDO officials believe the other tests are sufficient for ASPs to demonstrate a significant increase in operational effectiveness. GAO recommended that DHS assess ASPs against the full potential of current-generation equipment and revise the program schedule to allow time to conduct computer simulations and to uncover and resolve problems with ASPs before full-scale deployment. DHS agreed to a phased deployment that should allow time to uncover ASP problems but disagreed with the other recommendations, which GAO believes remain valid. The challenges DNDO has faced in developing and testing ASPs illustrate the importance of following best practices for investments in complex homeland security acquisitions and for testing of new technologies. GAO recently found that many major DHS investments, including DNDO's ASP program, had not met the department's requirements for basic acquisition documents necessary to inform the investment review process, which has adopted many acquisition best practices. As a result, DHS had not consistently provided the oversight needed to identify and address cost, schedule, and performance problems in its major investments. A primary lesson to be learned regarding testing is that the push to replace existing equipment with the new portal monitors led to an ASP testing program that until recently lacked the necessary rigor. Even for the most recent round of testing, DNDO's schedule consistently underestimated the time required to conduct tests and resolve problems uncovered during testing. In contrast, GAO has previously found that testing programs designed to validate a product's performance against increasing standards for different stages in product development are a best practice for acquisition strategies for new technologies. Aspects that improved the latest round of ASP testing could also, if properly implemented, provide rigor to DHS's testing of other advanced technologies.