This is the accessible text file for GAO report number GAO-08-1178T entitled 'Combating Nuclear Smuggling: DHS Needs to Consider the Full Costs and Complete All Tests Prior to Making a Decision on Whether to Purchase Advanced Portal Monitors' which was released on September 25, 2008. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Testimony: Before the Committee on Homeland Security and Governmental Affairs United States Senate: United States Government Accountability Office: GAO: For Release on Delivery: Expected at 9:30 a.m. EDT: Thursday, September 25, 2008: Combating Nuclear Smuggling: DHS Needs to Consider the Full Costs and Complete All Tests Prior to Making a Decision on Whether to Purchase Advanced Portal Monitors: Statement of Gene Aloise, Director: Natural Resources and Environment: GAO-08-1178T: GAO Highlights: Highlights of GAO-08-1178T, a testimony before the Committee on Homeland Security and Governmental Affairs, United States Senate. Why GAO Did This Study: The Department of Homeland Security’s (DHS) Domestic Nuclear Detection Office (DNDO) is responsible for addressing the threat of nuclear smuggling. Radiation detection portal monitors are key elements in our national defenses against such threats. DHS has sponsored testing to develop new monitors, known as advanced spectroscopic portal (ASP) monitors, to replace radiation detection equipment currently being used at ports of entry. ASPs may offer improvements over current generation portal monitors, particularly the potential to identify as well as detect radioactive material and thereby minimize both missed threats and false alarms. However, ASPs cost significantly more than current generation portal monitors, and testing of ASPs’ capabilities needs to be more objective and rigorous. Due to concerns about ASPs’ cost and performance, Congress has required that the Secretary of DHS certify that ASPs will provide a significant increase in operational effectiveness before obligating funds for full-scale ASP procurement. DHS is currently testing ASPs and anticipates a decision on certification in November 2008. This testimony addresses (1) the highlights of GAO’s September 2008 report on the life cycle cost estimate to deploy ASPs (GAO-08-1108R), and (2) preliminary observations from ongoing work reviewing the current program of ASP testing. What GAO Found: GAO’s independent cost estimate suggested that from 2007 through 2017 the cost of DNDO’s program to equip U.S. ports of entry with radiation detection equipment will likely be about $3.1 billion, but could range from $2.6 billion to $3.8 billion. GAO’s estimate was based on the anticipated costs of DNDO implementing its 2006 project execution plan, the most recent official documentation of the program. DNDO’s cost estimate of $2.1 billion to implement its project execution plan is unreliable because it omits major project costs, such as maintenance, and relies on a flawed methodology. For example, although the normal life expectancy of the standard cargo ASP is about 10 years, DNDO’s estimate considers only 8 years. According to DNDO officials, the agency is now following a scaled-back ASP deployment strategy rather than the 2006 project execution plan, and a senior DNDO official told GAO the ASP deployment strategy could change dramatically depending on the outcome of ongoing testing. GAO’s analysis indicated the cost to implement the scaled-back plans over the period 2008 through 2017 will be about $2 billion, but could range from $1.7 billion to $2.3 billion. However, frequent changes in DNDO’s deployment strategy make it difficult to assess ASP program costs. GAO’s recent report recommended that the Secretary of Homeland Security direct DNDO to update the project execution plan, revise its cost estimate, and communicate the revised estimate to the Congress so that it is fully apprised of the program’s scope and funding requirements. DHS agreed with the recommendations. DNDO has made progress in addressing a number of problems GAO identified in previous rounds of ASP testing. However, GAO’s ongoing review of the 2008 ASP testing program identified several potential areas of concern. First, the DHS criteria for “significant increase in operational effectiveness” appear to set a low bar for improvement—for example, by requiring ASPs to perform at least as well as current generation equipment when nuclear material is present in cargo but not specifying an actual improvement. GAO recently requested additional information from DNDO about the rationale behind these criteria, particularly in light of seemingly stricter criteria found in other documents. Second, the ASP certification schedule does not allow for completion of computer simulations that could provide additional data on ASP performance. While these computer simulations may have limitations, they also could provide useful data on ASP capabilities prior to the Secretary’s decision on certification. Finally, the test schedule is highly compressed and is running at least 8 weeks behind, leaving limited time for analysis and review of test results. Assuming that DHS addresses these concerns, the 2008 round of testing could provide an objective basis for comparing ASPs with current generation equipment. However, GAO recommended in March 2006 that DHS analyze the benefits and costs of deploying ASPs to determine whether any additional detection capability provided by ASPs is worth the cost, and would still question the replacement of current generation equipment with ASPs until DNDO demonstrates that any additional increase in security would be worth the ASPs’ much higher cost. To view the full product, including the scope and methodology, click on [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-1178T]. For more information, contact Gene Aloise at (202) 512-3841 or aloisee@gao.gov. [End of section] Mr. Chairman and Members of the Committee: I am pleased to be here today to discuss our work on the plans of the Department of Homeland Security's (DHS) Domestic Nuclear Detection Office (DNDO) to develop and deploy advanced spectroscopic portal (ASP) radiation detection monitors to the nation's ports of entry.[Footnote 1] Preventing radioactive material from being smuggled into the United States--perhaps to be used by terrorists in a nuclear weapon or in a radiological dispersal device (a "dirty bomb")--is a key national security objective. Today I will discuss our September report, which focuses on developing an independent life cycle cost estimate for replacing radiation detection equipment already deployed at U.S. ports of entry with ASPs,[Footnote 2] and our ongoing work reviewing DNDO's current program of ASP testing activities, which started in April 2008 and are scheduled to be completed in November 2008 under DNDO's latest plan. These tests are critically important because they will serve as the primary support for a congressionally mandated DHS Secretarial certification of the effectiveness of ASPs, also scheduled for November 2008. Radiation portal monitors--large stationary detectors through which cargo containers and trucks pass as they enter the United States--are an important component of the radiation detection system. The portal monitors in use today, known as polyvinyl toluene (PVT) monitors, are capable of detecting radiation but not identifying specific radioactive materials. To address this limitation, U.S. Customs and Border Protection (CBP) relies on handheld detection technologies, particularly radioactive isotope identification devices (RIID), to help CBP officers distinguish between dangerous and benign materials. [Footnote 3] CBP's standard operating procedures for use of radiation detection equipment include conducting primary inspections with PVTs to detect the presence of radioactivity, and secondary inspections with PVTs and RIIDs to confirm and identify the source and determine whether it constitutes a threat. Any vehicle triggering a PVT alarm is referred to a secondary screening area, where it is sent through a second PVT to confirm the original alarm. Whether the second PVT confirms the alarm or not, the vehicle, driver, and any passengers or cargo are scanned by a CBP officer with a RIID, which can detect radiation and also identify many of the most commonly used radioactive materials by name. All PVT alarms must be resolved--that is, CBP officers must investigate each alarm until they are convinced that the vehicle, occupants, and any cargo pose no threat and, if radioactive materials are found, that the vehicle occupants appear to have a legitimate reason to possess and transport them--before the vehicle, driver, and any passengers can be allowed to enter the United States. In contrast to PVTs, ASPs offer the potential to detect radiation and identify the source, reducing the need for secondary screenings of cargo containing benign radioactive materials. According to DNDO, the inability of the current generation PVT systems to identify the type of material causing an alarm results in the need to balance system sensitivity with the false and innocent alarm burden to the flow of commerce. DNDO also believes that CBP may use an inordinate amount of inspection resources for radiation detection at the expense of other missions, such as drug interdiction. To address these limitations, DNDO is sponsoring the development of ASPs with the expectation that they will minimize both missed threats and false alarms when deployed for primary screening and improve upon the RIID's identification performance and time required to correctly resolve primary alarms when deployed for secondary screening. ASPs cost significantly more than PVTs. DNDO's latest unit cost estimate (including deployment costs) is about $800,000 for the standard cargo version of the ASP and about $425,000 for the PVT standard cargo portal. Due to concerns about the performance and higher cost of ASPs relative to PVTs, the Congress has required that the Secretary of Homeland Security submit a report certifying that ASPs will provide a significant increase in operational effectiveness. [Footnote 4] Prior to primary and secondary deployment of the ASPs, the Secretary must submit separate and distinct certifications that address the unique requirements for operational effectiveness of each type of deployment. Our prior work on DNDO's efforts to develop and procure ASPs raised a number of serious concerns. In March 2006, we reported on, among other things, DNDO's efforts to develop ASPs; we recommended that DHS analyze the benefits and costs of deploying ASPs to determine whether any additional detection capability provided by ASPs is worth the additional cost.[Footnote 5] In October 2006, we concluded that DNDO's cost-benefit analysis did not provide a sound basis for its decision to purchase and deploy ASPs at an estimated cost at that time of $1.2 billion--for example, because DNDO relied on assumptions of the anticipated performance level of ASPs instead of actual test data. [Footnote 6] We also reported that DNDO did not assess the likelihood that ASPs would either misidentify or fail to detect nuclear or radiological material; rather, it focused its analysis on reducing the time necessary to screen traffic at ports of entry and the impact of any delays on commerce. We recommended, in part, that DNDO conduct further testing before spending additional funds to purchase ASPs. Similarly, in September 2007, we testified that DNDO's testing of ASPs at the Department of Energy's (DOE) Nevada Test Site did not represent an objective or rigorous assessment because DNDO used biased test methods that enhanced the apparent performance of the ASPs and did not test the limitations of the ASPs' detection capabilities.[Footnote 7] We recommended that DHS delay ASP certification and full-scale production decisions until all relevant tests and studies have been completed, limitations to these tests and studies have been identified and addressed, and results of the tests and analyses have been reported to the appropriate congressional committees. For reasons I am about to discuss, these recommendations are as important today as when we made them last year. My testimony today addresses (1) the highlights of our September 2008 report on our independent life cycle cost estimate for deploying ASPs at ports of entry, and (2) preliminary observations from our ongoing work reviewing the current program of ASP testing. For our September 2008 report, we contracted with a company with expertise in estimating the life cycle costs of major federal acquisitions. Together we developed an independent cost estimate of DHS's 2006 strategy to deploy radiation detection portal monitors, covering fiscal years 2007 through 2017, to ensure that the Congress has authoritative information on all the life-cycle costs associated with a full-scale acquisition of radiation portal monitors. To review ASP testing, we analyzed ASP test plans and interviewed senior DNDO officials responsible for managing the ASP program, and we observed testing conducted at the DOE's Nevada Test Site and Pacific Northwest National Laboratory. We also interviewed DOE, national laboratory, National Institute of Standards and Technology, and DHS officials. We conducted this work, including our work to date on ASP testing, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Summary: Our independent cost estimate suggests that from 2007 through 2017 the cost of DNDO's program to equip U.S. ports of entry with radiation detection equipment will likely be about $3.1 billion, but could range from $2.6 billion to $3.8 billion. We based our estimate on the anticipated costs of DNDO implementing its 2006 project execution plan, the most recent official documentation of the radiation portal monitor project's objectives, scope, schedule, costs, and funding requirements. According to the project execution plan, DNDO will buy and deploy multiple types of portal monitors, including ASPs designed to screen rail cars, airport and seaport cargo, as well as mobile ASPs to provide greater flexibility in screening commerce. In March 2008, DNDO estimated the total cost of deploying radiation portal monitors to be about $2.1 billion, but we believe the agency's estimate is unreliable because it omits major project costs, such as maintenance, and relies on a flawed methodology. For example, although the normal life expectancy of the standard cargo ASP is about 10 years, DNDO's estimate considers only 8 years. Furthermore, during the course of our review, DNDO officials told us the only ASP that the agency now plans to deploy is the standard cargo portal monitor. Although DNDO could not fully document this change, the agency provided a one-page summary of its current deployment plans. We performed a limited analysis of these summary data and determined that the cost to deploy standard ASP cargo portals over the period 2008 through 2017 will likely be about $2 billion, but could range from $1.7 billion to $2.3 billion. However, a DNDO official responsible for overseeing the agency's operations told us that even the deployments listed on the one-page summary could change dramatically depending on the outcome of ongoing testing. In our view, it is difficult to assess the costs of the ASP program because of the frequent changes in DNDO's deployment strategy. Our report recommended that the Secretary of Homeland Security direct DNDO to update the project execution plan, revise its cost estimate, and communicate the revised estimate to the Congress so that it is fully apprised of the program's scope and funding requirements. DHS agreed with our recommendations. While DNDO has made progress in addressing a number of problems we identified in previous rounds of ASP testing, we have identified potential areas of concern based on our ongoing review of the 2008 ASP testing program. First, the DHS criteria for "significant increase in operational effectiveness" appear to set a low bar for improvement. Most notably, the criteria for primary screening require ASPs to perform at least as well as current generation equipment when special nuclear material or medical or industrial isotopes are present in cargo, but they do not specify an actual improvement. We recently requested additional information from DNDO about the rationale behind these criteria, particularly in light of seemingly stricter criteria found in other documents. Second, the schedule leading up to ASP certification does not allow for completion of "injection studies"--a type of computer simulation for testing the response of the ASP threat identification algorithms to data on threat objects combined with stream-of-commerce data from a port of entry. While we recognize that injection studies have limitations, they could provide useful data on ASP capabilities, particularly for primary screening. Finally, the test schedule leading up to certification is highly compressed and is running at least 8 weeks behind, leaving limited time for analysis and review of test results. Assuming that DHS addresses these concerns, the 2008 round of testing could provide an objective basis for comparing ASPs with current generation equipment. However, consistent with our March 2006 recommendation that DHS analyze the benefits and costs of deploying ASPs, we would still question the replacement of current generation equipment with ASPs until DNDO demonstrates that any additional detection capability and increase in security provided by ASPs would be worth the ASPs' much higher cost. DNDO's Program to Deploy Radiation Detection Portal Monitors at U.S. Ports of Entry Is Likely to Cost About $3 Billion: As we discuss in our report, our independent cost estimate suggested the total cost of DNDO's program to equip U.S. ports of entry with radiation detection equipment will likely be about $3.1 billion, but could range between $2.6 billion and $3.8 billion. We based our estimate on the anticipated costs of DNDO implementing its 2006 project execution plan, the most recent official documentation of the radiation portal monitor project's objectives, scope, schedule, costs, and funding requirements. According to this plan, DNDO plans to buy and deploy several types of ASPs, including those designed to screen rail cars, airport cargo, and seaport cargo, as well as mobile ASPs that provide greater flexibility in screening commerce. The plan also provides for the deployment of several types of PVTs. Clearly, the numbers and types of portal monitors deployed will significantly affect the total cost of the radiation portal monitor project. In all, the project execution plan calls for the purchase of 2,754 portal monitors, with a total of 2,582 scheduled for deployment--approximately 1,034 ASPs and 1,548 PVTs--and 172 held in excess at the project's completion. In developing our estimate, we categorized radiation portal monitor project costs according to program phases--design and development, procurement, deployment, maintenance, and operational sustainment. We did not incorporate operational costs, in particular the cost of CBP officers operating the radiation detection equipment. DNDO and CBP believe that deploying ASPs will reduce the use of CBP staff resources for radiation detection tasks, but currently there are no usable estimates of how CBP's staffing would change with the deployment of ASPs. Our analysis included a period of 11 years: actual life-cycle expenses from fiscal year 2007, and estimated life cycle costs from fiscal year 2008 through 2017. DNDO's estimate of $2.1 billion to deploy radiation detection equipment (submitted as part of its budget request to OMB in March 2008) is unreliable because it omits major project costs and relies on flawed methodology. As a result, DNDO's cost estimates and budget requests for the radiation portal monitor project are too low, which could lead to significant cost overruns later in the project. DNDO's estimate contains the following major deficiencies: * It does not appear to include the costs of all variations of ASPs contained in DNDO's project execution plan. DNDO's current baseline considers only the standard ASP cargo portal and ignores the costs of other types of ASPs, such as those designed to screen rail cars, airport and seaport cargo, and mobile detectors. According to senior DNDO officials, the current approved project execution plan no longer reflects the agency's procurement and deployment plans for ASPs. In acknowledging that the project execution plan should be revised and updated, these officials told us that DNDO currently plans to field only the standard cargo ASP portal, mainly at high-volume ports of entry. (We used the 2006 project execution plan because it is the agency's only official plan for ASP deployment.) * DNDO's estimate considers only 8 years rather than 10 years, the operational life expected by the manufacturer of sodium iodide crystals, a key ASP component. DHS cost guidance maintains that a project's life cycle can be estimated over that period of time during which equipment will remain available before it is exhausted, that is, decayed or deteriorated. DNDO officials agreed that a 10-year life cycle cost estimate would have been more appropriate and said that they would have used a 10-year estimate had they not been constrained by OMB budget submission software, which limits the number of years of costs that can be included. * DNDO's cost estimate does not include all of the elements of the ASPs' life cycle, such as maintenance or operational sustainment. These costs are approximately $999.2 million and $364.9 million, respectively, under our cost estimate. * DNDO did not have detailed documentation of the costs to support its estimates. According to OMB, DHS, and GAO guidelines, such documentation is necessary to establish the basis of the estimates and to provide assurances that the estimates are credible. Assuming DNDO attempts to implement its authorized 2006 project execution plan rather than its scaled-back plan, our report estimated a $753 million budget shortfall for the radiation portal monitor project for fiscal years 2008 through 2012 (the years included in both our analysis and the DNDO analysis). Additionally, we estimated that DNDO will require another $833 million from fiscal year 2013 through fiscal year 2017 to complete the entire life cycle of the program. According to DNDO officials, our cost estimate will result in estimates higher than the program's current true cost because it is based on the outdated project execution plan. However, in the absence of more recent documentation, we believe our independent cost estimate must be based on the agency's most recent approved plan (the 2006 project execution plan). Furthermore, several official DNDO documents specify that multiple versions of ASPs will be deployed. For example, the agency's February 2006 expenditure plan submitted to the Congress foresees "several variants" of ASP systems being deployed, with standard cargo, rail, and automobile versions mentioned specifically. And DNDO's submissions to OMB for budget years 2008 and 2009 describe a program that includes land crossings, seaports, rail lines, airports, and other ports of entry. Finally, agency officials acknowledged the program requirements that would have been fulfilled by the discontinued ASPs remain valid, including screening rail cars, airport cargo, and cargo at seaport terminals, but the agency has no current plans for how such screening will be accomplished. These officials told us the technology to accomplish these requirements likely will not be ASP monitors and could be a totally new technology. We believe a comprehensive estimate of the cost to provide radiation detection equipment for U.S. ports of entry should account for meeting these objectives, even if DNDO decides that ASP technology is not suited to them. DNDO agreed in June 2008 to update its project execution plan so that we could better estimate the costs of the agency's current plans. DNDO also agreed to provide an updated estimate of the standard portal's life cycle costs and to meet with us to reconcile our two estimates. However, DNDO did not provide a revised project execution plan or cost estimate and instead provided in July 2008 a one-page spreadsheet of summary information outlining its plans to buy and deploy portal monitors for the 7-year period 2008 through 2014. DNDO's summary data indicate that during this time period the agency plans to deploy 717 ASPs and 1,005 PVTs. The summary data do not provide the breadth and depth of information needed to generate detailed and fully documented cost estimates. Furthermore, according to subsequent discussions with a senior DNDO official, if ongoing tests indicate the ASPs' performance warrants it, the agency may speed its deployment of ASPs over the next few months. In our view, the frequent changes in deployment plans and the lack of available cost documentation raise concerns about the overall management of the radiation portal monitor project and whether it is guided by a sound and stable strategy. Nonetheless, we used the agency's summary data to perform a more limited cost estimate for only the standard cargo portal. We found that from 2008 to 2017 the total program cost for buying and deploying standard cargo portals would likely be about $2 billion, but could range from about $1.7 billion to $2.3 billion. Our report recommended that the Secretary of Homeland Security direct the Director of DNDO to (1) work with the Commissioner of CBP to update the project execution plan to guide the entire radiation detection program at U.S. ports of entry, (2) revise DNDO's estimate of the program's cost and ensure that the estimate considers all the costs associated with its project execution plan, and (3) communicate this revised estimate to the Congress so that it is fully apprised of the program's scope and funding requirements. DHS agreed with our recommendations in its comments responding to our draft report. Preliminary Observations of Ongoing Testing of ASPs: We are currently reviewing the ongoing 2008 ASP testing program and process leading to a decision by the Secretary of Homeland Security on certification of ASPs, and we plan to issue a final report in November 2008, prior to the Secretary's decision on certification. Our work to date shows that DNDO has made progress in addressing a number of problems we identified in previous rounds of ASP testing. A particular area of improvement is in the performance testing at the Nevada Test Site, where DNDO conducts trials of the ability of ASPs to detect and identify radioactive materials, particularly those that could be used in a nuclear weapon. In 2007, we reported that DNDO had allowed ASP system contractors to adjust their systems after preliminary runs using the same radiological materials that would be used in the formal tests, potentially biasing the test results. In contrast, in our more recent work, we found that the plan for the 2008 round of performance testing stipulated that there would be no system contractor involvement in test execution, and no ASP system contractors were present at the test location on the day we observed performance testing. Furthermore, DNDO officials told us, and we observed, that they did not conduct preliminary runs with threat objects that were used in the formal tests. A further area of improvement in the performance testing concerns the use of handheld RIIDs. Specifically, DNDO used actual CBP officers who adhered to nearly all of the standard operating procedures in testing the RIIDs. This improvement addresses our concern from last year that DNDO did not objectively test the performance of the RIIDs because it did not use a critical CBP standard operating procedure that is fundamental to the equipment's performance in the field. Despite these improvements, we have identified potential areas of concern based on our ongoing review. Our concerns fall into three areas: * DHS criteria for "significant increase in operational effectiveness." DNDO, CBP and the DHS management directorate agreed on six criteria for the degree of improvement ASPs must provide over current generation equipment--four criteria for primary screening and two for secondary screening. These criteria appear to set a low bar for improvement. Most notably, the criteria for primary screening require ASPs to perform at least as well as current generation equipment when special nuclear material or medical or industrial isotopes are present in cargo, but they do not specify an actual improvement. Similarly, one of the criteria for secondary screening requires ASPs to reduce the average time to release conveyances but does not indicate a specific level of reduction. As such, the criteria leave open the possibility of a Secretarial decision in favor of certification even if ASPs do not provide a significantly higher probability of detection compared with current generation PVTs when deployed for primary screening and only a small reduction in the time required for secondary screening. We recently requested additional information from DNDO about the rationale behind these criteria, particularly in light of seemingly stricter criteria found in other documents such as the ASP performance specification, which requires an 80 percent probability of detection, at a 95 percent confidence, of government-designated representative threat objects. We are awaiting DNDO's response. * The extent to which the 2008 testing program provides a sound basis for determining ASPs' operational effectiveness. While the performance testing at Nevada Test Site could provide a useful comparison of the performance of the ASPs and current generation equipment for both primary and secondary screening given the improvements described above, DNDO does not plan to complete "injection studies" that could provide additional data on ASP performance prior to ASP certification. (Injection studies are a type of computer simulation for testing the response of the ASP threat identification algorithms to data on threat objects combined with stream-of-commerce data from a port of entry.) Rather, DNDO officials told us that testing using actual threat objects is sufficient for certification. While we recognize the limitations of injection studies such as the inability to exactly model threat objects hidden in commerce, reliance on performance testing but not injection studies for the Secretary's decision on certification will result in less data with which to evaluate the capabilities of ASPs, particularly for primary screening.[Footnote 8] According to officials from Pacific Northwest National Laboratory, which has a lead role in injection studies, computer simulations allow for an increased statistical confidence in the measured probability of detection compared with testing at Nevada Test Site. The ASP test and evaluation master plan similarly emphasizes the importance of injection studies for a complete test of radiation detection system performance, and a February 2008 DNDO presentation to us includes among its fiscal year 2008 objectives the use of preliminary results from injection studies to evaluate performance of ASPs for primary screening. However, injection studies have taken longer than projected due to a number of issues such as deficiencies in the stream-of-commerce data. * The schedule leading up to a decision on ASP certification by the Secretary of Homeland Security. The test schedule leading up to certification is highly compressed and is running at least 8 weeks behind the schedule DNDO provided to us in May 2008, leaving limited time for analysis and review of test results. Key phases of the test schedule include system qualification, followed by testing at Pacific Northwest National Laboratory to demonstrate that ASPs are ready to be integrated into the interdiction systems at U.S. ports of entry and concurrent testing at Nevada Test Site to compare the ASP performance in detecting and identifying threats with that of current generation equipment. According to DNDO, the systems qualification and performance tests have been completed, but only after being delayed due to problems with system qualification. The final two phases, field validation at four ports of entry operated by CBP and operational testing at one port of entry, have not yet started. Field validation was to have started yesterday--September 24--but as with integration and performance testing this phase has also been delayed, and CBP has not yet indicated when it will begin. Even before this latest delay, the time between the scheduled end of operational testing in early November and a Secretarial decision on certification on November 28 allowed less than a month for review and analysis of results. Furthermore, DHS officials have indicated that the Secretary may make a decision on certification on the basis of "quick look" (preliminary) rather than final reports if the quick look reports are favorable to ASPs. The limited time between completion of testing and a decision on certification, combined with the potential reliance on quick look reports, increases the risk of a decision in favor of certification being called into question later by a more thorough analysis and review of results. Assuming that DHS addresses these concerns--for example, by clarifying the criteria for significant increase in operational effectiveness, performing the injection studies, and delaying a Secretarial decision on certification--the 2008 round of testing could provide an objective basis for comparing ASPs with current generation equipment. However, consistent with our March 2006 recommendation that DHS analyze the benefits and costs of deploying ASPs to determine whether any additional detection capability provided by ASPs is worth the cost, we would still question the replacement of current generation equipment with ASPs, particularly considering the gaps identified by DNDO in the global nuclear detection architecture--essentially, an integrated system of radiation detection equipment and interdiction activities to combat nuclear smuggling in foreign countries, at the U.S. border, and inside the United States. As we testified before this committee earlier this year, such gaps include land border crossings into the United States between formal points of entry, small maritime vessels, and international general aviation.[Footnote 9] An updated cost-benefit analysis, based on the latest information on ASP costs and test performance, could help policymakers address any trade-offs in addressing these gaps versus replacing current generation radiation detection equipment already deployed at ports of entry. The current generation equipment has known limitations, particularly with regard to the limited ability to detect certain nuclear materials and the inability to distinguish between types of radiological material. However, the existing equipment provides at least some detection capability and may act as a deterrent, and there is no evidence we are aware of that it is impeding the flow of commerce. Mr. Chairman, this concludes my prepared statement. I would be pleased to respond to any questions that you or other members of the Committee may have. GAO Contact and Staff Acknowledgments: For further information on this testimony, please contact Gene Aloise at (202) 512-3841 or by e-mail at aloisee@gao.gov. Individuals making key contributions to this testimony include Joseph Cook, Jennifer Echard, Brian Octeau, Alison O'Neill, Tim Persons (GAO's Chief Scientist), Karen Richey, Benjamin Shouse, Kevin Tarmann, Eugene Wisnoski, and Ned Woodward. [End of section] Footnotes: [1] DNDO was established within DHS in 2005; its mission includes developing, testing, acquiring, and supporting the deployment of radiation detection equipment at U.S. ports of entry. [2] GAO, Combating Nuclear Smuggling: DHS's Program to Procure and Deploy Advanced Radiation Detection Portal Monitors Is Likely to Exceed the Department's Previous Cost Estimates, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-1108R] (Washington, D.C.: Sept. 22, 2008). [3] CBP, also part of DHS, began deploying radiation detection equipment in 2002, prior to DNDO's creation, under the radiation portal monitor project. CBP remains responsible for operating the equipment. [4] Public Law 110-161 (121 Stat. 1844, 2069). [5] GAO, Combating Nuclear Smuggling: DHS Has Made Progress Deploying Radiation Detection Equipment at U.S. Ports of Entry, but Concerns Remain, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-389] (Washington, D.C.: Mar. 22, 2006). [6] GAO, Combating Nuclear Smuggling: DHS's Cost-Benefit Analysis to Support the Purchase of New Radiation Detection Portal Monitors Was Not Based on Available Performance Data and Did Not Fully Evaluate All the Monitors' Costs and Benefits, [hyperlink, http://www.gao.gov/cgi- bin/getrpt?GAO-07-133R] (Washington, D.C.: Oct. 17, 2006). [7] GAO, Combating Nuclear Smuggling: Additional Actions Needed to Ensure Adequate Testing of Next Generation Radiation Detection Equipment, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-1247T] (Washington, D.C.: Sept. 18, 2007). [8] Injection studies as currently designed have less relevance to evaluating the use of ASPs for secondary screening because the studies will not compare ASP performance with that of the handheld RIIDs, a key component of the current generation equipment used for secondary screening. [9] GAO, Nuclear Detection: Preliminary Observations on the Domestic Nuclear Detection Office's Efforts to Develop a Global Nuclear Detection Architecture, [hyperlink, http://www.gao.gov/cgi- bin/getrpt?GAO-08-999T] (Washington, D.C.: July 16, 2008). [End of section] GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. Government Accountability Office: 441 G Street NW, Room LM: Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: