Summary
In September and October 2001, letters laced with Bacillus anthracis (anthrax) spores were sent through the mail to two U.S. senators and to members of the media. These letters led to the first U.S. cases of anthrax disease related to bioterrorism. In all, 22 individuals, in four states and Washington, D.C., contracted anthrax disease; 5 died. These cases prompted Congress to ask GAO to describe and assess federal agencies' activities to detect anthrax in postal facilities, assess the results of agencies' testing, and assess whether agencies' detection activities were validated.
The U.S. Postal Service, Centers for Disease Control and Prevention, and Environmental Protection Agency (EPA) conducted several interdependent activities, including sample collection and analytic methods, to detect anthrax in postal facilities in 2001. They developed a sampling strategy and collected, transported, extracted, and analyzed samples. They primarily collected samples from specific areas, such as mail processing areas, using their judgment about where anthrax would most likely be found--that is, targeted sampling. The agencies did not use probability sampling in their initial sampling strategy. Probability sampling would have allowed agencies to determine, with some defined level of confidence, when all results are negative, whether a building is contaminated. The results of the agencies' testing in 286 postal facilities were largely negative--no anthrax was detected. However, agencies did not use validated sample collection and analytical methods. According to the agencies, validated methods were not available in 2001. Thus, there can be little confidence in negative results. Validation is a formal, empirical process in which an authority determines and certifies the performance characteristics of a given method. Consequently, the lack of validation of agencies' activities, coupled with limitations associated with their targeted sampling strategy, means that negative results may not be reliable. In preparing for future incidents, the agencies have (1) made some changes based on what has been learned about some of the limitations of their sampling strategies, (2) made some revisions to their guidelines, (3) funded some new research, and (4) planned or conducted conferences addressing some of the issues GAO has identified. In addition, the Department of Homeland Security (DHS) has taken on the role of coordinating agencies' activities and has undertaken several new initiatives related to dealing with anthrax and other bio-threat agents. However, while the actions DHS and other agencies have taken are important, they do not address the issue of validating all activities related to sampling. Finally, the agencies have not made appropriate and prioritized investments to develop and validate all activities related to other bio-threat agents.
Recommendations
Our recommendations from this work are listed below with a Contact for more information. Status will change
from "In process" to "Implemented" or "Not implemented" based on our follow up work.
Director:
Team:
Phone:
Nabajyoti Barkakati
Government Accountability Office: Applied Research and Methods
(202) 512-6412
Recommendations for Executive Action
Recommendation: Given the lack of validated methods for detecting anthrax contamination in facilities, the Secretary of Homeland Security should develop a coordinated approach to (1) improve the overall process for detecting anthrax and (2) increase confidence in negative test results generated by that process. This approach would include working with agencies to ensure that appropriate validation studies of the overall process of sampling activities, including the methods, are conducted. Specifically, the Secretary should take a lead role in promoting and coordinating the activities of the various agencies that have the technical expertise related to environmental testing.
Agency Affected: Department of Homeland Security
Status: In process
Comments: DHS has begun working with agencies to develop a strategic plan for validation anthrax sampling activities. On March 27, 2007, prior to our March 29, 2007 testimony (GAO-07-687T), working with CDC, EPA, DOD, and the FBI, DHS developed a working draft of a strategic plan and road map. This plan was provided to us in July 2007. The plan outlined how individual agencies' efforts would lead to the validation of the overall anthrax sampling process. Also, DHS is chair and coordinator of the Validated Sampling Plan (VSP) workgroup, whose participants include CDC, EPA, DOD, and NIST, which begun meeting in 2006.
Recommendation: Given the lack of validated methods for detecting anthrax contamination in facilities, the Secretary of Homeland Security should develop a coordinated approach to (1) improve the overall process for detecting anthrax and (2) increase confidence in negative test results generated by that process. This approach would include working with agencies to ensure that appropriate validation studies of the overall process of sampling activities, including the methods, are conducted. Specifically, the Secretary should ensure that a definition of validation is developed and agreed on.
Agency Affected: Department of Homeland Security
Status: In process
Comments: As stated by DHS in its Interagency Strategic Plan for Validation of Environmental Sampling Methods Used in Detection and Cleanup of B. anthracis Contamination in Facilities, the Interagency Validates Sampling Plan Work group (VSP), organized by DHS in 2006, agreed upon a definition of validation in response to GAO's recommendation. The VSP workgroup adopted the International Standardization (ISO) 17025 definition of validation, which states that validation is "the confirmation by examination and the provision of objective evidence that the particular requirements for a specific intended use are fulfilled."
Recommendation: Given the lack of validated methods for detecting anthrax contamination in facilities, the Secretary of Homeland Security should develop a coordinated approach to (1) improve the overall process for detecting anthrax and (2) increase confidence in negative test results generated by that process. This approach would include working with agencies to ensure that appropriate validation studies of the overall process of sampling activities, including the methods, are conducted. Specifically, the Secretary should guarantee that the overall process of sampling activities, including methods, is validated so that performance characteristics, including limitations, are clearly understood and results can be correctly interpreted.
Agency Affected: Department of Homeland Security
Status: In process
Comments: DHS officials stated in May 2006, different agencies have responsibility for different aspects of validation, and DHS control over other agencies' actions and budgets is limited. However, DHS has developed a strategic plan and roadmap to coordinate agency (CDC, EPA, DHS, and DOD) activities to validate the overall anthrax sampling activities. Activities in the plan are projected to be finished at the end of FY 2010, subject to agency funding. The plan lists specific tasks and the lead agency that will carryout those tasks, as well as an estimate of projected funding requirements. DHS could use this plan as a way to track progress and encourage agencies to complete and fund their validation efforts.
Recommendation: Given the lack of validated methods for detecting anthrax contamination in facilities, the Secretary of Homeland Security should develop a coordinated approach to (1) improve the overall process for detecting anthrax and (2) increase confidence in negative test results generated by that process. This approach would include working with agencies to ensure that appropriate validation studies of the overall process of sampling activities, including the methods, are conducted. Specifically, the Secretary should see that appropriate investments are made in empirical studies to develop probability-based sampling strategies that take into account the complexities of indoor environments.
Agency Affected: Department of Homeland Security
Status: In process
Comments: The DHS strategic plan lays out a series of studies to take place, beginning in FY 2007, with the last activities planned for the end of FY 2010, depending upon the availability of funding. For example, a CDC effort to do a study of sampling, collection, and analysis methods--specifically, optimization and validation of wipes and vacuum methods--was in progress in FY 2008. An effort by DHS and EPA to optimize methods and procedures for aggressive air sampling is to start in FY 2008 and finish in FY 2009. Controlled chamber tests are to take place in 2009. Some studies are currently unfunded.
Recommendation: Given the lack of validated methods for detecting anthrax contamination in facilities, the Secretary of Homeland Security should develop a coordinated approach to (1) improve the overall process for detecting anthrax and (2) increase confidence in negative test results generated by that process. This approach would include working with agencies to ensure that appropriate validation studies of the overall process of sampling activities, including the methods, are conducted. Specifically, the Secretary should ensure that appropriate, prioritized investments are made for all biothreat agents.
Agency Affected: Department of Homeland Security
Status: In process
Comments: The DHS strategic plan lays out a series of studies to take place, beginning in FY 2007, with the last activities planned for the end of FY 2010, depending upon the availability of funding. For example, a CDC effort to do a study of sampling, collection, and analysis methods--specifically, optimization and validation of wipes and vacuum methods--was in progress in FY 2008. An effort by DHS and EPA to optimize methods and procedures for aggressive air sampling is to start in FY 2008 and finish in FY 2009. Controlled chamber tests are to take place in 2009. Some studies are currently unfunded.
Recommendation: Given the lack of validated methods for detecting anthrax contamination in facilities, the Secretary of Homeland Security should develop a coordinated approach to (1) improve the overall process for detecting anthrax and (2) increase confidence in negative test results generated by that process. This approach would include working with agencies to ensure that appropriate validation studies of the overall process of sampling activities, including the methods, are conducted. Specifically, the Secretary should ensure that agency policies, procedures, and guidelines reflect the results of such efforts.
Agency Affected: Department of Homeland Security
Status: In process
Comments: Since validation studies of the overall process for detecting anthrax, including sampling activities and the methods used, are still ongoing, agency policies, procedures, and guidelines have not been fully revised to reflect such efforts. However, according to the strategic plan, CDC is in the process of updating its sampling strategy guidance and expanding it to include the best use of sample decision aids that have been developed since 2001, such as the Visual Sample Plan and Building Restoration Operations Optimization Model (BROOM). As a component task of the VSP interagency work group, CDC and EPA are working together to develop and complete revised sampling strategy guidance, partly aided by funding from DHS. The guidance is to undergo additional revision by the end of FY 2008 while validation activities are expected to take until at least 2010 to complete.
