![[logo] US EPA](https://webarchive.library.unt.edu/eot2008/20090512003014im_/http://www.epa.gov/epafiles/images/logo_epaseal.gif){kind=logo}
Report on Environmental Data Quality at Superfund Removal
Actions in Region 9
 |
Office of Inspector
General
Audit Report |
| Report on Environmental Data Quality at Superfund Removal Actions in Region 9 | |
| E1SFF7-09-0058-8100223 | |
| September 4, 1998 | |
EXECUTIVE SUMMARY
INTRODUCTION
EPA's Emergency Response Program responds to threats posed by the sudden or unexpected releases of hazardous substances. Region 9's Emergency Response Office manages the majority of these responses, called removal actions, throughout California, Nevada, Arizona, Hawaii, and the Pacific Islands.
We performed this audit because of known risks to the quality of environmental data. Region 9's Superfund Program has experienced serious problems with environmental data quality that are likely to continue. Risks to environmental data are significant because data is the basis for EPA's decision making and enforcement actions.
This audit is also part of a national audit of field sampling. The results of this audit pertaining to field sampling will be included in a national audit report.
OBJECTIVE
The objective of the audit was to determine if Region 9 had sufficient procedures in place to ensure that environmental data was of known and acceptable quality for Superfund removal actions.
RESULTS IN BRIEF
|
Actions Audited
|
Our audit of five removal actions showed Region 9 did not have sufficient procedures over Superfund removal actions to ensure that environmental data was of known and acceptable quality. Also, the Region did not fully use EPA's planning process, called data quality objectives, to ensure its removal actions and corresponding data collection activities were effective and efficient.
Chapter 1: Quality System for Removal Actions Needs Improvement
Our audit of five removal actions showed the Region did not have a quality management plan which adequately documented and described the quality system for removal actions. Further, most of the site-specific quality assurance project plans (QAPPs) we reviewed:
- Were not based on the seven-step data quality objectives (DQO) process;
- Were not designed to prevent and detect inappropriate quality data;
- Did not include defensible or optimal plans for collecting data;
- Were not reviewed or approved by the Region's Quality Assurance Office; and,
- Were not implemented or monitored.
We believe the main reason the quality system was insufficient was because decision makers did not perceive data quality as a risk to decision making.
Consequently, the Region completed three removal actions without appropriate quality data for decision making. In addition, about 420 samples were analyzed for one removal action that were not used for the decision indicated in the sampling plan.
Chapter 2: Data Quality Objectives Could Improve Removal Planning and Data Collection
The Region undertook removal actions and corresponding data collection efforts without fully using the data quality objectives process, EPA's systematic planning method.
The DQO process provides two primary benefits: (1) better decisions, because they are based on the scientific method; and (2) more cost-effective sampling for environmental data.
| "Data cannot be evaluated as good or not good until the use of that data has been clearly
defined."
-Principles of Environmental Sampling, Second Edition, 1996 |
As a result, the Region completed five removal actions, costing more than $20 million, without sufficiently documenting important decision criteria or alternatives.
The five Superfund removal actions we audited are described in more detail in Appendix B. Appendix C explains the abbreviations we used. Appendix D lists the criteria applicable to data quality and removal actions.
POSITIVE ACTIONS TAKEN
Although we found deficiencies in the quality assurance system for removal actions, we noted that:
- The Region had a quality management plan that was approved by EPA's headquarters;
- The Region's quality management plan required QAPPs to be approved by its Quality Assurance Office;
- The Region's Emergency Response Office had established a "generic" QAPP that provided quality assurance guidance for its removal actions;
- The Emergency Response Office's technical contractor had a quality management plan that required significant quality assurance activities;
- The Emergency Response Office and Superfund Cleanup Branch had prepared site-specific QAPPs for all the removal actions we audited; and,
- All of the site-specific QAPPs we reviewed required the data to be validated, an important quality assurance activity.
RECOMMENDATIONS
In summary, we recommend that the Regional Administrator:
- Develop a quality management plan for removal actions that includes: (1) responsibilities for quality assurance functions; (2) training requirements; and (3) a selection basis and tracking system for environmental laboratories.
- Require the regional Quality Assurance Office to monitor compliance with quality management plans, including plans for the Emergency Response Office and its technical contractor.
- Amend the on-scene coordinators' responsibilities for QAPPs to include the development of DQOs and consideration of critical quality assurance activities. Also, ensure responsibilities include QAPP approval, implementation, and monitoring.
- Require the Quality Assurance Office to review and approve all QAPPs, including providing verbal approval at emergency actions, if necessary.
- Require on-scene coordinators to develop DQOs for all removal actions.
- Establish a minimum, mandatory training requirement for DQOs for regional personnel whose duties involve the collection, evaluation, or use of environmental data.
- Adopt a team approach to developing DQOs using facilitators, on-scene coordinators, quality assurance experts, statisticians, and technical experts.
REGION 9 COMMENTS
A draft report was provided to the Region on April 23, 1998, and the Region provided us its draft comments on July 24, 1998. We held an exit conference with regional officials on August 12, 1998 and the Region provided us its final comments on August 21, 1998.
In its final comments, the Region said it:
...is committed to early implementation of the audit recommendations and intends to complete substantially all major milestones in the recommendations before the end of the calendar year.
The Region agreed to develop a quality management plan to address data quality at removal actions. The quality management plan will provide for:
- A graded approach to quality assurance;
- Development of generic quality assurance project plans;
- Approval of quality assurance project plans by the Quality Assurance Office; and,
- Oversight of the implementation of the quality management plan by the Quality Assurance Office.
Also, the Region will initiate a training program to assure that all on-scene coordinators receive appropriate training on data quality objectives. The Region believed that these actions, together with other ongoing efforts, should greatly enhance the management of data quality at removal actions.
The Region said although it still had "some disagreements with specific conclusions, we do not believe that these disagreements detract from the overall message regarding the need to improve data quality." The Region also said that it "firmly believes that the five removal actions considered in preparation of the report were appropriate and protective of human health."
The Region's response to the audit report recommendations are summarized in Chapters 1 and 2 of our report. The Region's entire response is at Appendix A.