|
Title: |
Quality Requirements for Extramural
Agreements |
|
Presenter(s): |
Kevin Bolger QA Manager USEPA Region 5 (Session chair/presenter),
Amberina Khan QA Expert, USEPA Region 5, Larisa Leonova, USEPA Region 5, Louis Blume, QA Manager, USEPA |
|
Key Message(s): |
|
|
Abstract: |
This
session will provide an overview of USEPA quality requirements for extramural
agreements which involve environmental data operations. The
presentations will discuss:
|
|
Title: |
What Is It? Why Use It? And What’s in It for Me? |
|
Presenter(s): |
Jill
Lundell, Berta Oates, and Debbie LaCroix, Portage, Inc. |
|
Key Message(s): |
|
|
Abstract: |
Data
quality assessment (DQA) is the final step of the data life cycle. This cycle
consists of planning, implementation, and assessment. Although the planning
and implementation steps are of high importance, the last step, the DQA, is
of equal importance. The DQA is frequently overlooked, and is the topic of
discussion in this presentation. When a DQA is performed properly, then the
data attributes can uncover important characteristics at the site that are
likely to be missed with a “quick and dirty” number summary. Such
characteristics may include hot spots and trends in contamination. During
assessment, data are validated using documented quality procedures and
sampling and collection activities are compared to applicable Quality
Assurance Project Plan and/or Sampling Plan specifications. The DQA carefully
examines the data obtained from sample collection activities to determine if
project objectives established during the data quality objective (DQO)
process were met. Careful evaluation and appropriate statistical analysis of
data ensure that suitable actions will be taken based on the data collected. This
presentation will examine a completed DQA to reveal its components and
highlight the importance of careful evaluation and appropriate analysis of
data. The examination of rejected data for statistical analysis will be
discussed and real-world examples will be presented to emphasize the need for
completing a thorough DQA. |
|
Title: |
Oversight of Grants QA – Some Gaps Need
Filling |
|
Presenter(s): |
Marcus E. Kantz, USEPA Region 2 |
|
Key Message(s): |
|
|
Abstract: |
In USEPA
Region 2, we performed an Internal Audit of the management of water program
grants to find out whether the grants that involved the generation and/or use
of environmental data had either approved Quality Management Plans (QMPs) or
Quality Assurance Project Plans (QAPPs), as appropriate. For those that didn’t, we then aimed to
figure out why and what to do about it. Of the
202 grants in four water program branches, 149 involved the collection and/or
use of environmental data. For 47 of those
grants, the Project Officers (POs) had not ensured that an appropriate QMP or
QAPP had been approved. In other
words, approximately one-third of the grants that sponsored environmental
work occurred outside the watchful eye of the QA System. We determined
that 4 recurring issues were largely responsible for the improper application
of QA: uncertainty about the need to apply QA to pollution prevention (P2)
grants that utilize data only for determining the grants’ environmental
effectiveness; uncertainty about QA for multi-project grants; uncertainty
about QA for “repeating” grants; and inadequate follow-up once it’s clear
that a QAPP is needed. It is
important to note that the failure to apply QA for environmental
effectiveness assessments in P2 grants may have far reaching implications
beyond P2 grants. In the future, USEPA
will likely be required to demonstrate the environmental effectiveness of a
broad range of its programs and projects, including grants, contracts, and
in-house work. These effectiveness
assessments will require new QA oversight. Through
this presentation we hope demonstrate this audit finding as an Agency-wide
wakeup call for the need to apply QA oversight via the graded approach to any
program or project that attempts to demonstrate its environmental
effectiveness, especially if that is the only part of the program/project
that involves environmental data. |
|
Title: |
Modeling
Cryptosporidium Recovery |
|
Presenter(s): |
Michael J. Messner, USEPA Office of Water, Office of Ground
Water and Drinking Water |
|
Key Message(s): |
|
|
Abstract: |
USEPA's Long-Term 2
Enhanced Surface Water Treatment Rule (LT2) requires drinking water utilities
to monitor their source waters for Cryptosporidium. Monthly samples of source water are tested
using USEPA Method 1622 or 1623. These
methods are known to recover, on average, about 40% of Cryptosporidium
oocysts that are present. Oocysts not
observed and therefore not counted may be lost at each step of the
measurement process. Recently, USEPA
has obtained data from utilities monitoring under the LT2 Rule. Nearly 50 laboratories generated
measurement data, including results from analyses of matrix-spiked
samples. A hierarchical model
is developed to describe (a) how the "true" numbers of spiked
oocysts vary from sample-to-sample, (b) how laboratory-specific median
recoveries vary across labs, and (c) how recovery varies from
sample-to-sample within laboratories.
Bayesian Markov Chain Monte Carlo sampling is used to estimate model
parameters. A striking feature of
the data is the number of matrix spiked samples for which no oocysts are
identified. These zero-recovery
samples appear more often than predicted by the model, which suggests that
something other than a simple "low and variable" recovery is at
play. Occasionally, something seems to
happen to cause none of the numerous spiked oocysts to be observed. When each laboratory is allowed to have its
own probability of zero recovery as a special case, the model fit improved
significantly. Different laboratories
appear to have significantly different probabilities of zero recovery, and
different distributions of recovery, when not zero. This model and what we've learned about recovery have
implications for how one might "adjust for recovery" to better
understand Cryptosporidium occurrence in source water. The model may also help us understand the
value of more precise enumeration of spiked oocysts and the added value of
improved precision of the measurement method |
|
Title: |
Elevated Blood Lead Levels: Statistical Analysis meets Social Phenomena |
|
Presenter(s): |
Ronnie Levin, USEPA
Region 1, and Barry D. Nussbaum, USEPA Office of Environmental Information,
Office of Information Analysis and Access |
|
Key Message(s): |
|
|
Abstract: |
Reducing US children’s
blood lead levels is likely the major environmental public health achievement
of the last three decades. In fact,
with the marked decline in lead levels, current national random surveys
rarely identify children with elevated blood lead levels. Nonetheless, there
are almost 400,000 Identifying children with dangerously high blood lead levels is
moving from an analysis of statistical data to one investigating cultural
practices, past nutritional status, consumption of ethnic foods and remedies,
and site-specific factors, such as local industry. Developing plausible analytical
methodologies to incorporate these ‘anecdotal’ or ‘social’ data into
targeting efforts for lead poisoning reduction is our current challenge. |
|
Title: |
ORD’s Scientific Data Management (SDM)
Strategy: First Steps |
|
Presenter(s): |
Lynne Petterson, USEPA Office of Research and Development,
Office of Science Information Management |
|
Key Message(s): |
|
|
Abstract: |
The
purpose of this presentation is to summarize initial efforts in implementing
ORD’s Scientific Data Management (SDM) Strategy. The efforts include: Development of a ‘thin’ or high-level taxonomy
and its testing through paper prototypes in specific use cases;
Identification of value-added tool(s) and processes to assist researchers in
managing their scientific spreadsheets, and; Development and provision of
scientific Records Management training.
The three areas were identified as high priority tasks by the Near
Term Implementation Team for ORD’s SDM Strategy in 2007. The Near Term Implementation team consisted
of ORD researchers at the Branch Chief level and above. |
|
Title: |
A New Vision for Data Integrity |
|
Presenter(s): |
Charlie Appleby, USEPA Region 4 |
|
Key Message(s): |
|
|
Abstract: |
For
several years, an increase in the level of scrutiny of data quality has
required us to take a fresh look at how we ensure the integrity of the data
and information we generate. We are
all familiar with the phrase, “data of known and documented quality,” but
ensuring data integrity goes well beyond the implementation of a rigorous
quality system with documentation, QC measures, SOPs with corrective action
decision matrices and similar requirements.
This session will include presentations on the specific quality system
elements that are necessary to ensure data integrity and an auditor’s
experience with detecting the signs of unethical practices and ferreting out
the flaws in a quality system that allow such practices. The session will conclude with a
presentation and panel discussion of the vision that must be shared by
managers and scientists. This vision
must include an organizational culture with a strict ethical code that is
conducive to generating products and services with integrity, and the
responsibility that should be placed on new managers to carry the vision,
managing change as well as expectations for quality and integrity. The path to improved data integrity
naturally also leads to ensuring sound science, and as in any business
environment, performance at a high level of quality and reliability is the
surest way to ensure growth. |
Title:
|
QA Principles and Practices in the Everglades REMAP Project |
Presenter(s):
|
Charlie Appleby, USEPA Region 4
|
Key Message(s):
|
|
Abstract:
|
USEPA Region 4 occasionally performs environmental monitoring/assessment studies such as the Everglades Regional Environmental Monitoring and Assessment Program (REMAP). In-situ data were documented in the field, and eight analytical labs were contracted to perform sixty unique analyses for nutrients, anions, mercury, and physical parameters on samples of seven different matrices collected at over 250 Everglades sampling stations. Data quality in any survey study, but especially in one this size, must be part of project planning and execution from start to finish. The development and implementation of the quality system for Everglades REMAP Phase III are presented. One goal of this study was to produce data of known and documented quality that met pre-defined project goals and data quality objectives. For this $1.6 million project, the investment in quality asurance exceeded $100,000. Out of approximately 25,000 data points generated for this study, less than 0.01% were rejected as unuseable. |
|
Title: |
Stratified Sampling Design &
Field XRF to Reduce the 95% UCL for Residential Soil Lead |
|
Presenter(s): |
Deana M.
Crumbling, USEPA Office of Solid Waste and Emergency Response, Office of
Superfund Remediation and Technology Innovation |
|
Key Message(s): |
|
|
Abstract: |
Residential yards along a heavily traveled town street
were evaluated for soil lead (Pb) concentrations. Historic atmospheric deposition from a
nearby Pb battery smelting operation was the main source of Pb to the town. Prior sampling and ICP analysis nearly 10
years ago in these yards had yielded mixed and sometimes contradictory
results. Although these properties
were outside the expected zone of influence by air deposition, historical
soil sampling in this area had anomalously high concentrations. In 2007, a field-portable x-ray
fluorescence analyzer (FP-XRF) and a specialized sampling design were used to
resolve questions raised by the earlier data. Systematic planning defined three study questions: 1.
Is the average soil Pb concentration over a single property (defined
as the exposure unit) greater than the project-specific limit of 500 ppm? 2.
Can contamination of these particular properties be attributed to
truck activity to and from the smelter via this main thoroughfare? 3.
Can the spatial patterns of Pb distribution indicate the source. In addition to smelter truck traffic, three
more potential sources of Pb to the yards were postulated: a) general road
traffic from the era of leaded gasoline, b) lead paint from these older
houses, and c) air-deposited Pb from the smelter. A dynamic sampling and analysis design was constructed to
answer these questions in a single field effort. The design had to control for the effects
of micro-scale and short-scale matrix heterogeneity, and reduce statistical
uncertainty at the 500 ppm decision level to allow 95% statistical confidence
when deciding compliance. Sampling
density had to be high enough to detect any spatial patterning in lead
concentrations that could suggest the lead release mechanism. This case study presentation focuses on one yard to
explain how the triple combination of 1) an adaptive strategy to manage data
variability, 2) progressive CSM maturation over the course of field work, and
3) a stratified sampling & data analysis design allowed statistical
uncertainty (as reflected by the distance between the calculated mean and the
95% UCL) to be progressively reduced from 44% to 16% so that a highly
confident decision could be made that the property/exposure unit was
compliant with the 500 ppm action level. |
|
Title: |
Greening Quality Assurance: How to Use QA to Reduce Our Ecological
Footprint |
|
Presenter(s): |
Eugenia McNaughton, Manager, Quality Assurance Office, USEPA
Region 9 |
|
Key Message(s): |
|
|
Abstract: |
A Quality System (QS) and an Environmental Management
System (EMS) have similar objectives:
to achieve program objectives in the most efficient and appropriate
way. People organized in various ways,
be it in social, business, industry or government contexts, are reluctant to
change their practices until an obvious benefit to doing so can be
demonstrated. But both QS and |
|
Title: |
Reporting Limit Verification Sample |
|
Presenter(s): |
Robert P.
Di Rienzo ALS
Laboratory Group (formerly)
DataChem Laboratories, Inc. (801)
266-7700 |
|
Key Message(s): |
The use of
Reporting Limit Verification Sample (RLVS) will
|
|
Abstract: |
This presentation will discuss the use of a RLVS as a batch
quality control element. Data will be presented to establish the use of RLVS
to assess bias and precision at the reporting limit, to calculate control
limits at the reporting limits and to estimate the uncertainty at the
reporting limit. A new procedure for using this data to calculate limits of
detection will also be presented. |
|
Title: |
The Development and Implementation of a Virtually
Paperless Lab – Advantages and Disadvantages for a Core Faciltiy |
|
Presenter(s): |
Debora Andrews, USEPA Office of Research and Development, National Health and Environmental Effects Laboratory |
|
Key Message(s): |
|
|
Abstract: |
In the
ever-increasing climate of being environmentally friendly and doing more with
less, going paperless in the laboratory seems to be the wave of the
future. However, are we truly prepared
to go paperless? Can we find ways to
ensure our records are indeed as defensible as their paper cousins? What measures do we need to take in order
to insure the appropriate controls and long term storage of our current
electronic records? This
presentation focuses on an ELISA (enzyme linked immunosorbent assay) Core
Facility, based in USEPA’s National Health and
Environmental Effects Laboratory, in Advantages we have noted have been not only the reduction
of paper used, but also the ease of record transfer, locating records if
misplaced by the end user, and reduction in storage space. Disadvantages noted were lag time in
having everyone switch over to e-records, lack of a traditional paper trail
caused a few problems, and lack of electronic signatures, or ability to lock
the data. Solutions that were
implemented will be discussed, most notably, how to use existing Agency
standard software to work towards traceability and accountability in the
arena of e-records. This is an abstract for presentation which has been
reviewed by the USEPA; views expressed do not necessarily represent USEPA
policy. |
|
Title: |
USEPA Office Product Quality – Mapping Products/Services
Quality Features and associated Information Policies into an interactive
system to ensure implementation of the organization’s quality system |
|
Presenter(s): |
Kevin
Lorena Cedeno-Zambrano, USEPA Office of Environmental
Information, Office of Planning, Resources, and Outreach
|
|
Key Message(s): |
|
|
Abstract: |
The USEPA
Quality Policy, CIO 2106.0, approved October 20, 2008 provides a structure to
ensure effective application of the Agency quality program to Agency products
and services. Agency quality managers
may use the new policy to strengthen their Office/Region quality management
plans (QMPs) and better align application of quality principles to USEPA
business lines within their Office/Region.
To better inform the USEPA Office of Environmental Information’s (OEI)
improvement planning process, we conducted an office-wide review to better
identify OEI’s products and services and the associated major quality
features. OEI then mapped potential
information policies and procedures that may be applicable to each product
and service into a proposed interactive module for an updated electronic QMP
(eQMP). This presentation reviews the
OEI process for mapping products, services, and quality features into a
quality matrix. The draft interactive
module will demonstrate how it acts as a tool to identify applicable
information policies for products and services. The process, matrix, and proposed module
may be resources for others who want to conduct a similar analysis and
develop similar tools. |
|
Title: |
Role of an NHEERL QA Manager in Achieving
Quality Performance Objectives − Changes from 1999 to 2008 |
|
Presenter(s): |
Thomas Hughes, USEPA, Office of Research and Development, National Health and Environmental Effects Laboratory |
|
Key Message(s): |
|
|
Abstract: |
My career as a
USEPA Quality Assurance Manager (QAM) started on September 26, 1999 when I
was appointed the QA and Records Manager for the Experimental Toxicology Division
(ETD) in NHEERL, in the Office of Research and Development (ORD), on the
Research Triangle Campus in |
|
Title: |
Value-Added Verification: Building relationships and organizational
knowledge through Verification and Validation of management practices. |
|
Presenter(s): |
Margo Rice |
|
Key Message(s): |
|
|
Abstract: |
This presentation will explore current quality assurance
techniques which can be used in concert with Program Management activities or
on a per Project basis to ensure quality of product or project success in
addition to supporting the development of organizational knowledge,
information-sharing and skills development. Each quality assurance approach
represents a pairing of traditional methodology considerations for empirical
definition of process or product requirements with considerations for
presentation context, information collection and sharing. Specific techniques
include “Value-Added Auditing” and tailored approaches for Independant
Verification and Validation and Peer Review. The presentation will include
casestudies from use on projects managed by Project Performance Corporation
and work done in partnership with the Environmental Protection Agency. |
|
Title: |
A Quality Assurance Program Plan for the Risk Management Program:
An Example Document Implementing the New Environmental Policy |
|
Presenter(s): |
David R.
Taylor, Ph.D., C.Q.E, USEPA Region 9 |
|
Key Message(s): |
|
|
Abstract: |
The
recently approved Environmental Policy has implications for programs within
USEPA that do not collect what have been traditionally considered to be
“environmental measurements.” One such
program is the Risk Management Program whose purpose is to ensure that
companies or organizations storing, transporting, or using hazardous waste
prepare and implement Risk Management Plans.
USEPA Regions as well as its state, tribal and local organizations
perform inspections of these facilities to ensure that these plans are
current and that they are being implemented in a way that ensures the safety
of surrounding communities. These
inspections evaluate a variety of practices and policies, but no samples are
collected or analyzed. Although the
new Environmental Policy requires that a QA Plan be prepared, the current
USEPA QA guidance documents are not easily adapted to this type of
program. This talk will discuss how a
QA Plan was structured to describe this program’s Quality System, and
possible implications for other similar programs. |
|
Title: |
Framework for Providing Quality Assurance Guidance for
Computer-Based Software Models used in Research Projects and Regulatory
Support Activities |
|
Presenter(s): |
Eric S. Hall – USEPA Office of Research
and Development, National Exposure Research
Laboratory
|
|
Key Message(s): |
|
|
Abstract: |
Abstract: The US
Environmental Protection Agency (USEPA) relies on computer-based software models
to provide information on the effects of pollutants in the environment and on
human health. Some of the software models used by EPA are developed
outside of the agency and are obtained through contracts, cooperative
agreements, interagency agreements (IA), and grants. The elements of
quality are well-understood when applied to situations, such as determining
if the detection limit of a piece of equipment is sufficient to measure the
amount of a substance in the environment. When defining the elements
required to determine quality for software models, guidance such as EPA
QA/G-5M is limited because its focus is on developing quality assurance
project plans (QAPPs). The QA/G-5M describes how to plan for and
document quality-related processes associated with software models, but it
does not contain a step-by-step description of the key QA elements that
should be included when developing a software model and its associated
documentation. Most individuals are
not experts in software engineering and are not experienced in developing
quality assurance guidance for software models. The contracting officer
technical representatives (COTRs – also known as CORs, WAMs, WACORs, DOPOs,
TOPOs, or project officers) are the people who manage the technical details of
a project where model software is developed. The COTR generates the
contract scope of work (SOW), work assignment/task order/delivery order,
grant, cooperative agreement, or IAG for a project. The SOW is where
quality requirements for software models must be inserted. Software
models are a critical component of EPA’s scientific research and regulatory
support activities; therefore we need a standard (and flexible) framework to
provide guidance on how to implement quality assurance for software models.
A good software QA methodology help EPA to obtain models that are consistent in their operation, reliable, easy to use, documented, implement the requirements, and transparent (CREDIT). There
are three elements that must be analyzed to determine if a model meets
minimum quality standards: 1) model code listings; 2) behavior/response of
‘model executables’ (operational model software), and; 3) model documentation
(e.g., user manual/operations manual, design document, test manual, etc.).
EPA’s quality assurance staff needs guidance to advise COTRs in how to assess
the quality of a software model. The quality staff requires specific
details on how to apply and implement a quality regime for software models so
that they can assist COTRs in their project responsibilities. This
presentation provides an easy to understand QA methodology for software
models that can be applied to a wide variety of projects. |
|
Title: |
Leading Change: Elevating Environmental Field Sampling and
Measurement Activities to a Higher Level of Quality |
|
Presenter(s): |
Kenna Yarbrough, USEPA Office of Enforcement and Compliance
Assurance, Office of Criminal Enforcement, Forensics, and Training |
|
Key Message(s): |
|
|
Abstract: |
Since 2001, USEPA’s The presentation will discuss two concepts that are not
traditionally applied to field measurements and sampling. Actively implemented at the NEIC, these
concepts are a robust proficiency testing program for all field staff and the
evaluation and reporting of measurement uncertainty information/data for
environmental field measurements. Other options for improving field sampling and measurement
quality will be discussed. These
options do not require a rigorous 3rd party ISO accreditation for
the environmental field program.
Examples to be presented include the adoption/conformance to The NELAC
Institute (TNI) field measurement and sampling organization draft standards
and USEPA’s Regional Science & Technology (RS&T) Director’s Field
Operations Group (FOG) field measurement and sampling consensus
standards. There are five FOG
consensus standards that involve the following quality related subjects: evidence management; equipment/instrument
inventory, calibration and maintenance; personnel training and records of
training; document control; and records management. The presentation will provide examples of
the many options available to environmental field offices to continually
improve the quality of work and data provided to their customers. |
|
Title: |
Visualization Tools Change the Whole
Game on Locational Quality
|
|
Presenter(s): |
Pat Garvey, USEPA Office of Environmental Information,
Office of Information Collection |
|
Key Message(s): |
Key Messages
|
|
Abstract: |
A presentation of web mapping tool from Google Earth and Microsoft
Virtual Earth would be showcased on how these tools are changing the way
USEPA looks at, documents, and improves the quality of locational data. Tools to improve locational accuracy will
be showcased and ways for the users and public to improve the data will be
highlighted. The USEPA’s Facility Registry System will be used as the
example of the problem, and the solution and the means for greater
involvement in better accuracy of locational data at USEPA. |
|
Title: |
A Brief Discussion of Three Important Quality Management
Concepts |
|
Presenter(s): |
Julia K. Caprio
Geosyntec Consultants |
|
Key Message(s): |
|
|
Abstract: |
Quality Management also referred to as Total
Quality Management, is the organization wide management of quality.
Management consists of planning, organizing, directing, control, and
assurance. This presentation will discuss the importance of defining quality
within an organization in an effective manner such that all personnel
understand the benefits and empowerment of working within a quality oriented
system. Secondly this presentation
will discuss the importance of leadership including descriptions of effective
leaders and their key role in developing a culture of quality within an
organization. The last concept that will be discussed during this presentation
is what the role of a quality manager is within an organization. The quality
manager’s role has evolved from “policing” personnel activities to training
and guiding personnel with regard to quality integration into their work
practices. All of these concepts will
be described from the point of view of implementation within a service
oriented environmental consulting company. |
|
Title: |
ERLN Operational Framework |
|
Presenter(s): |
Terry Smith and Schatzi Fitz-James, Office of Solid Waste
and Emergency Reponse, Office of Emergency Management Sean
Kolb, Computer Sciences Corporation |
|
Key Message(s): |
|
|
Abstract: |
EPA’s
Environmental Response Laboratory Network (ERLN) is one of five laboratory
response networks under the Federal Integrated Consortium of Laboratory
Networks (ICLN). The ICLN‘s mission is
to coordinate federal response networks to provide timely, high quality and
interpretable results for early detection and effective consequence
management of acts of terrorism and other events requiring an integrated
laboratory response. The mission of
the ERLN is to provide known laboratory capabilities, capacities, and quality
data in a scalable, systematic, and coordinated response to environmental
emergencies providing preparedness, response, remediation and recovery
analytical support. ERLN laboratories will include federal, state, local and
commercial laboratories capable of analyzing environmental sample matrices
contaminated by toxic industrial chemicals (TICs), chemical warfare agents
(CWAs), biological agents, and radiochemical agents as a result of acts of
terrorism or an environmental emergency.
The ERLN
is faced with many unique challenges.
It must be able to provide analytical capabilities for a wide variety
of environmental matrices. It must be able to provide these analytical
capabilities to support a wide variety of capacity needs including an
individual short term response involving a few dozen samples to multiple long
term responses that require the analyses of tens of thousands of samples. It
must be able to provide these analytical capabilities to support a wide
variety of data needs including determining: (a) priorities for response
based on risks to human health and the environment, (b) the nature and extent
of contamination from the release of a hazardous material, (c) appropriate
cleanup goals and/or level, and (d) whether or not contaminated areas can be
returned to their intended use. This
paper discusses the operational framework EPA has established that enables it
to reliably access ERLN laboratory assets on an as-needed basis with
sufficient capacity and analytical capability to meet project-specific data
quality objectives. The key components
of the ERLN’s operational framework include a comprehensive quality system
based on industry-accepted standards, a stable sample management system
designed to maintain the integrity of samples, and a data reporting
requirement designed to provide consistent, reliable data of known and
documented quality to its ERLN customers. |
|
Title: |
Adapting a Quality
Management System to Meet Changing Requirements |
|
Presenter(s): |
Zachary Willenberg, Battelle Memorial Institute |
|
Key Message(s): |
|
|
Abstract: |
A
documented QMS is a necessity for most research
facilities and is required by many funding organizations, including the U.S.
Environmental Protection Agency (USEPA).
However, different funding organizations, even within the Federal
Government, have different quality requirements. For contract research organizations, whose
work can be supported by diverse government and industry sources, a QMS that
is rigorous yet flexible enough to address different requirements and demands
is essential, especially to ensure that the system provides sound
science. This presentation will compare
and contrast quality requirements from different organizations (e.g., USEPA,
U.S. Navy, U.S. Army Corps of Engineers, and Industrial Trade Organizations),
and discuss how those requirements are addressed in a flexible QMS that is
able to meet the varying demands of all these organizations. This presentation is based on case studies
describing such a QMS that has been in place for almost 10 years and that has
been successfully used and adapted over time to meet changing demands.
The
example cases to be covered include a strategy on dealing with document-heavy
ISO 17025 certification requirements, to allow an analytical group to be as
efficient as possible and prevent the certification requirements from
becoming burdensome on affected staff.
To address this situation, the QMS has been designed to
“compartmentalize” certain research groups and hold only those groups to
varying degrees of quality requirements.
This practice allowed two research labs to meet requirements of ISO
17025 for gas analyses and New Jersey Certification for dioxin/furan and PCB
analyses, without the entire organizational group being subject to such high
levels of quality assurance. Another
example case involved the successful application of project- or
program-specific quality management plan addenda. If a program or project has specific
quality management requirements, but those requirements are not sufficient to
warrant a new program-specific Quality Management Plan (QMP), then the
development of an addendum is used to expand certain sections of the overall
QMS QMP to meet those requirements.
For example, this practice was used in an ongoing USEPA program, for
which additional requirements for documentation handling and storage procedures
were needed, but the rest of the quality system was acceptable. Based on
these and other case studies, attendees will come away with specific
applications that they can easily adapt to their own organizations’ QMS and
that will enhance and develop their quality systems to ensure that results
delivered to their clients are based in sound science. |
|
Title: |
Environmental Response Laboratory
Network Data
Management Strategies |
|
Presenter(s): |
Schatzi
Fitz-James and Terry Smith, USEPA Office of Solid Waste and Emergency
Response, Office of Emergency Management Sean Kolb
and Lisa Modigliani, Computer Sciences Corporation |
|
Key Message(s): |
|
|
Abstract: |
The
Environmental Response Laboratory Network’s (ERLN’s) quality system’s
flexibility enables it to address various analytical services while
maintaining the integrity of the data used to make major decisions regarding
public health and environmental welfare.
The ERLN’s quality system is designed to: •
Assess data quality from the response phase
(e.g., field activities) to the laboratory; •
Detect problems through data assessment and establish corrective
action procedures that keep the process reliable; •
Document all aspects of the measurement process to improve
inter-laboratory data comparison; and •
To provide data that are technically sound and legally defensible. To address the
external framework (field activities) aspect of the quality system, USEPA
Office of Emergency Management has charged the Data Management Workgroup with
the task to develop a generic Quality Assurance Project Plan (QAPP) and a
QAPP for Emergency Response to provide guidance in establishing,
implementing, and using QA and quality control (QC) protocols for data
collection activities. This guidance
will define the data quality within a framework that also incorporates the
intended use of the data. ERLN
laboratories will be responsible for establishing and maintaining a
documented quality system, or internal framework, for planning, implementing, and assessing work performed and for carrying
out required quality assurance and quality control activities necessary to
ensure the integrity of the data they generate. A key feature of each laboratory’s
quality system is a set of standard operating procedures (SOPs) that facilitates consistency in the quality and
integrity of data through the consistent implementation of processes or
procedures within each ERLN laboratory.
This paper discusses data management strategies for the
data quality of the external and internal systems, which includes data flow,
data management, and data exchange throughout each phase of an ERLN
response. In addition, the data
requirements and the tools necessary are identified to assure compliance with
USEPA data standards. These
requirements are designed to ensure consistent analytical data of known and
documented quality across the ERLN. |
|
Title: |
Air Force’s Environmental Restoration Processes
Optimization (ERP-O) Program |
|
Presenter(s): |
Joann Socash for Ed Brown,
|
|
Key Message(s): |
|
|
Abstract: |
Environmental Restoration Processes Optimization (ERP-O)
allows the Air Force Center for Engineering and the Environment (AFCEE) to apply quality practices to Air Force
Environmental Restoration products and services, from concept to implenentaion.
ERP-O is one
component of the Air Force’s Performance-Based Manangement (PBM) initiative
developed by the AFCEE. The PBM
framework consists of eight main strategy components connected by open, real
time, communication that promotes trust among the restoration team
members. PBM provides a set of tools
that can be used either separately, as is increasingly common in current
business practice, or simultaneously.
Both approaches will benefit the project or program; however the most
significant results are realized when the components are coordinated. The Air
Force recognized that many of its cleanup actions were not performing as
originally expected and develop the ERP-O program for optimizing
performance. ERP-O encourages the
application of performance metrics and sound science. ERP-O includes (1) preparing and using
Conceptual Sites Models, (2) confirming adequacy of selected ARARs, (3)
defining Risk based on future planned use of resources, (4) promoting use of
decision logic, (5) establishing a clear and succinct exit strategy, (6)
promoting use of performance-based contracts (PBC) when applicable, and (7)
promoting implementation of Green and Sustainable remediation in the
operation of remedial systems and monitoring programs. ERP-O recognizes
that as a remediation progresses site conditions change and systems and
procedures designed for the original site conditions may no longer be
optimal. ERP-O may be implemented is
several phases. All Air Force
installations maintain an inventory of there remedial systems and the status
of the cleanup effort. In Phase I, a
team of experienced staff from various disciplines visits the installation to
review all active projects and provide recommendation. Recommendations from Phase I may be implemented
directly or moved to Phase II for a detailed engineering evaluation. Implementation of the recommendations is
accomplished by the installation in Phase III. The focus of the ERP-O is to gain the
maximum cleanup results and reduce overall long-term costs to the Air
Force. In doing this remedial action
is balanced with the long-term requirements for site monitoring. |
|
Title: |
Quality- A Contract Runs through It |
|
Presenter(s): |
Marion
Kelly, USEPA Office of Water, Office of Science and Technology Lynn
Walters, CSC |
|
Key Message(s): |
·
An approved QAPP must be in place for all USEPA
projects prior to any data gathering work or use. In most cases, USEPA projects involve the
use of contractors and grantees. ·
USEPA’s QAPP requirements need to be addressed in the
contract or grant vehicle, both at the procurement (solicitation) and
implementation stage. ·
There are multiple ways to accomplish this, with no
one-size-fits-all solution. Various
approaches are examined here. |
|
Abstract: |
USEPA Order CIO 2105.0 requires that Quality Assurance Project Plans (QAPPs), or equivalent
documents, be approved for all applicable projects and tasks involving
environmental data prior to any data gathering work or use, except under
circumstances requiring immediate action to protect human health and the
environment or operations conducted under police powers. Many
USEPA projects that involve the collection or use of environmental data are
supported by contractors. As a result,
requirements for QAPPs, or equivalent documentation, need to be addressed in
the contract prior to the contractor’s collection or use of data under the
project. These requirements are
addressed during the contract procurement process (i.e., solicitation and
award) and throughout contract implementation (i.e., when specific work
assignments, task orders, or delivery orders are issued and approved under
the contract). Although
the process sounds straightforward, there is no “right” or “wrong” way to
accomplish the requirement for having an approved QAPP in place prior to
initiation of contractor work.
Multiple approaches can be used, depending on the specific circumstances
of the project and the contract. In
this paper, we examine and suggest some “best management practices” for
addressing this requirement. |
|
Title: |
Geostatistical
Modeling as a Quality Management Tool to Address Uncertainty in
Decision-making for Large Scale Sediment Assessment and Remediation Projects |
|
Presenter(s): |
Judith Schofield, CSC |
|
Key Message(s): |
|
|
Abstract: |
As part of sediment assessment and remediation projects,
project planners are interested in describing the extent and nature of
contamination throughout their project site, as well as identifying areas
that exceed contaminant concentrations of concern and may therefore require
remedial action. Project planners want
to ensure that they are remediating all of the contaminated sediments; however,
they also do not want to spend limited resources on remediating relatively
clean sediments. Geostatistical models are facilitating these decisions while
providing an understanding of the uncertainty associated with the decisions. In support of several sediment assessment and remediation
projects being conducted by the US Environmental Protection Agency’s Great
Lakes National Program Office under the Great Lakes Legacy Act,
geostatistical tools are being used to evaluate site conditions and develop
remedial plans. Geostatistical models,
including kriging models, were developed in support of remedial
investigations and feasibility studies being conducted for the Geostatistical tools, developed in the mining industry, are
being practically applied in environmental assessment and remediation. These
techniques can be utilized to describe the extent and nature of
contamination, develop sampling designs, reduce uncertainty in decision
making, and facilitate cost effective remedial investigations. |
|
Title: |
Programmatic Synergy in Obtaining Cost Effective Data of
Known and Documented Quality |
|
Presenter(s): |
Louis Blume, USEPA |
|
Key Message(s): |
|
|
Abstract: |
The US Environmental Protection Agency’s (USEPA) Great
Lakes National Program Office has worked closely with USEPA’s Office of
Superfund, Office of Water, Regions 2 and 5, the States of New York,
Michigan, Wisconsin, Indiana, Minnesota and Ohio, as well as numerous
non-federal sponsors over the past five years, in the successful
implementation of the sediment remediation program under the Great Lakes
Legacy Act of 2002. The Act was
recently reauthorized on September 15, 2008 as the Great Lakes Legacy
Reauthorization Act of 2008 and appropriates annual expenditures of $50
million per year. The Legacy program’s
quality management policy mandates use of existing tools and expertise, where
feasible, for sample collection, analysis, and data management and
reporting. This policy provides a
challenge, as well as an opportunity, for several USEPA programs and States
to coordinate efforts for managing data. The presentations in this session will discuss aspects of
the data management collaboration in support of the Great Lakes Legacy Act
among USEPA programs and State agencies including: 1) the utilization of
Superfund’s contract laboratory program (CLP) for cost-effective sample
analyses. Sample analysis costs will be compared for several approaches
including through nationally competed contracts and through remedial
contractors and subcontracts; 2) the utilization of the Staged Electronic
Data Deliverable (SEDD) as a multi-stage standard for analytical data
reporting. The standard facilitates
sample specific quality control data reporting that provides transparency and
reproducibility. This standard,
combined with the use of Electronic data eXchange and Evaluation System
(EXES), streamlines data verification and provides electronically verified
data in customized electronic reports within 24-48 hours of data receipt from
the laboratory; 3) the challenges of utilizing these standardized approaches
with non-CLP laboratories, such as state laboratories and other non-federal
sponsors, and the potential this has for data reporting requirements for
other USEPA programs; and 4) the need for developing similar reporting
standards for associated field data and the means to conduct data validation
and data usability of merged field and laboratory data in support of project
level decisions. |
|
Title: |
A
business case for Implementing Uniform Federal Policy for Quality Assurance
Project Plans (UFP QAPPs) – The Region 2 Experience |
|
Presenter(s): |
Kevin Kubik , USEPA Region 2
|
|
Key Message(s): |
|
|
Abstract: |
Over a
three year period, benefits resulting from use of the UFP-QAPP in Region 2
include: improved QAPP quality; reduced preparation and review costs; and an
expanding customer base among Superfund, Brownfields, and non-Superfund
programs (Great Lakes National Program Office). The UFP-QAPP directive was issued by OSWER
for use on June 7, 2005 for implementation on all Federal Facility Hazardous
Waste Sites. Region 2 has adopted the
directive for all Region 2 CERCLA, RCRA, and Brownfields environmental data
collection projects. This past
year Region 2 has improved UFP-QAPP guidance usability by developing,
distributing, and conducting training on a suite of user-friendly tools and
an improved data deliverable process.
Electronic UFP-QAPP worksheet templates for Region 2 DESA Laboratory
and Contract Laboratory Program (CLP) method information as well as a Generic
QAPP template have been developed and distributed to customers. UFP QAPP
guidance and worksheet templates for use in Emergency Response initial
incident response and in transitional/follow-up activities were developed and
training presented to regional emergency responders, on-scene coordinators,
and contractors. In implementing the
UFP data review process, the Region is moving from an assembly line/cookie-cutter
data validation process to project specific data quality assessments. This
presentation will share Region 2‘s approach to customer friendly UFP
implementation improvements and describe the net efficiency and quality
benefits derived from increased use of the UFP. |
|
Title: |
Improving Customer Service Through Electronic Data Management & Paperless Data Delivery - The Region 2 Experience |
|
Presenter(s): |
Jennifer Feranda and Linda Mauel, USEPA Region 2 |
|
Key Message(s): |
|
|
Abstract: |
Region 2
has made great strides in efforts to improve customer service and
satisfaction, balanced with assuring the quality of environmental data used
in decision-making. This presentation
shares two contributing initiatives. The
Region has developed an innovative process which enables delivery of quality
assured analytical data, provided through the USEPA’s Contract Laboratory
Program (CLP), in a completely electronic format to Superfund, RCRA, and
Brownfields clients. The data delivery
process is provided in two stages: Stage 1) the delivery of preliminary data
in an Excel superset spreadsheet format following an initial automated data
review process to allow USEPA project managers to perform an initial
assessment of potential project needs.
A software tool (Data Assessment Tool (DAT) Editor) has been provided
to assist them in reading and pulling information they need from the Excel
files; and Stage 2) the delivery of the final quality assessed data, both in
the Excel superset spreadsheet format and a Form 1 PDF format on completion
of final validation and data quality assessment. With the
Agency’s change in database software from Lotus Approach to Microsoft Access,
Region 2 found an opportune time to develop a tracking database to manage
information related to the Quality Assurance (QA) oversight of Superfund
Projects. The intent of the database
is to: 1. Provide cradle-to-grave tracking
of environmental data collection projects from Quality Assurance Project Plan
(QAPP) development thru booking and
assignment of analytical services to data review and final archival of
project data; 2. Integrate with other automated
applications to minimize manual data entry and manipulation; 3. Generate necessary management
reports to document Regional
compliance with national policies on procuring analytical services (FASTAC,
ANSETS); and 4.
Minimize time and resources needed to input, track, manage, and
maintain environmental data and information. These
processes provide great improvement in timeliness for data delivery to
customers and data management. USEPA
resources to organize, copy, and mail hard copies of the data packages and to
manually enter database information have been saved. In addition, the paperless delivery
initiative has met |
|
Title: |
Improving Performance Data Though a
Life-Cycle Approach to Information Quality: Implementation of the Agency’s
Quality Policy |
|
Presenter(s): |
Joseph Greenblott and Judy Lieberman USEPA, Office of the Chief Financial Officer, Office of
Planning, Analysis and Accountability John Warren and Patricia Mundy USEPA, Office of Environmental Information, Quality Staff |
|
Key Message(s): |
|
|
Abstract: |
USEPA,
the Office of Management of Budget, Congress, and the public value Agency
performance data used for programmatic and Agency-wide planning,
decision-making, budgeting, and accountability. We highlight quality assurance issues for
data used for performance measures.
Planners and end-users involved in developing performance measures and
collecting, analyzing, and using performance data will follow a systematic
life-cycle approach for assuring performance data quality. Steps include the selection of performance
metrics; data collection planning; data collection, storage, and analysis;
and reporting and presentation. We
discuss how to apply the Agency’s new Quality Policy and the Plan-Do-Check-Act
lifecycle approach for performance data to:
We
present case studies to demonstrate how the approach can be applied to
existing performance data in two different USEPA programs. |
|
Title: |
Improving the Quality of Environmental Data Exchanges
Using Shared Services |
|
Presenter(s): |
Pat Garvey and John Harman, USEPA Office of
Environmental Information, Office of Information Collection
|
|
Key Message(s): |
The Office of Information Collection (OIC) offers many
services that program offices, regions, and labs can share to improve the
quality of data that is exchanged among environmental partners |
|
Abstract: |
The
Office of Information Collection (OIC) has services that support validation
of data collected and transferred to USEPA.
We work with USEPA program management and staff based on their
business requirements to leverage existing services This
session will include a review of services available in OIC, and a
presentation of business cases that demonstrate these options. This includes the following: ·
Terminology Services ·
Exchange Network Quality Assurance Services ·
Facility Registry and Substance Registry Services ·
Code Set Management Services (coming features) ·
Office of Water Case Study |
|
Title: |
Results of an Organizations Self-Assessment of Model Development and Application Practices |
|
Presenter(s): |
Lora S.
Johnson, Director of Quality Assurance, Office of Research and Devleopment,
National Exposure Research Laboratory |
|
Key Message(s): |
|
|
Abstract: |
The National Research Council issued a report on “Models in Environmental Regulatory Decision Making” (2007). This report, combined with the increasing importance of models to exposure research, prompted a comprehensive review of model selection, development, application, evaluation, and management practices within EPA’s National Exposure Research Laboratory. Results of the self-assessment will be described, including the need to more systematically incorporate quality assurance practices from the perspective of both software engineering and science. |
|
Title: |
Development, Evaluation and Application of Environmental Models |
|
Presenter(s): |
Noha Gaber, USEPA, Office of Research and Development, Office of the Science Advisor |
|
Abstract: |
The EPA Council for Regulatory Environmental Modeling recently published its Guidance on the Development, Evaluation and Application of Environmental Models. This document provides a simplified, comprehensive resource for modelers across the Agency on best modeling practices. When adhered to, the guidelines will help to ensure the quality, utility and regulatory relevance of the models that EPA develops and applies and the transparency of modeling analyses and model-based decisions. This presentation will provide details of the principles and best practices encouraged by the guidance and a discussion of the role of the QA community in implementing the guidelines. |
|
Title: |
Start with the End in Mind: Key Ingredients for Effectively Working ith Volunteer Collected Data |
|
Presenter(s): |
Danielle Donkersloot, New Jersy Department of Environmental Protection |
|
Key
Messages: |
|
|
Abstract: |
Well-designed monitoring programs have clear goals for their data – whether it’s building a solid water quality database for future trend analysis or documenting a pollution source in the watershed. Defining your data use, data users and developing a data communication plan is often an afterthought for many monitoring programs. It is vital to define your data users and your uses as you start to plan your monitoring program. All too often volunteers try to fit their data/information into classic data use scenarios. When asked about data use goals for their program, volunteer program coordinators will often respond "we want our data to be used by the state," when in reality they are action-oriented and more interested in protecting, improving, or making a difference in their favorite waterbody or watershed. The first part of this workshop will focus on the key ingredients to consider when you design a monitoring program. We will discuss the example of how the NJ Department of Environmental Protection (NJDEP) has defined the uses of volunteer collected data through the four-tiered approach for the agency and public use. Volunteer collected data is recognized as part of the State’s monitoring matrix and is viewed as a viable data source for the NJDEP. Even through the State has embraced the use of volunteer collected data, it has been difficult for the State to access and standardize the data of the many organizations throughout the state. The second part of the workshop will focus on the data management system the Department and volunteers have been working on for sharing different types of data and for presenting water quality information. We will discuss some of the strengths and short-falls of creating this type of system for volunteer use. These “lessons learned” apply to everyone in the water monitoring realm, not just the volunteers community. |
Title:
|
USEPA Region 6 Mobile Laboratory - A Field Analytical Vehicle |
Presenter(s):
|
Raymond Flores, F. Edward O'Neill, and Nicholas Gannon, USEPA Region 6 |
Abstract:
|
The USEPA Region 6 Mobile Laboratory is housed at the Houston Regional Laboratory. The mobile laboratory provides onsite analyses for multiple environmental programs including Superfund, RCRA, CWA and SDWA. Mobile Laboratory analyses provide quick turn around times and enhanced site characterization from increased station location samplings resulting in the ability to perform real time decision making by the Project Lead. Mobile Laboratory analyses meet the DQOs of the project and are supplemented by fixed base laboratory confirmation analyses. The use of the Mobile Laboratory by Region 6 programs including analytical methodology, quality controls, advantages, limitations, data deliverable and accreditation will be presented. |