V>EPA Revised Assessment of
Detection and Quantitation
Approaches
October 2004
-------
Engineering and Analysis Division
Office of Science and Technology
Office of Water (4303T)
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue, NW
Washington, DC 20460
EPA-821-B-04-005
October 2004
-------
Disclaimer
This document has been reviewed and approved for publication by the Engineering and Analysis
Division, Office of Science and Technology. Neither the United States Government nor any of its
employees, contractors, or their employees make any warranty, expressed or implied, or assumes any
legal liability or responsibility for any third party's use of or the results of such use of any information,
apparatus, product, or process discussed in this report, or represents that its use by such party would not
infringe on privately owned rights.
-------
Table of Contents
FOREWORD
CHAPTER 1: INTRODUCTION 1-1
1.1 Background
1.2 Clause 6 Settlement Agreement Requirements
1.3 EPA's Approach to Conducting this Assessment
1.4 Peer Review of the Agency's Assessment
1.5 Proposal and Request for Public Comments
1.6 Terminology used in this Document
CHAPTER 2: OVERVIEW AND HISTORY OF DETECTION AND QUANTITATION LIMIT
APPROACHES 2-1
2.1 Currie's Call for Standardization
2.2 Development of the MDL and ML as Practical Embodiments of Currie's Proposal
2.3 Other Detection and Quantitation Approaches
CHAPTER 3: ISSUES PERTAINING TO DETECTION AND QUANTITATION
3.1 Analytical Chemistry Approaches to Detection and Quantitation
3.2 CWA Regulatory Issues Affecting Detection and Quantitation
3.3 Statistical Issues
CHAPTER 4: EVALUATION CRITERIA
4.1 Criterion 1
4.2 Criterion 2
4.3 Criterion 3
4.4 Criterion 4
4.5 Criterion 5
4.6 Criterion 6
4.7 Consensus Principles
3-1
4-1
CHAPTER 5: ASSESSMENT
5.1 Detection Limit Approaches
5.2 Quantitation Limit Approaches
5-1
CHAPTER 6 FINDINGS AND NEXT STEPS 6-1
APPENDIX A: LITERATURE SEARCH REGARDING DETECTION AND QUANTITATION
LIMIT APPROACHES A-l
APPENDIX B: COMPUTATION OF DETECTION AND QUANTITATION LIMITS B-l
APPENDIX C: EXAMPLE CALCULATIONS C-l
-------
Foreword
EPA has assessed current procedures for determining the sensitivity of test methods and their
application to Clean Water Act (CWA) Programs. The assessment was required by a settlement
agreement with the Alliance of Automobile Manufacturers, et al. We announced the availability of our
preliminary assessment for public comment on March 12, 2003. This assessment discussed statistical,
chemical, and regulatory issues related to detection and quantitation and different approaches to
detection and quantitation. The Agency has revised the preliminary assessment Document to incorporate
public comment on that assessment.
In a related action on March 12, 2002, we proposed to revise EPA's method detection limit
(MDL) definition and procedure, and codify our minimum level (ML) procedure. The MDL and ML,
respectively and in order of increasing magnitude, are the EPA's embodiment of a detection and a
quantitation limit.
In this revised assessment, we have:
• Explained why and how we conducted this assessment (Chapter 1),
• Identified relevant concepts to include in the assessment (Chapter 2 of this document),
• Identified issues that may be relevant to the assessment from an analytical chemistry, statistical,
or regulatory perspective (Chapter 3),
• Used six criteria to evaluate the ability of each procedure or concept to support activities under
the Clean Water Act (Chapter 4),
• Assessed how well each concept meets the evaluation criteria (Chapter 5),
• Summarized our findings and discussed next steps (Chapter 6), and
• With real-world data and several different procedures, calculated and compared detection and
quantitation limits, and evaluated the theoretical and practical limitations of each concept
(Appendices).
Public comment on the preliminary assessment and the proposed regulatory revisions expressed
many divergent views that conflicted with the proposed revisions. Commenters noted that: (1) the MDL
does not adequately address analytical variability or systematic error (bias); (2) the MDL does not always
achieve a one percent (1%) false positive rate; (3) EPA should provide better guidance on the intended
use of the MDL and ML in compliance reporting; and (4) the MDL and ML are not appropriate for all
applications in CWA programs. Several commenters expressed support for two alternatives to the MDL
and ML that were submitted by a laboratory association and the U.S. Geological Survey, respectively.
Although none of the alternative procedures recommended by commenters fully satisfied EPA's needs
under the CWA, several procedures contain steps, such as blank correction, that EPA believes warrant
further consideration. There was no agreement among commenters as to which of the competing
alternatives or revisions to adopt. Commenters suggested that we work together to discuss mutual
concerns and possible solutions rather than proceed with the proposed revisions. We agree and recognize
that these concerns provide a strong starting point for a continued dialog with stakeholders.
Based on this new information, it is clear that there is a broad interest in improving current
procedures and uses, but no consensus for a specific procedure or procedures has emerged among the
laboratory, industry, regulatory or regulated communities. In addition, EPA sees merit in alternative
procedures suggested by commenters; however, none of these completely satisfy EPA's needs. Thus, we
believe that it is appropriate to withdraw the March 2003 proposed revisions, take final action on the
2003 assessment to complete the terms of the settlement agreement, and obtain additional stakeholder
-------
input. In a Federal Register notice published on September 15, 2004 [69 FR 55547], we announced that
a neutral party is exploring the feasibility of a process by which a broad group of stakeholders would
work together to define and address concerns about the way detection and quantitation limits are
calculated and used to support CWA programs. This stakeholder process would include stakeholders
representing constituencies such as citizens, environmental organizations, permit writers, regulators and
regulated industries. We trust that this stakeholder process will address the wide variety of views held by
stakeholders and lead to recommendations for possible improvements to current EPA procedures and/or
use of alternative procedures.
To facilitate open discussion and consideration of issues, we have made every effort to ensure
that this Revised Assessment Document does not prejudge the result of a future stakeholder process. We
look forward to further stakeholder participation in this process.
-------
Chapter 1
Introduction
1.1 Background
On June 8, 1999 (64 FR 30417), EPA promulgated (i.e., published in a final rule) Method
1631B: Mercury in Water by Oxidation, Purge and Trap, and Cold Vapor Atomic Fluorescence Spectro-
metry (the "method") for use in EPA's Clean Water Act programs. The method was developed
specifically to measure mercury at ambient water quality criteria levels and includes a method detection
limit (MDL; see 40 CFR part 136, Appendix B) of 0.2 nanograms per liter (ng/L).
Following promulgation, a lawsuit was filed challenging EPA on the validity of the method. The
basis of the challenge included several specific aspects of Method 1631 as well as the general procedures
used to establish the MDL and minimum level of quantitation (ML) published in the method. In order to
settle the lawsuit, EPA entered into a settlement agreement (the "Settlement Agreement") with the
Alliance of Automobile Manufacturers, Inc., the Chemical Manufacturers Association, and the Utility
Water Act Group (collectively the "Petitioners") and the American Forest and Paper Association
("Intervenor") on October 19, 2000. Under Clause 6 of the Settlement Agreement, EPA agreed to
perform an assessment of detection and quantitation limit concepts. The complete text of Clause 6 is
provided in Exhibit 1-1 of this chapter. A summary of Clause 6 is provided in Section 1.2. The summary
is followed by a description of EPA's approach to the assessment, including the material and data
evaluated (Section 1.3), the use of an independent peer review to evaluate the Agency's assessment
(Section 1.4), and EPA's March 2003 publication of and request for comment on the February 2003
assessment, and a related proposal concerning potential changes to detection and quantitation limit
procedures approved for use under the Clean Water Act (Section 1.5). A brief discussion of the
terminology used in this document is provided in Section 1.6.
1.2 Clause 6 Settlement Agreement Requirements
Clause 6 of the Settlement Agreement is titled Reassessment of Method Detection Limit and
Minimum Level Procedures. Clause 6 consists of five subclauses, a - b and d - f. (There is no subclause
c.)
1.2.1 Clause 6a
Clause 6a broadly defines the scope of the assessment and provides a schedule for completing
the initial phase. Specifically, Clause 6a requires EPA to:
• Sign and forward to the Office of Federal Register (OFR) a notice inviting public comment on a
reassessment of existing EPA procedures for determining the detection and quantitation limits of
contaminants in aqueous samples.
• Forward the notice to the OFR on or before February 28, 2003.
• Provide a period of at least 120 days for public comment on the notice.
1 - 1
-------
• At a minimum, include the MDL procedure published at 40 CFRpart 136, Appendix B, and the ML
procedure described in Section 17.8 of Method 163 IB, in the reassessment of detection and
quantitation limits.
• Invite comment on one or more alternative procedures for determining and describing test sensitivity.
Clause 6a also provides EPA with the option of proposing modifications to the existing procedures.
1.2.2 Clause 6b
Clause 6b requires that EPA obtain a peer review of its reassessment, and describes six specific
topics that must be included in the charge to the peer reviewers. Specifically, Clause 6b requires EPA to:
• Submit the reassessment of existing procedures (including any proposed modifications thereof) and
any evaluation of alternatives for peer review by experts in the field of analytical chemistry and the
statistical aspects of analytical data interpretation.
• Conduct the peer review in accordance with EPA's peer review policies.
• Prepare a charge to the peer review panel that requests the peer reviewers to consider:
»• Criteria for selection and appropriate use of statistical models
»• Methodology for parameter estimation
*• Statistical tolerance and prediction
*• Criteria for design of detection and quantitation studies, including selection of concentration
levels ("spiking levels")
*• Interlaboratory variability, and
*• Incorporation of elements of probability design.
1.2.3 Clause 6d
Clause 6d requires EPA to provide the Petitioners and Intervenor (the "litigants") with an
opportunity for review of the Agency's assessment concurrent with the Clause 6b peer review.
1.2.4 Clause 6e
Clause 6e requires EPA to provide the litigants with:
• An opportunity to meet periodically (i.e., every six months) to discuss the Agency's progress during
development of the assessment,
• A plan for performing the assessment on or before the second of these meetings, and
• Copies of relevant documents, where appropriate, in advance of these meetings.
1.2.5 Clause 6f
Clause 6f establishes a schedule and requirements concerning final action on the notice described
in Clause 6a. Specifically:
• On or before September 30, 2004 (since amended to November 1, 2004), EPA is to sign and forward
to the OFR a notice taking final action on the notice described in Clause 6a, and
• Coincident with publication of this notice of final action, EPA is to provide the litigants with an
opportunity to meet and discuss the implications of the final notice and/or the need for any
subsequent EPA action in light of the final notice.
1-2
-------
Exhibit 1-1. Full Text of Clause 6 of the Settlement Agreement
D. Reassessment of Method Detection Limit and Minimum Level Procedures
a. Un or before February £O, ZUUo, LPA shall sign and forward to the Uffice of the Federal Kegister for prompt
b.
FPA
"D P D M D L 40 C F R P 136
Appendix B, as well as the "minimum level" procedures, which is described in section 17.8 of Method 1 631 B. The
notice shall invite comment on trrt s evaluation of one or more alternative procedures for determining and
invite public comment for a period of no less than one hundred twenty ( I ZUJ days.
Prior to publishing the notice inviting public comment on LPA procedures for determining test sensitivity, LPA sha
/ \
alternatives for peer review by experts in the field of analytical chemistry and the statistical aspects of analytical
for selection and appropriate use of statistical models, methodology for parameter estimation, statistical toleranc
,„ ,„. . EpA ,
\ FPA'
II
e
the January 1998 Science Policy Council Handbook (EPA 100-B-98-QO) [SIC], including any subsequently-developed
FPA H
[c.
d.
e.
[Note - the correct document number for the Science Policy Council Handbook is EPA 100-B-98-001]
Note • there is no clause "6.c" in the Settlement Agreement]
re titi oners and Intervenor.
Inte rve nor with a period ic op portun ity to m ee t (i.e. , every six (Oj m on ths ) on the rtgency s pro gre ss. t F rt shall
P I A ' " "
reassessment/assessment of alternatives on or before the second such periodic meeting. Where appropriate, tr rt
P I
f.
On or before September 30, 2004 (Note! since amended to November 1,2004), EPA shall sign and forward to the
Uffice of the Federal Kegister for prompt publication a notice taking final action on the notice described in
subparagraph D.a. Coincident with publication of the notice of final action, LPA shall provide Petitioners and
Intervenor an opportunity to meet to discuss the implications of the final notice and/or the need for any subsequer
EPA action in light of the final notice.
s
it
1-3
-------
1.3 EPA's Approach to Conducting this Assessment
This document details the Agency's assessment of methodology for the determination of method
sensitivity, specifically: detection and quantitation limits. This assessment is being conducted in
accordance with a plan summarized in Section 1.3.1 and is based, in part, on an assessment of the data
described in Section 1.3.2.
1.3.1 Study Plan
EPA developed a technical approach for 1) conducting the assessment, and 2) complying with all
applicable requirements of the Settlement Agreement. The approach was documented in a draft study
plan that has since formed the general framework for the assessment described in this Assessment
Document. EPA also conducted a literature search to identify and review issues and concepts that should
be considered when developing the plan. A summary of this literature review is provided in Appendix A
to this Assessment Document.
The study plan described roles and responsibilities for implementing the plan, provided a
background discussion of detection and quantitation limit concepts, including the MDL and ML, and
outlined a series of 11 events associated with the Agency's assessment of detection and quantitation limit
approaches. The relationship between those planned events and this Assessment Document is
summarized in Exhibit 1-2 at the end of this chapter.
Although the Settlement Agreement did not require that EPA seek formal peer review on its draft
plan, the Agency chose to conduct a peer review of the draft plan. The peer review was initiated in
December 2001, conducted in accordance with EPA's peer-review policies, and performed by two
statisticians and two chemists. EPA reviewed the comments and recommendations offered by these
reviewers, and where appropriate, revised the plan to reflect the peer-review comments. EPA also
reviewed, and where appropriate, revised the plan to reflect comments provided by the petitioners
following their concurrent review.
1.3.2 Material and Data used in the Assessment
In order to perform the assessment described in this document, EPA sought to collect
documentation describing existing detection and quantitation limit concepts and procedures and data that
could be used to evaluate these concepts and procedures.
Documentation concerning the existing concepts and procedures was obtained by performing a
literature search as described in Appendix A to this Assessment Document, and where appropriate, by
purchasing copies of documents describing concepts or procedures from the organizations that published
them.
In performing this assessment, EPA hoped to identify a substantial amount of data containing
results of direct relevance to the determination of detection and low-level measurement capability. That
is, measurement results in the low concentration region. To date, EPA has been able to identify only six
data sets that were of use in fully evaluating variability in the range of analytical detection and
quantitation. Three of the six were developed by EPA for the express purpose of studying the
relationship between measurement variation and concentration across a wide variety of measurement
techniques and analytes. EPA refers to these data sets as "EPA's ICP/MS Study of Variability as a
Function of Concentration," "EPA's Multi-technique Variability Study" (also referred to as the "Episode
1-4
-------
6000 study"), and "EPA's GC/MS Threshold Study" (also referred to as "the Episode 6184 study"). In
all three cases, replicate measurement results from each combination of analyte and measurement
technique were produced by a single laboratory over a wide range and large number of concentrations.
The fourth data set was developed by the American Automobile Manufacturer's Association (AAMA)
for the purpose of estimating one particular kind of quantitation value. That quantitation value is called
an alternative minimum level (AML; see Gibbons et al, 1997). In the AAMA study, replicate results
were measured at a limited number of concentrations by multiple laboratories using EPA Method 245.2
(cold vapor atomic absorption; CVAA) for mercury and EPA Method 200.7 (inductively coupled
plasma/atomic emission spectroscopy; ICP/AES) for twelve other metals. The final two data sets were
jointly gathered by EPA and the Electric Power Research Institute (EPRI) to support interlaboratory
validation of EPA Methods 1631 and 1638.
The studies from which these six data sets were obtained are summarized in sections 1.3.2.1 -
1.3.2.6 below. Additional information about these studies can be found in Appendices B and C to this
Assessment Document.
In March 2003, EPA published an Assessment Document dated February 2003, and requested
comments on the assessment and additional data to support continued evaluation of detection and
quantitation limits. Three stakeholders commenting on the assessment also offered to provide EPA with
data that would substantiate their views or aid EPA in further evaluating detection and quantitation
procedures. These data are further described in Sections 1.3.2.7 - 1.3.2.8 and Section 1.3.3 below.
Although the petitioners offered specific suggestions for other data sets that they believed should
be considered in this assessment, EPA found that these data sets did not include a sufficient number of
results in the region of detection and quantitation to yield information for the assessment, overlapped
with data already used in the assessment, or exhibited signs of significant contamination that made the
data inappropriate for inclusion in the assessment. These data, and EPA's decisions regarding the data,
are discussed in Section 1.3.3 below.
1.3.2.1 EPA's ICP/MS Study of Variability as a Function of Concentration
The objective of the ICP/MS study was to characterize variability as a function of concentration
using EPA's draft Method 1638 for determination of nine metals by inductively coupled plasma with
mass spectroscopy (ICP/MS). The nine metals were silver, cadmium, copper, nickel, lead, antimony,
selenium, thallium, and zinc. The ICP/MS instrument used in this study averages triplicate scans to
produce a single measurement of each element at each concentration. Such averaging is typical of
ICP/MS design and use.
In preparation for the study, the ICP/MS was calibrated using triplicate scans averaged to
produce a single measurement of 100, 1,000, 5,000, 10,000, and 25,000 nanograms per liter (ng/L) for
each element. Originally, the instrument was calibrated using unweighted least squares estimates under
the assumption of linearity. Subsequently, the analytical results were adjusted with weighted least
squares estimates. Weighted least squares estimates are based on the knowledge that variability
(expressed as the standard deviation) increases with increasing analyte concentration.
1-5
-------
Although the instrumentation has the capability to provide intensity results for each of the three
scans at each concentration, averaging the three scans to produce a single measurement is the normal
operating mode, and the average was used to produce the measurements in this study. Draft Method
1638 specifies the use of average response factors rather than least squares estimation of a linear
calibration, although it does allow for the use of such procedures.
All nine metals were spiked into reagent water to produce solutions at concentrations of: 0, 10,
20, 50, 100, 200, 500, 1,000, 2,000, 5,000, 10,000, and 25,000 ng/L. Each solution was divided into
seven replicate aliquots for subsequent analysis. The aliquots were analyzed beginning with the blank
(zero concentration) followed by analyses from the highest to the lowest concentration. This sequence
was chosen to minimize carry-over effects and to allow the analyst to stop at the concentration that
returned zero results. Carry-over is caused by residual sample remaining in the inlet system of the
instrument, in this case, the ICP/MS. Carry-over can occur when analysis of a high-concentration sample
is followed by analysis of a relatively low-concentration sample, as could occur if the replicates were
analyzed in random order. Use of the highest to lowest analytical sequence ensured that each successive
concentration analyzed was close enough to the previous concentration that any effects of carryover
would be negligible and, therefore, would not compromise study results. (A more in-depth discussion of
the randomized design and the effects of carry-over issues is provided in Chapter 3, Section 3.3.8.2).
Results at multiple mass-to-charge ratios, or m/z's, were reported for each metal, although draft
Method 1638 specifies only one m/z for eight of the nine metals. For lead, m/z's 206, 207, and 208 are
specified. Only data associated with m/z's specified in draft Method 1638 were used in the ICP/MS
study.
1.3.2.2 EPA's Multi-technique Variability Study (the "Episode 6000 Study")
In 1997 and 1998, EPA conducted a study of variability vs. concentration for a number of
analytical methods. Five laboratories were employed for the analyses; each analyte and method
combination was tested by one of these laboratories. Details of the study design are described in EPA's
Study Plan for Characterizing Variability as a Function of Concentration for a Variety of Analytical
Techniques (July 1998). Based on the sampling episode number assigned to the study by the EPA
Sample Control Center, the study and results have become known as the Episode 6000 study and data.
The analytes and analytical techniques studied were:
• Total suspended solids (TSS) by gravimetry
• Metals by graphite furnace atomic absorption spectroscopy (GFAA)
• Metals by inductively-coupled plasma atomic emission spectrometry (ICP/AES)
• Hardness by ethylene diamine tetraacetic acid (EDTA) titration
• Phosphorus by colorimetry
• Ammonia by ion-selective electrode
• Volatile organic compounds by purge-and-trap capillary column gas chromatography with a
photoionization detector (GC/PID) and electrolytic conductivity detector (GC/ELCD) in series
• Volatile organic compounds by gas chromatography with a mass spectrometer (GC/MS)
• Available cyanide by flow-injection/ligand exchange/amperometric detection
• Metals by inductively-coupled plasma spectrometry with a mass spectrometer (ICP/MS)
1-6
-------
In this study, an initial (range finding) MDL was determined for each combination of analyte and
analytical technique using minor modifications to the MDL procedure at 40 CFR part 136. Specifically.
the modifications made the optional iterative step 7 of the MDL procedure mandatory and required the
spike concentration to be no more than a factor of three times the determined MDL (instead of a factor of
five times). During the study, however, two of the laboratories found that the reduction in the allowable
spike range necessitated an unreasonably large number of iterations. In continuing the study, EPA
returned to the spike-to-MDL ratio of five published in the 40 CFR part 136, Appendix B procedure.
After determining the initial MDL, each laboratory analyzed 7 replicate samples spiked at
concentrations that were 100, 50, 20, 10, 7.5, 5.0, 3.5, 2.0, 1.5, 1.0, 0.75, 0.50, 0.35, 0.20, 0.15, and 0.10
times the initial MDL. In a few instances, laboratories analyzed more than 7 replicates. As often as
possible, the replicate analyses at each concentration level were produced using the same calibration that
was used in determining the initial MDL. Where laboratory reports indicated that multiple calibrations
were conducted, each result was associated with its calibration in the data analysis.
Spiked aqueous solutions were analyzed in order from the highest concentration (100 times the
MDL) to the concentration at which 3 or more non-detects (zeros) were encountered among the 7
replicates, or the lowest concentration specified (0.1 times the MDL), whichever occurred first. This
analysis order (1) minimized carryover that could occur in some methods if a low-concentration sample
had followed a high-concentration sample (as may happen when samples are analyzed in random order),
and (2) prevented collection of a large number of zeros if the signal disappeared.
For methods that do not produce a signal for a blank, the signal will disappear somewhere below
the MDL, i.e., a zero will be reported. Laboratories were instructed that when three nondetects (out of
seven measurements) were reported, it was not necessary to move to the next lower concentration,
because it would be of no practical value to have laboratories measure seven zeros, move to a lower
level, measure seven zeros, etc.
A variant of the iterative procedure for determining the MDL was used for organic compounds
determined by chromatographic methods. Methods for organics normally list many (15 to 100) analytes,
and the response for each analyte is different. Therefore, to determine an MDL for each analyte, the
concentration of the spike would need to be inversely proportional to the response. Making a spiking
solution with 15 to 100 different concentrations is cumbersome and error prone. The approach used in
the study was to run seven replicates at decreasing concentrations until signal extinction, then select the
concentration(s) appropriate for the determining the MDL for each analyte according to the MDL
procedure. In some cases, the laboratories selected the concentrations, in others cases, EPA did. This
approach was generally applied for organics analysis. However, laboratories also had the option of using
some combination of the monotonically decreasing concentrations described above and a few selected
concentrations to achieve the desired spiking levels.
1.3.2.3 EPA 's GC/MS Threshold Study (the "Episode 6184 Study")
Data from the Episode 6184 study of variability vs. concentration were used to evaluate the
effect of GC/MS thresholds on the ability to identify semivolatile organic compounds at low
concentrations. Details of the design of this study are described in EPA's Study Plan for Characterizing
Error as a Function of Concentration for Determination of Semivolatiles by Gas Chromatography/Mass
Spectrometry (December 1998). Data were generated for 82 semivolatile organic compounds using EPA
Method 1625C (semivolatile organic compounds by GC/MS). MDLs were not determined for these
compounds. Instead, solutions of the analytes were prepared and analyzed at concentrations of 50.0,
1-7
-------
20.0, 10.0, 7.50, 5.00, 3.50, 2.00, 1.50, 1.00, 0.75, 0.50, 0.35, 0.20, 0.15, 0.10, 0.075 and 0.050 ng/jiL (or
|ig/mL). Each solution was injected into the GC/MS in triplicate with the mass spectrometer threshold
set to zero, and again in triplicate with the mass spectrometer threshold set to a level typical of that used
in routine environmental analyses. As with the ICP/MS study and the Episode 6000 study, and for the
same reasons described in Section 1.3.2.1, samples were analyzed in order from the highest to the lowest
concentration.
1.3.2.4 AAMA Metals Study of Methods 200.7 and 245.2
The American Automobile Manufacturers Association conducted an interlaboratory study of
EPA Method 200.7 (metals by ICP/AES) and Method 245.2 (mercury by CVAA). The study was
designed to estimate a quantitation value based on a concept termed the alternative minimum level
(AML) that had been described in the literature (Gibbons et al, 1997). Nine laboratories participated in
the study, and each reported data for the following 13 metals: aluminum, arsenic, cadmium, chromium,
copper, lead, manganese, mercury, molybdenum, nickel, selenium, silver and zinc. Study samples were
analyzed by EPA Method 200.7 for 12 of the metals. Mercury was determined by EPA Method 245.2.
As part of the study design, the nine laboratories were randomized prior to the start of the study.
Five sample matrices (including reagent water) were studied, including four wastewater matrices that are
representative of the automotive industry. Starting from a blank, or unspiked sample, all target analytes
were spiked at four concentrations to yield a total of five concentrations per matrix. Concentrations
ranged from 0.01 to 10 fig/L for mercury and selenium on the low end, and from 2.0 and 1000 fig/L for
mercury and selenium on the high end. In addition, the concentrations were matrix-dependent. The same
concentration ranges for each metal by matrix combination were used for all five weeks of the study.
Matrix A (reagent water) was analyzed in all nine laboratories, and three laboratories analyzed
each of the other four matrices. All analyses were repeated weekly over a five-week period. As a result,
a total of 6,825 observations were obtained, which includes 2,925 observations for matrix A (9 labs x 13
metals x 5 spike concentrations x 5 weeks), and 975 observations (3 labs x 13 metals x 5 spike
concentrations x 5 weeks) for each of the other four matrices (6,825 = 2,925 + (975 x 4)). There were
two missing values for chromium in matrix A from laboratories 1 and 9.
1.3.2.5 Method 1631 Interlaboratory Validation Study
The Method 1631 interlaboratory validation study was conducted by EPA to evaluate
performance of the method and to gather data to evaluate existing performance specifications, including
detection and quantitation limits. To accommodate stakeholder interests and expand the scope of the
study, the Electric Power Research Institute (EPRI) funded the distribution of additional samples to study
participants.
This jointly funded study involved an international community of twelve participating
laboratories and one referee laboratory. Each participating laboratory analyzed four different matrices,
each containing mercury at a concentration selected to allow for characterization of method performance
across the measurement range of the method. Each of the 12 participating laboratories was provided with
13 sample pairs (a total of 26 blind samples). These included 1 filtered effluent pair, 1 unfiltered effluent
pair, 4 filtered freshwater pairs, 1 filtered marine water pair, 1 unfiltered marine water pair, and 5 spiked
reagent water pairs. All 12 laboratories received and analyzed the same sample pairs (a total of 312
analyses). To measure the recovery and precision of the analytical system, and to monitor matrix
interferences, the laboratories were instructed to analyze matrix spike and matrix spike duplicate samples
1-8
-------
on specified field samples for each filtered and unfiltered matrix, spiked at 1 -5 times the background
concentration of mercury determined by analysis of an unspiked aliquot of the sample. The laboratories
were instructed to perform all other QC tests described in Method 1631, including the analysis of blanks,
and to conduct MDL studies in reagent water following the procedure at 40 CFR part 136.
1.3.2.6 Method 1638 Interlaboratory Validation Study
The Method 1638 interlaboratory validation study was conducted by EPA to evaluate
performance of the method and to gather data that would allow revision of existing performance
specifications, including detection and quantitation limits. To accommodate stakeholder interests and
expand the scope of the study, the Electric Power Research Institute funded the distribution of additional
samples to study participants.
A total of eight laboratories (and a referee laboratory) participated in the study. The study was
designed so that each participating laboratory would analyze sample pairs of each matrix of interest at
concentrations that would span the analytical range of the method. Each laboratory was provided with 11
sample pairs (a total of 22 blind samples). These included 1 filtered effluent pair, 1 unfiltered effluent
pair, 4 filtered freshwater pairs, and 5 spiked reagent water pairs. All eight laboratories received and
analyzed the same sample pairs (a total of 176 analyses). To measure the recovery and precision of the
analysis, and to monitor matrix interferences, the laboratories were instructed to analyze a matrix spike
and matrix spike duplicate of specified field samples in each filtered and unfiltered matrix, spiked at 1-5
times the background concentration of the analytes determined by analysis of an unspiked aliquot of the
sample. The laboratories were instructed to perform all other QC tests described in Method 1638,
including the analysis of blanks, and to conduct MDL studies in reagent water following the procedure at
40 CFR part 136.
1.3.2.7 American Council of Independent Laboratories Data
The American Council of Independent Laboratories (ACIL) is a trade association representing
independent, commercial scientific and engineering firms. Its members are professional services firms
engaged in testing, product certification, consulting, and research and development. On behalf of its
membership, ACIL submitted comments on EPA's proposal. To substantiate their comments, ACIL
provided EPA with data summary tables consisting of blank analyses used to calculate detection limits.
The data provided were performed by a single laboratory using Method 200.7 for five analytes. Because
only blank sample analyses were available, not all detection and quantitation limit procedures could be
assessed using the data. However, comparisons of the detection limit procedures submitted by ACIL and
the US Geological Survey were performed based on these data and are discussed in Appendix C. In
addition, because blanks were analyzed approximately two to three times per week, a comparison of
long-term to short-term variability was also performed using these blank data. ACIL also submitted an
alternative procedure for estimation of a detection limit, which is summarized in sect. 2.3.3 of this
document.
1.3.2.8 U.S. Geological Survey Method Detection Limit Data
To assist EPA's assessment of their long-term MDL (LT-MDL) procedure, the US Geological
Survey (USGS) provided data from blank sample analyses. These data represented a combination of 78
metals, methods and matrices, and were analyzed approximately twice per month. Unlike the blank data
provided by ACIL, these blanks were collected in the field, and, therefore, include more sources of
variability. As with the ACIL data, it was not possible to assess all detection and quantitation limit
1-9
-------
procedures using the blank data set because some procedures require use of samples spiked at one or
more concentrations. The LT-MDL procedure is summarized in sect 2.3.4 of this document.
USGS also submitted spiked sample results along with the blank data. These spikes were limited
to a single concentration, and did not sufficiently characterize the region of interest to allow for full
evaluation of detection and quantitation levels.
1.3.3 Data Considered but not Used in this Assessment
The Petitioners and mtervenor to the Settlement Agreement suggested ten specific data sets that
EPA should consider in its assessment of detection and quantitation limits. EPA evaluated each of these
data sets to determine if the design of the study, including the concentrations targeted in the study, would
provide sufficient data for evaluating measurement variability in the region of interest (i.e., at
concentrations below, at, and above the region of detection and quantitation). If such data were
available, EPA further evaluated the data set to ensure that it was of sufficient quality to support the
Agency's assessment. Four of the ten data sets met these requirements and were used in EPA's
assessment. Table 1 identifies each of the data sets suggested by the petitioners along with a brief
rationale for using or excluding the data from this assessment, additional discussion is in Appendix B.
After EPA published the February 2003 Assessment Document for comment, ACIL submitted
data as described in Section 1.3.2.7 above, and three commenters offered to provide EPA with additional
data that would enhance EPA's assessment. EPA requested the data offered by each of these
organizations, but received a response from only two of the three (Laucks Testing Laboratories and
USGS). After evaluating the data, EPA determined that the data from Laucks Testing Laboratories was
not useful because it was incomplete. The Laucks data unfortunately did not include the data from
extraction to detection which is needed to compare detection and quantitation approaches. Most of the
data sent by USGS was useful and is described in Section 1.3.2.8.
Table 1. Data Sets Suggested by Petitioners and Commenters
Dataset Source
and Year
AAMA
1996-1997
AAMA
1996-1997
EPA/EPRI
1997-1998
EPA/EPRI
1997-1998
ACIL
2002-2003
USGS
2002-2003
EPRI
1987
EPRI
1990
Analytes and
technology
Metals by ICP/AES
(200.7)
(245.2)
Mercury by CVAF
(1631)
Metals by ICP/MS
(1638)
Metal, by ICP/AES
(200.7)
Metals by ICP/MS
and GFAA
Metals by GFAA
(EPA 200)
Metals by ICP/AES
(EPA 200.7)
EPA's Use of Datasets
U S 1 3 2 4
U S 1 3 2 4
Used in this assessment and described in Section 1 .3.2.5
U S 1 3 2 6
Used in this assessment and described in Oection 1 .3.L..I
Used in this assessment and described in Oection 1 .3.2.8
Not used in this assessment because or insufficient low-level data
N t d tn t b f - ff t 1 -1 1 d t
1-10
-------
Dataset Source
and Year
EPRI
1994
AAMA
1996-1997
EPRI
1996
MMA
2000-2001
. y
Tt' r""9
2003
Analytes and
technology
Ai Be Ti by GFAA
(EPA 200)
PCBs by GC/ECD
(608 2)
Cd As Cr by GFAA
(EPA 200)
Aroclors 1016 and
1260 by GC/ECD
M
Hexachlorocyclo-
pen tadiene,
4-Nitropheno. (OSW
8270)
EPA's Use of Datasets
concentrations in the region of interest
concentrations in the region of interest
N EPA' E 6000 S
concentrations in the region of interest
N (5)
(7) " MDL
Oamples spiked with low levels of Mroclors exhibited average recoveries ^OUU/0,
N D
"
det/ quant procedures.
1.4 Peer Review of the Agency's Assessment
In August 2002, EPA conducted a formal peer review of the Agency's assessment. This peer
review, which satisfied requirements in Clause 6b of the Settlement Agreement, was conducted in
accordance with EPA's peer review policies described in the Science Policy Council Handbook (EPA
100-B-OO-OOl). The review was performed by two experts in the field of analytical chemistry and two
experts in the statistical aspects of analytical data interpretation. Each reviewer was provided with a
draft version of this Assessment Document, which documented the Agency's approach to the assessment
and the Agency's preliminary findings and conclusions. Reviewers also were provided with copies of all
data evaluated in the assessment, statistical programs used to analyze the data, and copies of the detection
and quantitation concepts and procedures evaluated by EPA. In accordance with the Agency's peer
review policies, the reviewers were provided with a written 'charge' intended to ensure the evaluation
would meet EPA needs.
In its charge to the peer reviewers, EPA requested a written evaluation of whether the assessment
approach described by EPA is valid and conceptually sound. Reviewers also were asked to consider and
address eight specific questions pertaining to the adequacy of the concepts and issues considered, the
evaluation criteria developed by EPA, EPA's assessment and conclusions, the data used to perform the
assessment, suggested improvements to the procedures discussed, and EPA's consideration of
interlaboratory vs. intralaboratory issues. Comments from peer reviewers were generally supportive of
EPA's assessment and its presentation of the assessment in the Assessment Document. Where
appropriate, EPA revised that Assessment Document to reflect specific suggestions and comments
offered by the peer reviewers. The revised version of the Assessment Document, reflecting peer
reviewer comments, was completed in February 2003, and made available through a public notice on
March 12, 2003 (see section 1.5 below). Copies of all materials associated with the peer review,
1-11
-------
including the peer review charge, the materials provided to the peer reviewers for review, complete
copies of the peer reviewers' comments, and detailed EPA responses to each of the comments were
provided in the public docket supporting the Agency's March 2003 assessment.
1.5 Proposal and Request for Public Comments
On February 28, 2003, the EPA Administrator signed two notices for publication in the Federal
Register. These notices fulfilled EPA's obligations under Clause 6(a) of the Settlement Agreement and
were published in the Federal Register on March 12, 2003.
The first of these notices announced the availability of EPA's assessment of detection and
quantitation procedures that are applied to analytical methods used under the Clean Water Act. It also
announced that results of the assessment could be found in the "Technical Support Document for the
Assessment of Detection and Quantitation Concepts" (EPA 821-R-03-005, February 2003), requested
public review and comment on the assessment. The full text of this notice was published at 68 FR
11791, March 12, 2003.
The second notice requested comment on proposed revisions to the detection and quantitation
definitions and procedures at 40 CFR part 136. The proposed changes were based on the assessment and
on stakeholder comments received over the years. The full text of this notice was published at 68 FR
11770, March 12, 2003.
1.5.1 Summary of Changes Proposed in March 2003
EPA proposed a number of technical and editorial changes to the definitions specified at 40 CFR
136.2 and to the procedure specified at 40 CFR 136, Appendix B. A detailed description of those
changes can be found in the March 12, 2003 public notice (68 FR 11770). Briefly, those proposed
changes included:
• A revised definition of the term "detection limit" at 40 CFR 136.2(f) to explicitly equate the term
with the "method detection limit" specified in 40 CFR 136, Appendix B; and a revised definition
of the term "method detection limit" included in Appendix B to provide technical clarifications
and more clearly equate the term with the "critical value" described by Currie (1968, 1995) and
the Limit of Detection described by the American Chemical Society (Keith et al., 1980;
McDougal et al., 1983). Those concepts are further described in Chapter 2 of this assessment
document.
• An expanded Scope and Application discussion in the codified MDL procedure to recognize that
there are a variety of purposes and analytical methods for which the MDL procedure may be
employed. The proposed revisions provided examples of four common uses of the MDL
procedure (i.e., demonstrating laboratory capability with a particular method; monitoring trends
in laboratory performance; characterizing method sensitivity in a particular matrix; and
establishing an MDL for a new or revised method for nationwide use.) The proposed revisions
also clarified that the procedure may not be applicable to certain test methods such as those used
to measure pH or temperature.
• Proposed modifications to the considerations for estimating the detection limit in Step 1 of the
codified MDL procedure and to the specifications for establishing the test concentration range in
Step 3 of the codified procedure.
• Proposed deletion of the optional procedure for calculating a 95% confidence interval estimate
for the MDL.
1-12
-------
• Proposed changes to the iterative procedure to mandate its use when determining an MDL for a
new or revised method or when developing a matrix-specific MDL, but allow it to remain
optional when determining an MDL for other purposes, such as verifying lab performance.
• Proposed addition of a new procedural section to address the treatment of suspected outliers.
• Proposed deletion of the discussion of analysis and use of blanks included in Section 4(a) of the
codified procedure.
• Proposed changes to the optional pre-test described in Section 4(b) of the procedure to improve
the utility of results from this test.
• Editorial changes to the codified version of the MDL. Examples of these editorial changes
include addition of a summary section, clarifications, reorganization of steps, simplified
presentation of calculations, and deletion of the reporting section.
Proposed addition of a definition of the ML at 40 CFR 136.2
• Proposed addition of a procedure (including a definition) of the ML to 40 CFR 136, Appendix B
• Explicit allowance of alternative detection and quantitation procedures, provided that the
resulting detection and quantitation limits meet the sensitivity needs for the specific application.
The objective of this proposed allowance was to provide greater flexibility in establishing or
improving the sensitivity of methods for use under CWA and facilitate approval of analytical
methods from other agencies or organizations that utilize alternate detection and quantitation
concepts.
In addition to requesting comment on the assessment and the proposed revisions, EPA also
specifically requested comment on several aspects of the proposal, including alternative actions that
could have been taken. With respect to the ML, for example, EPA explicitly sought comment on the
proposed addition of the ML definition to 40 CFR 136.2 and procedure to 40 CFR 136, Appendix B vs
an alternative option of not incorporating the definition at 40 CFR 136.2, but instead continuing to
specify the ML on a method-by-method basis. EPA encouraged commenters to support their views with
data or information that would assist the Agency in making a final decision.
1.5.2 Impact of Comments on the Assessment
EPA provided a 120-day period following publication of the notices for submission of comments
(from the date of publication of the notices to July 10, 2004). In response to requests from stakeholders,
EPA re-opened this comment period on July 16, for an additional 30 days (68 FR 41988).
During the comment periods, EPA received comments from 126 individuals or organizations
representing the diversity of the stakeholder community on this issue. They included 23 laboratories, 31
water treatment plants, 3 federal agencies, 11 state and county agencies, 23 industrial firms, 3 instrument
manufacturers, 19 trade organizations, 4 consultants, 8 individuals, and the law firm representing the
petitioners. Comments offered by these groups addressed more than 25 different issues. A complete
summary of the comments and EPA's responses to those comments can be found in Appendix B to this
Assessment Document. These comments are discussed at various locations throughout this document,
and include discussion of:
• Additional detection and quantitation limit procedures suggested by commenters. (Chapters 2, 3,
and 5)
• Public comments received on chemical, regulatory, and statistical issues, along with EPA's
consideration of these issues in light of the comments received. (Chapter 3)
• Comments received on each of the evaluation criteria used in EPA's assessment and EPA's
response to those comments. (Chapter 4)
1-13
-------
• Potential process for additional stakeholder involvement on the evaluation of detection and
quantitation limit procedures (Chapter 6)
Appendix C contains a detailed analysis of the detection and quantitation limit procedures evaluated
through computation of limits using the data described in Section 1.3.2. This analysis has been revised to
reflect new data and comments on the original version of the assessment, which was published as
Appendix C to the February 2003 version of EPA's Assessment Document.
1.6 Terminology used in this Document
We use the term "quantitation" in this document because of its common usage among analytical
chemists, even though we recognize that the term "quantification" (i.e., the act of quantifying) is the term
listed in most dictionaries. Also, when referring to detection and quantitation, we use the words
"approach" or "concept" to refer, generically, to the procedures used to establish detection and
quantitation limits or the theories on which those procedures are based. We use the word "limit" rather
than "level" to indicate that the detection and quantitation concepts are directed at the lowest
concentration or amount at which an analyte is determined to be present (detection) or may be measured
(quantitation). In choosing the word 'limit' we do not mean to imply any sense of permanence. We
recognize that measurement capabilities generally improve overtime, and that detection or quantitation
'limits' established today may be superseded by future developments in analytical chemistry.
Although the Settlement Agreement refers to the word "sensitivity" to describe detection and
quantitation limits, we have avoided such use of the term "sensitivity" in this document because the term
is widely used by analytical chemists to describe something other than detection and quantitation
capabilities. Traditionally, analytical chemists have referred to the term "sensitivity" as meaning
instrument signal units per concentration units, such as is given for a calibration slope or a response
factor. For example, in ion selective potentiometry, the sensitivity is 59 millivolts per decade change in
concentration for monovalent species and half that for divalent species. Sensitivity is a performance
characteristic, but it differs from detection limits. For example, one might compare the sensitivity of
instruments. Obtaining a sensitivity of 10,000 counts per ppb indicates a properly functioning Sciex 250,
while a Perkin-Elmer 6000's sensitivity would be 100,000 counts per ppb. Another performance
characteristic of sensitivity is that it may vary in an expected pattern as with mass to charge ratio in mass
spectrometry or atomic number for x-ray fluorescence spectrometry.
1-14
-------
Exhibit 1-2. Relationship of Assessment Document to
Assessment of Detection and Quantitation Limit Approaches
Event 1, Develop a detailed plan for responding to Clause 6 the Settlement Agreement: Tm. <»v<»ntwas oomPi<»t<»ci m Apm
2002 when the draft plan was revised to reflect peer review and Litigant comments.
EVent 2, Identify and eXplOre iSSUeS tO be COnSidered: The Settlement Agreement identified six specific issues that should be
n
development of the technical approach, trft identified a number of other issues that should be considered during the
assessment, tr ft listed and described each of these issues in the study plan and noted that identification of issues is likely to
be a dynamic process, in that as a suite of issues is identified and discussed, other issues may surface. Finally, trA stated
its intent to re are an "issue a er" that full ex lained and discussed each of the identified issues Cha ter 3 of this
Assessment Document serves the function of the issue paper described in the plan.
Event 3, Develop criteria against which concepts can be evaluated: Aner tuny considering an relevant issues, EPA developed
a suite of criteria that could be used to evaluate the suitability of various detection and quantitation procedures for use in C WA
prog ram s. Ch apte r 4 of this ftssessment LJocu me nt provide s and de scrib es th e crite ria selected by t r ft a fter its
Event 4, Evaluate existing procedures for establishing detection and quantitation levels: EPA svaiua.sd oxis.mg diction
and quantitation limit concepts used or advanced 1) by voluntary consensus standards bodies (VCoDs), 2) in the published
literature, OJ by L r A. A s per the term s of the Oettlem ent Agreement, the ML) L and ML were explicitly ta rgeted for inclusion.
tr ft committed to evaluating concepts published by fto I M International and loL/ and to consider approaches and procedures
offered by other organizations such as the American Chemical Oociety (ACo) and the International Union of Pure and Applied
Chem is try ^IUr ftC j, as well as other approach es that have been adop ted by tr ft for use in other program s or that were
identified during tr A s review of the published literature. Chapter 2 describes the concepts that trA evaluated in the
assessment. Where appropriate, these approaches also are discussed in context to the issues that are identified and
discuss ed in C ha pter O. C ha pte r O p res en ts the re suits of tr ft s assessm en t of e ach ap pro ach again st the e valu ation criteria
established in Chapter 4. Appendices D and C of this document present additional details of trA s assessment of each
approach, using the data described in Chapter I, Oection I .O.
EVent 5, DeVelOp and evaluate alternative prOCedureS: EPA planned to develop and evaluate alternative procedures and
modifications to existing procedures only if the Agency s assessment of existing procedures suggested that modifications or
alternatives to the existing procedures were needed, trft noted that its primary objective in developing such alternatives ^or
modifications) would be to address deficiencies noted in tvent 4 and improve the performance of the procedures that best
meet the criteria established in tvent 3. In accordance with the plan and with tPA s findings during the assessment, this
Assessment Document includes suggested modifications to the existing MDL and ML procedures.
Event 6, Conduct peer review of the Agency's assessment: EPA documented results of the Agency's assessment m a draft
Mssessment Do cum ent tha t wa s com pie ted in ftugust, ZUUZ . trft co nducted a form a I peer review of the assessment in
accordance with the Agency s peer-review policies and guidance. I he peer review was performed by two experts in the field
of analytical chemistry and two experts in the statistical aspects of analytical data interpretation.
7-11, ActiOnS taken follOWing peer revieW. After considering peer review comments, EPA revised its assessment and
the draft Assessment Document to reflect peer review comments. In March 2003, trA published two r K notices that met the
term s of Oettlem ent ftgree ment Clause Da. Comments were received on those notice s over a 4 m on th period ending in
August 2003. trA evaluated all comments received, and revised its assessment as appropriate to reflect these comments.
I his docum ent details this revised a sse ssm ent.
1 - 15
-------
Chapter 2
Overview and History of
Detection and Quantitation Limit Approaches
It is not possible to measure the concentration of a substance in water all the way down to zero.
As an analogy, consider the following example: imagine measuring an object less than 16th of an inch in
length with a ruler marked in l/16th-inch increments. How well can the length of the object be measured
using only the ruler? Similar issues arise as chemists try to measure ever smaller concentrations of
substances in water. In response to the challenges associated with measuring low concentrations,
chemists have defined numerical values that provide points of reference for reporting and using
measurement results. These values are usually referred to as detection and quantitation limits. This
chapter provides an overview of detection and quantitation approaches and procedures in analytical
chemistry and their use in Clean Water Act applications.
2.1 Currie's Call for Standardization
Since 1968, most of the literature regarding detection and quantitation has referenced the work of
Dr. Lloyd Currie, recently retired from the National Institutes of Science and Technology (NIST,
formerly the National Bureau of Standards). In 1968, Currie published a paper in which he reviewed the
then current state of the art regarding detection and quantitation, presented a three-tiered concept, and
demonstrated his concept with operational equations for a single laboratory. In his paper, Currie
reviewed eight existing definitions for the concept of detection, and reported that when these eight
operational definitions were applied to the same data, they resulted in numerical values that differed by
nearly three orders of magnitude. These results made it impossible to compare the detection capabilities
of measurement methods using available publications. Currie proposed standardizing the terminology
using theoretical definitions that he called the critical value, the detection limit, and the determination
limit. (In 1995, writing on behalf of International Union of Pure and Applied Chemistry (IUPAC), Currie
used the term "quantification limit" instead of his original term "determination limit." Substantial
agreement with the International Organization for Standardization (also known as "ISO") on the meaning
and language of detection and quantitation was achieved later, although some "subtle differences in
perspective" remain [Currie, 2000]). His purpose for these definitions was to create a system in which
the standard documentation of any measurement method would include a statement of capabilities that
were directly comparable to any other method for measuring the same substance.
Currie used terms from statistical decision
theory as the basis for his three-tiered system. In 1968
and 1995, Currie defined the critical value as the
measured value at which there is a small chance that
the concentration in the sample is zero. Consequently,
any measured result greater than or equal to the critical
value is considered evidence that the sample contains
the substance of interest. Currie was careful to
emphasize that the decision as to whether the substance
has been detected is made by comparing the
measurement result to the critical value. Figure 2-1
shows a critical value selected such that measurements
greater than the critical value have less than a 1 %
0.4
0,3
0.2
Critical Value •
1216
Concent ration
Figure 2-1
2-1
-------
Critical Valu
•^Detection Limit
chance of being associated with a sample that does not contain the substance of interest. The area under
the curve to the right of the critical value represents the probability that a measured value will exceed the
critical value. The area under the curve to the left of the critical value represents the (much greater)
probability of observing a value that is less than the critical value when the true concentration is zero.
Currie (1968 and 1995) used the term detection limit to refer to a true concentration that has a
high probability of generating measured values greater than the critical value. That is, measurements on
samples that contain concentrations equal to the
detection limit have a high probability of exceeding the
critical value and are, therefore, unlikely to result in a
decision that the substance is not detected in the
sample. In Currie's concept, the critical value and the
detection limit are related and functionally dependent,
but it is clear that the detection decision is made on the
basis of comparing sample by sample measurements to
the critical value. While Currie's terminology is
consistent with standard statistical decision theory, it is
in all likelihood responsible for a great deal of
confusion among chemists and others who may
associate the term 'limit' with some sort of decision
point. Currie (1995) states: "The single, most important
application of the detection limit is for planning. It
allows one to judge whether the CMP (Chemical
0.3
0.4
0.3
0.2
0.
o.o.
Q
4 6
Concentration
2
Figure 2-2
Measurement Process) under consideration is adequate for the detection requirements. " Figure 2-2
shows a detection limit selected such that 99% of the measurements on a sample containing this
concentration are expected to be above the critical value. The bell-shaped curve centered at the detection
limit illustrates how likely various measurement responses are when the concentration of the substance in
a sample is equal to the detection limit. That is, the figure shows the probability density of values
measured in a sample with a true concentration equal to the detection limit. The area under the curve to
the left of the critical value is equal to 1% of the total area, while the area to the right is equal to 99%.
Currie (1968, 1995) defined the determination limit, later renamed the quantification limit, as
(quoting Currie, 1995) "performance characteristics that mark the ability of a CMP to adequately
'quantify' an analyte. " Quantification limits "serve as benchmarks that indicate whether the CMP can
adequately meet the measurement needs. The ability to quantify is generally expressed in terms of the
signal or analyte (true) value that will produce estimates having a specified relative standard deviation
(RSD) commonly 10%. " This translates into a quantification limit equal to a multiplier of 10 times the
standard deviation (a measure of measurement variability) at the limit. The multiplier of 10 (equal to the
inverse of the 10% RSD) is arbitrary, but has been used widely. IUPAC selected 10 as a "default value"
(Currie, 1995), implying other values are possible. In papers published in 1980 and 1983, the American
Chemical Society's Committee on Environmental Improvement also recommended the use of a multiplier
of 10 for determining quantitation limits (see MacDougall, et al, 1980 and Keith, et al, 1983).
Measured concentrations greater than the quantitation limit are considered to be reliable by chemists,
although from a statistical perspective, any measured value, along with knowledge of the precision of the
measurement, is useful.
Currie's goal of having method developers publish directly comparable descriptions of detection
and quantitation capability remains elusive more than thirty years after publication of his first paper on
this topic. Even if Currie's three-tiered concept were used, the treatment of related issues causes
2-2
-------
difficulty in comparing methods. Some of these issues include interlaboratory variability, selection of
appropriate statistical models, design of detection and quantitation capability studies, and statistical
prediction and tolerance. These and other issues are discussed in Chapter 3 of this document.
2.2 Development of the MDL and ML as Practical Embodiments of
Currie's Proposal
In 1981, staff at EPA's Environmental Monitoring and Support Laboratory in Cincinnati, Ohio,
published a procedure for determining what they referred to as a method detection limit (MDL) (Glaser
etal, 1981). The MDL functions as a practical, general purpose version of Currie's critical value. The
MDL was subsequently promulgated for use in CWA programs on October 26, 1984 (49 FR 43234) at 40
CFR part 136, Appendix B. Prior to formal development of the MDL in 1981, the EPA Office of Water
had included the term "minimum level" (ML) or "minimum level of quantitation" in some methods for
analysis of organic pollutants. These methods were proposed on December 3, 1979 and subsequently
promulgated on October 26, 1984, along with the MDL. Additional information about the MDL and ML
is provided below in Sections 2.2.1 and 2.2.2.
2.2.1 Method Detection Limit
Conscious of the definitions provided by Currie and others, Glaser et al. (1981) stated "[t]he
fundamental difference between our approach to detection limit and former efforts is the emphasis on the
operational characteristics of the definition. [The] MDL is considered operationally meaningful only
when the method is truly in the detection mode, i.e., [the] analyte (the substance of interest) must be
present." Expanding on this reasoning, Glaser et al. (1981) developed MDL estimates for methods that
produce a result of zero for blanks, such as EPA Methods 624 and 625 for determination of organic
pollutants by gas chromatography/mass spectrometry (GC/MS). Blank variability exists, whether or not
it can be detected by measurement processes. Failure to detect this variability may be attributed to
insufficient sensitivity of the measurement process or, as is the case with some measurement processes,
thresholds that are built into equipment which censor measurements below certain levels. Currie's
critical value is dependent on the ability to estimate measurement variability of blank samples. In cases
where the substance is not detected in direct measurements on blanks, an alternative approach to
estimating blank variability must be used. One option is to estimate measurement variability at
concentrations that represent the lowest possible levels where a signal can be detected. This is the basic
approach of the MDL, which provides a general purpose, straightforward, operational procedure for
estimating a quantity analogous to the Currie critical value when measurement processes applied to blank
samples do not produce detectable signals. More complex statistical procedures for estimating blank
variability are possible and maybe preferable from a rigorous statistical perspective, but the MDL has
been found to be satisfactory by chemists in a wide range of applications.
In 1984, the MDL became a regulatory option for wastewater discharge permits authorized under
the Clean Water Act. To determine the MDL, at least seven replicate samples with a concentration of the
pollutant of interest near the estimated detection capabilities of the method are analyzed. The standard
deviation among the replicate measurements is determined and multiplied by the ^-distribution for n-1
degrees of freedom (in the case of 7 replicates, the multiplier is 3.143, which is the value for 6 degrees of
freedom). The decision to base the MDL on a minimum of seven replicates reflected a consensus among
EPA chemists and statisticians that a requirement of seven replicates is not overly burdensome for
laboratories and that laboratories could reasonably be expected to perform the analyses in a single batch.
2-3
-------
Both the MDL concept and the specific definition at part 136 have been used within EPA by the
Office of Ground Water and Drinking Water (OGWDW), the Office of Solid Waste (OSW), the Office
of Emergency and Remedial Response (OERR), and others. The MDL also has been used outside of
EPA in Standard Methods for the Examination of Water and Wastewater, published jointly by the
American Public Health Association (APHA), the American Water Works Association (AWWA), and
the Water Environment Federation (WEF), and in methods published by the ASTM International, and
elsewhere.
Some members of the regulated industry and others have criticized the MDL because:
• There are some inconsistencies between the definition and the procedure
• It does not account explicitly for false negatives
• It does not always yield a 1 % false positive rate
• It does not sufficiently account for blank bias
• A prediction or tolerance limit adjustment is not provided
• It does not account for interlaboratory and temporal intralaboratory variability, and
• It allows discretion in the use of the optional iterative procedures
These issues are discussed later in this document.
2.2.2 Minimum Level of Quantitation
The minimum level of quantitation (ML) was originally proposed on December 5, 1979 (44 FR
69463) in footnotes to Table 2 of EPA Method 624 and to Tables 4 and 5 of EPA Method 625. The ML
was defined as the "level at which the entire analytical system must give recognizable mass spectra and
acceptable calibration points" (in the footnote to Table 2 in Method 624) and as the "level at which the
entire analytical system must give mass spectral confirmation " (in the footnote s to Tables 4 and 5 in
EPA Method 625).
Between 1980 and 1984, EPA also developed Methods 1624 and 1625 and promulgated these
methods along with the final versions of EPA Methods 624 and 625 on October 26, 1984 (49 FR 43234).
The definitions of the ML in the promulgated versions of EPA Methods 1624 and 1625 were the "level at
which the analytical system shall give recognizable mass spectra (background corrected) and acceptable
calibration points " (in footnote 2 to Table 2 in Method 1624) and as the "level at which the entire
GC/MS system must give recognizable mass spectra (background corrected) and acceptable calibration
points" (in footnotes 2 to Tables 3 and 4 in Method 1625).
As EPA developed additional methods over the next decade, the definition of the ML was
generalized to "the lowest level at which the entire analytical system must give a recognizable signal and
acceptable calibration point for the analyte" (see, e.g., Section24.2 of EPA Method 1613 at 40 CFRpart
136, Appendix A). In generating actual numerical values for MLs, the lowest calibration point was
estimated from method development studies and included in the methods, although a specific calculation
algorithm was not used. EPA methods that include the ML generally specify the number of calibration
standards to be used and the concentrations of those standards. As a result, laboratories using those
methods calibrate their analytical systems with a multi-point calibration (i.e., calibrate using a series of
standards at different concentrations over the range of the instrument) that includes a standard at the
lowest calibration point listed in the method (i.e., the ML).
2-4
-------
In response to a need to establish a compliance evaluation threshold when the water quality-
based permit limit is below the detection limit of the most sensitive analytical method published at 40
CFR part 136, EPA refined the definition of the ML in 1994 as 10 times the same standard deviation
used to calculate the MDL1. Because the MDL is commonly determined as 3.14 times the standard
deviation of seven replicate measurements, the ML was commonly calculated as 3.18 times the MDL.
(The figure of 3.18 was derived by dividing 10 by 3.14; if more than 7 replicates were used to determine
the MDL, both the MDL and the ML multipliers are adjusted accordingly, based on values from the t-
distribution.) This calculation makes the ML analogous to Currie's quantification limit and the
American Chemical Society's limit of quantitation (LOQ), which is defined as ten times the standard
deviation of replicate or low concentration measurements (MacDougall, et al., 1980 and Keith, et al.,
1983).
To simplify implementation of the ML, the definition also was expanded to state that the
calculated ML is rounded to the whole number nearest to (1, 2, or 5), times 10°, where n is an integer.
The reason for this simplification is that calibration of an analytical system at some exact number (e.g.,
6.27) is difficult and prone to error, whereas rounding to the whole number nearest to (1, 2, or 5) x 10n
provides a practicable value. The most recent definition of the ML is "the lowest level at which the
entire analytical system must give a recognizable signal and acceptable calibration point for the analyte.
It is equivalent to the concentration of the lowest calibration standard, assuming that all method-
specified sample weights, volumes, and cleanup procedures have been employed. The ML is calculated
by multiplying the MDL by 3.18 and rounding the result to the number nearest to (1, 2, or 5) x 10", where
n is an integer," and this definition was contained in the version of EPA Method 1631 that was
promulgated on June 8, 1999 (64 FR 30417) (see Section 17.8 of EPA Method 1631 Revision B).
The ML will generally be somewhat lower than Currie's quantitation limit, even when similar
sample sizes and estimation procedures are used. This is because the standard deviation used to calculate
the ML will generally be smaller than the standard deviation at the lowest concentration at which the
relative standard deviation is 10%. This is due to the fact that, in almost all cases, standard deviation is
non-decreasing with increasing concentration, e.g., it generally tends to increase as concentration
increases.
Some members of the regulated industry and others have criticized the ML because it:
• Does not account for interlaboratory and temporal intralaboratory variability, and
• Is based on a multiple of the estimated standard deviation which is assumed to be constant in the
region of detection and quantitation, rather than a fitted model as suggested by the regulated industry.
These concerns are discussed later in this document.
The refined definition of the ML first appeared in EPA's 1994 draft National Guidance for the Permitting,
Monitoring, and Enforcement of Water Quality-based Effluent Limitations Set Below Analytical Detection/
Quantitation Levels". The draft guidance was very controversial and never finalized. However, the refined
definition of the ML has remained in use for newer analytical methods.
2-5
-------
2.3 Other Detection and Quantitation Approaches
To expand somewhat on Currie (1968), standardizing the operational definitions of detection and
quantitation would benefit society by making it easier to compare and select measurement methods based
on low-level measurement capability and requirements in particular applications. Unfortunately, in spite
of agreement on general principles and definitions advanced by Currie and his supporters, consensus on
procedures that would result in comparable detection and quantitation estimates has been elusive.
Sections 2.3.1 - 2.3.3, which are by no means an exhaustive list of the various approaches advanced to
date, highlight approaches that have been most widely advanced for environmental applications.
2.3.1 EPA Approaches
Over the years, a number of detection and quantitation limit approaches have been developed,
suggested, or used by EPA in responding to differing program mandates. In part, this situation reflects
actual differences in the mandates, and in part, it reflects the fact that no concept advanced to date has
emerged as a clear 'winner' that meets all needs for all situations. Approaches that have been used or
suggested by EPA include the:
• MDL and ML (described in Sections 2.2.1 and 2.2.2)
• Instrument detection limit (IDL)
• Practical quantitation limit (PQL)
• Estimate quantitation limit (EQL)
• Contract-required detection limit (CRDL) and contract-required quantitation limit (CRQL)
Instrument Detection Limit EPA methods for analysis of metals have historically included an instrument
detection limit, or IDL. Functionally, the IDL is similar to the MDL except that the IDL includes
temporal variability (it is determined on 3 non-consecutive days) and does not include all sample
processing steps (the IDL characterizes the detection capabilities of the instrument as opposed to the
method). Because IDLs do not reflect the entire measurement process and, for the most part, have been
used only for measurement of metals, EPA did not consider the IDL as a potential alternate to the MDL
when conducting the assessment described in this Assessment Document.
Practical Quantitation Limit: The practical quantitation limit, or PQL, was established in the 1980s by
EPA's drinking water program as the lowest concentration at which reliable measurements can be made.
The PQL is defined as "the lowest concentration of an analyte that can be reliably measured within
specified limits of precision and accuracy during routine laboratory operation conditions" (52 FR
25690, July 8, 1987). The PQL is a means of integrating information on the performance of approved
analytical methods into the development of a drinking water regulation. The PQL incorporates the
following:
• Quantitation,
• Precision and bias,
• Normal operations of a laboratory, and
• The programmatic need to have a sufficient number of laboratories available to conduct compliance
monitoring analyses of drinking water samples.
EPA uses two main approaches to determine a PQL for an analyte under the Safe Drinking Water
Act (SOWA). One approach is to use the data from Water Supply (WS) studies (e.g., laboratory
performance evaluation studies conducted by the Agency as part of the certification process for drinking
2-6
-------
water laboratories). The PQL is established at the concentration at which at least 75% of the laboratories
in the study, or the subset representing EPA Regional laboratories and state laboratories, obtain results
within some predetermined percentage of the true value of the test samples (e.g., ±30%). This approach
is used in most cases when WS data are available to calculate a PQL. The WS data approach was used to
determine the PQLs for Phase V inorganic chemicals such as antimony, beryllium, cyanide, nickel and
thallium (July 17, 1992; 57 FR 31776), as well as many other contaminants regulated under the SDWA.
In the absence of WS data, the second approach that EPA uses is the multiplier method. In this
approach, the PQL is calculated by multiplying the EPA-derived MDL by a factor between 5 and 10.
The exact multiplier varies and sometimes depends on the degree of concern about the specific
contaminant (i.e., based on a human health risk assessment for consumption of drinking water).
Application of the PQL has been traditionally limited to drinking water. Furthermore, the PQL
may not be related to the lowest quantitation limit because 1) the PQL is associated with the analyte and
may have been determined irrespective of a specific analytical method (e.g., using data from a variety of
methods approved for that analyte at 40 CFR part 141), 2) the performance evaluation (PE) samples
from which it is derived contain pollutant concentrations that may be well above the true limit of
quantitation, 3) the multiplier used to calculate a PQL when PE data are not available is somewhat
dependent on concerns about risks from human exposure to contaminants in drinking water, and 4) the
resulting PQLs may be too high for purposes other than the Safe Drinking Water Act (e.g., other EPA
programs). In addition, because EPA has privatized the performance evaluation program for drinking
water laboratory certification, it is not yet clear that appropriate data will be available in the future.
Based on these facts, EPA did not conduct an assessment of the PQL for CWA applications.
In the late 1980s, EPA's Office of Solid Waste (OSW) adopted a different version of the PQL as
a quantitation limit. No procedure for establishing the limits was given; instead values were extrapolated
from the Contract Laboratory Program CRQLs (see below). Since 1994, OSW has actively removed the
term "PQL" from its revised methods, replacing it with the term "estimated quantitation limit" (EQL).
The term PQL and the original numerical values remain in a few older OSW guidance documents.
Lowest Concentration Minimum Reporting Level (LCMRL) and Minimum Reporting Level (MRL):
Recognizing the potential for improvements over the PQL approach, and mindful that confidence in
quantitation depends on measurement precision as well as accuracy, EPA's Office of Ground Water and
Drinking Water has recently developed a standardized procedure for the determination of the "Lowest
Concentration Minimum Reporting Level (LCMRL)" and a companion procedure for laboratories to
establish their ability to quantify analytes at a "Minimum Reporting Level" (MRL).
The Lowest Concentration Minimum Reporting Level (LCMRL) is defined as the lowest true
concentration for which the future recovery is predicted to fall, with high confidence (99%), between 50
and 150% recovery. A result below the LCMRL is an estimated value that does not satisfy these data
quality objectives. However, it may be appropriate to report "estimated" data (i.e., below the LCMRL),
depending upon the objectives of the study being conducted. The proposed LCMRL procedure is an
iterative process that uses results from three or more different concentrations, of at least five to seven
replicate reagent water samples at each concentration. The average recovery, standard deviation, number
of replicates, and Student's lvalue are used to calculate a prediction interval of results that takes into
account accuracy and precision at the level tested. For a concentration level to pass criteria, the
prediction interval of results must be contained within the boundaries of a predefined quality control
interval.
2-7
-------
The Agency also has developed a procedure for use in the drinking water program which permits
laboratories to confirm that they are capable of meeting a required MRL during their initial
demonstration of capability. The MRL validation procedure will involve the analysis of one set of at
least seven replicate reagent water samples spiked at the required MRL. To be validated at the MRL, the
calculated prediction interval of results must be contained within the predefined quality control interval.
The Agency anticipates using standardized LCMRL/MRL procedures to support the monitoring
required under the Safe Drinking Water Act for unregulated contaminants. Requirements for this
monitoring are expected to be proposed in the Federal Register late in 2004. This proposal will include a
full description of the LCMRL/MRL procedures.
Estimated Quantitation Limit EPA's Office of Solid Waste has defined the EQL as:
"The lowest concentration that can be reliably achieved within specified limits of
precision and accuracy during routine laboratory operating conditions. The EQL is
generally 5 to 10 times the MDL. However, it may be nominally chosen within these
guidelines to simplify data reporting. For many analytes the EQL analyte concentration
is selected as the lowest non-zero standard in the calibration curve. Sample EQLs are
highly matrix dependent. The EQLs in SW-846 are provided for guidance and may not
always be achievable. " (see SW-846, Chapter 1).
As noted in most newer SW-846 methods, the EQLs are provided for guidance and may not
always be achievable. Because the EQL is not rigorously defined and is guidance, because the EQL may
be based on the MDL, and because the EQL can be the lowest calibration point and would, therefore,
overlap the ML, EPA did not consider the EQL further in its assessment of detection and quantitation
approaches.
Contract-Required Detection and Quantitation Limits: EPA's Superfund program has adopted the use of
contractually-required limits that are based on consensus among analytical chemists about levels that can
realistically be achieved in commercial laboratories using a contractually-specified method. Laboratories
that participate in the Superfund Contract Laboratory Program (CLP) are required to demonstrate that
they can achieve the specified CRDLs and CRQLs. The CRDLs are consensus values that apply to the
analyses of metals using CLP methods. The CRQLs apply to organic analytes and are based on the
concentration of the lowest non-zero calibration standard specified in the CLP methods, in a fashion
analogous to the original derivation of the ML. Because few CWA applications involve the use of the
CLP methods, EPA did not consider the CRDL or the CRQL as viable alternatives to the MDL and ML
when conducting the assessment described in this document.
2.3.2 Industry-supported Approaches
The regulated community has demonstrated an interest in detection limit approaches since EPA
first promulgated the MDL and ML for use in CWA programs in 1984 (49 FR 43234). As part of that
rule, EPA promulgated Methods 601 through 613, 624, 625, 1624, and 1625 for organic compounds at 40
CFR part 136, Appendix A and EPA Method 200.7 for metals by inductively coupled plasma
spectrometry (ICP) at 40 CFR part 136, Appendix C. EPA also promulgated the MDL procedure at 40
CFR part 136, Appendix B. The Virginia Electric Power Company (VEPCO) brought suit against EPA,
challenging the Agency's use of the MDL in the promulgated methods. In a settlement, EPA agreed that
2-
-------
the MDL would be applicable only to the 600-series organic methods, as these methods already
contained MDL values; i.e., it would not be applicable to EPA Method 200.7. The settlement agreement
did not preclude future use of the MDL by EPA or the right of VEPCO to bring suit in such future use.
After the VEPCO settlement, the regulated community, mainly through efforts of the Electric
Power Research Institute (EPRI), remained involved in detection and quantitation approaches to be used
under EPA's CWA programs. The first approaches that industry advanced were the compliance
monitoring detection level (CMDL) and compliance monitoring quantitation level (CMQL) (Maddalone,
et al, 1993). The CMDL/CMQL were variants of EPA's MDL/ML that attempted to adjust for
inter laboratory variability.
The regulated community continued its efforts to develop alternative detection and quantitation
approaches with development of the "alternate minimum level" (AML) in the mid-1990s (Gibbons et al.,
1997). The AML is based on statistical modeling of standard deviation versus concentration, which
requires large amounts of data.
Most recently, the regulated community has funded development of the interlaboratory detection
estimate (IDE) and interlaboratory quantitation estimate (IQE). The IDE and IQE have been balloted and
approved by ASTM's Committee D-19 for water as Standard Practices D-6091 and D-6512, respectively.
These approaches take into account nearly all sources of variability to arrive at detection and quantitation
limits that are higher, on average, than the limits produced by other approaches (see Appendix C of this
Assessment Document). Because the regulated community has shifted support from the CMDL/CMQL
? Why? and the AML to the IDE and IQE, and because EPA is not aware of other organizations that
currently advocate the earlier approaches, EPA did not consider industry approaches other than the
IDE/IQE in its assessment of possible alternatives to the MDL and ML.
As with all other approaches advocated to date, the IDE and IQE have fallen short of being ideal
approaches for detection and quantitation for all organizations and applications. To date, EPA is not
aware of a demonstrated implementation of the IDE or IQE in the development of an analytical method.
Specific concerns that have been raised about the IDE and IQE are that:
• They contain an allowance for false negatives that may be inappropriate,
• The IDE and IQE are based on the use of prediction and/or tolerance intervals, which in some cases
may yield conservative (high) estimates,
• The IDE and IQE require a large amount of data in order to be able to model variability versus
concentration, including data generated in multiple laboratories, and
• The complexity and expense the statistical procedures involved in calculating an IDE and IQE could
be a barrier to innovation and method development.
These concerns are discussed in detail later in this document.
In December 2002, the Inter-Industry Analytical Group (IIAG) submitted a proposal to EPA that
recommends (1) a sensitivity test intended to "replace the MDL as a test of whether an individual
laboratory is performing adequately," and (2) an interlaboratory validation study design intended to
characterize precision and accuracy of methods used for regulatory compliance.
IIAG's proposed sensitivity test includes the provision that EPA first determine the lowest
calibration point of a method, prescribe a dilution of that calibration point as the spike level (e.g., at one-
half or two-thirds the lowest calibration point), specify a required number of replicates, and set a quality
2-9
-------
control acceptance criterion. HAG asserts that this test would provide all laboratories with a single spike
level and an "unambiguous pass or no-pass test." EPA solicited comment on approaches that might be
considered appropriate for such determinations (i.e., the lowest calibration point of a method, an
appropriate dilution, a number of replicates, and an acceptance criterion for standard deviation between
measurements of the replicates). HAG'S proposed "full range" validation study is intended to determine
precision and bias across the entire working range of an analytical method (i.e., from a blank to the upper
end of the working range) and would account for variability between laboratories. HAG recommends
that results of such a study be used to establish an interlaboratory method detection level.
At the time HAG'S submitted the sensitivity test and full range validation study, EPA did not
have the opportunity to evaluate IIAG's proposal against the criteria discussed in Chapter 4, but included
the complete text of the recommendations in the regulatory record supporting the February 2003
Assessment Document. EPA is including an assessment of this proposal in Chapter 5 of this document.
2.3.3 Approaches Advocated by the Laboratory Community and Voluntary Consensus
Standards Bodies
In 1980 (MacDougall et al., 1980) and 1983 (Keith et al., 1983), the American Chemical
Society's Committee on Environmental Improvement (CEI) advanced approaches for the Limit of
Detection (LOD) and Limit of Quantitation (LOQ). The ACS LOD is defined as the lowest
concentration level that can be determined to be statistically different from a blank. The recommended
value for the LOD is three times the standard deviation of replicate measurements of a blank or low-level
sample. The LOD is roughly equivalent to the MDL in numerical terms and conceptually equivalent to
Currie's critical value.
The ACS LOQ is defined as the level above which quantitative results may be obtained with a
specified degree of confidence. The recommended value for the LOQ is 10 times the standard deviation
of replicate measurements of blanks or low-level samples. Because the LOD and LOQ are still used by
the analytical community, they have been included in EPA's reassessment of detection and quantitation
approaches.
In the mid-1980s, the ACS CEI introduced the concept of the Reliable Detection Limit (RDL)
and the Reliable Quantitation Limit (RQL). The RDL and RQL were attempts at simplification of the
LOD and LOQ. Both the RDL and the RQL involved applying a multiplier to the standard deviation
derived from replicate measurements of a low-level sample. Neither concept received acceptance by the
analytical community. Because the RDL and RQL are no longer being advanced by ACS, they were not
considered for evaluation in EPA's assessment of detection and quantitation approaches.
In 1999 (Currie, 1999a and 1999b), IUPAC and ISO reached substantial agreement on the
terminology and approaches documented by Currie (1995), although "subtle differences in perspective"
of the organizations remain (Currie, 2000). IUPAC and ISO have not, to date, published methods that
include limits reflecting these standards. Similarly, although ASTM International adopted the IDE in
1997 and the IQE in 2000, ASTM International has not included any IDE or IQE values in methods
approved through the ASTM ballot process. On the other hand, ISO and ASTM International have
published methods that employ the MDL. Because IUPAC and ISO have approved the critical value,
detection limit, and quantification limit, and because ASTM International has approved through ballot
the IDE and IQE, EPA has included these approaches in its assessment of detection and quantitation
approaches.
2-10
-------
At the ACS Annual Meeting held in August, 2002, CEI members discussed the issue of detection
and quantitation, with the objective of determining if the LOD and LOQ approaches should be re-visited.
At that meeting, several members suggested that the committee consider adopting a sample-specific
detection limit approach in which the ratio of instrument signal to background noise is used to estimate a
detection limit for each analyte in each sample analyzed. EPA did not include the signal-to-noise ratio
concept in this assessment because its application is limited to specific types of measurement techniques,
such as gas chromatography/mass spectrometry. Limitations of this concept for use in general
environmental chemistry are best illustrated by the fact that it would not apply to any of the techniques
traditionally used to determine the "conventional pollutants" cited in the Clean Water Act (the only
pollutants cited by name in the Act), i.e., biochemical oxygen demand (BOD), total suspended solids
(TSS), fecal coliforms, and pH.
During the comment period on the February 2003 assessment document, the American Council
of Independent Laboratories (ACIL) submitted a procedure that was developed to address bias that may
arise in the estimation of detection limits under certain conditions. The ACIL procedure separates
estimation of a detection limit into two cases: analyses that always produce a numeric result, even so-
called "blank" samples (i.e., zero analyte added), and analyses that do not always produce a numeric
result, i.e. blank samples appear to produce no signal. We will call these Case I and Case II. Analysis of
samples for metals with inductively coupled plasma optical emission spectroscopy (ICP-OES) is an
example of ACIL Case I; analysis of samples for PCBs with gas chromatography and mass spectrometry
(GC/MS) is an example of ACIL Case II.
For Case I analyses, ACIL suggests making use of the numeric results obtained from the analysis
of blank samples which laboratories routinely run as a quality control measure. ACIL provided a
detailed set of instructions for conducting the analyses and doing the MDL calculation. Differences
between the ACIL Case I calculation and the EPA MDL calculation include: (1) use of blanks rather than
low-level spikes to estimate standard deviation, (2) the calculation of both a critical level and a long-term
MDL, where the MDL is based on adding the mean of the blank results to 2 times the product of the
standard deviation and t-statistic, (3) a bias offset correction that adds the mean of the blank results to the
calculated critical level and MDL, (4) recommends the use of results from a minimum of 20 analyses,
and (5) analysis over the course of a year from routine daily operations (rather than on one day). ACIL's
Case I procedure is similar, but not identical, to the USGS procedure that is described in Section 3.3.4 of
this document. The ACIL Case I procedure has no explicit limits on the amount of contamination
allowed in the blanks before a laboratory is considered to be "out of spec."
For Case II, blanks cannot be used to estimate the standard deviation because they do not provide
a response. Thus, Case 2 recommends an iteration of multiple low level spikes somewhat similar to the
requirements in the EPA MDL procedure. However, the calculation of an MDL from the results of these
spiking experiments differs significantly from the EPA MDL procedure. The procedure also specifies a
sensitivity check for which some of the details are not as explicit compared to the Case I part of the ACIL
detection limit procedure.
2.3.4 Approaches Advocated by Other U. S. Government Agencies and Other Governments
Within the U.S., EPA found that other Federal agencies tend to rely on the detection and
quantitation limit approaches described above or on variants of those procedures. For example, the USGS
National Water Quality Laboratory (NWQL) began using the EPA MDL procedure in 1992. USGS
2-11
-------
NWQL has since developed a variant of the MDL called the long-term MDL (LT-MDL) that has been in
routine use since 1999. The LT-MDL determination ideally employs at least 24 spiked samples prepared
and analyzed by multiple analysts on multiple instruments over a 6- to 12-month period at a frequency of
about two samples per month.
Unlike EPA programs that rely on hundreds of commercial, Federal, State, and local laboratories
for sample analysis, the samples collected for USGS water programs are analyzed by the USGS National
Water Quality Laboratory in Denver, Colorado. As described by USGS, the long-term MDL is based on
many of the same fundamental assumptions as the MDL, namely:
1. Normal data distribution,
2. Constant standard deviation from the spike concentration down to zero, and
3. Best-case detection condition (because LT-MDLs typically are determined by spiking the analyte
in a clean matrix, e.g., reagent water).
The three primary differences between the EPA MDL and the USGS LT-MDL procedures are
the (1) larger minimum number (24) of spike samples, (2) longer time period, and (3) mixing of
instruments and analysts in the determination of the LT-MDL. Because the MDL and LT-MDL
approaches otherwise are so similar, EPA did not evaluate the long-term MDL approach in the February
2003 assessment. Instead, EPA considered the underlying differences between the two approaches
(namely the effects of temporal, instrument, and analyst variability) in its assessment of issues (see
Chapter 3).
In the LT-MDL procedure the low-level spike used for each analyte and instrument is
recalibrated at least once a year or when an anomaly occurs. USGS has enhanced the LT-MDL
procedure by using their large volume of uncensored blind laboratory blank data, which also is collected
yearly, as a reality-check on the spike-based LT-MDL. In cases where the standard deviation used to
calculate an LT-MDL based on blind blank data is significantly different (especially when greater) than
the standard deviation used to calculate the spike-based LT-MDL, the blank data are used to calculate the
LT-MDL. Blind blank data also are used to evaluate whether the calculated LT-MDL requires an off-set
correction for blank bias, i.e, LT-MDL = (s * Student t) + median or mean blank concentration. This
offset is similar, but not identical, to the ACIL Case I procedure described in Sect. 2.3.3 of this
document. The LT-MDL offset correction compensates for a blank distribution that is not centered on
zero (as assumed by the EPA MDL formula).
The NWQL has found that this blank bias correction to the LT-MDL is especially important for
blank-limited analytes, including some metals, total organic carbon, phenol, and nutrients. The NWQL
also uses a data reporting convention that incorporates a higher reporting level (called the laboratory
reporting level; LRL) that is set at two or more times the LT-MDL. However, this convention also
includes reporting of data between the LT-MDL and LRL.
Outside the U.S., EPA found that the European Union (EU) relies on the terminology and
conventions developed by Currie, IUPAC, and others (Eurachem, 2000). The EU advocates reporting all
results along with an estimate of the uncertainty associated with each value. In its discussion of the
issue, the EU indicates that use of the term 'limit of detection' only implies a level at which detection
becomes problematic and is not associated with any specific definition. Instead, the EU focuses its
attention on ways to estimate uncertainty, basing its approach on the ISO Guide to the Expression of
Uncertainty in Measurement (1993). However, the EU also notes that the use of uncertainty estimates in
2-12
-------
compliance statements and the expression and use of uncertainty at low levels may require additional
guidance. The United Kingdom's Valid Analytical Measurement Programme (VAM) has adopted a
similar approach that is based on both the ISO and the Eurachem guidance (Barwick and Ellison, 2000).
Because these approaches are focused on estimating uncertainty rather than at establishing or defining
limits for detection and quantitation, EPA did not consider the European approaches in this assessment.
2-13
-------
Chapter 3
Issues Pertaining to Detection and Quantitation
As part of the Settlement Agreement concerning EPA's reassessment of detection and quantitation
limit approaches, EPA considered several specific issues pertaining to these approaches. These issues
included:
Criteria for selection and appropriate use of statistical models,
• Methodology for parameter estimation,
Statistical tolerance and prediction,
• Criteria for design of detection and quantitation studies, including selection of concentration levels
("spiking levels"),
Interlaboratory variability, and
• Incorporation of elements of probability design.
In developing the plan for conducting this assessment, EPA identified other issues that should be
considered. With the exception of the first issue, these issues are discussed in this chapter and include:
Concepts of the lower limit of measurement (discussed in chapter 2),
• The need for approaches that can support CWA programs, including:
method performance verification at a laboratory,
method development and promulgation,
National Pollutant Discharge Elimination System (NPDES) applications,
non-regulatory studies and monitoring,
descriptive versus prescriptive uses of lower limits to measurement, and
use of a pair of related detection and quantitation procedures in all OW applications
Censoring of measurement results,
• Sources of variability (including, but not limited to interlaboratory variability),
False positives and false negatives,
• Measurement quality over the life of a method,
Matrix effects,
• Background contamination,
Outliers,
• Instrument non-response,
Accepting the procedures of voluntary consensus standards bodies (VCSBs),
• National versus local standards for measurement,
Ease of use (i.e., ability of study managers, bench chemists, and statisticians to do what is required by
a detection or quantitation limit procedure),
Cost to implement the procedures, and
• Laboratory-specific applications.
These issues are organized into three subsections that follow. Section 3.1 discusses the issues
that are primarily driven by analytical chemistry concerns, Section 3.2 discusses the issues that are
3 - 1
-------
primarily driven by CWA regulatory considerations, and Section 3.3 discusses issues that are primarily
driven by statistical concerns. Table 3-1, at the end of this chapter, provides a summary of the issues
discussed in Sections 3.1 - 3.3.
3.1 Analytical Chemistry Approaches to Detection and Quantitation
This section explains the key analytical chemistry issues involved in the development of detection
and quantitation limits. These include: (1) nonzero sample blanks, (2) instrument censoring, (3) matrix
effects, (4) analyte recovery, and (5) temporal variability of the measurement system.
3.1.1 Nonzero Sample Blanks
Analytical chemists rarely state that a sample contains zero concentration of a substance of
interest. Even when the sample is created in a laboratory for the purpose of containing as little substance
of interest as possible (a blank), analytical chemists recognize that some small residual amount of the
substance may be present and contribute to the measurement result. The inability of a laboratory to
reduce the concentration of a substance in the blank is often the limiting factor in attempts to make
measurements at ever lower levels.
A classic example of this potential problem was published by Patterson in the late 1960s and
1970s (e.g., Patterson and Settle, 1976). Patterson demonstrated that the majority of concentrations of
lead reported in the literature for such diverse matrices as urban dust, open ocean waters, and biological
tissues were in error by several orders of magnitude. The source of the "gross positive errors" (or
"positive bias" from blanks) was contamination introduced during sample collection, handling, and
analysis. Interlaboratory studies of the day designed to determine consensus values for reference
materials were, in fact, determining the consensus values for background contamination across
laboratories. Patterson recognized the value in running blank samples (samples thought not to contain the
substance of interest) to demonstrate that the sample collection, handling, and analysis processes were not
introducing contamination. Patterson subsequently developed the techniques for "evaluating and
controlling the extent and sources of industrial lead contamination introduced during sample collecting,
handling, and analysis" that form the basis of the "clean techniques" used for metals analysis today, and
that are incorporated in several EPA analytical methods, including EPA Method 1631 for measurement of
trace-level mercury.
The most common analytes for which contamination problems are encountered in environmental
measurements are metals, primarily zinc because of its ubiquity in the environment. With the exception
of some volatile organic compounds, such as methylene chloride and acetone, that are used as solvents in
laboratories, contamination in the measurement of organic compounds is less of a problem than
contamination of samples for metals analyses. Therefore, for determination of metals, a blank is usually
included or compensated in the calibration whereas, for organics, except for the solvents, the
concentration in the blank is generally assumed to be zero and there is no compensation of the calibration.
3 -2
-------
Measurement methods designed to determine substances at very low concentrations may include
requirements for the preparation and analysis of a variety of blanks that are designed to identify the extent
and the sources of contamination. Analysts understand that "blank" does not necessarily mean zero
concentration, and through careful control and evaluation, it is possible to make measurements for which
the blank contribution is sufficiently small to be considered negligible.
In the February 2003 version of this document, EPA noted that useful detection and quantitation
limit approaches should address the potential contribution of the blank through both the design of the
study that generates the detection and quantitation limit estimates and the evaluation of the study results.
Stakeholders commenting on EPA's 2003 assessment of these approaches added that, for many blank
analyses, there is a measurable response (blank bias) that can be attributed to reagents, sample vessels,
and other contamination sources, and that the MDL procedure failed to take these blank responses into
account. Several commenters suggested that the mean of the blank results should be added to the formula
used to calculate the MDL. The American Council of Independent Laboratories (ACIL) submitted a
procedure to use blanks rather than spikes to estimate a detection limit. The USGS submitted a long term
MDL procedure that uses either the mean or median of blank results as a lower bound reality check on the
MDL whenever an MDL computed from the low level spiking experiments is sufficiently less than the
blank results.
Following a careful evaluation of these comments and further consideration of this issue, EPA
recognizes that, under certain conditions, it may be appropriate to account for blanks in establishing
detection and quantitation limits with certain limitations. For example, a procedure to handle blanks
should account for negative results, and should limit and control sample and laboratory contamination.
Negative blanks are possible and can be caused by a blank-subtracted calibration in which the result for
the calibration blank is greater than the results for the blanks used to establish the MDL. If such negative
blanks were to result in a negative mean blank, adding the mean blank result to the formula could result in
an unattainably low MDL. Conversely to eliminate unnecessarily high MDLs, laboratories also would
need to ensure that the results of blank samples are not excessive. The laboratory would need to use
"clean" and other techniques to control contamination to the lowest possible levels and/or use a second or
higher order calibration function to preclude high results for a calibration blank from exerting undue
influence on the sample results. In addition to working out some of the details of necessary bounds on
blank correction and contamination, differences between the procedures submitted by USGS and ACIL
need to be evaluated.
3.1.2 Analytical Instrument Thresholds: Data Censoring
Certain analytical instruments typically employ "thresholds" to eliminate spurious or background
signals so that analysts can be relieved of the burden of removing or compensating these small signals.
As a result, the instrument itself may return a response of zero to a blank (a "non-response"). As an
example, gas chromatograph/mass spectrometer (GC/MS) instruments often contain thresholds below
which no instrument signal is reported. With no instrument signal reported, no measurement result can
3 -3
-------
be reported, and the instrument will report zero to indicate the lack of a signal. To understand how
instrument thresholds are used, it may be helpful to think of background static heard on a citizen-band
(CB) radio or a walkie-talkie. The static is present, but it has no meaning. Turning the "squelch" knob to
filter out the static also may make it impossible to hear the caller. In the context of detection, increasing
the instrument threshold may cause the instrument to miss a substance of interest at a low level.
In 1997, EPA conducted a study of 82 semivolatile (acid and base/neutral) organic compounds
measured by EPA Method 1625 in order to observe the performance of a GC/MS instrument both with
and without application of an instrument threshold (sometimes known as the "Episode 6184 study"); see
Chapter 1, Section 1.3.2.3). In the study, solutions at up to 17 concentration levels were analyzed with
the threshold on (i.e., low-level signals are automatically eliminated) and with the threshold off (i.e., there
is no suppression of signals). Samples were analyzed at decreasing concentrations, including a blank
concentration level, with triplicate determinations at each concentration. With the threshold turned on, all
of the measurements made on the blank were reported as zero. This is not surprising, given the purpose
of the instrument threshold. Without the threshold off, only 27 of 230 measurements on the blank (11%)
were reported as zero, and no negative results were reported.
Instrument thresholds have a direct and indirect impact on estimating detection and quantitation
limits. The main direct impact is that it is not possible to estimate the standard deviation of measurements
at zero. However, by definition, the standard deviation at zero is required to calculate the Currie critical
value (CRV; Lc). The EPA MDL procedure was constructed to deal with this problem by providing a
way to estimate a standard deviation at a low concentration, and including instructions for determination
of a concentration as close to zero as is possible that will generate a measurement.
To calculate an MDL using the 40 CFR 136, Appendix B, MDL procedure, it is necessary to find
the lowest concentration at which the analytical system will return results. Many laboratories have run
repeated measurements in order to find this concentration. The challenge of finding this lowest
concentration manifested itself in EPA's variability versus concentration (Episode 6000) studies.
Technologies for determination of organic, conventional pollutants, and metal analytes were evaluated in
the Episode 6000 studies. The MDL procedure suggests iteration until the calculated MDL is within a
factor of five of the spike level. For the Episode 6000 studies, EPA instructed laboratories to use a factor
of three instead of five in an attempt to more narrowly define the lowest spike level at which
measurements could be made. This change to a factor of three also was suggested by one of the peer
reviewers charged with evaluating EPA's 2003 assessment of detection and quantitation limits, who
noted:
"However, the use of as much as five times the critical level for the spike concentrations
could be problematic. The inflation of the MDL by using a spike at the critical level is
only 25% for a method with a high-level CVof20% (this and other calculations here are
done with the Rocke andLorenzato 1995 variance function assuming a sample size of 7).
A spike concentration of 3 times the critical level inflates the MDL to a value 140%
higher, which even there may be tolerable. Use of a value 5 times the critical level gives
an inflation of over 280%. ..."
3 -4
-------
Following some theoretical example calculations that are not reproduced here, the peer reviewer's
comment continues with:
"Thus, I would recommend that the procedure be altered to use concentrations that are
no more than 3 times the detection limit, and perhaps to permit concentrations lower then
the critical level, including possibly blanks" (Rocke, 2002).
The reviewer's calculations suggest that the MDL may be strongly inflated for a spike level of
five times the MDL, but only moderately inflated at a spike level of three times the MDL. However,
during the Episode 6000 studies, several laboratories asked for relief from the factor of three requirement
because a large number of iterations were required to meet it. In response, EPA reinstated the factor of
five for these laboratories. If the reviewer's example calculations are correct and a practical procedure for
determining the MDL using the factor of three were implemented, it could exacerbate the concern from
the regulated community that MDL values are too low.
Several stakeholders commenting on EPA's 2003 assessment suggested that approaches to
detection and quantitation should address methods that do not always produce an instrument response, e.g.
so-called blanks never produce a response because of electronic censoring by the instrument, and that
EPA's approaches do not do so. These stakeholders prefer that the MDL not be applied to methods for
which an identifiable analyte signal cannot be established using method blanks, where pattern recognition
is required (e.g., Method 608 for PCBs as Aroclors), or where the method requires more than one signal
for an analyte to be positively identified (e.g., the use of multiple ions in GC/MS methods). EPA
recognizes that additional guidance needs to be developed for these methods. One commenter, the
American Council of Independent Laboratories, submitted a draft set of procedures designed that partially
addressed methods that do not produce an instrument response at zero concentration. EPA evaluated the
ACIL procedure, which involves a complex set of iterative spiking experiments, and found that it needs
further refinement. EPA agrees that this issue warrants further examination.
3.1.3 Matrix Effects
"Sample matrix" is a term used to describe all of the substances, other than the substance(s) of
interest, present in an environmental sample. In the case of a wastewater sample, this would include the
water itself, as well as any other dissolved or suspended materials.
"Matrix effect" is a term used to describe a situation in which a substance or combination of
substances in the sample (other than the substance [s] of interest) influence the results of the measurement.
Positive interferences may inflate the results for the substance or make it difficult to distinguish one
substance from another. However, unless the positive bias from the matrix is consistent and predictable,
the measurement result may be unreliable. Negative interferences may suppress the results for the
substance to the point that the results cannot be distinguished from background instrument noise.
3 -5
-------
In some cases, finding a matrix effect indicates that the analyst should select a more appropriate
method. For example, a colorimetric method for the measurement of sulfide may be a poor choice for the
analysis of a sample that is very cloudy or darkly colored. In other cases, characteristics of the sample
such as its pH may destroy the substance of interest, effectively preventing analysis for that substance.
Nearly all of the newer analytical methods approved at 40 CFR part 136 describe the preparation
and analysis of quality control samples that are designed to indicate the presence of matrix effects (e.g.,
matrix spike and/or matrix spike duplicate samples). Many of these methods also contain techniques for
addressing matrix effects. Further, EPA has developed guidance documents that amplify the discussions
in those methods (e.g., Guidance on Evaluation, Resolution, and Documentation of Analytical Problems
Associated with Compliance Monitoring, June 1993, EPA 821-B-93-001). For determination of mercury
by EPA Method 1631 that is the subject of the Settlement Agreement, additional guidance on resolving
matrix interferences to achieve specified detection and quantitation limits is provided in EPA's Guidance
for Implementation and Use of EPA Method 1631 for the Determination of Low-Level Mercury (March
2001, EPA 821-R-01-023). Following the techniques in the methods and guidance will usually reduce
adverse effects of the sample matrix on detection/quantitation limits and measurement results.
3.1.3.1 Allowance for Matrix Effects in Detection and Quantitation Limits
Some stakeholders have suggested that detection and quantitation limits should be determined in
"real-world" matrices, rather than in reference matrices intended to simulate method performance in a
particular medium (water, soil, biosolids, tissue). Some commenters on EPA's 2003 assessment believe
that any form of a detection limit study should require demonstration of the lowest level of detection
achievable in an interference-free matrix and should relate this to what is actually detectable in a highly
complex matrix. One commenter stated that EPA should provide an objective set of procedures that a
permittee can follow to avoid liability when faced with an MDL or ML it legitimately cannot achieve
because of the unique nature of its effluent. EPA notes that permittee liability was not a goal or purpose of
our assessment of detection and quantitation approaches and issues. Although EPA recognizes stakeholder
concerns about the matrix effects, there are several problems associated with the approach suggested by
some commenters. These problems include:
• Many "real-world" matrices contain the target pollutant at levels well above the detection or
quantitation levels, making it impossible to characterize what can and cannot be detected at low
levels. Diluting the sample to dilute the target pollutant concentration is an option. However, this
also has the potential to dilute any interferences that might be present, thereby defeating the
purpose of using the real-world matrix.
• Use of a reference matrix to establish detection and quantitation limits allows the results to be
reproduced (i.e., confirmed) by an independent party; such a confirmation may not be possible
with many real world matrices that may be subject to seasonal, diurnal, or other types of
variability.
3 -6
-------
Few environmental analyses are conducted on actual samples of reagent water or other reference
matrices and there may be matrix-specific limitations to the sensitivity of any given analytical
method. From a practical standpoint, it would be very impractical to evaluate method sensitivity
in every possible matrix to which a method might be applied, or to establish a subset of all
possible matrices that would satisfy the concerns of every regulated discharger.
The cost of determining detection and quantitation limits in every possible matrix would be
prohibitive.
Because of these difficulties a reference matrix (or reference matrices) is an appropriate and
practical first choice to establish detection and quantitation limits. And the procedures for defining these
limits should allow for evaluation of data collected in the specific matrices of concern. Laboratories or
data users are most able to determine which matrices might be considered to be "highly complex" based on
the matrices that are typically analyzed in a given laboratory. EPA's detection and quantitation procedures
do not preclude laboratories from determining MDLs in matrices other than reagent or "blank" matrices,
and the Agency encourages laboratories and others to determine matrix-specific MDLs when all efforts to
resolve matrix interferences have been exhausted. The existing procedure at 40 CFR 136, Appendix B,
includes a discussion regarding determination of matrix-specific MDLs for this reason. Laboratories
usually are very capable of eliminating or compensating matrix interferences if tasked to do so. However,
given the degree of concern about this issue it is appropriate for all parties to continue to search for
additional solutions to the "real world" matrices issue.
3.1.3.2 Repository of Reference Matrices
Two of the four peer reviewers charged with evaluating EPA's assessment of detection and
quantitation limit approaches suggested that EPA create a repository of reference matrices, similar to those
developed by NIST, and that these reference matrices be used to challenge a test method and to establish
detection and quantitation limits (Cooke, 2002 and Wait, 2002). EPA has considered such a repository
from time to time and again in response to this suggestion, but has been unable to resolve all of the issues
surrounding such a repository. Some of these issues are:
• The stability of aqueous samples,
• The holding times necessary to assure stability,
• The argument that no matrix from a given industrial discharge in an industrial category or subcategory
reflects the characteristics of another discharge in that or other industrial categories or subcategories,
• The cost of maintaining such a repository, and
• The potential conflict with NIST and with non-governmental organizations that provide reference
matrices.
3 -7
-------
Given these issues, it is appropriate to leave the development and maintenance of standard
reference materials (SRMs) and certified reference materials (CRMs) to NIST and the commercial
marketplace. These reference materials are a useful means of challenging a test method and EPA has
suggested in recent methods that reference matrices be analyzed, when available, as an additional QC
measure. For example, when EPA developed an appendix to Method 1631 for application to matrices
other than water, EPA specified use of a quality control sample (QCS) with the statement that "many
certified reference materials (CRMs) are available for total mercury in plants, animals, fish, sediments,
soils, and sludge" and the requirement that "recovery and precision for at least one QCS per batch of
samples must meet the performance specifications provided by the supplier."
Although SRMs and CRMs could be useful in establishing detection and quantitation limits,
practical considerations are likely to preclude their use for this purpose in most situations. This is because
the materials would need to contain the analytes of interest at levels that are near the detection limit (e.g.,
within 1 to 5 times the concentration of a determined MDL). Such concentrations are unlikely to occur in
an SRM produced by NIST or a CRM produced by a vendor, and diluting the CRM/SRM would diminish
matrix effects, as indicated in Section 3.1.3.1.
As an alternative to using standard reference materials, EPA commonly tests its analytical methods
on a variety of real-world matrices, and allows for this variability in the QC acceptance criteria for the
matrix spike (MS) and matrix spike duplicate (MSB) samples. For example, EPA published performance
data in Table 3 of EPA Method 163 IB for reagent water, fresh water, unfiltered and filtered marine water,
and unfiltered and filtered secondary effluent, and allowed for the variability among these matrices in the
QC acceptance criteria for the MS/MSD in the method. ASTM Committee D 19 allows this approach in
development of QC acceptance criteria for methods (see Section 6.5.1.1 of ASTM D 5847: Standard
Practice for Writing Quality Control Specifications for Standard Test Methods for Water Analysis)
3.1.4 Recovery of Analytes from the Sample Matrix
In the preceding two sections, we discussed bias (errors) from blank contamination and matrix
effects. Errors from recovery effects ("recovery bias") are discussed in this section. Recoveries are a
measure of the amount (usually expressed as a percentage) of analyte that is recovered from the sample
matrix and measured by the analytical system. Chemists sometimes use the phrase "accuracy of the
method" when listing the percent recovery of an analyte. A goal of analytical chemistry is to achieve
recoveries as close as possible to 100%. When this is not achieved, recovery correction may be used. The
purpose of recovery correction is to adjust the measured concentration for the amount by which the
measured concentration differs from the true concentration (if known). Recovery "factors" are initially
determined by analysis of a sample containing a known (spiked) amount of the analyte. These factors are
applied to measurements of samples with an unknown amount of the analyte in the same or a similar
matrix. To illustrate the potential need for recovery correction, consider analytes, such as organic bases
3 -8
-------
(e.g., benzidine) and acids (e.g., phenols) in a water sample, that are either not totally (100%) recovered in
the extraction process, or are adsorbed on the surface of a GC column at very low (nanogram) levels. As a
result, the measured concentration of these analytes is always less than the true concentration in the water
sample. These incomplete recoveries have led some developers of detection and quantitation limit
approaches to believe that these limits should be recovery corrected (i.e., that the detection or quantitation
limit should be adjusted inversely proportional to the recovery). For example, if an analyte is recovered at
50%, the detection and/or quantitation limit should be doubled, and the amounts measured in unknown
samples also should be doubled to allow for recovery correction.
Several stakeholders have stated that understanding this "recovery bias" is particularly important
when reporting results near the limit of detection, and is critical when reporting quantifiable results. These
stakeholders believe that even if recovery bias is not controlled at the detection level, the approach to
determining detection and quantitation limits should compensate for it.
Few of the traditional approaches to establishing detection and quantitation limits include
procedures for recovery correction. For example, the issue was not addressed by Currie in his original
proposal of a critical value or quantitation limit. Similarly, neither EPA's MDL and ML nor the American
Chemical Society's LOD and LOQ, all of which are based on the approaches advanced by Currie, include
a mechanism for recovery correction. When Currie introduced his critical value, he defined it as "the
minimum significant value of an estimated net signal or concentration, applied as a discriminator against
background noise" (Currie, 1995). Because the critical value is defined as a measured concentration rather
than a true concentration, a recovery correction is not included.
The use of recovery correction, however, has been included in several of the most recently
developed approaches for detection and quantitation. For example, the minimum detectable value (MDV)
adopted by ISO and IUPAC, and the interlaboratory detection estimate (IDE) and interlaboratory
quantitation estimate (IQE) adopted by ASTM include procedures for recovery correction. The IQE also
contains a further correction that we have termed a "bias" correction.
In the ISO minimum detectable value approach, recovery is treated as a linear function versus
concentration, and an extrapolation is used to estimate the recovery at zero concentration. This projection
of the regression line to zero concentration can lead to errors because, depending on the intercept (in
concentration units), the recovery at zero concentration can be positive, zero, or negative, resulting in an
inflated minimum detectable value, an minimum detectable value very close to zero, or a negative
minimum detectable value. For further details, see the section titled "Negative detection limits for the
ISO/IUPAC MDV" in Appendix C to this Assessment Document, and the data in Table 2 of that appendix.
3 -9
-------
The IDE and IQE fit recovery versus concentration in a way analogous to the fitting in the
minimum detectable value. The difference is that an unweighted model is used in the minimum detectable
value, whereas the linear model in the IDE and IQE is weighted as determined by the model of standard
deviation versus concentration that is used in calculating the IDE and IQE. (If this model is the constant
model, the weighting is the same as for the minimum detectable value.) The IQE, but not the IDE,
includes an additional correction for the bias associated with an estimate of the true standard deviation at
each concentration as compared to the measured standard deviation at each concentration. In this context
(a "bias" correction to the IQE), "bias" means the amount by which the estimated sample standard
deviation differs from the true population standard deviation. This use should not be confused with a
common use of "bias" in analytical chemistry measurements to denote the deviation of a result from the
true value (usually expressed as percent.)
Recovery correction may be appropriate if (1) when developing method detection and quantitation
limits, the recovery is consistent across laboratories, matrices, and conditions, and (2) the relative
variability (as relative standard deviation) remains constant as the recovery decreases. These two
conditions are rarely observed. The first requirement (consistent recovery) would need to be tested under a
variety of conditions because, if the recovery varies among laboratories, matrices, and analytical
conditions, then a detection and/or quantitation limit would need to be developed for each of these
conditions. EPA's experience is that poor recovery is rarely consistent; i.e., if one laboratory measures a
recovery of 40%, another laboratory may measure 20%, or 60%, but not exactly 40%.
Although some stakeholders disagree, EPA believes that the normal condition in environmental
analytical measurements is that variability (as standard deviation) between sample results remains
approximately constant as the recovery decreases (i.e., the relative precision [as RSD] is poorer at low
recovery). For example, if the RSD is 10% at 100% recovery, the RSD may be 50% at 50% recovery, and
may be 100% at 10% recovery. For examples of the effect of poor recovery on precision, see the quality
control (QC) acceptance criteria for the semivolatile organic compounds in Table 8 of EPA Method 1625
(see 40 CFR part 136, Appendix A). This increase in relative variability is not the result of measurements
being made at lower levels, as is the normal case, but is a result of variability in the extraction
(partitioning) process. One stakeholder commenting on EPA's 2003 assessment stated that EPA's
statement that variability remains approximately constant as recovery decreases may not hold true in all
cases, and recommends that, if recovery falls within a specified level (e.g., less than 70%), detection limits
should be adjusted accordingly. EPA acknowledges that there may be instances in which this general
condition does not hold true.
EPA has traditionally viewed recovery correction with great caution, and has preferred to require
that laboratories analyze quality control samples to demonstrate that analytes are recovered within an
acceptable level. For example, EPA's Office of Water methods require that laboratories prepare and
analyze both a reference matrix and a sample matrix that have been spiked with the analytes of interest,
and that these analytes be recovered within method-specified acceptance criteria. If the recovery criteria
are met, then samples analyzed in the batch are considered to be reliable within the overall level of error
associated with the method, and results are reported without correcting for the recovery. Measurements of
dioxins/furans, PCBs, and pesticides can be made to very low (femtograms per liter) concentrations, with
no decrease in recovery compared to recoveries observed at much, much greater (microgram per liter)
concentrations. (One microgram is equivalent to one million femtograms). The ability to measure
dioxins/furans, PCBs or pesticides down to these low concentrations demonstrates that recoveries for these
compounds do not decrease with decreasing concentration. There also are chemicals, such as the
nitrophenols and benzidine, that are not recovered reliably at sub microgram per liter levels. But these
instances are known and recognized in the instructions for conducting these measurements.
3 - 10
-------
MDLs are established and listed in methods based on the determined (measured) concentration
(not the spike concentration), and laboratories and others that are required to verify MDLs, verify based on
the determined concentration. If EPA estimated and listed MDLs in methods based on the spike (true)
concentration, logic would require that the true (recovery corrected) concentration be used for regulatory
compliance with the result that all results, not just the MDL, would be greater than nonrecovery corrected
results.
Recovery-correction techniques are employed in some Agency methods. Most notably are those
methods that employ isotope dilution techniques, in which a stable, isotopically labeled analog of each
target analyte is spiked into each sample. Because of their structural similarity to the analytes of interest,
the labeled analogs are assumed to behave exactly like their unlabeled analogs (the target analytes).
Because the recovery of the labeled analog will be similar to that of the target analyte, the technique allows
for recovery correction of each target analyte and is particularly useful in highly complex matrices. In
these methods, recovery correction techniques are specified as part of the procedures for calculating and
reporting results and are dependent on the one-to-one relationship of the target analyte and the labeled
analog. Inclusion of an additional procedure for recovery-correction in determining detection or
quantitation limits for such methods could result in double-counting of analytical bias.
Another procedure for dealing with bias (errors) from blank contamination, matrix effects, or
errors from recovery effects ("recovery bias"), is to assure that the detection or quantitation limits which is
determined meet the data users data quality needs for both precision and accuracy, without any correction.
As described previously in chapter 2, EPA's drinking water program is developing an approach to setting
quantitation levels called the minimum reporting level or MRL. The MRL addresses these issues by
setting a data quality objective for minimum and maximum permitted inaccuracy arising from these
effects.
The issue of bias in determination of detection or quantitation limits be it from blanks, matrices or
other than 100% recovery of an analyte is a longstanding issue. All parties should continue to
collaboratively work to develop other solutions or approaches to mitigate bias effects.
3 - 11
-------
3.1.5 Temporal Variability of Analytical Measurements
As with most other areas of technology, instruments continue to improve. Instrument
manufacturers and laboratories are increasing data processing power, speed of analysis, and the reduction
of chemical or electronic "noise." Any of these instrument improvements can be expected to improve the
measurement of concentrations of environmental pollutants. This process can be illustrated for a variety of
EPA methods. A case in point is EPA Method 1613 for determination of poly chlorinated
dibenzo-p-dioxins and polychlorinated dibenzofurans. Development of this method began in 1988. At the
time, commercially available high resolution mass spectrometer systems were able to achieve a detection
limit of approximately 4 pg/L and a ML of 10 pg/L. By the time that EPA proposed the method in 1991,
the Canadian government published its own version that included a quantitation limit 5 pg/L. By the time
EPA promulgated Method 1613 in 1997, many laboratories performing the analysis had replaced or
supplemented their old instruments with newer models. As a result, many laboratories performing
analyses using Method 1613 routinely measure sample results at levels 10 times lower than those analyzed
routinely only 10 years earlier.
Although there is no such thing as a "perfect" measurement, the idea that "practice makes perfect"
(i.e., analytical results get better with practice) applies to the quality of measurements made with a given
method over time. We can demonstrate this using simple techniques like laboratory control charts. The
improvements are a result of experience, as well as improvements in equipment over time. EPA expects
changes in performance when new staff are trained. For this reason, many EPA methods specify that "start
up tests" be repeated each time new staff arrive. It is not unusual to see slight increases in measurement
variability as new staff are trained followed by a decrease back to normal level after analysts become
sufficiently experienced with the analytical method. .
The use of quality control (QC) charts as a means of tracking method and laboratory performance
as a function of time is described in EPA's Handbook for Analytical Quality Control in Water and
Wastewater Laboratories (referenced in the 40 CFR part 136, Appendix A methods). Although these
charts are instructive in tracking improvement or stability, they have two significant drawbacks: (1) they
do not establish an absolute limit within which an analysis must be operated and (2) continued
improvement can lead to unusually stringent limits that, eventually, will not be met. As long as absolute
QC acceptance criteria (limits), such as those found in EPA methods, are established and as long as there is
a recognition that stringent limits may be an artifact of improvement beyond what is routinely achievable,
QC charts can be instructive in identifying statistically significant losses of, or improvements in, analyte
responses in the region of interest. ASTM Committee D 19 adopted the philosophy of establishing
absolute limits for analytical methods in approving Standard Practice D 5847.
Stakeholders commenting on EPA's 2003 assessment of procedures for characterizing the
detection and quantitation limits of analytical methods expressed concern that EPA's MDL and ML are
determined using a single batch of samples representing a "snapshot" in time, and do not account for the
temporal variability that can occur in a laboratory from day to day (e.g., due to use of multiple analysts and
instrumentation, changing laboratory conditions). Although the codified version of the MDL does not
preclude laboratories from incorporating temporal variability into the procedure (e.g., it allows the use of
more than 7 replicates and does not require that the replicates be analyzed in a single batch), many users
understand the MDL to be a single batch procedure. EPA encourages, where appropriate, use of data
gathered over an extended period of time to calculate an MDL because measurement capabilities tend to
improve and laboratory conditions tend to vary. Detection and quantitation limit calculations can be
supported by procedures that allow laboratories to affordably characterize such changes and
improvements.
3 - 12
-------
3.2 CWA Regulatory Issues Affecting Detection and Quantitation
Section 3.2.1 provides a brief overview of Clean Water Act activities that involve chemical
measurements and are, therefore, directly impacted by detection and quantitation limit approaches.
Specific issues to be considered in the context of these CWA applications and EPA's regulatory
obligations are discussed in Sections 3.2.2 - 3.2.6.
3.2.1 Detection and Quantitation Limit Applications Under CWA
The Clean Water Act directs EPA, States, and local governments to conduct a variety of data
gathering, permitting, and compliance monitoring and enforcement activities. Many of these activities
depend directly on environmental measurements and, therefore, are affected, both directly and indirectly,
by detection and quantitation limit approaches. Stakeholders commenting on EPA's assessment of
detection and quantitation procedures stated that, because of the differing technical demands and
regulatory and laboratory uses of detection and quantitation levels, the procedures for determining these
values should be based on sound science. These stakeholders urged EPA to consider the implications of
each technical decision it makes regarding determination of detection and quantitation values on the
practical implementation of its regulations.
Several commenters believe that EPA, permit holders, and laboratories would be better served if
different approaches to detection and quantitation were taken for each use. Commenters specifically cite
uses as a start-up test in a single laboratory, as a value characterizing a given analytical method, as a test
for approving a method modification or alternate test procedure (ATP), compliance monitoring, and as a
permit compliance level. The Inter Industry Analytical Group, in particular, has recommended the
following 3-part approach:
A sensitivity test (as a test of start-up proficiency),
• A long-term MDL approach ( for laboratory reporting),
Full-range validation study (such as the ASTM IDE/IQE) for validation of new methods and for
setting quantitation levels that will be used as permit compliance levels.
3.2.1.1 Method Development and Promulgation
Section 304(h) of the Clean Water Act (CWA; the "Act") requires EPA to promulgate test
procedures (analytical methods) to be used for data gathering to support certification, permitting, and
monitoring under the Act. These methods are promulgated at 40 CFR part 136, and include methods
developed by EPA as well as those developed by other organizations, such as the publishers of Standard
Methods for the Examination of Water and Wastewater, as well as AOAC-International, ASTM
International, the U.S. Geological Survey, instrument manufacturers, and others. Upon request by a
laboratory, permittee, instrument manufacturer, or other interested party, EPA also considers alternate
testing procedures (ATPs). If EPA deems these ATPs to be acceptable for use, they may be published at
40 CFR part 136. A primary objective in promulgating methods developed by EPA and by other
organizations is to provide the regulatory community, permittees, and laboratories with multiple options so
that they may choose the method that yields the best performance at the lowest cost for the application.
3 - 13
-------
In recent years, EPA has focused on developing methods for promulgation at 40 CFR part 136
where no other methods are available that meet an immediate or anticipated regulatory need. The National
Technology Transfer and Advancement Act of 1995 (NTTAA) encourages government agencies to
consider methods published by voluntary consensus standards bodies (VCSBs), such as Standard Methods
and ASTM International, when VCSB methods are available. EPA accepts that many of these methods
have been through a sufficient level of testing, peer review, and scientific acceptance to warrant proposal if
they meet EPA's regulatory needs. When an individual laboratory, permittee, or other organization
submits a request for approval of an alternate test procedure, however, EPA generally requires that the
procedure be subjected to a level of testing that demonstrates that the method provides sensitivity,
accuracy, and other measures of performance comparable to an approved method.
The lack of widespread consensus on detection limits has led organizations that develop methods
to use different approaches, and many organizations have changed approaches over the years. Some
stakeholders, who commented on the 2003 assessment, believe that method-specific detection and
quantitation limits should account for interlaboratory variability, and therefore should be based on
interlaboratory data. Other stakeholders believe that such a requirement would be overly restrictive and
burdensome, resulting in fewer approved methods and technologies. The result is that a number of
different approaches for detection and quantitation are embodied in the methods approved at 40 CFR part
136. The vast majority of the approved methods include the MDL which, as noted in Section 2.2.1, has
been used by several EPA Offices, Standard Methods, AOAC, ASTM, and others. Other approaches
embodied in the methods at 40 CFR part 136 include, but are not limited to:
1) a method "range" that is usually not defined, but is often interpreted as the lower end of the
range in which pollutants either can be identified or quantified,
2) an "instrument detection limit" that has been defined by a variety of procedures, but is intended
to capture instrument sensitivity only,
3) an "estimated detection limit" that may be based on best professional judgement, single
laboratory data, or some other source of information,
4) a "practical quantitation limit," that has typically been determined according to one of the
scenarios described in Section 2.3.1, and
5) "sensitivity" that is an undefined concept similar in result to the MDL.
A solution to this lack of consensus would be to require that all methods promulgated at 40 CFR
part 136 contain uniform approaches for detection and quantitation. However, taking such action would be
disingenuous and confound methods promulgation because:
• To date, no single detection and quantitation limit approach has emerged to meet the needs of all
organizations for all applications.
• If EPA's were to select an approach that differs from those of other organizations, those organizations
would be required to conform their method to accommodate the EPA approach. Doing so would mean
that these organizations would have to invest additional laboratory resources to develop detection and
quantitation limits that conform to OW definitions.
• If outside organizations decided against conforming their approaches to that of EPA, fewer methods
would be promulgated at 40 CFR part 136. This would result in fewer options for the regulatory,
permittee, and laboratory communities.
If EPA selected an approach that has burdensome procedures for developing detection and quantitation
limits, it could discourage development of innovative technology or method modifications.
3 - 14
-------
Given these issues and EPA's desire to encourage the development of improved measurement
techniques, and provide the stakeholder community with a variety of measurement options whenever
possible, it would be counter-productive to allow method developers the choice of only one detection or
quantitation limit approach, or to only promulgate those methods that contain this single approach.
However, there are real benefits to standardization, all new methods developed by EPA for promulgation
at 40 CFRpart 136 should reflect such standardization, and EPA should strongly encourage outside
organizations to include these standardized approaches in their methods. However; there was no clear
consensus as to what this standardized approach should be. Industry advanced IDE/IQE procedures but
others did not necessarily support them.
3.2.1.2 Verification of Laboratory Performance
Just as sensitivity is important for evaluating method performance, it is important to verify that a
laboratory using a method can achieve acceptable levels of sensitivity for making measurements. Such
demonstrations can take many forms and should be viewed in the context of the decision to be made. The
analytical methods published at 40 CFR part 136 are designed for monitoring compliance with CWA
permits. Most pollutants in permits have a numeric limit, and compliance with this limit is determined by
laboratory analysis of samples from the waste stream or water body regulated by the limit. The laboratory
that conducts such analyses must be able to demonstrate that its detection or quantitation limits are low
enough to assure reliable measurements.
Thus, even where a method describes the sensitivity measured or estimated by the developer or the
organization that published the method, some means are needed to demonstrate that a given laboratory can
achieve sufficient sensitivity to satisfy the regulatory decision (e.g., monitoring compliance).
The EPA MDL procedure provides a means for verifying laboratory performance and has long
been used in this fashion by EPA and various other Federal and State agencies as a measure of method
sensitivity. Other procedures may be employed, including analysis of reference materials containing the
analytes of interest at concentrations that are at or below the regulatory limits of interest, spiked samples
that are similarly prepared (e.g., matrix spikes), or laboratory performance evaluation samples such as
those used in laboratory accreditation studies. Several commenters on EPA's 2003 assessment
recommended that a simple "sensitivity" test (e.g., determination of analyte recovery in a sample
containing a low-spike concentration of the analyte) be used to evaluate or establish laboratory
performance. Although at least two commenters submitted some ideas for conducting such a test, none
were sufficiently or clearly detailed. However; EPA is open to consideration of approaches to verify lab
performance.
The IDE and IQE were advanced by the regulated industry and subsequently approved by ASTM
International as a means of characterizing the performance of a method in laboratories that participate in an
interlaboratory study. These approaches were developed to establish detection and quantitation limits that
could be met by any laboratory that participated in the study. However; the IDE/IQE cannot be used to
verify individual laboratory performance.
3 - 15
-------
Developers of the IDE/IQE have recognized that an analogous approach is desirable for single-
laboratory application and have begun work on a within-laboratory detection estimate (WDE), to be
followed by a within-laboratory quantitation estimate (WQE). As with the IDE/IQE, these approaches are
intended to capture a wide range of sources of variability such as temporal variability, and include a
prediction or tolerance limit (or both), but will not include interlaboratory variability. EPA would
consider such single laboratory approaches if and when they are adopted by a voluntary consensus
standards body, such as ASTM International. EPA will explore approaches for lab performance
verification through the stakeholder process.
3.2.1.3 National Pollutant Discharge Elimination System
The National Pollutant Discharge Elimination System (NPDES) serves as a means by which EPA,
States, and Tribes control point source releases of pollutants into the nation's waters. Under this system,
individual facilities are issued NPDES permits that provide effluent limitations that restrict the quantities,
discharge rates, and concentrations of pollutants that may be legally discharged. Typically, these
limitations are based on technology-based standards. If, however, these technology-based limits are not
adequate to protect the water quality standard designated for the facility's receiving water, stricter controls
are warranted. In such cases, NPDES permits must contain "water quality-based" controls.
Development and Implementation of Technology-based Controls (Effluent Guidelines)
EPA promulgates national effluent limitations guidelines and standards under the authority of
Clean Water Act Sections 301, 304, 306, 307, 308, and 501. The regulations allow the discharge of
pollutants from normal industrial processes when the discharges have been treated using various levels of
available and affordable treatment technologies. Functionally, these industry-specific guidelines establish
standards for the quality of wastewater discharges to waters of the United States. They are generally
stated in the form of concentration-based limits for selected substances that are not to be exceeded. For
example, the maximum oil concentration in wastewater separated from oil pumped out of an offshore well
and discharged on any single day shall not exceed 42 milligrams per liter (mg/L). This form is called a
numeric effluent guideline limit or numeric limit.
Development and Implementation of Water Quality-based Controls
States designate water-quality standards for various bodies of water within their boundaries. Each
standard consists of a designated use, criteria to support that designated use, and an anti-degradation
policy. Examples of designated uses include public water supply, recreation, and propagation offish and
wildlife. A discharge that causes, has reasonable potential to cause, or contribute to an excursion of an
applicable water quality standard requires a water-quality based limit. Such a water-quality based limit
shall be established at levels that derive from and comply with applicable water-quality standards and
must be consistent with the assumptions and requirements of any available waste load allocation for the
discharge, approved by EPA pursuant to 40 CFR 130.7.
A special case occurs when the water quality-based effluent limit is less than the detection limit of
the most sensitive analytical method. This case is addressed in Section 3.2.3 below, on compliance
evaluation thresholds.
3-16
-------
Permit Compliance Monitoring
Under Clean Water Act Sections 318, 402, and 405, NPDES permits are issued to owners of
facilities that discharge wastewater to waters of the United States (e.g., coastal areas, lakes, rivers,
streams, certain wetlands, etc.). Specific discharge limits are established either for individual facilities or
for classes of facilities. Individual permits are established for industries with many site-specific issues
that determine the substances discharged, such as the pharmaceutical industry in which the specific drugs
produced could influence the water quality. NPDES permits generally specify the use of measurement
methods promulgated at 40 CFR part 136 under the Clean Water Act Section 304(h) for purposes of
compliance monitoring and other reports submitted under NPDES permits.
Detection plays a role in compliance monitoring because of concerns with measurement results at
the low end of any analytical method. All measurement results are variable. At the low end of most
measurement methods, there comes a point at which a particular measurement result is unacceptably likely
(a policy decision) to have come from a sample in which the substance of interest is absent (zero
concentration). Such a measurement result would be below the critical value defined by Currie (1995)
and in common usage, would be called below detection. In practice, the reporting limit may be set equal
to a critical value, detection limit, or quantitation limit. Assuming that the reporting limit is a detection
limit of 1 mg/L of oil and grease, the measurement result would be reported as "less than 1 mg/L of oil
and grease."
Stakeholders are particularly concerned with the use of the detection and quantitation limits for
compliance purposes (e.g., judging whether a discharger is in compliance or whether a waterbody
complies with its water-quality standards), for which a high level of reporting consistency and confidence
in the data is required. Several commenters on EPA's assessment stated that procedures used to determine
these limits should provide the certainty required to make regulatory decisions.
Several commenters suggested that there should be a single compliance benchmark for detection
of each analyte that is independent of laboratory or method capabilities; laboratories used for compliance
reporting would be required to demonstrate that they can detect at or below this level. These commenters
note that such an approach would be particularly useful and appropriate for analytes with water quality (or
other) standards set below current technological capabilities.
3.2.1.4 Non-Regulatory Studies and Monitoring
EPA conducts a variety of non-regulatory studies and monitoring activities to support its Clean
Water Act programs. These activities range from long term surveys, such as the Great Lakes Water
Quality Surveys that are conducted each spring and summer to monitor trends in water quality against
established baselines, to short-term studies that are used to establish baselines, model pollutant cycles, and
guide direction for future study and policy. Examples of such studies include the National Study of
Chemical Residues in Fish that was conducted in the late 1980s (a follow-up to that study is currently
underway), and the Lake Michigan Mass Balance Study conducted in the early 1990s.
When designing a study or monitoring program, EPA uses information about detection and
quantitation limits, along with information on the risks associated with the pollutant(s) of interest and the
cost of measurement, to select an appropriate method for measuring the pollutant. Accepting all
positively valued measurement results and selecting a measurement method with a detection limit lower
than the level of concern for the substance being measured would provide some assurance that
measurement results associated with that concentration would be positively valued. Selecting a
3-17
-------
measurement method with a quantitation limit lower than the level of concern for the substance being
measured would generate measurement results that are easier to explain to the data user and the general
public.
3.2.2 Descriptive versus Prescriptive Uses of Lower Limits to Measurement
The literature on detection and quantitation generally assumes that these procedures are
descriptive, as opposed to prescriptive. In other words, detection and quantitation studies are described as
characterizing the current performance of a laboratory or laboratories using a method to measure a
substance rather than specifying specific performance benchmarks that a laboratory must meet to
demonstrate and maintain proficiency. Two possible reasons for this treatment are: (1) the intended
audience includes laboratory staff and measurement methods developers who wish to make new methods
useable by as many laboratories as possible, and (2) the author may have an institutional reason for not
attempting to control variability and thus lower detection and quantitation limits. On the other hand, the
technology-based and water quality-based effluent limitations programs administered by EPA's Office of
Water have an institutional goal of protecting human health and the environment. Providing this
protection requires that the Agency be able to measure pollutants at ever lower concentrations.
Establishing stringent standards and a compliance scheme for laboratories is one way to more rapidly
develop the ability to measure at these concentrations. A prescriptive strategy concerning detection and
quantitation limits would be to:
• Determine the detection and quantitation limits at multiple laboratories.
• Establish a detection limit and a quantitation limit for the method that is based on some
performance of these laboratories. These limits could be established as the limits reported by the
mean or median laboratory, or by some other criterion, such as the pooled value of the limits
achieved by all laboratories or the limits that are met by a certain percentage of the laboratories.
• Use the established detection and quantitation limits as performance standards that must be
demonstrated by laboratories that practice the method.
Such an approach is consistent with other performance standards included in EPA methods, such as
standards for instrument calibration, recovery of spiked reference and matrix samples, etc.
The use of such an approach would help ensure that prescriptive detection and quantitation limits
(i.e., performance standards) reflect the capabilities of multiple laboratories, rather than a single state-of-
the-art research laboratory. Of course, it is possible that even when multiple laboratories are used to
establish performance standards for detection and quantitation, some laboratories initially may not be able
to achieve these standards. However, most laboratories facing this problem would try to improve and
achieve these standards by investing in staff training, improved equipment, a stronger quality assurance
program, or clean room practices and higher quality maintenance and operations.
There is a risk that some members of the laboratory community will not be able to meet the
standard, either because they are not willing to invest the resources necessary to do so or for other reasons.
That risk should be considered when using a prescriptive approach to detection and quantitation (i.e.,
establishing limits that act as performance standards). Conversely, the risk of using a descriptive
approach is that it can result in detection and quantitation limits that reflect a broad community of
laboratories, including those that have made little if any effort to control contamination and variability at
these low levels, thus raising detection and quantitation limits to a level that is higher than desired for
protection of public health and the environment.
3-18
-------
3.2.3 Compliance Evaluation Thresholds
When technology-based effluent limitations are developed, the limits typically are at or above the
quantitative measurement capabilities (e.g., the ML) of one or more analytical methods that are available
to support compliance monitoring. Therefore, it is possible to monitor and evaluate permit compliance at
concentrations with an accepted degree of measurement certainty.
A situation that arises frequently in addressing water quality-based limits is that permit limits may
be set below the detection or quantitation limit of the most sensitive, approved analytical method. This is
particularly true for pollutants that are toxic in extremely low concentrations or that bioaccumulate. A
recommended approach for these situations is to include in the permit, the appropriate permit limit derived
from the water quality model and the waste load allocation for the parameter of concern, regardless of the
proximity of the limit to the analytical detection level, along with an indication of the specific analytical
method that should be used for monitoring (See Technical Support Document for Water Quality-based
Toxics Control, EPA/505/2-90-001, March 1991). Both the MDL and ML have been used as reporting
limits or compliance evaluation thresholds in NPDES permits. EPA promulgated regulations for NPDES
permits for dischargers to the Great Lakes basin that require the use of the ML for compliance assessment
purposes (See Appendix F, Procedure 8, Part B of 40 CFR 132). EPA has recommended for most
permitting situations that the compliance level be defined in the permit as the ML (See Technical Support
Document for Water Quality-based Toxics Control, EPA/505/2-90-001, March 1991). Outside of the
Great Lakes basin, it is important to note, however, that states that implement the NPDES permits
program have not always followed EPA's guidance. The inconsistent use of the MDL and ML as
reporting limits or compliance evaluation thresholds in NPDES limits suggest that EPA should
develop additional implementation guidance.
From a technical standpoint, a one-sided limit that addresses false positives only, such as Currie's
critical value or EPA's MDL, is the most appropriate approach for producing a compliance evaluation
threshold for the situation in which the WQBEL is less than a detection limit in the most sensitive
analytical method because the one-sided limit allows measurement to the lowest possible level while
protecting a discharger from the risk of a false violation. For example, consider the situation in which
2,3,7,8-tetrachlorodibenzo-/?-dioxin (dioxin) is to be evaluated against the ambient water quality criterion
of 13 parts-per-quintillion (ppqt). The most sensitive analytical method approved at 40 CFR part 136 is
EPA Method 1613, with an MDL of 4 parts-per-quadrillion (ppq) and an ML of 10 ppq. The MDL is
more than 300 times greater than the ambient criterion. Therefore, if dioxin is detected in the receiving
water as a result of a discharge (i.e., the measurement result is greater than the MDL of 4 ppq), there has
been an exceedance of the ambient criterion. Use of the ML as a compliance evaluation threshold is
appropriate because it is the point at which the measurement could be considered reliable.
3.2.4 National versus Local Standards for Measurement
In accordance with the Settlement Agreement, EPA is re-examining the approaches of detection
and quantitation used with methods approved for use at 40 CFR part 136. The Clean Water Act
authorizes States and local governments to implement permits, with the requirement that they be at least as
protective (stringent) as the national standards established by EPA. EPA recognizes that some States have
implemented approaches to detection and quantitation that are either specific to that State, result in lower
numeric limits in discharge permits, or both. Given that State and local governments use different
approaches, a change by EPA with regard to this assessment of detection and quantitation procedures may
have an unanticipated impact on those States and local governments.
3-19
-------
3.2.5 Cost and Implementation Issues
Detection and quantitation limit procedures are typically employed by organizations that develop
methods and by laboratories that use the methods. Method developers typically include governmental
organizations such as EPA, NOAA, USGS, and DOE, or voluntary consensus standards bodies (VCSBs)
such as the American Public Health Association (APHA), ASTM International, AOAC-International, and
ISO/IUPAC. Method developers also may include manufacturers of instruments or supplies used in
testing. Users of methods generally are the laboratories performing tests to assess and assure product
quality, to support regulatory compliance monitoring, or to support scientific studies.
Method development requires a more diverse set of skills than method use because such
development generally demands an understanding of quality systems, statistics, and analytical
technologies. Personnel working for the method developer generally include a project manager,
measurement analysts, who are experienced in several measurement technologies or very experienced in a
specific, complex technology, and at least one statistician. Operating laboratories typically will not have a
statistician, and the breadth and dept of the analyst experience may be less than in a method development
laboratory, because an operating laboratory is focused on obtaining reliable results in the analysis of a
given sample using a well-tested measurement technology.
3.2.5.1 Implementation of a Detection/Quantitation Limit Procedure by a Method Developer
The basic resources available to the method developer are time, money, and the technical skills of
the developers's staff. The fundamental decision for implementing a detection or quantitation procedure
is whether that procedure is intended to characterize the performance of the method at a well-performing
laboratory or the performance of the method across a group of laboratories. If the procedure is intended to
characterize the performance of the method across a group of laboratories, it is also necessary to decide if
there will be some way to compare the performance of individual laboratories to the group performance
standard. There are serious time, cost, and skill issues with each of these decisions. Ordering these
decisions from the least resource intensive to the most, they are characterizing the performance of the
method: (1) at a well-performing laboratory, (2) at a group of laboratories, or (3) at a group of laboratories
with comparisons of individual laboratories. Other costs for the method developer could include
planning, data management, reference laboratory services, and whether laboratories are willing to
volunteer for the study or if their services must be purchased.
An independent decision is whether to assume a simple model for measurement variability and
limit the number of test concentrations, iterate assuming a simple model, or to design a study of the
relationship between measurement variation and the concentrations of the substances measured by the
method. This decision will greatly influence the number of samples measured in the study and the
required skill of the personnel who design, conduct and interpret the results of the study. If the
laboratories do not volunteer for the study, then the direct cost for measuring these samples or blanks
ranges from a few dollars per sample to more than $1,000 per sample for some analytes. Until the
relationship between measurement results and standard concentrations becomes well known, such studies
will require the active participation of professional statisticians in design, implementation, and analysis.
3.2.5.2 Implementation of a Detection/Quantitation Limit Procedure by a Laboratory
A laboratory may implement detection or quantitation procedures for its own quality control
purposes, because of regulatory requirements, or to participate in a round robin study for a VCSB or some
other organization. When participating in the study of another organization, the laboratory may
3-20
-------
voluntarily accept some cost of the study for marketing purposes, professional development, or to
benchmark the performance of the laboratory.
In each case, a detection or quantitation limit approach will be of little utility if it is not capable of
being implemented by the laboratory. An advantage of straightforward approaches such as the EPA
MDL, the ACS limit of detection, and the ISO/IUPAC critical value is that they can, in principle, be
understood by analysts expected to use the approach. Likewise, the procedures described for
implementing the MDL approach are straightforward and have been implemented by thousands of
laboratories. In contrast, the ASTM IDE and IQE procedures are highly complex and, as a consequence,
are beyond the capability of most environmental testing laboratories.
Highly complex procedures are usually more costly to implement than simple procedures. As
noted in Section 3.1.5, method performance generally improves over time. This means that a detection
and quantitation limit approach should be supported by procedures that will allow individual laboratories
and other organizations to affordably characterize such improvement. Mandating interlaboratory studies
using complex detection and quantitation procedures means that laboratories lacking statistical support
staff, and seeking to develop new techniques or modify existing techniques to achieve improved
measurement sensitivity would have to rely on, and perhaps even pay, other laboratories to demonstrate
the sensitivity of their procedures. This limitation has the effect of hindering method development and
improvement.
3.2.6 Use of a pair of related detection and quantitation procedures in all Clean Water Act
applications.
In Section 3.2.1, we discussed several different applications for detection and quantitation limits
under the Clean Water Act. To review, these applications are:
• Method development and promulgation,
• Method performance verification at a laboratory,
• Technology-based effluent guidelines development,
• Water quality-based effluent limits development,
• Permit compliance monitoring, and
• Non-regulatory studies and monitoring.
In the 2003 assessment, EPA argued that although EPA could develop a separate detection and
quantitation approach for each of these applications and attempt to define and evaluate each of these
approaches in our re-examination of detection and quantitation approaches, the resulting matrix of
applications and approaches would cause confusion for stakeholders, such as regulators, permittees, and
the laboratory community. To minimize this confusion, EPA suggested that a single pair of related
detection and quantitation procedures could meet the needs of all CWA applications. Some commenters
disagreed with this approach and recommended that at least two distinct procedures should be used, one
for method development and one for verifying laboratory performance.
3.2.7 Accepting the Procedures of Voluntary Consensus Standards Bodies
In February 1996, Congress enacted Public Law 104-113 (15 USC 3701), the National
Technology Transfer and Advancement Act (NTTAA). This act directs "federal agencies to focus upon
increasing their use of (voluntary consensus) standards whenever possible, thus reducing federal
procurement and operating costs. " The Act gives Federal agencies discretion to use other standards
3-21
-------
except where the use of voluntary consensus standards would be "inconsistent with applicable law or
otherwise impractical."
The NTTAA is implemented by Federal agencies based on the policies described in Circular
A-l 19 from the Office of Management and Budget (OMB). The current version of this OMB circular was
published in the Federal Register on February 19, 1998 (63 FR 8546). Neither the NTTAA nor Circular
A-l 19 require that agencies replace existing government standards with standards from a voluntary
consensus standard body (VCSB). If EPA already has standards in place for detection and quantitation
approaches, EPA is not obligated by NTTAA to replace these with VCSB standards. Although some
stakeholders commenting on EPA's 2003 assessment encouraged EPA to allow use of alternative
procedures for determining detection and quantitation levels, commenters in general did not support
eliminating continued use of the MDL or ML.
Circular A-l 19 also discusses the effect of the policy on the regulatory authorities and
responsibilities of Federal agencies. The circular states that:
"This policy does not preempt or restrict agencies' authorities and responsibilities to
make regulatory decisions authorized by statute. Such regulatory authorities and
responsibilities include determining the level of acceptable risk; setting the level of
protection; and balancing risk, cost, and availability of technology in establishing
regulatory standards. However, to determine whether established regulatory limits or
targets have been met, agencies should use voluntary consensus standards for test
methods, sampling procedures, or protocols."
Thus, EPA is responsible for establishing the levels of risk and protection, not only for the regulatory
limits applied to discharges, but also to the risks of decision errors (e.g., false negatives or false positives)
in the detection and quantitation approaches applicable under the Clean Water Act.
Finally, Circular A-119 describes two types of technical standards: performance standards and
prescriptive standards. A performance standard is defined as:
"a standard... that states requirements in terms of required results with criteria for
verifying compliance but without stating the methods for achieving required results. " In
contrast, a prescriptive standard is one "which may specify design requirements, such as
materials to be used, how a requirement is to be achieved, or how an item is to be
fabricated or constructed."
Neither the NTTAA nor Circular A-119 direct agencies to favor performance standards over
prescriptive standards, or vice versa. EPA believes that the current MDL procedure is a prescriptive
standard, in that it specifies both the design of the MDL study and how the requirement to establish
method sensitivity be achieved. There is some obvious flexibility or opportunity for judgement in
employing the MDL procedure, and much of the historical debate over the utility of the MDL procedure
would suggest that it may not be prescriptive enough. The alternative detection and quantitation
approaches evaluated in this document, including the approaches submitted by commenters on the 2003
assessment, also are prescriptive, not performance, standards.
To effect a performance-based approach to estimating detection and quantitation limits, an option
that EPA may consider is to allow method developers, laboratories, and others the choice of any one of a
variety of approaches to establishing these limits, including the existing MDL procedure or a VCSB
3-22
-------
standard. Thus, establishing method sensitivity could be considered a performance standard under
NTTAA and Circular A-l 19, rather than a prescriptive standard. That these different approaches
(prescriptive standards) yield different answers would be immaterial if EPA evaluates the answers relative
to a specific decision, i.e. the benchmark becomes a performance rather than a prescriptive standard. That
evaluation should not be divorced from knowledge of the decision to be made (e.g., the regulatory limit
for a given pollutant).
3.2.8 Alternative Procedures
One of the peer reviewers who evaluated a draft version of the February 2003 assessment
document noted that:
"EPA has stated in the TSD that one primary procedure is needed for clarity and to avoid
confusion among stakeholders. If alternate procedures are needed, the EPA Clean Air Act
system of reference and equivalent methods has worked well, and could be a model for
EPA to follow under the Clean Water Act. " (Cooke, 2002)
EPA currently assesses and has approved at 40 CFR Part 136 methods that employ an alternative
procedure for establishing method sensitivity. This approval process includes an overall evaluation of the
suitability of the method in entirety and thus includes the detection or quantitation approach used to
establish the performance specifications listed in the method.
The peer reviewer is referring to the system of reference methods used under the Clean Air Act.
This system is similar to the existing "alternate test procedure" (ATP) program for analytical methods
currently used within the Office of Water. The difference between the ATP program and the case of the
procedures for establishing detection and quantitation limits is that in an ATP program, the goal is clear
and agreed upon (i.e. is a method appropriate for CWA applications), whereas there remain fundamental
theoretical issues surrounding the relative merits of the various detection and quantitation approaches that
are the subject of this document.
For example, when a test procedure is developed for use in the Clean Air Act or Clean Water Act
programs, the reference method is designed to measure Analyte X, in Matrix Y, at some concentration
related to a regulatory need (e.g., a compliance limit) or environmental study. Alternative procedures may
be capable of making measurements of Analyte X in Matrix Y, at the level of concern, using completely
different instrumentation. Thus, the demonstration of equivalency between the reference method and a
possible alternative method is judged using a metric that consists of Analyte X, Matrix Y, and the level of
concern, as well as other aspects of method performance.
In contrast, the primary differences between the EPA MDL/ML approaches and potential
alternatives such as the ASTM IDE and IQE are related to the philosophical differences of how detection
and quantitation limits should be derived and applied. These differences are described at length in this
chapter and the rest of the Assessment Document. Therefore, EPA does not believe that a variant of
existing ATP programs is likely to be an effective model for assessing other detection and quantitation
procedures.
A stakeholder commenting on EPA's 2003 assessment recommended that EPA adopt alternative
procedures in Appendix B of 40 CFR 136 as site-specific alternatives to the MDL and ML when such an
alternative is determined to be necessary by a discharger and/or regulatory agency (e.g., in special cases
when more scientifically rigorous procedures are needed). As noted previously, EPA has reviewed and
3-23
-------
approved at 40 CFR Part 136 methods that employ an alternative procedure for establishing method
sensitivity as part of an overall evaluation of the suitability of the method. EPA has done so without need
of any revisions to appendix B at 40 CFR part 136.
3.3 Statistical Issues
This section provides a brief explanation of the key statistical issues involved in the development
of detection and quantitation limits.
3.3.1 Sources of Variability
Various known and unknown sources of variability will influence measurements made by a
laboratory using a specific method. These sources may include random measurement error, differences in
analysts, variations between different equipment manufacturers and models, variations in analytical
standards, routine fluctuations in equipment performance, and variations in facility conditions (e.g.,
varying levels of background contributions).
There are several ways in which some of these sources of variability can be controlled. One is a
strong quality assurance (QA) program that includes use of: 1) trained and qualified staff, 2) properly
maintained equipment, 3) standards that are fresh and properly prepared and stored, 4) written standard
operating procedures and methods for all sample handling, analysis, and data reduction/reporting
activities, 5) procedures for monitoring ongoing laboratory performance, and 6) quality control (QC)
samples and QC acceptance criteria to ensure that the laboratory systems are in control. The EPA
methods promulgated at 40 CFR part 136 require the use of qualified staff, appropriately cleaned and
calibrated equipment, and properly prepared standards. Many of these methods also provide detailed
steps for performing all sample handling and analysis activities, and detailed requirements for analysis of
specific quality control samples with corresponding QC acceptance criteria.
Even when prescribed EPA method requirements and guidance are used, however, it is not
possible to completely eliminate all variability that can occur within or between laboratories. Even with
procedures in place to control quality and reduce variability, it should be recognized that some
laboratories, analysts, and instruments may achieve lower detection and quantitation limits than others.
Ultimately, some laboratories may not be capable of meeting low-level measurement requirements without
some effort to improve operations.
Many of these sources of variability are considered in establishing detection and quantitation
limits for analytical methods used under EPA's Clean Water Act programs because these detection and
quantitation limits are first established in single-laboratory studies, then evaluated or verified in multiple
laboratories, and, where necessary, further evaluated in an interlaboratory study. These studies include
evaluation of method performance characteristics, including detection and quantitation capabilities, in
multiple laboratories using multiple matrices, analysts, instrumentation, reporting activities, standards, and
reagents. Although detection and quantitation are not evaluated in the various matrices used in these
studies, EPA's MDL procedure includes instructions for determination matrix-specific MDLs.
Some stakeholders commenting on EPA's assessment of approaches to detection and quantitation
believe that accounting for these sources of variability when determining detection and quantitation limits
is necessary because relative variability increases as the lower sensitivity limits of a method are
approached. Some stakeholders believe, for example, that a methodology for detection and quantitation
has to address the variability that occurs across laboratories (interlaboratory variability) using the same
3-24
-------
analytical method. Other stakeholders believe, however, that interlaboratory variability is not an issue
because detection and quantitation decisions are made in a single laboratory. Some stakeholders believe
that procedures should address the long-term variability that can occur within a single laboratory over
time. As discussed in section 3.1.5 of this chapter, EPA encourages, where appropriate, gathering data to
address temporal variability. EPA acknowledges that interlaboratory variability is very important during
the methods development process and should be incorporated, as appropriate, during the process. EPA
also recognizes that within lab variability should be considered when establishing laboratory performance.
Over the years, stakeholders have noted that the variability that can result from application of
analytical methods to different matrices also should be addressed by procedures for determining method-
specific detection and quantitation limits. However, it has been EPA's experience that matrix effects
typically can be overcome using various sample processing procedures. In EPA's interlaboratory
validation studies of the 600-series wastewater methods, the recoveries of some organic analytes from
real-world matrices were closer to 100% than were recoveries from a reagent water matrix. This effect is
thought to be attributable to dissolved solids in the real-world matrix that, in effect, "salt out" the organic
compounds. EPA does not believe it is appropriate or feasible to aggressively pursue matrix effects in
establishing detection and quantitation limits (i.e., EPA has not attempted to find worst-case matrices in
order to maximally exacerbate matrix effects). Instead, EPA considers the type of matrices that would be
regulated under the Clean Water Act (e.g., the effluents that are discharged from properly designed and
operated secondary treatment plants). Further discussion of matrix effects can be found in Section 3.1.3.
Because detection and quantitation limits focus exclusively on the capabilities of the measurement
process, a source of variability that is not considered in any of the detection and quantitation limits is the
variability that is associated with sample collection. If the sample is not representative of the population
from which it was collected, then the variability associated with measurements made in the region of
detection or quantitation may be immaterial. For example, EPA's Technology Innovation Office
conducted a study to characterize the effects of sampling variability on measured results. In that study,
results from seven discrete samples collected within a two-foot distance of one another were evaluated.
Each sample was analyzed for the explosive TNT on-site using a colorimetric test kit, and in a laboratory
using EPA SW-846 Method 8330 (high-performance liquid chromatography). Analysis of the results
from these measurements indicated that 95% of the total variability was due to sampling location and only
5% was due to differences between the analytical methods. Put another way, differences in sampling
location caused 19 times more uncertainty in the data results than did the choice of analytical method,
over a distance of only 2 feet (Crumbling, 2002). While this result may not be typical, and EPA does not
mean to diminish the importance of understanding measurement error in the region of detection and
quantitation, EPA believes it is important to understand it in the context of the overall sampling and
analysis error.
3.3.2 False Positives and False Negatives
3.3.2.1 False Positives and False Negatives in Making Detection Decisions
In this section, we discuss the impact of detection, quantitation, and reporting levels on false
positive measurement results and false negative measurement results. The definitions of false positives
and false negatives are directly related to the concepts of critical value and detection limit used by Currie
(1995). These terms were adapted from statistical decision theory to establish the framework for decision
making with regard to detection of analytes. The critical value (Lc), as defined by Currie, is the point at
which the detection decision is made. That is, measured values that are less than the critical value are
judged to be not statistically different from blanks ("not detected"). Measured values that are no less than
3-25
-------
the critical value are judged to be statistically different from the blanks ("detected"). Denoting measured
values that are less than the critical value as non-detects constitutes censoring and is discussed in more
detail in Section 3.3.5.
The critical value is defined such that when the analyte is not present in a sample, there is a small
possibility that a measurement will exceed the critical value. A measurement that indicates the critical
value has been exceeded is, therefore, the result of one of two circumstances: (i) the analyte is present in
the sample; or (ii) the analyte is not present in the sample and, by chance, the measurement has exceeded
the critical value. The occurrence of (ii) is defined in statistics as Type I error ("false positive"). A
measurement that is less than the critical value occurs when: (iii) the analyte is not present in the sample;
or (iv) the analyte is contained in the sample at the hypothesized concentration but the measurement
procedure fails to indicate its presence. The occurrence of (iv) is defined as the Type II error ("false
negative").
The following table summarizes possible situations:
Decision
Concentration =C,
where C>0
Concentration =0
State of the Sample
Concentration —C,
where C>0
Analyte Present
Correct (i)
Type II error (iv)
Concentration^!)
Analyte Not Present
Type I error (ii)
Correct (iii)
Calculating the probability of a Type I error only requires assumptions regarding the distribution
of observations under the hypothesis that the concentration is equal to zero. In the terminology of
statistical decision theory, Concentration = C, where C>0 corresponds to a true value is referred to as the
"Alternative Hypothesis" (see, e.g. Introduction to Mathematical Statistics, by Hogg and Craig, 5th
edition, [1995]). When C is hypothesized, assumptions need to be made about the distribution of
observations at Concentration=C for the probability of Type n error to be evaluated.
In analytical chemistry, the probability of Type I error is often called the "false positive" rate and
the probability of Type II error is often called the "false negative" rate. The statistical alternative
hypothesis should be specified before introducing the false negative rate. An error common to some
published discussions of false negative rates and detection and quantitation concepts is to state that use of
Currie's detection limit as a reporting limit or action level will somehow "control" the rate of false
negatives. This is both incorrect and counter-productive, because a single level cannot control false
negative rates.
Currie introduced the idea of a Detection Limit, Ld, in place of a statistical alternative. The
Detection Limit is not a part of the detection decision process (i.e., is the concentration in the sample
statistically different from the blank?). The Detection Limit is defined such that when the true
concentration of an analyte is equal to the Detection Limit, there is a small probability that a measured
value will be less than the Critical Value (detection decision-making level in this case), and thereby result
in the false negative decision.
3-26
-------
One of the peer reviewers of EPA's 2003 Technical Support Document (the TSD) stated:
"Also, to reemphasize, the single most problematic issue when developing a detection
limit is correction for false negatives. I took from the TSD (in §3.3.6) an implicit
emphasis on LC-type values such as the MDL [when correctly calculated, as in (I)], as
motivated by an underlying sort of practical/environmental conservatism that essentially
removes false negatives from the estimator's development. I am willing to accept this
interpretation. I suspect the fray will continue, however, since there seems to be a fair
amount of confusion on the issue in the analytical chemistry literature. The bottom line
from my reading of the TSD is that, in effect, we are calculating an LC, but using
terminology that makes some readers think it's an LD. I can accept the argument that
false negative errors are not the critical issue here, and hence that the approach is
reasonable (once correct calculations are undertaken). But, the Agency should put forth
an effort to overcome this confusion in terminology. (I expect they will ask me how, and
in reply I'd suggest emphasizing that an LC calculation is a form of decision limit, not a
detection limit. But here I suspect many users will still confuse the terms, or reverse
their meaning, or not see the difference, or who knows what else? I don't know how
winnable this battle is...) " (Piegorsch, 2002)
To illustrate the intent of Currie's detection limit, consider a case where the detection decision-
making level is set equal to Currie's critical value, and a sample is spiked at a true concentration equal to
Currie's detection limit. Given a large number of measurements on this sample, about 99% of the
measurement results will be reported as being measured above the detection decision-making level, and
1% of the measurement results will be reported as being measured below this level. Knowledge of the
lowest true concentration that will routinely produce acceptable results (e.g., Currie's detection limit) can
be used to determine if the measurement method meets the needs of a study. For instance, a study
concerned with a wastewater treatment technology that is not expected to be effective at concentrations
below 10 mg/L may call for a relatively inexpensive measurement method capable of detecting the analyte
at 10 mg/L, rather than a more expensive measurement method capable of measuring a hundred times
lower.
3.3.2.2: Effect of Bias on Rates of False Positives
The presence of bias in a method can have a strong effect on the rate of false positives associated
with detection limit estimates. For example, in defining the critical level, Currie assumed that blank
results follow a Normal distribution centered about zero (0). However, for some methods and analytes,
this assumption may not hold due to factors that can and should be controlled, such as calibration errors
and high background contamination. In many cases, bias can lead to either under- or over-estimation of
detection limits. In cases such as these, not taking bias into account when determining detection and
quantitation limits (using the mean or median of the results, for example) may influence false positive
rates.
3.3.3 Use of Multiple Replicates
Existing detection/quantitation procedures are based on estimating the standard deviation of blank
or spiked replicates. Statistical estimates tend to be less variable when the number of replicates increases.
Some commenters on EPA's 2003 assessment believed that use of only seven replicates over a short
period of time results in a substantial underestimation of the MDL. However EPA's MDL procedure does
not limit the maximum number of samples that the laboratory may use to estimate the MDL; the procedure
3-27
-------
simply sets a minimum number of seven replicates. Laboratories may choose to improve their estimates
of the standard deviation that is used to calculate the MDL by analyzing more than seven replicates.
3.3.4 Statistical Prediction and Tolerance
To define a critical value, a detection limit, or a quantitation limit, different descriptive
terminology is used to distinguish differences in the numeric value of the limit. The following example
uses a critical value, but the questions motivating detection and quantitation limit decisions may be
phrased in a similar fashion.
In setting a critical value, do we want a critical value that tells us how likely it is that:
• A measurement result was produced by measuring a blank sample,
• The next measurement result will be produced by measuring a blank sample, or
• The next [pick any number] of measurement results will be produced by measuring a blank sample?
In statistical terms, these three objectives may be addressed, respectively, by application of methodology
for determining:
• Percentiles;
• Prediction intervals; and
• Tolerance intervals.
Percentiles are fairly straight forward to interpret, i.e., they specify the percentage of a distribution
that falls below a given percentile value. Prediction and tolerance intervals are, in effect, confidence
intervals on percentiles and can be somewhat more difficult to understand and apply. There are many
excellent textbook and literature references that present the theory and application of prediction and
tolerance intervals such as Hahn and Meeker, Statistical Intervals, 1991, and Pratt and Gibbons, Concepts
of Non-parametric Theory, 1981. Hahn and Meeker describe at length the different statistical intervals
including their properties, applications, and methodology for constructing the intervals. Pratt and Gibbons
have an excellent discussion of tolerance intervals that is general in application due to the non-parametric
perspective, i.e., no distributional assumptions are required for the results to be valid.
One of the peer reviewers of EPA's 2003 assessment stated:
"Tolerance intervals are inappropriate for environmental monitoring. The main issues
here are 1) is the true concentration greater than some specified safe action level, with
sufficient confidence, and 2) what interval of possible concentrations is consistent with
one or a series of measurements, with a specified degree of confidence? Both are
statements about a given sample or series of samples, and not about the hypothetical
variability of future estimates. Suppose that one has a sample of 10 observations with
mean concentration of 1 ppb and standard deviation of 0.5 ppb. Then the estimated 99%
critical level is (2.326)(0.5) — 1.2 ppb. One may choose to use a t-score instead of a
normal score so that the chance that a future observation will exceed this level is in fact
99%. In this case, the critical level estimate would be (3.250)(0.5) — 1.6ppb. This does
actually correspond to a prediction interval for future observations from a zero
concentration sample.
3-28
-------
"If one asked instead for a 95% confidence interval for the .99 percentage point of the
true distribution of measurements (assuming normality) when the true quantity is zero,
this can be calculated approximately using a chi-squared distribution and covers the
interval (0.9 ppb, 2.4 ppb). It does not, however, make sense to use 2.4 ppb as a
threshold, since the chance of a future observation exceeding 2.4 ppb when the true mean
concentration is 0 is about .0005, far smaller than the intended false-positive limit of
.01." (Rocke, 2002)
Another of the peer reviewers of this assessment stated:
"the operational definition as taken from pp. 5-2/5-3 of
MDL = t 0.99 (df) S
does not correspond to a confidence statement that I can interpret.... This should be
replaced, although I agree that a number of statistical quantities could be used; this is
where the "fray " seems to be most boisterous. (By the way, the TSD, and I, should be
more careful in the use of statistical terminology. We both refer often to confidence
"intervals, " when in fact the quantity of interest is a confidence limit — or tolerance
limit, etc. — on some underlying parametric quantity.)...
"If we accept the TSD's argument on p. 3-25 that the practical value of tolerance limits is
limited, then the MDL should be viewed as a prediction limit. And if so, it must contain
an additional term as per Gibbons (1994, p. 98):
\
n
"One caveat: although I think the prediction limit argument is acceptable, if the use of
tolerance limits rather than prediction limits is in fact desired, then Gibbons' (1994, p.
99) presentation or an equivalent approach should be used instead to correct the MDL
calculation." (Piegorsch, 2002)
Similarly, Hahn and Meeker describe situations in which the various intervals or limits are
appropriate to use. (As noted by the peer reviewer, the terms "intervals" and "limits" are sometimes used
interchangeably). They also give examples of the sort of applications that are suitable for each type of
limit although the decision to use a particular type of limit in a given application is not determined strictly
by theoretical considerations. It is also a matter of judgment.
Prediction intervals contain results of future samples from a previously sampled population with a
specified level of confidence. Prediction limits are not estimators of parameters such as means or
percentiles. For example, a prediction interval may be constructed to contain future sampling results
expressed as a mean or standard deviation of a future sample or all of a certain number of individual
future sampling results.
3-29
-------
While the theoretical construct underlying Currie's critical level is clear and straightforward, EPA
recognizes that estimating this level from limited data is less straightforward and the choice of an
appropriate statistical methodology involves policy judgements that might legitimately differ for different
uses of the MDL.
3.3.4.1 Tolerance Intervals
Tolerance intervals contain a specified proportion of a population of measured values with a given
statistical confidence level. For example, we say that a proportion, P, of a population is contained within
the intervals (L1; L2) with (l-a)100% confidence. Random variables that are the lower and upper ends of
the interval, L! and L2, respectively, are referred to as tolerance bounds. A tolerance bound is therefore
the endpoint of an interval of random length that is determined on the basis of having a specified
probability of 1-a that its coverage of the population is at least equal to a specified value P. The quantity
1-a is referred to as the confidence level for the interval and P is the minimum proportion of the
population contained in the interval. Tolerance bounds are not estimators of values such as a mean or a
percentile but rather bounds that are always guaranteed to contain the desired value at some level of
statistical confidence. Pratt and Gibbons discuss this and other properties that affect the utility of
tolerance intervals and create difficulties in their interpretation and application.
In effect, the determination of what, if any, interval to use is a policy decision. The choice should
consider how easy it is to estimate the interval you want under the conditions that exist. As Pratt and
Gibbons point out, the interpretation of tolerance intervals (and analogously, prediction intervals) can be
problematic, especially when issues of sample size and the choice of confidence level come into play.
Pratt and Gibbons cite examples where the interplay of sample size and high percentile and confidence
levels make tolerance intervals useless.
3.3.4.2 Use of Tolerance and Prediction in Setting Detection and Quantitation Limits
Statistical intervals can be, and have been by a number of authors, adapted for use in setting
detection and quantitation limits. The basic approach requires a functional definition of detection or
quantitation that includes a statistical term or terms. An interval could then be constructed about the
statistical term which could be used to assess the detection or quantitation limit, or make an adjustment to
a calculated value that would result in the detection or quantitation limit. For example, most detection
limit estimators are functionally dependent on an estimate of standard deviation of measurement error. A
statistical interval could be constructed about the standard deviation and the length of the interval could be
used to assess the detection limit. The end points of the interval could be used as the basis for an
adjustment (upward or downward) in the calculated limit.
The error rates in ASTM's IDE Standard Practice are based on statistical tolerance intervals (i.e.,
the nominal Type 1 error rate is 5% (5%=100%-95%), and the nominal Type 2 error rate is 10% (10% =
100%-90%)). Several stakeholders have commented that the use of a tolerance interval approach can
protect, at a 99% level of confidence, against false positives and false negatives, and that tolerance
intervals become increasingly important with a decreasing sample size. For example, if the sample
standard deviation is determined with 7 measurements and all sources of variance are properly represented
in the 7 measurements, then there is approximately a 5% chance that the true population standard
deviation will be more than two times the sample standard deviation. For a typical ICP determination of
20 or more elements this means that at least one is likely to have a calculated MDL two times lower than
it should be. Obviously the false positive rate for this element will be large.
3-30
-------
The use of prediction and/or tolerance limits in setting detection and quantitation limits should be
evaluated in the context of the specific application and policy considerations. In practice, the effect of
adjustment of detection and quantitation limits by use of prediction and tolerance intervals can be quite
large, depending on the amount of data available and the choices of percentiles and confidence levels.
3.3.5 Censoring Measurement Results
Measurement results are often reported as less than some detection, quantitation, or reporting limit
(see Section 3.2.1.3, Permit Compliance Monitoring) without providing a single best estimate for the
numeric result. For example, if a direct reading of the measurement results indicates a concentration of 3
mg/L and the reporting limit for the substance is 5 mg/L, the laboratory may only report that the
measurement result is less than 5 mg/L. Statisticians call this suppression of results that are less than a
specified amount "censoring." Reasons for the practice of censoring relate directly to issues surrounding
the development of detection and quantitation limits (i.e., the premise that measurement results below
certain low levels may not be useable for certain purposes).
Some data users prefer to use the actual measurement results (even if they are negative values),
rather than to censor the results at a reporting or detection limit, because censoring data at such a limit
loses information about low-level measurements and can introduce bias into the data set. If all low values
are eliminated, then the average (mean) of the remaining data would have a positive bias. In other words,
while negative or extremely low values may be considered problematic by some, they are of value to
statisticians and modelers, because they convey useful information about the distribution of results.
Some programs, such as EPA's Superfund Contract Laboratory Program, require laboratories to
report measurement results in conjunction with a qualifier that the result is below a specified detection,
quantitation, or reporting level. In the example provided in the first paragraph of this section, the
laboratory might report both a measured value of 3 mg/L and a reporting limit of 5 mg/L. Under certain
assumptions, measurement results below the specified reporting level could then be used to calculate
averages and statistical estimates that would be superior to estimates calculated using censored data.
Although the Superfund approach provides the greatest degree of flexibility for data users, it
should be used with care. First, data users who choose to use values reported below a detection or
quantitation limit need to have a firm understanding of the limitations of those data. Second, and as noted
in Section 3.2.1.3, Permit Compliance Monitoring, reporting data below a detection or quantitation limit
can lead to misinterpretation.
One of the peer reviewers that evaluated EPA's 2003 assessment of detection and quantitation
limit approaches noted that European Union (EU) has adopted another variant for reporting or censoring
data.
"In this case, the EU has adopted EPA Method 1613B (for analysis ofdioxins andfurans)
as well as EPA's MDL approach. However, the EU has further specified that the MDL be
used as an Upper Bound reporting limit where all non-detects are found in the analysis of
human or animal foodstuff. This forces laboratories to achieve levels available with
modern instrumentation, otherwise, the Upper Bound reporting level is above the
regulatory compliance level, and the data (or foodstuffs) are rejected" (Cooke, 2002).
3-31
-------
EPA agrees that this approach, which yields a "worst-case" (or highest possible) estimate of the
pollutant concentration, can serve as an incentive to the analytical and regulated community to pursue
measurements at the lowest levels which analytical methods are capable of achieving. However, EPA also
cautions that this approach effectively censors measurements made below the MDL and could yield an
overestimate of the concentration of the analyte of concern.
Several stakeholders have requested that EPA provide specific guidance and procedures regarding
data censoring and reporting, particularly when data are reported for compliance evaluation. EPA notes
that the decision to censor data is a data reporting and data use policy issue, not a laboratory issue. This
holds without regard to what detection or quantitation limit approach is used. The EU approach reflects a
similar point of view, in that it relies on the MDL as a detection approach, but also establishes this limit as
the reporting level for non-detects to better encourage development of lower MDLs. However, EPA also
recognizes that laboratory methodologies and data reporting and use policies are interrelated.
3.3.6 Outliers
Outliers are extreme or aberrant measurement values that, on inspection, do not follow the
characteristics of a set of data. Outliers maybe generated by a number of causes, such as errors in
following an analytical procedure, errors in recording results, or the result of extreme random variation in
a properly operating process. For example, if a new measurement method is being tested but the
laboratory fails to follow the procedure correctly when analyzing some samples, the associated
measurement results may stand out as outliers. A graphic example is provided in Figure 3-1, which shows
measured concentrations of aluminum versus spike concentrations for analytical results obtained using
EPA Method 1620. At a spike concentration of 250 i-ig/L, one of the measured values is approximately
750 fJ-g/L. This result visually stands out from the rest of the values, and may be an outlier.
3-32
-------
, UOIL
Spike Concentration
Figure 3-1
Stakeholders commenting on EPA's assessment of detection and quantitation procedures
generally believed that outliers should be identified and removed from data used to determine detection
and quantitation limits. Commenters added that, although it would be helpful to have specific instructions
for identifying outliers, application of the instructions should be optional (i.e., to the discretion of the data
user).
A common process for identifying potential outliers is to apply one or more statistical procedures
for identifying values far from the mean (average) of the data. An example of such a procedure is
described in ASTM Practice D-2777.
Because extreme values can be expected to occur on occasion, it may not be appropriate to
exclude them from the results used to develop detection or quantitation values. As recommended in the
ASTM procedure, the first step is to contact the laboratory to try to determine and resolve the cause. A
review of the analyst's records associated with the measurement may establish whether the extreme value
was caused by failure to follow the method or by some rare event associated with the method. If the
method under study was not followed, or there is a known or suspected analytical error, it is appropriate to
exclude the measurement result from the detection or quantitation analysis. If the measurement result is a
rare event associated with the method under study it may also be appropriate to exclude the measurement
result from the results in the study. EPA believes that results that are associated with spurious errors that
cannot be corrected will invalidate the measurement and should not be incorporated into the MDL
3-33
-------
determination.
3.3.7 Detection and Quantitation Studies
3.3.7.1 Study Design
The issues associated with the design of detection and quantitation studies include:
• how effectively a selection of spike concentrations can be used to correctly determine which
model type should be used to model variability,
• the extent to which the distance between spike concentrations can impact estimates of detection
and quantitation limits,
• how to reduce the influence of uncontrollable factors in the measurement process (probability
design),
• how complete to make the design factors in terms of the physical measurement process, and
• how flexible to make the design factors in terms of the physical measurement process.
Spike Concentrations and Modeling
If a model under consideration cannot be described by the number of spike concentrations in the
design, then it is not possible to tell if the model is appropriate. To take the simplest example, it is not
possible to describe the slope of a line associated with linearly increasing variation from a single spike
concentration. Two well-spaced spike concentrations would allow you to estimate a slope, but would
provide no idea of the variability of the estimate. Three well-spaced spike concentrations represent the
minimum requirement for estimating the linear relationship and the variability of that relationship.
Clayton et al. (1987) describe the relationship between the spread of the spike concentrations, the
number of spike concentrations, and the number of replicate measurements with regard to estimated
variability when a linear model is used. While the specific equation used in this paper does not apply to
all models, it indicates principles that do apply. Increasing the number of replicate measurements and
reducing the spread of the spike concentrations are all expected to reduce estimated variability along with
the associated detection and quantitation limits. However, one of the components of variability associated
with detection and quantitation is that associated with estimating the calibration relationship. To account
for this source of variation, it maybe appropriate to cover the entire calibration range. On the other hand,
many replicates at a high concentration may improperly weight the data in favor of high detection and
quantitation estimates.
It is also important to note that modeling of variability introduces modeling error, and direct
measurements of the variance in the region of interest may provide a more appropriate estimate of
variability, especially where the change in variance over this region is small.
3-34
-------
Probability Design
The process known as randomization is an important statistical consideration in the design and
interpretation of experimental studies. Randomization involves the allocation of experimental units to
factors and treatments under study according to a design determined by probability. Randomization
avoids bias and systematic errors that can occur in studies where randomization is not used.
Randomization is discussed in classic texts such as Statistics for Experimenters, by Box, Hunter, and
Hunter (197 8).
In studies of measurement methods, randomization can be used in the process of creating spike
concentration solutions and the ordering of analyses. However, randomization has practical drawbacks,
particularly with regard to studies designed to establish detection or quantitation limits. For example,
consider a simple study involving the analyses of samples spiked at five concentrations of the analyte of
interest, with five replicate samples analyzed at each concentration. A total of 25 analyses are required
for the study, and the analyses of the samples can be organized in a 5 by 5 matrix. A random number is
assigned to each block in the matrix as a means of randomizing the order of the replicates at each
concentration.
By virtue of this randomized design, a sample with a high concentration of the analyte of interest
may end up being analyzed immediately prior to a sample with a very low concentration of the analyte.
Unfortunately, this can lead to problems that result from the "carry-over" of analyte within the
instrumentation from one analysis to the next. When carry-over occurs, the apparent concentration of the
low-concentration sample can be inflated because some of the high-concentration sample 1 may be carried
into the low-concentration sample 2. In the context of a study designed to establish "how low you can go"
(i.e., establishing a detection limit), carry-over of the analyte into a low-concentration sample may
compromise the results by inflating the result for low-concentration sample 2, but not inflating the results
for other low-concentration samples because the randomized design did not cause them to be analyzed
immediately following a high-concentration sample.
Analysts are aware of the potential for carry-over and generally take steps during routine analyses
to minimize the chance that it will occur. Examples of steps that can minimize carry-over problems
include analyzing "cleaner" samples before "dirtier" samples, and interspersing "blanks" between samples
when possible or practical. Obviously, the intentional segregation of low and high concentration samples
defeats the purpose of the randomized design. Interspersing blanks between the samples can be effective,
as well as blocking similar concentrations together and randomizing blocks. But in order to ensure that
the blanks do not have other effects on the results, blanks would be needed between each sample or block
analysis, and this would greatly increase the cost of the study (e.g., 25 samples and 24 blanks would be
required in case of pure randomization). Although this was done for the Episode 6000 study, this
approach would not be practical in most cases. Therefore, despite the statistical benefits, in practice,
randomization of the sample analysis sequence can be difficult to apply in detection and quantitation limit
studies.
In the Agency's studies of variability as a function of concentration discussed in Sections 1.3.2.1 -
1.3.2.3 of this document, EPA chose to use a non-random design to avoid carry-over problems and to limit
the potential difficulties with measurements at very low concentrations. For example, if there was no
instrument response at concentration X, then it would be unlikely that there would be a response at a
concentration of X/2. In the non-random design, EPA permitted the analyst to stop analyses of ever-
lower concentrations, whereas a randomized design would have required that all the samples be analyzed,
even when there was no instrumental response for many of those samples.
3-35
-------
One of the peer reviewers evaluating EPA's 2003 assessment commented that the effects of carry-
over could have been mitigated by studying variability around the calibration line rather than the mean of
the replicates. However, carry-over affects subsequent samples differently. The effect of the carry-over
cannot be mitigated, regardless of whether variability is studied around the calibration line or the mean of
the replicates, unless the amount of carry-over is known and can be subtracted from the affected (low-
concentration) sample. This subtraction has limitations because of error accumulation and because the
amount of carry-over cannot be determined precisely without extensive studies at multiple concentrations.
Completeness
The physical measurement process can be studied using rough approximations or it can be studied
more rigorously. A rough approximation could use the available components of a method as applied to
convenient samples. A more rigorous study would use a complete, specific, and well-defined
measurement method with all sample processing steps. The appropriate level of study will probably
depend on the purpose of the study.
Measurement procedures (methods) may be more or less strictly designed. Variability in what is
allowed in the procedures may add to variability in the measurement results. To the extent that
permutations of a method's procedures are not expected to be used in a particular detection or quantitation
study, EPA recommends that this information be included in the report on the study results. While there
may be physical/chemical reasons for extrapolating the results of a variability study on one set of
procedures to permutations of those procedures, there is no statistical basis for making such an
extrapolation. Statistical theory by itself is only able to describe conditions that have been observed. On
the other hand, a knowledge of the underlying physics of the measurement process can guide the
completeness of the modeling process when statistical procedures fail. For example, the Rocke and
Lorenzato model in the linear or log-log domain may be the best general characterization of a physical
measurement process. Therefore, this model can be applied to data to produce a complete answer when
statistical procedures fail to deduce the "correct" model.
3.3.7.2 Criteria for the Selection and Appropriate Use of Statistical Models
Detection and quantitation limits maybe based on statistical models of the relationship between
measurement variation and the concentration of a substance in the sample. Results are produced by
adding varying known amounts of the substance to the sample ("spiking"), making replicate measurements
at each concentration, and modeling the variability of the results as a function of concentration. This
section summarizes the history of modeling variability versus concentration, considers criteria for
selecting models, and discusses current practices with regard to available data.
3.3.7.2.1 Short History of Modeling Measurement Results
Over time, a number of different models have been used to estimate measurement variation.
Currie (1968) modeled variation in radiochemical measurement methods using a procedure associated
with counting large numbers of distinct objects which are appropriately modeled with the Poisson
distribution. However, he relied on large sample sizes and standard normal distributions to describe all
other types of measurement methods. Hubaux and Vos (1970) developed a procedure based on an
estimated calibration relationship that uses smaller sample sizes to estimate Currie's detection and
quantitation limits. Again, measurement results were assumed to follow standard normal distributions, but
it was also assumed that measurement variation was constant throughout the range of interest. Similarly,
3-36
-------
Glaser et al. (1981) suggested that measurement variation increases linearly with concentration, but they
did not provide estimators under this theory because they believed that measurement variation is usually
approximately constant in the range of detection. Glaser et al. (1981) did suggest that, when appropriate
data were available, a linear regression analysis of the relationship over the analytical range be performed.
Clayton et al. (1987) discussed transforming the measurement results (using logarithms or square root
functions). Gibbons et al. (1991) suggested that measurement variability may be proportional to
concentration. Rocke and Lorenzato (1995) proposed a model motivated by physical characteristics of
measurement processes, in which measurement variability is approximately constant at low
concentrations, but changes in a continuous mathematical manner to a relationship where variability
increases as concentration increases.
Figure 3-2 illustrates the fundamental analytical measurement models in linear and logarithmic
domains. The models are applicable to nearly all analytical measurements; we will not deal with the
exceptions because they represent a small percentage of cases. As can be seen from the top two graphs,
response is a linear function of concentration in both the linear and log domains. The middle two graphs
and the bottom two graphs are those most pertinent to the discussion of detection and quantitation.
3-37
-------
Concentration
Linear Domain
100 , ,
O
a.
0 33 <0 tO SO 100
Qsnoentration
vs. Concentration
Log-Log Domain
100 -,
10 -
Of
(A
c
o
Q.
3 o-i-I
CC
QJOI -
DflOl
OJOt O.I 1 10 100 1000
Concentration
SD vs. Concentration
Linear Domain
10 -,
4. -
0 2! 10 60 80 100
Concentration
10D ,
10 -
SD v& Concentration
Log-Log Domain
OD1
Qfil 0.1 1 10 tQO 1QQQ
Concentration
RSD vs. Concentration
Lit ear Domain
80 -
*Q -
20 -
0
0 30 40 60 m 100
Concentration
RSD v& Concentration
Log-Log Domain
1000 ,
10 -
001 0.1 1 10 100 1000
Concentration
3.3.7.2.2
Figure 3-2
Detection Limits Using Variability at Low Concentrations
The middle two graphs in Figure 3-2 show variability versus concentration and show the model
postulated by Rocke and Lorenzato. The flat (constant) portion of the graph in the linear domain is
difficult to see because it occurs near the origin, but it can be seen easily in the log domain. Most
detection approaches (e.g., Currie's critical value and detection limit; EPA's MDL; the ACS LOD) are
constructed assuming that the flat (constant) region of the variability versus concentration relationship
holds true, although the graph is rarely displayed (a horizontal line would be singularly uninteresting).
Detection approaches such as Currie's critical value, detection limit, LOD, and MDL are constructed by
multiplying the standard deviation in the flat region by some constant.
3-38
-------
Contention and differences of opinion occur in determining how to arrive at an "appropriate"
standard deviation and what to do with the standard deviation when you have it. Currie's critical value
and EPA's MDL use a multiple of the standard deviation in a similar manner (a ^-statistic adjusted for the
number of replicates used for Currie's critical value; 3.14 for 7 replicates in EPA's MDL). The IDE uses
an additional upward adjustment based on a statistical tolerance limit calculation.
3.3.7.2.3 Quantitation Limits Using Standard Deviation Multiples and Models of Standard
Deviation versus Concentration and RSD versus Concentration
Both the limit of quantitation (LOQ) advanced by Currie and the American Chemical Society's
Committee on Environmental Improvement and EPA's minimum level of quantitation (ML) result from
multiplication of the standard deviation by a factor of 10, again assuming a flat portion of the variability
versus concentration graph. This factor of 10 is directed at achieving a relative standard deviation (RSD)
of 10 percent. An advantage of this approach is that a quantitation limit is produced, regardless of what
the RSD turns out to be.
For example, it is known that the determination of 2,4-dinitrophenol by EPA Method 625
produces highly variable results and that 10 percent RSD cannot be achieved at any concentration level for
this compound. Multiplying the standard deviation of replicate measurements of low-level samples results
in a quantitation limit that is considerably higher than the quantitation limits for other compounds
measured by Method 625. The RSD at this quantitation limit could be 30, 50, or 70 percent. Limiting the
RSD associated with the quantitation limit to some arbitrary value (e.g., 30%, as with the ASTM IQE)
could prohibit the use of EPA Method 625 for determination of 2,4-dinitrophenol. If 2,4-dinitrophenol
were present at a high concentration in a discharge, it would not be reported. Although it could be argued
that a more precise method should be used for determination of 2,4-dinitrophenol, determination of
pollutants by a large suite of different methods would be quite costly with little meaningful benefit.
Increasing precision (i.e., decreasing measurement error) would be critical only if the concentration at
issue was near enough to a compliance limit that measurement error could influence the compliance
determination. On the other hand, having widely varying RSDs for different analytes within the same
method may be confusing to permitting and enforcement authorities who may not appreciate the subtleties
of reporting violations in light of the underlying RSDs.
Another means of arriving at a limiting RSD is to graph RSD versus concentration, as shown in
the bottom two graphs of Figure 3-2. This approach is used by the ASTM IQE. It has the advantage that a
model is fit to data, rather than using a point estimate such as the Currie and ACS LOD or the EPA ML.
However, this approach requires considerably more data than are necessary for approaches based on point
estimates. In addition, how a model is selected can play a major role in the outcome.
3.3.7.2.4 Criteria for Selecting Models
Both statistical and graphical procedures have been proposed for selecting between models for
predicting measurement results based on spike concentrations.
Statistical Criteria
While statistical criteria are available for choosing between models of similar types, the currently
available criteria are not satisfactory for choosing between the wide variety of models considered for the
relationship between measurement variation and spike concentration, based on EPA's studies. More
technically, statistical criteria include using: (1) the simplest model to obtain statistical significance, (2)
3-39
-------
the model with the smallest estimated variability, and (3) the model with the smallest likelihood ratio.
Given the wide variety of models considered for detection and quantitation, there are problems associated
with each of these procedures. Data that obviously do not follow the model may produce statistically
significant results, variability may be estimated with weights that make the various estimates
incomparable, and the likelihood function may not be comparable between models.
Graphical Criteria
Graphical criteria may be susceptible to some subjectivity in their application, but they are
currently the best available method for choosing between models. At the most basic level, the primary
graphical criteria is for the form of the model to be suggested by the available data. To consider the
quality of the graphical analysis, it is useful to see if some small number of data are overly influential in
determining if a model does or does not fit. Given the ability of the human eye to discern deviations from
a straight line rather than from a curved line, a useful technique is to plot the data so that they will indicate
a straight line if they follow the model of interest.
Both graphical and statistical criteria will be strongly affected by the number and choice of spike
concentrations used to fit the different models. Too few spike concentrations will lessen the statistical
power of significance tests for slope and curvature from which decisions on the type of model will be
made. In addition, the amount of subjectivity with which decisions are made using graphs increases when
fewer concentration levels are used. For example, the judgement of whether a residual plot depicts
"random scatter" is essentially impossible when only five concentration levels are used (i.e., the residual
plot will include only five points). The number of results from which standard deviations are calculated
will also have an effect on how models are selected. This set of results may include analysis of multiple
replicates at a single laboratory or analysis of one or more replicates from multiple laboratories. If data
are obtained from too few laboratories or replicates, the standard deviation estimates will be less reliable,
which could lead to incorrect model selection based on statistical or graphical criteria.
3.3.7.2.5 Assessment of Current Models
EPA plotted variability versus concentration data to evaluate the extent to which real data from
measurement methods used under the Clean Water Act would conform to a number of different models.
For details of how data sets were selected and how data were collected within the data sets, see Appendix
B, Characterizing Measurement Variability as a Function ofAnalyte Concentration for a Variety of
Analytical Techniques, of the February 2003 Technical Support Document (EPA-821-R-03-005, February
2003). Four sets of composite scatter plots for all combinations of analytical technique, analyte, and study
were produced. These sets include:
1. Measurement versus Spike Concentration,
2. Log Measurement versus Log Spike Concentration,
3. Observed Standard Deviation versus Spike Concentration,
4. Log Standard Deviation versus Log Spike Concentration, and
5. Relative Standard Deviation (RSD) versus Log Spike Concentration.
There are hundreds of scatter plots in each set, sorted by the source, measurement technique, and
study. The first set of scatter plots can be used to evaluate how well measurement results match the
spiked concentration. If the assumed straight line model is true, then the relationship outlined by the
plotted data will be approximately linear. These relationships are plotted using log-log plots so that small
deviations from the straight line can be visualized easily. All the graphs are contained in attachments to
3-40
-------
Appendix B of the Technical Support Document (EPA-821-R-03-005, February 2003).
The plot of observed standard deviations versus spike concentrations can be used to evaluate the
reasonableness of the constant variation and/or linearly increasing variability models (Currie, 1968,
Hubaux and Vos, 1970, and Glaser et al, 1981). If the constant model for standard deviation is true, there
would be no apparent relationship between the standard deviation and spike concentration. If the straight-
line model for standard deviation is true, plots are expected to indicate an approximately linear
relationship. Analogously, the standard deviation/spike concentration versus spike concentration is
expected to show a straight-line relationship when variability is proportional to the spike concentration
(Gibbons et al., 1991). The log-log plots of standard deviation versus spike concentration are expected to
indicate if log or square root transformations may be appropriate (Clayton et al., 1987) or to display a
shape that approximates a "hockey stick" when it is appropriate to use the model proposed by Rocke and
Lorenzato (1995). With the Rocke and Lorenzato model, variability near zero will be approximately
constant, but will increase proportionally with concentrations in the higher concentration range.
Because the large number of resulting plots makes it difficult to draw general conclusions, for the
most part, conclusions must be considered on a case-by-case basis.
3.3.7.3 Methodology for Parameter Estimation
Along with various approaches of detection and quantitation and models for measurement, a
number of specific procedures have been suggested for estimating model parameters. Maximum
likelihood and least squares are two generally applicable statistical methods that can be used in estimating
model parameters. There are advantages and disadvantages to both that must be weighed in particular
cases. A standard statistical practice for evaluating the quality of an estimation procedure is to calculate
the precision and bias, usually best understood by examining a plot of residuals from a fit to a function.
All else being equal, the estimation procedure resulting in the greatest precision and least bias is preferred.
In some cases, precision and bias can be calculated based on the assumptions behind the estimation
procedure. In other cases, it is either necessary or convenient to estimate precision and bias using
simulations. From a general theoretical perspective, the maximum likelihood estimation methodology is
preferable because it generates estimates that are generally best with regard to properties of precision and
bias (especially for larger sample sizes), while also being approximately normally distributed.
Unfortunately, maximum likelihood methodology sometimes can be problematic because the method
requires the solution of complex equations. Least squares estimation is generally more tractable, and thus
is more generally applicable, although the estimates that result may not be as desirable from a theoretical
statistical perspective.
What can sometimes be overlooked in considering estimation and model fitting is that direct
measurement of variation of the blank or low level concentration may be the most cost-effective and least
difficult method to implement especially where variability does not change much over the region of
interest.
3-41
-------
Chapter 4
Evaluation Criteria
This chapter presents the criteria developed by EPA as a means for evaluating and selecting
acceptable detection and quantitation limit approaches for use in Clean Water Act (CWA) programs.
These criteria reflect EPA's careful consideration of the issues identified and discussed in Chapter 3,
including EPA's needs under CWA programs. A total of six criteria were established, and are discussed in
Sections 4.1 - 4.6. The six evaluation criteria are:
Criterion 1: The detection and quantitation limit approaches should be scientifically valid.
Criterion 2: The approach should address demonstrated expectations of laboratory and method
performance, including routine variability.
Criterion 3: The approach should be supported by a practical and affordable procedure that a
single laboratory can use to evaluate method performance.
Criterion 4: The detection level approach should identify the signal or estimated concentration at
which there is 99% confidence that the substance is actually present when the analytical method is
performed by experienced staff in well-operated laboratories.
Criterion 5: The quantitation limit approach should identify the concentration that gives a
recognizable signal that is consistent with the capabilities of the method when a method is
performed by experienced staff in well-operated laboratories.
Criterion 6: Detection and quantitation approaches should be applicable to the variety of decisions
made under the Clean Water Act (CWA), and should support state and local obligations to
implement measurement requirements that area at least as stringent as those set by the Federal
government.
Section 4.7 presents additional principles recommended by stakeholders commenting on EPA's
assessment.
4.1 Criterion 1
Criterion 1: The detection and quantitation limit approaches should be scientifically valid.
The concept of scientific validity is widely accepted but loosely defined. For the purposes of this
evaluation, a detection/quantitation approach or methodology will be considered scientifically valid if it
meets the following conditions:
• It can be (and has been) tested,
It has been subjected to peer review and publication,
• The error rate associated with the approach or methodology is either known or can be estimated,
Standards exist and can be maintained to control its operation (i.e., it is supported by well-defined
procedures for use), and
It has attracted (i.e., achieved) widespread acceptance within a relevant scientific community.
4- 1
-------
While EPA acknowledges that other measures could be established to demonstrate scientific
validity, EPA has adopted the conditions cited because they reflect those discussed by the U.S. Supreme
Court as pertaining to assessments of scientific validity when considering the admissibility of expert
scientific testimony2. These conditions also are directly relevant to EPA's needs.
Some stakeholders supported the use of objective criteria for determining scientific validity, but
questioned the appropriateness of using criteria that were designed for courts and juries to support
scientific decisions made by scientific experts. EPA carefully reviewed the Court's conditions for
demonstrating the scientific validity of an expert's reasoning or methodology, and believes that these
conditions are appropriate for demonstrating the scientific validity of any scientific approach or
methodology, including those that might be used to establish detection and quantitation limits under
CWA. EPA further believes these criteria are consistent with the EPA Science Policy Council's
assessment factors for evaluating the quality of scientific and technical information (EPA 100/B-03/001,
June 2003), including the extent to which technical information and data are peer reviewed and
appropriately tested. However, EPA is willing to consider alternative or supplemental criteria for
evaluating scientific validity as it moves forward with the stakeholder process.
Stakeholders agree that detection and quantitation levels should be based on sound scientific
principles, and note that low-cost and/or simple approaches should not be selected if inaccurate or
unmeasurable limits may result. Stakeholders also noted that some of the conditions listed above (e.g., the
condition that an approach or methodology should have attracted widespread acceptance within a relevant
scientific community) have the potential for favoring concepts already adopted and required by regulatory
agencies. EPA agrees that this is a valid concern, and therefore, will consider the overall validity and
practicality of new approaches.
4.2 Criterion 2
Criterion 2: The approach should address realistic expectations of laboratory and method
performance, including routine variability.
As discussed in Chapter 3 of this Assessment Document, the detection and quantitation limit(s)
for an analyte in an analytical method can be established from a single-laboratory study, multiple single-
laboratory studies, or an interlaboratory study.
Early methods developed by EPA under Clean Water Act programs, and nearly all methods
developed by EPA under Safe Drinking Water Act programs, were developed by an EPA research
laboratory in Cincinnati, Ohio with specialized experience in the analytical chemistry of drinking water.
This laboratory also established method detection and quantitation limits which, in many instances,
initially could not be achieved in other laboratories. Over time, however, the difficulty in achieving these
limits was overcome as analysts gained experience with the use of these new methods.
Stakeholders have suggested that detection and quantitation limits be developed using data from
multiple laboratories in order to account for the routine inter- and intra-laboratory variability that can
occur over time. Although compliance measurements are made in single laboratories, EPA agrees that
detection and quantitation limits in methods that will be widely used by many laboratories should consider
2Daubert v. MerrellDow Pharmaceuticals, 509 U.S. 579 (1993) and Kumho Tire Co. v.
Carmichael, 526 U.S. 137 (1999)
4-2
-------
these sources of variability. For this reason, after the development in a single laboratory of a new or
modified analytical method with an initial estimate of detection and quantitation limits, EPA's Office of
Science and Technology evaluates and verifies these limits in multi-laboratory studies.
Voluntary consensus standards bodies (VCSBs) such as ASTM International have historically
used interlaboratory studies to establish method performance. Over the past 5 to 10 years, ASTM
International has been developing interlaboratory and single-laboratory approaches for detection and
quantitation. Single-laboratory studies at a specialized research laboratory may produce detection and
quantitation limits that are lower than those produced by studies that gather data from many laboratories
that may or may not be experienced with the method. EPA believes that a realistic expectation of method
and laboratory performance likely lies somewhere in between that provided by a specialized single-
laboratory study and that provided by an interlaboratory study with no pre-qualification requirements.
Estimates of detection and quantitation limits should consider the inherent variability of the measurement
process, but not be based on the lowest common denominator, e.g., data from inexperienced or unqualified
analysts and laboratories.
EPA expects that laboratories must meet some minimum standards of performance and experience
with a method, and sets performance criteria in methods. Examples of such criteria include measures to
demonstrate that a laboratory is producing accurate results at a concentration of interest (i.e., analysis of
reference standards or spiked samples), measures to demonstrate that results are not biased by
contamination (i.e., analysis of blanks), and measures to demonstrate that the laboratory can detect
pollutants at low concentrations (i.e., at the method detection limit). It is likely that laboratory
performance will improve (and variability will be lower) when laboratories are required to meet specified
performance criteria in order to report results.
A further consideration concerning routine variability of laboratory performance is the means for
rejection of outliers to more accurately estimate routine variability. True outliers can occur in laboratory
data, and some means of resolving outlier issues should be included. Statistical procedures are available
for the identification of candidate outlier values. Once a candidate outlier has been identified, evaluation
of the value from a QA/QC perspective (e.g., some procedural error or quality control error has occurred)
should be the basis of exclusion of the value from a data set. In cases where no cause for the outlier has
been identified, it may reasonable to reject an outlier on statistical grounds, but every effort should be
made to justify the exclusion on technical grounds.
In examining each approach against this criterion, EPA will evaluate whether the approach can be
used to provide realistic expectation of laboratory performance. As part of this assessment, EPA will
examine the sources of variability captured by the approach, and the degree to which the statistics that
underlie the approach realistically reflect these sources of variability.
4.3 Criterion 3
Criterion 3: The approach should be supported by a practical and affordable procedure that a single
laboratory can use to evaluate method performance.
Any approach or procedure for determining detection and quantitation limits at a single laboratory
should be simple, with detailed instructions, and cost-effective to implement (i.e., it should be reliable and
"laboratory-friendly"). Laboratories that use detection or quantitation procedures range from large
laboratories and laboratory chains with a wide range of technical capabilities, to small laboratories
operated by one or a few people with limited statistical skills. While this range of laboratory capability
4-3
-------
places a premium on simplicity and ease, EPA agrees with stakeholders that data reliability and quality are
also important. A suitable approach or procedure for detection and quantitation incorporates the right
balance between the need for valid data and the need for the procedure to be simple and inexpensive to
perform. EPA also believes that if a procedure is complicated, it will be prone to error in use. Similarly,
if a procedure requires investment of extensive resources that cannot be billed to the client, laboratories
will have a disincentive to use the procedure. Therefore, if EPA wishes to encourage development and
use of innovative techniques that improve measurement performance or lower measurement costs, the
Agency should consider practicality and affordability as significant, if not equal, considerations to
scientific validity.
After evaluating each of the issues discussed in Chapter 3 of this document, EPA concluded that
successful implementation of CWA programs depends on the ability of laboratories to easily and
affordably:
• demonstrate that a method works in a particular matrix at the levels of concern (i.e., demonstrate the
absence of matrix effects),
• characterize improvements in measurement capabilities in terms of detection and quantitation
capabilities, and
• characterize the detection and quantitation capabilities of new methods.
A matrix effect is an interference in a measurement that is caused by substances or materials in
the sample other than the analyte of interest that are not removed using the procedures in the method or
other commonly applied procedures. In the context of detection and quantitation, matrix effects may
manifest themselves by precluding measurements at levels as low as could be measured were the
interference not present. From a practical perspective, it is not possible to test the detection and
quantitation capability of an analytical method in every possible matrix in which it may be used. At a
minimum, it is unlikely that EPA or any other organization or laboratory could possibly identify and
obtain samples of every matrix to which the method might be applied, and even if such a feat were
possible, the cost and logistics of doing so would be prohibitive.
The situation for characterizing matrix effects on detection and quantitation is similar to the
situation for characterizing matrix effects on measurement performance at higher concentration levels. In
the latter case, EPA typically uses one or more spiked real-world or reference matrices (e.g., reagent
water, sand, diatomaceous earth) to establish QC acceptance criteria that verify performance of the
method at mid-to-high concentrations. Each analytical method includes QC acceptance criteria for such
real-world and reference matrix spikes, along with a suite of quality control requirements designed to
verify that failures are attributable to the matrix rather than to an analytical system that is out of control.
EPA would prefer to utilize detection/quantitation concepts that allow for similar characterization of
detection/quantitation capabilities in representative matrices and that are supported by simple, cost-
effective procedures that would allow individual laboratories to evaluate the effects of specific matrices
on these capabilities on an as needed basis. Because methods approved at 40 CFR part 136 already
contain a suite of quality control procedures and QC acceptance criteria that control laboratory
performance, EPA believes that it is not necessary to verify detection and quantitation limits in each and
every batch of each and every matrix analyzed. Rather, such testing can be done on an as-needed basis
when it is suspected that matrix interferences may preclude reliable measurements at low levels.
4-4
-------
Another consideration influencing the need for simplicity and practicality is that measurement
capabilities generally improve over time. As is discussed in Section 3.1 of this document, and as has been
noted by stakeholders, this is attributable to a variety of factors, including:
• increased staff experience with a given technique,
• technological upgrades or improvements in the instrumentation used for analysis, and
• development of new instrumentation or techniques that improves detection/quantitation, precision, or
bias.
In each case, the improvements may not be observed across the entire laboratory community. In the case
of increased staff experience, for example, it is obvious that a laboratory that specializes in one type of
analysis, such as low-level mercury measurements, will develop greater experience with these analyses
than a laboratory that rarely performs these measurements. Likewise, it is easy to see how one or a few
laboratories that concentrate their business on a particular type of analysis might be willing to invest
significant resources in new or upgraded equipment to improve performance, whereas laboratories that
rarely perform such analyses would not find such upgrades to be cost-effective.
Improvements in measurement capability, including the development of new methods, may create
a dynamic decision-making process, in that measurements at lower levels may allow EPA and States to
identify and measure previously undetected pollutants. Such improvements offer a means for monitoring
and controlling (i.e., regulating) the discharge of previously unregulated, but harmful, pollutants.
Therefore, it is in the best interest of the environment for EPA to encourage the development and use of
improved environmental analysis procedures and equipment by providing practical and affordable
procedures for evaluating method performance.
In evaluating this criterion, EPA will favor affordable and easy-to-use approaches and procedures
that allow analysts to 1) determine matrix-specific variations when necessary, based on realistic data, and
2) demonstrate lower detection and quantitation limits associated with improvements in measurement
capabilities. Procedures for establishing the detection capabilities of new methods or associated with
improved measurement capabilities should be practical enough to encourage such development. However,
EPA recognizes that some uses for detection and quantitation limits may require a more comprehensive
approach involving multiple laboratories. These procedures should specify the nature, minimum number,
and concentration levels of the samples to be used, and the corrective action to be taken if the resulting
detection or quantitation limit is inconsistent with the data from which it is derived.
4.4 Criterion 4
Criterion 4: The detection level approach should estimate the theoretical concentration at which there
is 99% confidence that the substance is actually present when the analytical method is
performed by experienced staff in a well-operated laboratory.
Any approach to establishing levels at which detection decisions are made should be capable of
providing regulators, the regulated community, and data users with a high level of confidence that a
pollutant reported by a well-operated laboratory as being present really is present. Historically,
approaches to making detection decisions have set the criterion for detection at 99 percent confidence
(i.e., with 99% confidence that the analyte concentration is greater than zero). This criterion results in the
probability of a false positive i.e., that a pollutant will be stated as being present when it actually is not
(this is a Type I error), of one percent. The procedure also should be capable of generating a detection
level when the substance of interest is not present in a blank and/or when instrument thresholds are used
4-5
-------
in routine operation. A well-operated laboratory is a laboratory that routinely monitors performance
through QC analyses, control charts, and other measures to rapidly identify and correct deteriorating or
poor performance, and with analysts experienced with method sample preparation, analysis, and detection
procedures.
In evaluating this criterion, EPA will favor approaches and procedures that reflect routine
analytical conditions in a well-operated laboratory.
4.5 Criterion 5
Criterion 5: The quantitation limit approach should identify the concentration that gives a
recognizable signal that is consistent with the capabilities of the method when a method is
performed by experienced staff in well-operated laboratories.
Measurement capabilities among laboratories vary depending on a number of factors, including,
but not limited to, instrumentation, training, and experience. Similarly, measurement capabilities among
different analytical methods vary depending on a number of factors, including the techniques and
instrumentation employed and the clarity of the method itself. In evaluating different approaches to
estimating quantitation limits, EPA will give preference to those approaches that strike a reasonable
balance between using either state-of-the art laboratories or a highly varied community of laboratories to
establish quantitation limits.
Historical approaches to recognizing laboratory capabilities in establishing detection and
quantitation limits have varied between two extremes of establishing the limit in a state-of-the-art research
laboratory to reflect the lowest possible limit that can be achieved, and establishing the limit based on
statistical tolerance intervals calculated from a large number of laboratories with varying levels of
experience, instrumentation and competence. Generally, use of the former has been employed to serve as
a goal or performance standard to be met by other laboratories, whereas use of the latter treats the limit,
not as a performance standard that needs to be met by each laboratory, but rather as a characterization of
the performance of the capabilities of a population of laboratories at the time of method development.
Historical approaches to recognizing method capabilities also have varied between those that
allow the error expressed as relative standard deviation, or RSD among low-level measurements to vary,
depending on the capabilities of the method, and those that fix this error (RSD) at a specific level.
Initially, Criterion 5 stated that the "quantitation limit should identify a concentration at which
the reliability of the measured result is consistent with the capabilities of the method when a method is
performed by experienced staff in a well-operated laboratory. " Reviewers from within EPA questioned
the criterion's implication that measurements below a quantitation limit could be considered unreliable. A
similar concern was expressed by one of the peer reviewers charged with evaluating EPA's assessment
and an earlier draft of this Assessment Document. This reviewer noted that:
"almost all implementations of limits of quantitation have nothing to do with whether the
measurements are actually quantitative, " and that "any level at which the instrument can
be read, and at which there is a reliably estimated standard deviation is a level at which
quantitation is possible " (Rocke, 2002)
4-6
-------
The peer reviewer suggested that Criterion 5 might be rewritten as:
"the quantitation limit should identify a concentration at which the instrument yields a
measurable signal at least 99% of the time, and which is no smaller than the detection
level. Such a quantitation limit will often be the same as the detection level. "
EPA agrees that this is a valid perspective, in that if the pollutant is identified and the analytical system
produces a result (i.e., a measurable or recognizable signal), quantitation occurs. Although this
interpretation of a quantitation limit has validity, implementation of such an approach would require that
all values generated by an analytical system be reported, along with an estimate of the uncertainty
associated with each value (e.g., the "reliably estimated standard deviation" mentioned by the peer
reviewer). As noted in Section 2.3.4, several organizations, including the European Union, are developing
procedures for estimating the uncertainty associated with measured results. If successful, such an
approach would eliminate many of the data censoring concerns discussed in Section 3.3.5. Given the
difficulty in achieving consensus on an appropriate means of establishing a quantitation limit, however,
EPA believes that it would also be difficult to obtain consensus on an appropriate means for estimating the
uncertainty associated with each result measured on each environmental sample. In addition, analytical
chemists have used and perceive that they understand a quantitation limit to mean the lowest
concentration at which an analyte can be identified and quantified with some degree of certainty. This
understanding necessarily involves use of the sound judgment of a qualified analytical chemist.
Therefore, EPA will continue to monitor developments on this subject, and if appropriate, re-
evaluate this issue if and when it becomes practical and widely accepted by the laboratory, regulatory, and
regulated communities. In the meantime, EPA believes that the traditional approach of defining a
quantitation limit at some level above the detection limit provides a data user with a reasonable degree of
confidence in the measured value without requiring that laboratories develop and report individual
estimates of uncertainty. Criterion 5 reflects this belief.
In evaluating the approaches, EPA will give preference to those approaches that strike a
reasonable balance between using either state-of-the art laboratories or a highly varied community of
laboratories to establish quantitation limits.
4.6 Criterion 6
Criterion 6: Detection and quantitation approaches should be applicable to the variety of decisions
made under the Clean Water Act, and should support State and local obligations to
implement measurement requirements that are at least as stringent as those set by the
Federal government.
The Clean Water Act requires EPA to conduct, implement, and oversee a variety of data gathering
programs. As noted in Section 3.2 of this Assessment Document, these programs include, but are not
limited to:
• Survey programs to establish baselines and monitor changes in ambient water quality,
• Screening studies to identify emerging concerns and establish the need for more in-depth assessment,
• Effluent guideline studies to establish technology-based standards for the control of pollutants in
wastewater discharges,
4-7
-------
• Toxicity and environmental assessment studies to establish water quality-based standards for the
control of pollutants in wastewater, and
• Risk assessment studies designed to characterize and evaluate human health and environmental risks
associated with various water body uses.
In addition, EPA needs to evaluate detection limit or quantitation capabilities for methods
approved at 40 CFR part 136 for the following applications:
• Ambient and effluent permitting and compliance monitoring under NPDES and the pretreatment
program and under State and local programs,
• Quality control in analytical laboratories, and
• Method development, promulgation, and modification.
In theory, EPA could evaluate each of these applications independently and identify a detection and
quantitation limit approach that is best suited to each application, as recommended by some stakeholders
commenting on EPA's assessment. In the 2003 assessment, EPA stated that this would increase
confusion, record keeping burdens, and laboratory testing burdens. EPA also stated that data generated
under a single procedure can be used for development of detection and quantitation limits that are
applicable to more than a single use. For example, the data used to determine the capabilities of multiple
laboratories using a given method may also be used to develop method-specific detection and quantitation
limits. For these reasons, EPA recommended the adoption of a single pair of related detection and
quantitation procedures used to address all or most Clean Water Act applications. Some stakeholders
recommend the use of different approaches for different CWA applications. For example, these
stakeholders would prefer a more rigorous approach to determining detection and quantitation limits for
method development than for verifying laboratory performance. They would like to include a procedure
that is based on a multilaboratory approach rather than a single laboratory approach to define detection
and quantitation capabilities of analytical methods. EPA recognizes that the complexity and statistical
rigor appropriate for a detection and quantitation approach for method development and validation would
be greater than that needed for demonstrating laboratory proficiency. EPA plans to seek additional
stakeholder input on whether different approaches are needed for different CWA purposes (see Chapter
6).
Although EPA prefers to identify a manageable set of detection and quantitation limit approaches to
meet CWA needs, EPA believes that any reasonable approach advanced by other organizations should be
acceptable for use provided it meets the needs of the specific application for which it would be used.
Allowing use of detection and quantitation approaches developed by other organizations provides the
stakeholder community with increased measurement options that may help reduce measurement costs or
improve measurement performance for specific situations. This approach also is consistent with the intent
of the National Technology Transfer and Advancement Act.
The Clean Water Act authorizes State or local governments to implement specific aspects of the Act,
with the provision that they do so in a way that is at least as protective (i.e., stringent) as the national
standards put forth by EPA. Therefore, this criterion is intended to ensure that any detection and
quantitation limit approach adopted by the Office of Water is sufficiently clear and defined to ensure
consistency with approaches adopted by State or local governments.
Finally, it is important to differentiate between detection and quantitation limit approaches and
compliance evaluation thresholds. Detection and quantitation limit approaches pertain to measurement
process thresholds. In contrast, compliance evaluation thresholds are used to support wastewater
-------
discharge limits established in National Pollutant Discharge Elimination System (NPDES) or pretreatment
program permits. Such limits are usually expressed as either a maximum concentration of pollutant
allowed in the discharge or a maximum mass of pollutant allowed to be discharged in a specific time
period.
Ideally, and in most cases, analytical methods are available to allow for detection and quantitation of
pollutants at concentrations that are lower than the discharge levels needed to protect or restore the quality
of the receiving water. When such measurement capability does not exist (e.g., analytical methods are not
available that can reliably measure at levels necessary to protect receiving water), permitting authorities
must decide how to evaluate and report pollutant concentrations at these levels. Historically, EPA has
recommended that in such cases, the permitting authority include the water quality-based limit in the
permit, but establish the compliance evaluation threshold at the quantitation limit of the most sensitive
available method.
In examining each approach against this criterion EPA will consider 1) the applicability of various
detection/quantitation approaches to the variety of data gathering decisions that must be made under the
CWA, including those that do and those that do not involve compliance monitoring, and 2) the ability of
the approaches to support State and local obligations for implementing the CWA. As discussed in
Chapter 6, EPA believes that additional discussion about this criterion is appropriate based on negative
comments from stakeholders regarding the use of a single pair of detection and quantitation limit
approaches to meet all CWA needs.
4.7 Consensus Principles
Some stakeholders commenting on EPA's assessment of approaches to detection and quantitation
expressed their support of a set of "consensus principles" submitted by 36 signatories representing
industry and laboratory communities. EPA agrees with certain consensus principles such as the principle
that detection and quantitation levels should be based on sound scientific principles, and that low-cost
and/or simple approaches should not be used if invalid data will result (see Criterion 1 above). As another
example, EPA incorporated routine variability, the rate of false positives, precision, and matrix effects in
several criteria, and considered these aspects in its assessment of detection and quantitation concepts.
Some of these consensus principles are included in the criteria discussed in this chapter. Other consensus
principles have clarified or highlighted existing aspects of approaches to detection and quantitation and
provide a framework for additional consideration.
For ease of consideration, the consensus principles recommended by commenters have been separated
by EPA into technical and policy considerations and include:
Technical Considerations
• Detection and quantitation levels must be based on sound scientific principles. Low-cost and/or
simple approaches must not be selected if inaccurate compliance determinations or unmeasurable
permit limits may result.
• The definition of "quantitation" must account for both precision and bias.
• Detection limit procedures must take into account the variability and bias of method blank results.
• False positives (Type I errors), false negatives (Type II errors), and precision must all be addressed by
detection concepts and reporting of analytical results for regulatory purposes.
4-9
-------
• Precision, bias, and qualitative identification (where appropriate) must all be addressed by the
definition and concepts of quantitation and by the reporting of analytical results for regulatory
purposes.
• Detection limit procedures must include procedures for ongoing demonstration of sensitivity,
preferably incorporated into the routine analytical quality control as a check against false negatives.
• Detection and quantitation levels must take into account routine inter- and intra-laboratory variability
within a laboratory over time.
• In its procedures for establishing detection and quantitation levels, EPA must develop guidance on
how to account for the effects of various matrices.
Policy Considerations
• The Lc, LD, and LQ are three distinct points, each of which has unique criteria that must be satisfied.
For consistency with international standards, EPA must adopt the definitions of Lc (critical value), Lj,
(detection limit), and LQ (quantification limit) of IUPAC (International Union of Pure and Applied
Chemistry) that are being adopted by international standards organizations (e.g., the International
Organization of Standardization (ISO)).
• The definitions of and procedures for determining detection and quantitation levels must take into
account that quantitation levels are used as regulatory compliance levels in NPDES permits.
• EPA should specify consensus standard procedures for establishing significant figures and for
rounding data.
• EPA must strive for consistency across all EPA offices (the Office of Water, Office of Research and
Development, Office of Ground Water and Drinking Water, and Office of Solid Waste and
Emergency Response) in defining and applying detection and quantitation levels.
4- 10
-------
Chapter 5
Assessment
This chapter summarizes EPA's assessment of various detection and quantitation limit approaches
against the evaluation criteria established in Chapter 4. Assessments of detection limit approaches are
presented in Section 5.1 and include an assessment of the:
• EPA method detection limit (MDL; Section 5.1.1),
• ASTM International interlaboratory detection estimate (IDE; Section 5.1.2),
• American Chemical Society (ACS) limit of detection (LOD; Section 5.1.3),
• International Organization for Standardization/International Union of Pure and Applied Chemistry
(ISO/TUPAC) critical value (CRV; Section 5.1.4),
• ISO/IUPAC minimum detectable value (MDV; Section 5.1.5),
• American Council of Independent Laboratories (ACIL) Critical Value ( ACIL Lc; Section 5.1.6),
• United States Geological Survey (USGS) Long-term Detection Limit (USGS LT-MDL; Section
5.1.7), and
• Inter-industry Analytical Group (IIAG) Sensitivity Test and Full-Range Validation Study (Section
5.1.8).
Assessments of quantitation limit approaches are presented in Section 5.2 and include an assessment of
the:
• EPA minimum level of quantitation (ML; Section 5.2.1),
• ASTM International interlaboratory quantitation estimate (IQE; Section 5.2.2),
• ACS limit of quantitation (LOQ; Section 5.2.3), and
• ISO/IUPAC LOQ (section 5.2.4).
A brief summary of the evaluation is presented in Tables 5-1 (detection limit approaches) and 5-2
(quantitation limit approaches).
EPA's 2003 assessment of detection and quantitation limit approaches focused on approaches
developed or published by ASTM International, the American Chemical Society (ACS), ISO/IUPAC, and
EPA. Stakeholder commenting on the initial assessment suggested that EPA should include additional
approaches in the next assessment. In addition to the initial four approaches, EPA has included three
additional approaches in this Revised Assessment document. These approaches are: the long-term MDL
developed by USGS, a new detection limit procedure developed by the American Council of Independent
Laboratories (ACIL), and a paired approach involving a sensitivity test and full-range validation study
submitted by the Petitioners (the Inter-industry Analytical Group). Several commenters advocated these
as approaches that more realistically reflect measurement variability. These additional approaches are
discussed and assessed in Sections 5.1.6 - 5.1.8 of this chapter.
Some stakeholders commenting on EPA's 2003 assessment believed that the evaluation criteria
used by EPA were written to favor the MDL and ML over other approaches to detection and quantitation.
EPA disagrees. The criteria were written to reflect EPA's needs for detection and quantitation approaches
under the CWA, and it is not necessary that an acceptable approach meet all of these criteria under all
conditions. Because the MDL and ML were developed to address EPA's needs, it should not be
surprising that the MDL and ML procedures generally meet the criteria EPA set out to assess detection
and quantitation procedures. EPA has frankly assessed the MDL and ML against these criteria and notes
5- 1
-------
that the MDL and ML procedures do not meet all of these criteria under all operating conditions (see
Sections 5.1.1 and 5.2.1 below). Due to the variability and unpredictability inherent in measurement
science, it is unlikely that any procedure would meet all of EPA's criteria under all conditions. However,
EPA is open to further discussions with stakeholders about the appropriateness of the evaluation criteria
described in Chapter 4, in particular, the issue of whether EPA should adopt different approaches for
different applications, as discussed in Chapter 6.
5.1 Detection Limit Approaches
Sections 5.1.1 through 5.1.8 describe EPA's assessment of eight detection limit approaches. Each
discussion is divided into two major subsections. The first subsection describes the approach and, where
applicable, the procedure that supports the approach. The second subsection details EPA's assessment of
the approach based on the five criteria established in Chapter 4 for evaluating detection limit approaches.
Note: Of the six assessment criteria in Chapter 4 four (Nos. 1, 2,3 and 6) pertain to both detection and
quantitation limit approaches. One criterion (No. 4) pertains only to detection limit approaches,
and one criterion (No. 5) pertains only to quantitation limit approaches. Therefore, the following
discussion of each detection and quantitation limit approach applies only the five applicable
criteria.
5.1.1 Evaluation of the MDL
Section 5.1.1.1 is an overview of the MDL approach and the procedures used to implement the
approach. Section 5.1.1.2 describes EPA's assessment of the MDL against the five evaluation criteria that
apply to detection limit approaches.(i.e., Criteria 1-4, and Criterion 6).
5.1.1.1 Description of the MDL Approach and Procedure
As promulgated at 40 CFR part 136, Appendix B, the MDL is defined as:
"the minimum concentration of a substance that can be measured and reported with 99%
confidence that the analyte concentration is greater than zero and is determined from
analysis of a sample in a given matrix containing the analyte."
A six-step procedure is given in Appendix B, with an optional seventh step to verify the
reasonableness of the MDL determined in the first six steps. The procedure is intended for use by
experienced analytical chemists. A brief summary of the MDL procedure is as follows:
1. The analyst makes an estimate of the detection limit based on one of four options: the instrument
signal to noise ratio; three times the standard deviation of replicate blank measurements; a break in the
slope of an instrument calibration curve; or known instrument limitations.
2. The analyst prepares a volume of reagent water that is as free of the target analyte as possible (if the
MDL is to be determined in reagent water).
3. The analyst prepares a sufficient volume of spiked reagent water (or of an alternate matrix) to yield
seven replicate aliquots that have a concentration of the target analyte that is at least equal to or in the
same concentration range as the estimated detection limit (it is recommended that the concentration of
the replicate aliquots be between 1 and 5 times the estimated detection limit).
5-2
-------
4. All of the replicate aliquots are processed through the entire analytical method.
5. The variance (S2) and standard deviation (S) of the replicate measurements are determined, as follows:
S2 -
n - 1
where:
X; = the analytical results in the final method reporting units obtained from the n sample aliquots and
2 refers to the sum of the X values from i=l to n, and
i =1 to n
6. The MDL is then determined by multiplying the standard deviation (S) by the Student's ^-statistic at a
99% percentile for n-1 degrees of freedom. If seven replicates are used, the Student's t-value is 3.143.
This information is used to calculate the MDL as follows:
MDL - V 1,1-« =0.99) C5)
where:
MDL = the method detection limit
Vi i-«= 99) = ^e Student's t-valne appropriate for a 99% confidence level with n-1 degrees of
freedom, and
S = the standard deviation of the replicate analyses.
A 95% confidence interval for the determined MDL maybe calculated from percentiles of the chi
square over degrees of freedom distribution (x2/df).
7. The optional iterative procedure to verify the reasonableness of the MDL involves spiking the matrix
at the MDL that was determined in Step 6, and analyzing another seven replicates spiked at this level.
The F-ratio of the variances (S2) is determined and compared with the F-ratio found in the table,
5-3
-------
which is 3.05. If S2A/S2B>3.05, the analyst is instructed to respike at the most recently calculated
MDL and process the samples through the procedure starting with Step 4. If S2A/S2B<=3.05, then the
pooled standard deviation is determined (S2A is the larger of the two variances). The pooled standard
deviation is then used to calculate the final MDL as follows:
MDL - 2.681 x Spooled
where 2.681 is equal to t(12 j.a =
'(12, 1-a =.99).
The 95% confidence interval around the final MDL may be determined using the chi squared
distribution.
The MDL procedure given at 40 CFRpart 136, Appendix B is described as being applicable to 1)
a wide variety of sample types, ranging from reagent water containing the analyte of interest to wastewater
containing the analyte of interest, and 2) a broad variety of physical and chemical measurements.
5.1.1.2 Assessment of the MDL Against the Evaluation Criteria
The following five subsections discuss the MDL approach and procedure in the context of the five
evaluation criteria that concern detection limit approaches (i.e., Criteria 1-4, and Criterion 6).
5.1.1.2.1 Criterion 1: The detection and quantitation limit approaches should be scientifically valid.
For the purposes of evaluating scientific validity, EPA is using the conditions discussed by the
Supreme Court in Daubert v. Merrell Dow Pharmaceuticals (1993) andKumho Tire Co. v. Carmichael,
(1999) (see Chapter 4, Criterion 1).
Condition 1: It can be (and has been) tested. The MDL procedure meets this condition. Over the years,
as stakeholders have sought to improve upon or identify alternative procedures, the MDL has been the
subject of a number of studies and comparisons, including this assessment. As a result, the MDL is one of
the most widely tested detection limit procedure in the history of detection approaches. (See Appendix A
for a list of literature references concerning the MDL and other detection limits.)
Critics of the MDL have noted that the detection limit produced with the MDL procedure can vary
depending on the spike level used. It is true that an initial MDL may be calculated using any spike level,
regardless of how high. Although a high initial spike level will result in an initially high MDL, the self-
correction check in the MDL procedure requires the final spike level to be within a certain range of the
reported (i.e. final) MDL. Specifically, Step 1 of the MDL procedure focuses the spiking level on the
lowest concentration at which measurements can be made, and the factor of 5 requirement in Steps 3 and
4 assure that the determined MDL will be at or near this concentration. Therefore, the requirements
included in Steps 1, 3 and 4 guard against an artificially high MDL being produced due to the choice of a
high initial spike level. EPA also recognizes the concern that the iterative procedure in step 7, which
provides a reality check on the results obtained in steps 1 - 6 is optional. EPA will consider whether
additional guidance on this aspect of the procedure is needed.
5-4
-------
In preparation for the assessment of detection and quantitation approaches, EPA tested the MDL
procedure with 10 different techniques, at decreasing spike concentrations, to evaluate this concern and
determine how well the procedure characterized the region of interest. Results of the study suggest that,
although the calculated MDL could vary depending on the spike level used, the MDL procedure is capable
of reasonably estimating the lowest level at which measurements can be made when the factor of 5
requirement is met.
One of the stakeholders commenting on EPA's 2003 assessment suggested that the MDL failed to
meet this condition because EPA should have tested it in "real world" matrices. EPA does not agree with
this suggestion for several reasons. First, it is not practical or possible to test detection limits in every real
world matrix, and there is no consensus as to which real world matrix would represent an appropriate real
world matrix for testing. Second, many real world matrices contain the target pollutant at levels well
above the detection or quantitation limit, making it impossible to characterize what can and cannot be
detected at low levels. In theory, the sample could be diluted to dilute the target pollutant, but in practice
sample dilution would also likely dilute any interferences that might be present, thereby defeating the
purpose of using a real world matrix. The current EPA approach, which exhaustively tests the MDL
procedure in a reference matrix using multiple techniques and ten different concentrations that span the
entire region of interest, is more than adequate to constitute "testing" of the MDL procedure. On the other
hand, where data suggests that matrix interferences may significantly affect achievable quantitation and
detection limits, this should be considered by a permit writer on a case by case basis.
Condition 2: It has been subjected to peer review and publication. The MDL meets this condition. Prior
to promulgation by EPA, the MDL approach and supporting procedure was published by Glaser et al. in a
peer-reviewed journal (Glaser, et al., 1981). The MDL procedure has been included at 40 CFR part 136,
appendix B since 1984. Values resulting from this procedure have been included, published, and tested in
many analytical methods since promulgation, including methods published by EPA and other Federal
agencies, and by consensus standards organizations and trade associations such as ASTM International,
and APHA, AWWA, and WEF.
Condition 3: The error rate associated with the procedure is either known or can be estimated. The error
rate is specified by a, with a suggested value of 0.01 (1%). Therefore, the MDL meets this condition. In
addition, the Step 7 of the MDL procedure suggests calculating a 95% confidence interval for the
determined MDL, providing additional estimation about the uncertainty (i.e., error) of the MDL
determined using the procedure.
The US Geological Survey (USGS) provided a dataset of spiked and blank sample data that EPA
used to evaluate the error rate associated with the MDL. (Error rates associated with the ACIL and USGS
detection limit procedures also were evaluated and are discussed in Sections 5.1.6 and 5.1.7.) Although
the sample size was insufficient to conclusively demonstrate the error rate of the MDL, the results suggest
the actual error rate is close to the intended 1%. In this case, the observed mean error rate was 2.9%.
Readers are referred to Appendix B for a discussion of two factors affecting this estimate - relatively small
sample size and some added long-term variability.
In the 2003 assessment, EPA suggested deleting the procedure for calculating the 95% confidence
interval because it appeared to be rarely, if ever, used. No commenters specifically agreed with this
suggestion, but several commenters responded that it should be retained. One commenter, arguing in
favor of the procedure, stated that "It has long been recognized that a 95% confidence level is appropriate
to establish standards and other regulatory requirements." Considering these comments, EPA now
believes there is no compelling reason to remove this procedure.
5-5
-------
Condition 4: Standards exist and can be maintained to control its operation. The MDL approach is
supported by a clearly defined, published procedure to control its operation. The procedure gives the
steps to be followed and instructs the analyst to use the entire measurement process. Hundreds, if not
thousands, of laboratories have successfully implemented the MDL procedure since its promulgation in
1984. EPA has found that when laboratories are required to perform MDL studies as part of an
interlaboratory study, the results reported by the laboratories are generally consistent. EPA has observed
similar consistency in use of the MDL by laboratories required to perform the procedure to demonstrate
proficiency with a method. Therefore, the MDL meets this condition.
Notwithstanding the preceding, the MDL procedure would be improved with additional guidance,
particularly with respect to initial spike levels, handling outliers, the optional reasonableness step (Step 7),
and multi-analyte test methods. The MDL procedure does not contain a discussion of outliers. It may be
helpful to clarify that 1) results should be discarded only if the results are associated with a known error
that occurred during analysis (e.g., the replicate was spiked twice) or through a statistically accepted
analysis of outliers, and 2) that laboratories should not simply select the best seven results of a dataset.
The optional step involves iterative testing to verify that the determined MDL is reasonable; EPA has
observed that few organizations bother to perform this step. EPA also has observed that when a method
involves a large number of analytes, it can be difficult to get all analytes to pass the iterative test in the
same run. The MDL procedure would benefit from guidance on how and when to address each of these
issues.
Condition 5: It has attracted widespread acceptance within a relevant scientific community. The MDL
meets this condition. The MDL has been used experimentally since 1980 and in a regulatory context since
1984. The MDL procedure is the most widely used and, therefore, the most widely tested detection limit
procedure in the history of detection approaches. Within EPA, the MDL has been used by the Office of
Research and Development, Office of Science and Technology, Office of Ground Water and Drinking
Water, Office of Solid Waste, Office of Emergency and Remedial Response, and other offices. The MDL
also has been used outside of EPA in methods published by ASTM International, in Standard Methods for
the Examination of Water and Wastewater- jointly published by the American Public Health Association
(APHA), the American Water Works Association (AWWA), and the Water Environment Federation
(WEF), and in methods elsewhere. Although the MDL has been criticized, it is the most widely used
approach of detection within the environmental chemistry community.
Stakeholders commenting onEPA's 2003 assessment of detection and quantitation procedures
noted that the extent to which the MDL has been used is a result of EPA's approval and inclusion of the
procedure in 40 CFRpart 136, and does not necessarily demonstrate that the MDL procedure produces an
accurate assessment of detection. EPA agrees that the extent of use could be attributed, in part, to
promulgation of the procedure at 40 CFR part 136. For this reason, EPA has not relied on widespread use
of the MDL as a sole or over-riding argument for its continued use. Rather, EPA views widespread use of
the MDL as one of many factors to be considered when evaluating which concept or concepts best meet
the Agency's needs under the Clean Water Act. For example, EPA agrees that the ability of a procedure
to produce an accurate assessment of detection capabilities is an important consideration, and addresses
this issue repeatedly throughout the assessment. In this chapter, for example, the ability of a procedure to
produce an accurate assessment of detection capabilities is addressed in
• Criterion 1, condition 3, which concerns error rate,
• Criterion 1, condition 4, which concerns use of standards to control operation of the procedure,
• Criterion 2, which addresses the ability of the procedure to realistically reflect laboratory and
method performance, and
5-6
-------
• Criterion 4, which addresses the ability of the approach to identify the concentration at which
users can be confident a substance reported as present is really present.
5.1.1.2.2 Criterion 2: The approach should address realistic expectations of laboratory and method
performance, including routine variability.
The MDL procedure is designed to demonstrate laboratory performance with a given analytical
method, and can be applied to a broad variety of physical and chemical methods. The procedure also
recognizes the importance of analyst experience and explicitly directs the analyst to employ all sample
processing and computation steps given in the analytical method when determining the MDL.
When the MDL procedure is followed as intended (i.e., all sample processing and analysis steps
of the method that are applied to routine analyses are included in determination of an MDL), the
demonstrated MDL will include some of the routine variability associated with the laboratory and the
method.
Stakeholders commenting on EPA's assessment stated that, because the MDL procedure is
performed in a single laboratory, on the same day, by the same analyst, in a single matrix, using a
minimum of 7 replicates, the procedure does not account for all sources of variability. These commenters
believe that the procedure does not address inter- or intralaboratory, long-term, concentration range,
analyte/method, or matrix variability. EPA notes that the MDL procedure does not include the
restrictions noted by these stakeholders (e.g., users are not restricted to use of only seven replicates; to
analysis of all replicates on the same day; or to determination of MDLs only in reagent water). The MDL
procedure includes, for example, instructions for determining a matrix-specific MDL and specifies that the
procedure requires a complete, specific, and well-defined analytical method. However, EPA also
recognizes that in practice the MDL procedure may be performed in the manner described by these
comments and that doing so will limit the amount of routine variability reflected in the results.
The MDL procedure provides users with the flexibility needed for multiple applications. For
example, if a laboratory desires to evaluate its performance using a single method to analyze a particularly
difficult matrix over a period of time (e.g., one year), the MDL procedure allows such an evaluation.
However in some cases, the MDL procedure might benefit with specific provisions for including sources
of variability that may not be addressed when following the minimum requirements of the MDL
procedure.
Stakeholders commenting on EPA's assessment directed most of their concern at the lack of long-
term variability in the MDL procedure. These commenters pointed to the American Council of
Independent Laboratories (ACIL) procedures for calculating the critical level and long term-MDL (LT-
MDL) and to the US Geological Survey's (USGS) procedures for generating their LT-MDL. These
procedures include the collection of blanks over a long period of time to include this source of variability.
The commenters stated that the lack of long-term variability leads to underestimates of Currie' s critical
value (Lc), and one commenter included sets of blanks collected over 3 months to demonstrate this effect.
EPA assessed the effect of long-term variability on calculated limits by simulating multiple 7-
replicate subsets from the full dataset offered by the commenter, and compared these short-term critical
levels to the critical level calculated using the full data set. Although the range of days from which the
sets of 7 replicates were simulated varied from between one week to greater than 3 weeks, a graphical
5-7
-------
analysis of the data did not reveal any effect of time on the resulting Lc. The total number of blanks also
did not seem to have an effect on the percentage of short-term Lc results that exceeded the overall Lc.
Details of this assessment are provided in Appendix C, along with possible reasons why expected
differences were not observed.
As noted in Section 3.3.3 of this RAD, a larger number of replicates will yield better estimates for
standard deviations, and therefore, better estimates of Currie's Lc and EPA's analogous MDL. However
the analysis performed in Appendix C demonstrates that MDLs estimating Lc based on 7 replicates are not
biased low. These values are merely less precise than those based on a larger number of replicates. As
noted previously, the current MDL procedure does not restrict laboratories to using 7 replicates (to the
contrary, the procedure specifies a minimum of 7 replicates), nor does it restrict laboratories to performing
the replicates on a single day. Laboratories that wish to perform more tests or to conduct their tests over a
longer period of time should be encouraged to do so.
Due to the variability inherent in measurement science, instrumentation, and the humans
conducting analyses, laboratories may routinely obtain detection limits that are lower or higher than those
obtained in another laboratory. Thus, when an MDL is determined during method development, it is
important to determine that MDL in more than one laboratory to ensure the MDL published in the method
reflects demonstrated expectations of method performance in a community of laboratories. It is not
necessary for this community to include the entire universe of all possible laboratories that might desire to
practice the method. Rather, during the stages of method development and validation, this community
only should include well-operated laboratories with analysts who are experienced with the techniques
used in the method, and have some familiarity conducting all of the steps in the new method before
generating MDLs that will be published with the new method.
In recent years, EPA's Office of Science and Technology has used single-laboratory studies to
develop an initial estimate of the MDL for a new or modified method, and has verified these MDLs in
interlaboratory studies or by conducting additional single-laboratory studies in other laboratories. For
example, when EPA initially drafted Method 1631 for measurement of mercury, EPA estimated the MDL
to be 0.05 ng/L based on results produced by a contract research laboratory. Additional single-laboratory
MDL studies conducted in other laboratories suggested that the MDL should be raised to 0.2 ng/L to
better reflect existing capabilities of the measurement community. During EPA's interlab oratory study,
each laboratory was asked to conduct an MDL study. Every laboratory in the interlaboratory study met
the MDL of 0.2 ng/L (laboratory MDLs ranged from 0.04 to 0.18 ng/L), the value published in the
promulgated version of Method 1631.
The MDL procedure addresses demonstrated expectations of laboratory and method performance,
including routine variability, and users should not be restricted to the minimum requirements of the MDL
procedure. If the MDL procedure is employed for method development purposes, it should be performed
in multiple laboratories to ensure that it adequately demonstrates expectations in a community of qualified
laboratories.
5.1.1.2.3 Criterion 3: The approach should be supported by apractical and affordable procedure that
a single laboratory can use to evaluate method performance.
The MDL procedure is among the most practical and affordable procedures that have been
suggested for determining detection limits because of the reasonable number of minimum replicates
(seven) and the relative ease with which the spiking experiments can be designed and the resulting data
5-
-------
analyzed. The MDL is designed for use by a single laboratory, and can be performed by a single analyst
using a single instrument. And the MDL procedure also allows MDLs from several analysts or
instruments within a laboratory, or between laboratories to be pooled and provide an estimate of the range
of MDLs that might be routinely expected.
Use of the optional iterative procedure would increase the number of analyses by at least seven
each time the procedure is implemented. If the procedure is implemented two times in reagent water, a
minimum of 14 analyses are required. If the procedure is implemented two times in an alternative matrix,
EPA estimates that 17-20 analyses maybe required, given the possible need to determine the background
concentration of the analyte in the alternative matrix. In any of these scenarios, the entire MDL
determination can be performed in a single analytical batch (most EPA methods specify batch sizes of 20
samples).
5.1.1.2.4 Criterion 4: The detection level approach should estimate the theoretical concentration at
which there is 99% confidence that the substance is actually present when the
analytical method is performed by experienced staff in a well-operated
laboratory.
The MDL meets this condition as described under Section 5.1.1.2.1, Condition 3 of this document
in many cases. However, EPA recognizes that there are cases where this does not hold, and that users of
the MDL procedure see this as a significant problem. EPA sees merit in blank correction procedures
developed by ACIL and USGS to address these cases. In future stakeholder consultations, EPA plans to
discuss these and other alternative solutions to this problem.
5.1.1.2.5 Criterion 6: Detection and quantitation approaches should be applicable to the variety of
decisions made under the Clean Water Act, and should support State and local
obligations to implement measurement requirements that are at least as
stringent as those set by the Federal government.
The MDL meets this criterion. The MDL has been applied to a variety of decisions under the
CWA since 1984. In addition, many States and others have adopted the MDL in their own programs.
5.1.2 Evaluation of the ASTM International Interlaboratory Detection Estimate (IDE)
The interlaboratory detection estimate (IDE) was published in 1997 by ASTM International as
standard D6091. The IDE was developed with support from members of the regulated industry to provide
a comprehensive detection limit procedure that addressed the concerns of the regulated industry,
statisticians, and analysts involved in ASTM Committee D19 on water.
A brief summary of the procedure is given in Section 5.1.2.1, and Section 5.1.2.2 presents EPA's
assessment of the IDE against the five criteria established for evaluating detection limit approaches (i.e.,
Criteria 1-4, and Criterion 6).
5-9
-------
5.1.2.1 Description of the IDE Approach and Procedure
ASTM Designation D 6091 is the Standard Practice for 99 %/95 % Interlaboratory Detection
Estimate (IDE) for Analytical Methods with Negligible Calibration Error. As stated in the practice:
"The IDE is computed to be the lowest concentration at which there is 90 % confidence
that a single measurement from a laboratory selected from the population of qualified
laboratories represented in an interlaboratory study will have a true detection probability
of at least 95 % and a true nondetection probability of at least 99 % (when measuring a
blank sample)."
The IDE is determined and verified using a procedure containing 5 major steps with
approximately 53 substeps and conditions. The full text of the IDE procedure is available from ASTM
International. The five major steps and their functions are given in Section 6 of the IDE procedure and are
as follows:
1. Overview of the procedure.
2. IDE Study Plan, Design, and Protocol - in this section, the task manager (study supervisor) chooses
the analyte, matrix, and analytical method. Details are given for range finding; the concentrations to
be used in the study; the study protocol (ASTM Practice D 2777 is suggested); the allowable sources
of variation; and the number of laboratories, analysts, and days over which the study will be
conducted.
3. Conduct the IDE Study, Screen the Data, and Choose a Model - after the study data are collected and
screened according to ASTM Practice D 2777, interlaboratory standard deviation (ILSD) versus
concentration data are tabulated and one of three models is fit to the data. The first attempt is at
fitting a constant model. If the attempt fails, a straight-line model is attempted. If the straight-line
model fails, an exponential model is fitted. After fitting, the model is evaluated for reasonableness
and lack of fit. If the model fails, the study supervisor determines if a subset of the data should be
analyzed or if more data are needed.
4. Compute the IDE - the IDE is computed using the ILSD model selected in Step 3 to estimate the
interlaboratory standard deviation at a true concentration of zero and at the IDE, using a mean
recovery model to transform measured and true concentrations. The IDE is computed as a one-sided
90 % confidence upper statistical tolerance limit.
5. Nontrivial Amount of Censored Data - this section addresses the effect of "non-detects" or "less-than."
Suggestions are given to see if uncensored data can be obtained from the laboratories or if the study
needs to be augmented with additional data. Suggestions are given for fitting a model to data that
contain less than 10 % non-detects or less-than to produce an IDE.
5.1.2.2 Assessment of the IDE Against the Evaluation Criteria
The following five subsections discuss the IDE approach and procedure in the context of the five
evaluation criteria that concern detection limit approaches.
5- 10
-------
5.1.2.2.1 Criterion 1: The detection and quantitation limit approaches should be scientifically valid.
Condition 1: It can be (and has been) tested. The Electric Power Research Institute provided input into
the design of EPA Method 1631 and 1638 Validation Studies for the purpose of calculating IDEs and
IQEs. EPRI also calculated IDEs and IQEs based on these data. These two datasets include a total often
metal analytes and therefore do not cover a wide range of analytical techniques and methods. Other than
these two datasets, EPA is not aware of any organization, including ASTM International, that has
conducted a study to test the procedure as written (i.e., designed and implemented an interlaboratory study
that involves estimating an initial IDE [IDE0] and multilaboratory analyses of multiple concentrations of
each matrix of interest surrounding IDE0). Developers of the approach performed limited testing of the
approach on 1) simulated data sets and 2) real-world data sets generated for other purposes. However,
these real-world data sets are of limited value for testing the IDE because the concentration ranges
associated with the data are above the low-level region of interest. As part of this reassessment, EPA
tested a variant of the IDE procedure on single-laboratory data sets designed for characterization of an
analytical method in the region of detection. Despite the lack of comprehensive testing, the procedure can
be tested, and therefore meets part of this condition. Specifically, the IDE meets the condition that it can
be tested, but it only partially meets the condition that it has been tested.
Condition 2: It has been subjected to peer review and publication. Although the IDE has not been
published in the peer-reviewed scientific literature, the IDE has undergone extensive review and ballot by
members of ASTM Committee D 19, many of whom are qualified peer reviewers. Therefore, although the
IDE does not meet this condition in the sense of formal peer review and publication, it meets the intent of
this condition (i.e., submission to scrutiny of the scientific community). In addition, the IDE was
reviewed by four peer reviewers as part of EPA's assessment of detection and quantitation limit
approaches.
Condition 3: The error rate associated with the procedure is either known or can be estimated. In theory,
expert statisticians could estimate the error rate of the IDE. However, the IDE procedure is extremely
complex from an analytical chemistry and statistical perspective. As a result, it is unlikely that the error
rate could be estimated by the typical users of the analytical method to which it would be applied, or even
by the typical developers of an analytical method. Moreover, EPA found the model selection procedure to
be highly subjective, a situation likely to yield different IDEs from the same data set, depending on the
staff involved in performing the calculations. In practice, such conditions make it impossible to estimate
the actual error associated with the IDE. Therefore, the IDE does not meet this condition.
One of the four peer reviewers charged with evaluating EPA's assessment of detection and
quantitation limit approaches concurred with EPA's assessment of the IDE, specifically stating, "I agree
that the IDE procedure as outlined is so complex as to make simple determination of error rates
associated with it untenable. " (Piegorsch, 2002)
One stakeholder, however, stated that concerns about the complexity and subjectivity in the IDE
(and IQE) procedures were unimportant, in part, because IDEs calculated using different models were
generally very close, and in part because "user-friendly software is available that will automatically
perform the IDE and IQE calculations." To consider the merit of this comment, EPA calculated single-
laboratory variants of the IDE using each of the four major model types using the Episode 6000 data set,
and true interlaboratory IDEs for each model type using the Method 1631 and 1638 interlaboratory study
data sets. Results of these calculations, along with the RSDs between the different IDE values obtained
for each analyte, are presented in Appendix B. Based on the calculated RSDs, there is a large amount of
variability between the single-laboratory variants of the IDEs calculated using the different models.
Generally, the IDEs calculated using the constant model were much greater than those calculated using the
5- 11
-------
other models. The hybrid model generally yielded the lowest IDEs, and the IDEs calculated using the
hybrid and exponential models were quite similar for some analytes, but quite different for others. While
one might hope that the variability between models would decrease if interlaboratory variability were
included in the calculations (as designed), EPA found this was not the case. To the contrary, RSDs
between the IDEs calculated from the interlaboratory datasets suggest that variability between model
estimates appears to increase when the additional variability between laboratories is included.
To evaluate the commenters' statement that the complexity and subjectivity of the procedures was
not important because the calculations can be automatically performed using "user-friendly software,"
EPA evaluated the two software packages offered by the commenter. One package was a DOS-based
program called "QCalc" and the other was an Excel spreadsheet that calculates IDEs based on Excel
functions, macros, and the Solver add-in function. EPA calculated single laboratory variants of the IDE
for a random subset of 20 analytes from the Episode 6000 study using 1) the QCalc package, 2) the Excel
spreadsheet, and 3) the suite of SAS programs EPA has been using to calculate IDEs as part of this
assessment. To ensure that differences between results were due to the programs themselves, the same
data were used for each program. Results of this comparison are provided in Appendix B to this Revised
Assessment Document.
One immediate problem was that comparisons could not be made between IDEs calculated using
QCalc and the other software packages for all of the models because the QCalc package only performs the
IDE calculation using two of the models (exponential and hybrid). The ASTM IDE procedure suggests
that one of three models be used (constant, linear, and exponential). No explanation was provided as to
why the software was limited to two models instead of three, or why one of the two models (i.e., the
hybrid model) used in the software was not one of the three models recommended by ASTM. (The hybrid
model used in QCalc is recommended by ASTM for calculation of an IQE but not for an IDE.)
Although similarities were generally observed among the various software packages when the
same model type was applied to the same set of data, EPA did observe strong differences in the values
calculated using the hybrid model across the various software programs. The Excel values generated
using the hybrid model were slightly higher than those determined using EPA's programs and
approximately twice as high as those determined using QCalc. Possible explanations for these differences
are given in Appendix C.
Perhaps the most significant problem with the assumption that use of the automated software
packages alleviates the complexity and subjectivity in the IDE procedure is that the various packages do
not always select the same model for the same set of data. ASTM's IDE procedure (D 6091) specifies that
the fitting to the constant model should be attempted first. If this fitting fails, a straight-line model should
be attempted, and if that fails, the exponential model should be fitted and evaluated for reasonableness and
lack of fit. EPA's SAS programs were coded to preferentially select the constant, linear, and exponential
models for the IDE, according to this scheme. However, QCalc and Excel packages each follow a
different scheme. As a result, the EPA and QCalc programs selected the same model type to calculate the
IDE for only 1 of the 20 analytes, the Excel and QCalc programs selected the same model type for only 6
of the 20 analytes, and the Excel and EPA programs selected the same model type for only 1 of the 20
analytes. Details and possible explanations for these underlying differences can be found in Appendix C.
Based on these differences in selecting and fitting models, it does not appear that the two
available software programs remove all complexity and subjectivity from the IDE calculation. Instead,
they appear to introduce new issues by using steps not included in the ASTM procedures. The results
support EPA's conclusion that such conditions make it impossible to estimate the actual error associated
with the IDE, and that the IDE, as currently constructed, does not meet this condition 3.
5- 12
-------
Condition 4: Standards exist and can be maintained to control its operation. The IDE approach and
procedure is supported by a published procedure (standard) to control its operation. The procedure gives
the steps to be followed in determining the IDE and instructs the study supervisor how to gather the data
and compute an IDE.
There are several "gray areas" in the published procedure. The most significant of which is in the
description of model selection. The procedure provides insufficient guidance on use of residual plots to
evaluate and select models and, as a result, selection of the model maybe very subjective, especially if the
number of concentrations is low. The problems noted in preceding Condition 3 concerning the use of
different model selection strategies among three different programs (the QCalc and the Excel software
packages provided by a commenter and EPA's SAS programs) is a direct reflection of the subjective
nature of model selection likely to result from the lack of guidance in the procedure. The discussion of
what model to use after rejecting the exponential and linear model is also very vague. The Rocke and
Lorenzato (hybrid) model is mentioned, as well as models with more than one coefficient. Much of the
data evaluated by EPA have tended to suggest the exponential model, based on the statistical tests
discussed. However, those data have almost always shown residual "patterns" when using this model,
which would then lead to consideration of other models. In addition, fitting the constant model is never
discussed in detail. Most likely, this is done by simply calculating a mean (weighted if necessary) of the
variances from the different concentrations; however, such calculations are never explicitly stated.
The IDE standard gives procedures that are inconsistent with procedures in the IQE standard, even
though the two approaches should be consistent for a given analyte with a given method. For example, the
exponential model figures prominently in the IDE procedure, where it is one of the three main models
discussed. The Rocke and Lorenzato model is not discussed in the IDE procedure, but it figures
prominently in the IQE procedure. In theory, a single model should support the definition of both the
detection and quantitation limits for a given analyte by a given method. As another example, the IDE
procedure includes a multiplier to account for bias in estimating the true standard deviation with the
sample standard deviation, but the IQE does not.
Although the IDE is supported by a published procedure, EPA found that the procedure will not
adequately control its operation because of the degree of subjectivity involved in implementing the
procedure and inconsistencies with its IQE counterpart. Therefore, the IDE does not meet this condition.
Condition 5: It has attracted widespread acceptance within a relevant scientific community. The IDE was
published by ASTM, International in 1997. ASTM, International is a voluntary consensus standards
organization that constitutes part of the relevant scientific community, however, seven years after
publication no new or revised ASTM standard has included detection limits using the IDE approach. EPA
is not aware of an IDE that has been published in the open literature or in an analytical method. Thus, the
IDE partially meets this criterion.
5.1.2.2.2 Criterion 2: The approach should address realistic expectations of laboratory and method
performance, including routine variability.
The IDE procedure, D6091, is designed to reflect expectations of interlaboratory performance,
including routine variability. The procedure contains extensive instructions for dealing with unusual
conditions, including sources of variability and outliers. However, EPA studies of a single-laboratory
variant of the procedure suggested that the procedure may not always work as intended. For example,
model selection based upon hypothesis tests (as described in Section 6.3.3.2 of D6091) almost always
indicated that the exponential model should be used, even when the data seemed to be show constant or
approximately linear error, while examination of residual plot indicated "systematic behavior" (i.e., non-
5- 13
-------
random deviations from the model) for the exponential and linear models. Information about single-
laboratory (or within-laboratory) variability is very important because assessments of laboratory
performance is based on the variability (uncertainty) of the dat produced at that laboratory. Compliance
measurements are made in a single laboratory and the results are reported with the uncertainty (variability)
associated with that dataset.
Another concern with the IDE procedure is that use of the non-mandatory appendices in ASTM D
6512 to determine the fit of a model may produce results that differ from those that would be obtained by
using the default procedures for testing model fit that are built into off-the-shelf statistical software, such
as those used in the Excel spreadsheets discussed in Section 5.1.2.2.1. Such observations, along with the
concerns described in Section 5.1.2.2.1, condition 4, lead EPA to believe that, while the IDE approach
addresses demonstrated expectations of laboratory and method performance, the IDE procedure does not
adequately do so. Therefore, the IDE only partially meets this criterion.
5.1.2.2.3 Criterion 3: The approach should be supported by a practical and affordable procedure that
a single laboratory can use to evaluate method performance.
The IDE procedure is designed for use by an ASTM International study supervisor or task
manager and not as a procedure that a single laboratory can use to evaluate method performance. EPA is
aware that ASTM Committee D 19 is developing a Within-laboratory Detection Estimate (WDE), but the
WDE is presently only in the formative stages. The WDE may meet this criterion, but the IDE does not.
Regarding cost, the IDE procedure would be the most costly of the procedures that EPA has
evaluated because of the time it would take to understand and implement the procedure, and requirements
for: 1) estimation of IDE0, 2) interlaboratory data, 3) extensive statistical intervention in determining the
correct model, and 4) possible reanalyses if the resulting IDE does not meet the criteria in the procedure.
5.1.2.2.4 Criterion 4: The detection level approach should estimate the theoretical concentration at
which there is 99% confidence that the substance is actually present when the
analytical method is performed by experienced staff in a well-operated
laboratory.
By definition, the IDE is designed to achieve "a true detection probability of at least 95 % and a
true non detection probability of at least 99 %." Although the 99% probability of a "true nondetection" is
equivalent to the 99% confidence that the substance is actually present given in Criterion 4, ASTM
International also included the simultaneous requirement for a 95% probability of a "true detection." The
developers are using the IDE as a means to control the rates of both false positive and false negative
results, in essence, making the IDE analogous by definition and formulaic construction to the detection
limit (DL) defined by Currie (1968). The IDE accomplishes this goal by using a tolerance limit that
increases the IDE well above the point at which the detection decision would be made. For a discussion
of this issue, see Sections 3.3.6 (false positives and false negatives) and 3.3.7 (prediction and tolerance
intervals) in Chapter 3 of this document.
As noted in Section 2.1 of Chapter 2 of this document, Currie (1968) used the term detection limit
(subsequently termed the minimum detectable value} to refer to a true concentration that has a high
probability of generating measured values greater than the critical value. That is, measurements on
samples that contain concentrations equal to the detection limit have a high probability of exceeding the
5- 14
-------
critical value and are, therefore, unlikely to result in a decision that the substance is not detected in the
sample. However, the detection decision is made on the basis of comparing sample measurements to the
critical value. With regard to his definition of the "detection limit, " Currie (1995) states "The single, most
important application of the detection limit is for planning. "
When the allowance for false negatives and the prediction and tolerance limits are taken into
account, the resulting IDE is raised to the point at which the probability of a false positive is less than .01
by several orders of magnitude. This protection against false positive results is excessive and would yield
numerical values of little practical value for making the detection decision.
Although there is an estimate of Carrie's Lc included in the IDE procedure, it is unclear where
the detection decision is made (it really should be an ICE/IDE procedure). If one focuses on the IDE and
not the Lc estimate, this criterion not met. Therefore, it is not clear whether the IDE would meet this
criterion (No. 4).
5.1.2.2.5 Criterion 6: Detection and quantitation approaches should be applicable to the variety of
decisions made under the Clean Water Act, and should support State and local
obligations to implement measurement requirements that are at least as
stringent as those set by the Federal government.
EPA's comparison of detection limits produced by various detection limit approaches shows that
the median IDE is considerably higher than ACS, ISO/IUPAC, and EPA detection limits. Although the
IDE could be applied to some decisions to be made under the CWA, it may not be appropriate for all uses.
The IDE is an implementation of Currie detection level or minimum detectable value, and may in practice
yield results higher than these levels. At best, the IDE only partially meets this criterion.
5.1.3 Evaluation of the ACS Limit of Detection
The limit of detection (LOD) was developed by the Committee on Environmental Improvement
(CEI) of the American Chemical Society (ACS). ACS is a professional society for chemists and other
scientists and the publisher of a number of scientific journals. It is not a voluntary consensus standards
body (VCSB), nor does it develop or publish analytical methods. In 1978, the ACS/CEI began addressing
concerns about the lack of useful standards for interlaboratory comparisons. In 1980, the Committee
published its "Guidelines for Data Acquisition and Data Quality Evaluation in Environmental Chemistry"
(MacDougall, et al, 1980), which included the approaches of the LOD and the limit of quantitation
(LOQ).
5- 15
-------
5.1.3.1 Description of the ACS LOD
The 1980 "Guidelines" define the LOD as:
"... the lowest concentration of an analyte that the analytical process can reliably detect.
... The LOD in most instrumental methods is based on the relationship between the gross
analyte signal St, the field blank Sb, and the variability in the field blank ab. "
and construct the formal relations using the equation:
where Kd is a constant. ACS recommended a minimal value of 3 for Kd. Thus, the LOD is 3o above the
gross blank signal, Sb. In the 1 980 publication, the ACS stated that at Kd = 3, there is a 7% risk of false
negatives and false positives. Given that the LOD is 3o above the blank, however, EPA believes that the
risk of false positives is somewhat less than 1%.
In 1983, the ACS Committee published "Principles of Environmental Analysis" (Keith et al,
1983). That publication occurred after the 1981 paper on the Method Detection Limit (MDL), and
ACS/CEI stated that the LOD is numerically equivalent to the MDL as Sb approaches zero. However,
neither the 1980 nor 1983 ACS publications provide a specific procedure for estimating the LOD, nor do
they provide a minimum number of observations needed to estimate the gross blank signal or the
variability term ob.
5.1.3.2 Assessment of the LOD Against the Evaluation Criteria
The following five subsections discuss the LOD approach and procedure in the context of the five
evaluation criteria that concern detection limit approaches (i.e., Criteria 1-4, and Criterion 6).
5.1.3.2.1 Criterion 1: The detection and quantitation limit approaches should be scientifically valid.
Condition 1 : It can be (and has been) tested. Testing of the ACS LOD is hampered by the lack of a
supporting procedure for establishing an LOD, and a conceptual dependence on the variability associated
with measuring blanks. For example, there is no detailed instructions, similar to those in the IDE and the
MDL procedures, to govern the minimum number of analyses needed to characterize the variability of a
blank sample. Because many environmental chemistry techniques yield a zero, or possibly even negative,
value when a blank sample is analyzed, and because the LOD approach is based on the standard deviation
of these results, directly testing the LOD in such techniques will yield a zero or negative value. One
solution for testing is to rely on ACS' 1983 statement that the LOD is conceptually equivalent to the MDL
as the blank signal approaches zero, and employ the MDL procedure as a means for indirectly testing the
LOD approach. EPA believes that use of the MDL procedure is a viable means for testing the approach;
therefore, the LOD meets this condition.
Condition 2: It has been subjected to peer review and publication. The LOD meets this condition because
the LOD definition was published in the peer-reviewed journal Analytical Chemistry in 1980 and 1983.
Condition 3 : The error rate associated with the procedure is either known or can be estimated. The error
rates can be estimated, so the LOD meets this condition. The error rate for both false positives and false
negatives is stated to be 7 % in the 1 980 Analytical Chemistry article. However, EPA believes that,
because the LOD is stated to be 3 times the standard deviation of replicate measurements of a blank, the
5- 16
-------
false positive rate is overstated and is actually somewhat less than 1 % whereas the false negative rate
depends on the true concentration in the sample.
Condition 4: Standards exist and can be maintained to control its operation. The LOD does not meet this
condition, because it lacks a clearly defined procedure for estimating the important terms required to
derive it. Although it may be possible to derive LOD values from data used to derive EPA MDL values,
there is no procedure giving explicit instructions on the use of replicate blanks, replicate spiked samples,
or a minimum recommendation for the number of replicates.
Condition 5: It has attracted widespread acceptance within a relevant scientific community. Because
ACS does not develop and publish analytical methods, it is difficult to determine the degree of acceptance
of the LOD. EPA has not specifically investigated the numbers of papers published in ACS journals that
include LOD values, and EPA's literature search for detection and quantitation approaches did not uncover
a large number of citations that promote the LOD in particular. However, ACS LOD values have
appeared in the technical literature. Given that ACS is a relevant scientific community, and that use of the
LOD has appeared in the technical literature, the LOD meets this condition.
5.1.3.2.2 Criterion 2: The approach should address realistic expectations of laboratory and method
performance, including routine variability.
The LOD approach is designed to address realistic expectations of laboratory and method
performance, including routine variability, and thus appears to meet this criterion. Unfortunately, ACS
has not published a procedure to implement the approach. In other words, the LOD addresses
demonstrated expectations of laboratory and method performance in theory, but in practice, provides no
direct means for performing these demonstrations. Therefore, EPA believes the ACS LOD only partially
meets this criterion.
5.1.3.2.3 Criterion 3: The approach should be supported by a practical and affordable procedure that
a single laboratory can use to evaluate method performance.
The ACS LOD approach does not meet this criterion, because it is not supported by a clearly
defined procedure for establishing the LOD.
5.1.3.2.4 Criterion 4: The detection level approach should estimate the theoretical concentration at
which there is 99% confidence that the substance is actually present when the
analytical method is performed by experienced staff in a well-operated
laboratory.
The 1983 publication associated the LOD with the "99% confidence level when the difference (St
- Sb) > 3o." Therefore, the LOD meets this criterion.
5.1.3.2.5 Criterion 6: Detection and quantitation approaches should be applicable to thevarietyof
decisions made under the Clean Water Act, and should support State and local
obligations to implement measurement requirements that are at least as
stringent as those set by the Federal government.
In the absence of a procedure for determining LOD values, the ACS LOD does not meet this
criterion because it cannot be used in a regulatory context unless it is assumed to be functionally
equivalent to the MDL (i.e., use the MDL procedure to establish an LOD).
5- 17
-------
5.1.4 Evaluation of the IUPAC/ISO Critical Value (CRV)
The critical value (CRV) was developed by the International Union of Pure and Applied
Chemistry (IUPAC) and the International Organization for Standardization (ISO). IUPAC and ISO are
professional societies for chemists and other scientists. ISO develops and publishes analytical methods
through its Task Groups. In 1995, Lloyd Currie of the National Institute for Standards and Technology
(NIST; formerly the National Bureau of Standards) published a signature discussion of IUPAC approaches
for detection and quantitation (Pure andAppl. Chem. 67:10, 1699-1722). Although refined during the
intervening years (see Currie, L.A, J. Radiochem. And Nuclear Chem. 245:1, 145-156, 2000), the CRV
approach remains basically as described in 1995.
5.1.4.1 Description of the ISO/IUPAC Critical Value (CRV) Approach and Procedure
The 1995 article states that the critical value (Lc) is:
"... the minimum significant value of an estimated net signal or concentration, applied as
a discriminator against background noise. This corresponds to a 1-sided significance
test. "
For a normal distribution with known variance, Lc reduces to:
where:
1-a is the false positive error rate, recommended at 5 % (a = 0.05), and
o0 is the standard deviation at zero concentration
If o0 is estimated by s0 (replicate measurements of a blank), z(1.a) is replaced by the Student's t-
value. For 7 replicates (6 degrees of freedom), the Student's f-value is 1.943, where a = 0.05.
5.1.4.2 Assessment of the CRV Against the Evaluation Criteria
The following five subsections discuss the CRV approach and procedure in the context of the five
evaluation criteria that concern detection limit approaches (i.e., Criteria 1-4, and Criterion 6).
5.1.4.2.1 Criterion 1: The detection and quantitation limit approaches should be scientifically valid.
Condition 1 : It can be (and has been) tested. The lack of a supporting procedure for establishing the
CRV, coupled with its conceptual dependence on the variability of blank measurements makes testing of
the approach difficult. For example, if blank measurements fail to produce a response, it is impossible to
calculate a CRV because the standard deviation of multiple zero results is zero. One solution for testing
the approach is to assume that the CRV is about equivalent to the MDL as the blank signal approaches
zero, and use a slightly modified version of the MDL procedure to test the CRV approach. The slight
modification involves selecting a Student's f-value based on a = 0.05 instead of a = 0.01, for n-1 degrees
of freedom. EPA believes this is a reasonable assumption, and therefore, that the MDL procedure is a
viable means for testing the CRV approach. Therefore, the CRV meets this condition.
5- 18
-------
Condition 2: It has been subjected to peer review and publication. The IUPAC/ISO definitions meet this
criterion. Moreover, it is likely that these definitions have received greater peer review than any of the
other approaches.
Condition 3: The error rate associated with the procedure is either known or can be estimated. The error
rate is specified by a, with a suggested value of 0.05 (5%). Therefore, the CRV meets this condition.
Condition 4: Standards exist and can be maintained to control its operation. The CRV is defined in the
various publications by Currie. However, EPA's search of the literature and the ISO web site found no
standard for control of the approach. Therefore, the CRV does not meet this condition.
Condition 5: It has attracted widespread acceptance within a relevant scientific community. Because
IUP AC and ISO are international bodies, it is difficult to determine the degree of acceptance of the CRV
in the U.S. and the world community. EPA has not counted the number of papers in published journals
that include CRV values, but EPA's literature search for detection and quantitation approaches did not
produce many citations that promote the CRV in particular. Therefore, it is difficult to determine if the
CRV meets this condition.
5.1.4.2.2 Criterion 2: The approach should address realistic expectations of laboratory and method
performance, including routine variability.
The CRV approach is designed to account for the variability of measurements of the blank in the
context of a "chemical measurement process" (method). Unfortunately, neither ISO, IUP AC, nor Currie
have published a procedure to implement the approach. As a result, the CRV addresses realistic
expectations of laboratory and method performance in theory, but in practice, provides no direct means for
demonstrating this performance. Therefore, the CRV partially meets this criterion.
5.1.4.2.3 Criterion 3: The approach should be supported by a practical and affordable procedure that
a single laboratory can use to evaluate method performance
The CRV approach is not supported by a clearly defined procedure for establishing a CRV.
Therefore, the CRV does not meet this criterion.
5.1.4.2.4 Criterion 4: The detection level approach should estimate the theoretical concentration at
which there is 99% confidence that the substance is actually present when the
analytical method is performed by experienced staff in a well-operated
laboratory.
CRV suggests a = 0.05, resulting in 1-a of 0.95 or 95 % probability of detection . Therefore, the
CRV does not meet this criterion.
5.1.4.2.5 Criterion 6: Detection and quantitation approaches should be applicable to the variety of
decisions made under the Clean Water Act, and should support State and local
obligations to implement measurement requirements that are at least as
stringent as those set by the Federal government.
In the absence of a procedure for establishing CRVs, the CRV approach does not meet this
criterion because it cannot be used in a regulatory context.
5-19
-------
5.1.5 Evaluation of the IUP AC/ISO Detection Limit
The detection limit or minimum detectable value (MDV) was developed by IUP AC/ISO and
published in the same papers as the CRV (Section 5.1.4)
5.1.5.1 Description of the IUP AC/ISO Detection Limit Procedure
The 1995 publications define the minimum detectable value (detection limit) as follows:
"The Minimum Detectable Value (MDV) ... [is] ... the net signal (or concentration) of that
value (LD) for which the false negative error is p, given Lc (or a). " (see the CRV for Lc)
For a normal distribution with known variance, LQ reduces to:
where:
z is the score variable
1-P is the false negative error rate, recommended at 5 % (P = 0.05), and
OD is the standard deviation at the detection limit
Earlier publications refer to the minimum detectable value as the detection limit. To avoid
confusion in terminology and to help distinguish the ISO/IUPAC approach from the MDL, LOD, and
CRV, the ISO/IUPAC detection limit in this assessment will be referred to as the Minimum Detectable
Value, abbreviated as MDV.
5.1.5.2 Assessment of the ISO/IUPAC MDV Against the Evaluation Criteria
The following five subsections discuss the ISO/IUPAC MDV approach and procedure in the
context of the five evaluation criteria that concern detection limit approaches (i.e., Criteria 1-4, and
Criterion 6).
5.1.5.2.1 Criterion 1: The detection and quantitation limit approaches should be scientifically valid.
Condition 1 : It can be (and has been) tested. The lack of a supporting procedure for establishing the
MDV makes testing of the approach difficult. However, the MDV probably can be tested using data
similar to those used to generate MDL values. Therefore, the MDV meets this condition.
Condition 2: It has been subjected to peer review and publication. The IUP AC/ISO definitions meet this
condition; moreover, it is likely that this definition has received greater peer review than any of the other
approaches.
Condition 3 : The error rate associated with the procedure is either known or can be estimated. The error
rates are specified by a and P, both with suggested values of 0.05 (5 %). Therefore, the error rate is
known.
Condition 4: Standards exist and can be maintained to control its operation. The MDV is defined in the
various publications by Currie. However, EPA's search of the literature and the ISO web site found no
standard for control of the approach. Therefore, the MDV does not meet this criterion.
5-20
-------
Condition 5: It has attracted widespread acceptance within a relevant scientific community. Because
IUPAC and ISO are international bodies, it is difficult to determine the degree of acceptance of the MDV
in the U.S. and the world community. EPA has not specifically investigated the number of papers in
published journals that include MDV values, but EPA's literature search for detection and quantitation
approaches did not uncover a large number of citations that promote the MDV in particular. Therefore, it
is difficult to determine if the CRV meets this criterion.
5.1.5.2.2 Criterion 2: The approach should address realistic expectations of laboratory and method
performance, including routine variability.
The MDV approach is designed to account for the variability of measurements of the blank in the
context of a "chemical measurement process" in the sense that it is used in concert with a critical value
that is based on blank measurement variability. The MDV is the true concentration that is used in the
planning of method evaluation and development. The actual detection decision is made at the critical
value (CRV) which is determined from measured values. The approach of a true concentration MDV and
its associated allowance for false negatives is of little practical value in making the actual detection
decision. Therefore, the MDV does not meet this criterion. The allowance for false negatives in a
regulatory context is discussed in greater detail in Chapter 3.
5.1.5.2.3 Criterion 3: The approach should be supported by apractical and affordable procedure that
a single laboratory can use to evaluate method performance
The MDV approach is not supported by a clearly defined procedure for establishing MDV values.
Therefore, the MDV does not meet this criterion.
5.1.5.2.4 Criterion 4: The detection level approach should estimate the theoretical concentration at
which there is 99% confidence that the substance is actually present when the
analytical method is performed by experienced staff in a well-operated
laboratory.
The allowance for false negatives reduces the probability of false positives to a value smaller
than 1 % by several orders of magnitude. . This protection against false positive results is excessive and
would yield numerical values of little practical value for making the detection decision. Perhaps more
importantly, as noted by Currie (1995) and discussed in Section 5.1.2.2.4 of this document, the detection
decision is made on the basis of comparing sample measurements to the critical value. Therefore, the
MDV does not meet this criterion.
5.1.5.2.5 Criterion 6: Detection and quantitation approaches should be applicable to the variety of
decisions made under the Clean Water Act, and should support State and local
obligations to implement measurement requirements that are at least as
stringent as those set by the Federal government
In the absence of a procedure for establishing MDV values, the MDV approach does not meet to
meet this criterion because it cannot be used in a regulatory context.
5.1.6 Evaluation of the American Council of Independent Laboratories (ACIL) Critical Value
During the comment period on the February 2003 assessment document, the American Council of
Independent Laboratories (ACIL) submitted a procedure that was developed to address errors, which are
referred to as "bias", that may arise under certain conditions when estimating detection limits. The ACIL
5-21
-------
procedure separates estimation of the detection limit into two cases; cases where analyses always produce
a numeric result (i.e., even so-called "blank" samples produce a signal), and cases where tests do not
always produce a numeric result (i.e., blank samples appear to produce no signal). Blanks that do not
produce a signal may do so either because they really are blanks, or the instrument is suppressing the
signal. For convenience, EPA refers to these as Case I and Case II, respectively. Analysis of metals with
inductively coupled plasma optical emission spectroscopy (ICP-OES) is an example of ACIL Case I, and
analysis of organic pollutants with gas chromatography/ mass spectrometry is an example of ACIL Case
II. Although the ACIL procedure appears to be a work-in-progress, it has some interesting approaches for
the use of blanks, and is similar in some respects to the USGS LT-MDL procedure.
5.1.6.1 Description of the ACIL Approach and Procedure
For Case I analyses, ACIL offers procedures for calculating a limit that approximates Currie's
critical value (Lc) and procedures for calculating a limit that approximates Currie's detection limit (LD).
As discussed in Chapter 2 and noted again in Section 5.1.5 above, Currie's LQ was designed to account
for the variability of measurements of the blank in the context of a "chemical measurement process" in the
sense that it is used in concert with a critical value that is based on blank measurement variability. The LD
is the true concentration that is used in the planning of method evaluation and development. The actual
detection decision is made at the critical value (Lc), which is determined from measured values. The
approach of a true concentration Lj, and its associated allowance for false negatives is of little practical
value in making the actual detection decision. For this reason, EPA focused its assessment of ACIL' s
procedure on the ACIL version of Currie's critical value rather than the ACIL version o
For Case II analyses, ACIL suggests a procedure that does not rely on the Currie Lc and LD
framework. Instead, the procedures involve picking an initial spike value, adjusting that level up or down
based on whether the analyte was detected, and spiking seven replicates at the new level.
A brief description of each procedure is provided below.
ACIL's Case I Critical Value (ACIL Lc)
As with EPA's MDL, the ACIL Lc is an attempt to approximate Currie's critical value. Whereas
EPA's MDL is based on the standard deviation of blank samples spiked with low levels of the target
analyte, ACIL's Case I detection limit is based on the standard deviation of the blank samples run as part
of the laboratories ongoing QC program. (Because some methods will not yield a result when blanks are
analyzed, ACIL's Lc procedure is accompanied by a spiked sample approach that can be used with those
methods.)
Although ACIL does not formally define ACIL Lc, a footnote 2 to the procedure describes it as
"very similar to Currie's critical level, Lc (Anal. Chem. Vol. 40 No 3, March
1968, p586). It is the level at which there is a given confidence that a result can
be distinguished from the blank."
Key features of the ACIL Case I detection limit are as follows:
• The procedure relies on the use of blanks (instead of low-level spikes) to estimate standard
deviation.
• When a sufficient number of blanks are used in the calculation, the mean blank result is added
5-22
-------
into the calculation to account for high bias exhibited in the blanks.
• ACIL states that at least 7 blanks should be used, but recommends more (as many as 100). If the
number of replicates is small, ACIL recommends using a tolerance interval calculation for
estimating ACIL Lc. Instead of defining exactly what constitutes a "small" number of replicates,
ACIL loosely defines it as fewer than 20 or 30. The confidence level for the tolerance interval
also is not specified. If the tolerance level approach is used, the mean blank result is not included
in the calculation (unlike the calculation used when there are more than 20 to 30 results).
• If multiple instruments are to be used for the same test and will have the same reporting limit, a
minimum of 7 blank results from each instrument should be used, and the results should be
combined to generate the standard deviation.
• It is acceptable (and expected) that some results will have negative values, and these negative
values should not be censored. Outlier removal is allowed, using a statistically accepted test, if
appropriate cautions are taken to guard against excessive or inappropriate rejection of data.
• ACIL provides a verification procedure that is based on comparing the variance of the blank
results to results from a new set of blanks.
• ACIL suggests reporting all results that meet or exceed the ACIL Lc.
The formula for ACIL Lc is:
LC= ^+(/0.99,M-l *J)
Where A is the mean of blank results
s is the standard deviation of blank results, and
n is the number of blank results
ACIL's Case II Detection Limit
For Case n analyses, ACIL's procedures involve picking an initial spike value, adjusting that level
up or down based on whether the analyte was detected, and spiking seven replicates at the new level.
Details of this procedure are as follows:
• Unlike the procedures used for methods that yield numeric results, ACIL Case II procedures
would use spiked samples to determine the detection limit for methods that do not always yield
numeric results.
• An initial spike value is chosen based on prior experience. (Detailed guidelines are not provided.)
• One replicate at this level is analyzed; if the analyte is detected, a new sample should be prepared
at !/2 the initial spike value. If the analyte is not detected at the original level, a new sample
should be prepared at 2x the initial spike value. This process is repeated to find the lowest level
that can be detected
• Once that level is identified, a minimum of 7 replicates spiked at the lowest level at which that
analyte was detected are analyzed, and the replicates must be analyzed in three different batches.
If the analyte is detected in all replicates, the Case IIMDL is set to this spike value. If the analyte
is not detected in all 7 replicates, at least 7 additional replicates are prepared and analyzed at
twice this value. If the analyte is detected in all 7 replicates spiked at this higher concentration,
the Case II MDL is set to this higher spike value. This process is repeated until the analyte is
detected in all 7 replicates.
5-23
-------
• The ACIL procedure includes a verification step that consists of spiking the reference matrix at 1
to 3 times the Case IIMDL (or 1 to 4 times for multi-analyte methods) to verify that the analyte(s)
can be detected. If not, the test is repeated at increasing spike levels until detection, and setting
the Case II MDL to the level where the analyte(s) were first detected.
• ACIL suggests reporting all results that meet or exceed the Case n MDL.
5.1.6.2 Assessment of the ACIL Lc against the Evaluation Criteria
The following five subsections discuss the ACIL Lc approach and procedure in the context of the
five evaluation criteria that concern detection limit approaches (i.e., Criteria 1-4, and Criterion 6).
5.1.6.2.1 Criterion 1: The detection and quantitation limit approaches should be scientifically valid.
Condition 1: It can be (and has been) tested. Although ACIL had not conducted an exhaustive study to
test the ACIL Lc, ACIL did apply data generated from member laboratories to the procedure in order to
calculate ACIL Lc values. ACIL also compared those values with values produced by EPA's MDL using
the same procedure. The results of these tests are included in the public docket supporting this
assessment. As part of its own assessment, EPA also tested the procedure using data obtained from the
U.S. Geological Survey. In this testing, EPA generated ACIL Lc values, compared those values with
values produced by other procedures, and calculated error rates associated with each of the values. Given
these studies, the ACIL Lc meets this condition.
Condition 2: It has been subjected to peer review and publication. The ACIL procedure was developed to
support ACIL's comments on EPA's 2003, assessment, and it has been subjected to limited peer review
within ACIL's member community. Although ACIL references publication of the procedure on the ACIL
website, EPA made repeated attempts to locate the procedure on the website over a period of several
months, and was unable to locate it. Given the limited peer review beyond the member community, and
the lack of publication in a publicly accessible medium, the ACIL procedure does not meet this criterion.
Condition 3: The error rate associated with the procedure is either known or can be estimated. The ACIL
procedure meets this condition. According to the formula used for estimating ACIL Lc, the error rate, is
specified by a, with a suggested value of 0.01(1%). EPA was able to evaluate this error rate using a small
set of data provided by the US Geological Survey. The data included spiked and blank sample results for
18 pollutants, most of which were analyzed by multiple methods, yielding 75 unique analyte/method
combinations. For each combination, 25 - 52 blanks were provided. EPA used these blanks to calculate
the ACIL Lc, and compared the results of individual blanks with the calculated ACIL Lc. (Details of this
assessment are provided in Appendix C.) In theory, no more than 1% of the blanks should have produced
a result that exceeded the ACIL Lc. Although the sample size was insufficient to conclusively
demonstrate the error rate of the ACIL Lc, the results suggest the actual error rate is close to the estimate
of 1 %. In this case, the observed mean error rate was 1.9%, and the highest error observed for any
method/analyte combination was only 3.8%. Given the small sample size, failure of a single blank could
(and did) result in a 3.8% failure rate, suggesting that this study may yield an error rate that is larger than
that which would be observed in a larger study. Regardless, it is clear that the ACIL Lc meets this
condition because the estimated error rate is given as part of the procedure, and the actual error rate can be
calculated through studies such as the one described above.
5-24
-------
Condition 4: Standards exist and can be maintained to control its operation. The ACIL Lc is supported by
a written procedure (standard) to control its operation. However, the procedure appears to be in draft
form, is somewhat difficult to follow and interpret, and contains inconsistencies and ambiguities that are
typical of a draft document. In particular, the instructions for Case II are not as clear or detailed as those
for Case I.
As an example of the inconsistencies, a footnote to the ACIL Lc states that a tolerance interval
will be a more reliable estimate of the ACIL Lc if the number of blanks is small (i.e., fewer than 20 or 30).
This implies that the tolerance interval calculation and preferred ACIL Lc will converge as the number of
blank results increases. However, this is not the case. The tolerance interval calculation will almost
always yield a higher result than the preferred ACIL Lc calculation. The only way that the tolerance
interval calculation will result in an ACIL Lc that is either lower or equal to the original ACIL Lc is when
blank contamination is high (unlike the preferred ACIL Lc calculation, the tolerance interval calculation
does not include the mean of the blanks). It is unclear why the reliability of one calculation compared to
the other depends on the number of blank results.
An example of the ambiguities in the procedure is that the alternative calculations, such as the
tolerance interval calculation, are presented as suggestions instead of requirements. This could lead to
confusion, as now written, if, as ACIL recommends that, the ACIL Lc be used as a reporting limit.
A different type of ambiguity in the procedure concerns the lack of sufficient detail to ensure
consistent application. For example, it is not clear exactly when the tolerance interval calculation is to be
used because the procedure defines small as 20 - 30 samples. When would 20 samples be sufficient and
when would 30 samples be sufficient? Moreover, the tolerance interval calculation does not specify the
confidence level used. In an example, both 99% and 95% are given as possibilities. In comparison, the
critical value calculated in ASTM's IDE sets the confidence level at 90%. Setting the confidence level at
99% will yield an ACIL Lc value between 11% and 37% higher than one calculated at 95%, based on the
numbers of blank results for which the tolerance interval approach is suggested.
Given these problems, the current ACIL procedure does not meet this condition.
Condition 5: It has attracted widespread acceptance within a relevant scientific community. The ACIL
Lc was supported by a large number of commenters, most of whom came from the ACIL member
community or the environmental laboratory community. Of note, however, is that supporters included
instrument vendors, consultants, and several members of the industrial community, including the Inter-
industry Analytical Group which offered its own approach to detection and quantitation and which has
been highly supportive of the ASTM IDE and IQE approaches. Therefore, EPA believes that the ACIL Lc
meets this condition.
5.1.6.2.2 Criterion 2: The approach should address realistic expectations of laboratory and method
performance, including routine variability.
The ACIL Lc is designed to address realistic expectations of laboratory and method performance,
including temporal variability, instrument variability, analyst variability, and high bias observed in blank
results. Based on EPA's analysis of the ACIL Lc presented in Appendix C, EPA believes that the
approach meets this criterion provided it is interpreted and applied consistently. (Concerns about the need
for clarification of the procedure are described in Section 5.1.6.1, Condition 4).
5-25
-------
5.1.6.2.3 Criterion 3: The approach should be supported by a practical and affordable procedure that
a single laboratory can use to evaluate method performance
The ACIL Lc meets this criterion. It is similar to the EPA MDL procedure, but it relies on the use
of QC data generated during routine laboratory operations, thereby making it even more cost effective
than the MDL.
5.1.6.2.4 Criterion 4: The detection level approach should estimate the theoretical concentration at
which there is 99% confidence that the substance is actually present when the
analytical method is performed by experienced staff in a well-operated
laboratory
Footnote 2 to the ACIL procedure describes the ACIL Lc as "very similar to Currie's critical
level, Lc (Anal. Chem. Vol. 40 No 3, March 1968, p586). It is the level at which there is a given
confidence that a result can be distinguished from the blank." According to the formula used for
estimating ACIL Lc, the error rate is specified by a, with a suggested value of 0.01(1%). This alpha value
means that, if the analyte is not present in the sample, it will be reported as present (i.e., a false positive)
no more than 1% of the time. In lay terms, this suggests 99% confidence that, if a substance is reported as
present, it really is present. Therefore, the ACIL Lc meets this criterion.
5.1.6.2.5 Criterion 6: Detection and quantitation approaches should be applicable to the variety of
decisions made under the Clean Water Act, and should support State and local
obligations to implement measurement requirements that are at least as
stringent as those set by the Federal government
If EPA's interpretation of the ACIL procedure is correct, the ACIL Lc appears to meet this
criterion.
5.1.7 Evaluation of the USGS Long-term Detection Limit (USGS LT-MDL)
The USGS National Water Quality Laboratory (NWQL) began using the EPA MDL procedure in
1992. USGS NWQL has since developed a variant of the MDL called the long-term MDL (LT-MDL) that
has been in routine use by the NWQL since 1999. The procedure for calculating the LT-MDL is
described in Section 5.1.7.1 below. Section 5.1.7.2 describes EPA's assessment of the LT-MDL against
the five evaluation criteria that concern detection limit approaches (i.e., Criteria 1-4, and Criterion 6).
5.1.7.1 Description of the USGS Approach and Procedure
As described in the USGS Open-File Report 99-193, the LT-MDL is a modification of the EPA
MDL designed to "capture greater method variability," thereby leading to higher detection limits than
those obtained using the EPA MDL procedure. As described by USGS, and noted in Chapter 2, the LT-
MDL is based on many of the same fundamental assumptions as the MDL, namely:
1. Normal data distribution,
2. Constant standard deviation from the spike concentration down to zero, and
3. Best-case detection condition (because LT-MDLs typically are determined by spiking the analyte in a
clean matrix, e.g., reagent water).
5-26
-------
The LT-MDL is determined using low-level spikes of reagent water. The three primary
differences between the EPA MDL and the USGS LT-MDL procedures are:
1. Larger minimum number (24) of spike samples,
2. Longer time period, and
3. Combining results from different instruments and analysts in the determination of the LT-MDL.
The USGS Open File Report does not provide an example of the exact calculation used for the
LT-MDL. EPA originally presumed that the standard deviation of the results from the 24 spiked sample
analyses is multiplied by the Student's f-value appropriate for 23 degrees of freedom (t=2.499).
However, USGS comments submitted in response to EPA's assessment of detection and
quantitation approaches included a copy of a presentation from the USEPA Region 6 12th Annual Quality
Assurance Conference, in Dallas, Texas in August 2002. That presentation provided significant additional
information on the calculation of the LT-MDL. Specifically, the LT-MDL uses "F-pseudosigma" (F0) in
place of S, the sample standard deviation, used in the EPA MDL calculation. F-pseudosigma is a non-
parametric measure of variability that is based on the interquartile range of the data. The LT-MDL may
be calculated using either the mean or median of a set of long-term blanks, or from long-term spiked
sample results, such that:
LT-MDL = M +
where:
M = mean or median of blank results
n = number of spiked sample results, and
F0 = F-pseudosigma, a nonparametric estimate of variability calculated as:
F _ Qi- Ql
1.349
where:
Q3 and Q! = the 75th percentile and 25th percentile of spiked sample results, respectively.
USGS believes that the use of F0 provides an estimate that is more robust and not influenced by outliers.
Like the EPA MDL, the LT-MDL is designed to limit the chance of a false positive result to < 1%.
However, the LT-MDL is designed to be used in conjunction with a "laboratory reporting level" (LRL) as
part of an overall reporting scheme for the NWQL. As described by USGS, the LRL is set as a multiple of
the LT-MDL. The multiplier varies, depending on the mean/median recovery of the analyte in the spiked
samples used for the LT-MDL. If the mean or median recovery is 100%, then the multiplier is 2. At 75%
mean or median recovery, the multiplier increases to 2.7, and at 50% recovery, the LRL multiplier
increases to 4. In each of these cases, the multiplier is essentially equivalent to dividing twice the LT-
MDL by the mean recovery (i.e., 2.7 LT-MDL ~ 2 LT-MDL/75%).
The LRL is designed to achieve a risk of < 1 % for both false negatives and false positives. The
reporting scheme used at the NWQL with the LT-MDL and LRL does not censor results at the LRL, and
the laboratory reports all results between the LT-MDL and the LRL with a lab-specific flag.
5-27
-------
The USGS presentation from the 2002 meeting describes how USGS enhanced the LT-MDL
procedure by using their large volume of uncensored blind laboratory blank data as a reality-check on the
LT-MDL derived from spiked reagent water samples. In cases where the standard deviation used to
calculate an LT-MDL based on blind blank data is significantly different (especially when greater) from
the standard deviation used to calculate the spike-based LT-MDL, the blank data are used to calculate the
LT-MDL. Blind blank data also are used to evaluate whether the calculated LT-MDL requires an off-set
correction for blank bias, i.e, [LT-MDL = (S x Student's f) + median or mean blank concentration]. This
offset is similar, but not identical, to the ACIL Case I procedure described in Section 2.3.3 of this
document. The LT-MDL offset correction compensates for a blank distribution that is not centered at zero
(an assumption in the EPA MDL procedure).
The NWQL has found that this blank bias correction to the LT-MDL is especially important for
blank-limited analytes, including some metals, total organic carbon, phenol, and nutrients. In practice, the
NWQL recalculates the LT-MDL annually, and compares the results between years using Levene's test of
equal variance, which they have found to be less influenced by departures from normality than the F-test -
an important consideration given that the LT-MDL is based on a non-parametric estimate of variability.
5.1.7.2 Assessment of the USGS LT-MDL against the Evaluation Criteria
The following five subsections discuss the USGS approach and procedure in the context of the
five evaluation criteria that concern detection limit approaches (i.e., Criteria 1-4, and Criterion 6).
5.1.7.2.1 Criterion 1: The detection and quantitation limit approaches should be scientifically
valid.
Condition 1: It can be (and has been) tested. The LT-MDL meets this condition.
USGS has tested and used the LT-MDL since October 1998. Evaluation and use of the LT-MDL
began with four methods in use by the NWQL for low-level volatiles by GC/Ms, trace metals by
ICP/AES, Kjeldahl nitrogen, and phosphorus. According to the Open File Report, the LT-MDL was
scheduled for testing in 17 additional methods, including semivolatile organics, organochlorine pesticides,
organophosphorus pesticides, pesticides analyzed by EIPLC, metals by ICP/MS, metals by GFAA, and ion
chromatography.
EPA used a combination of blank and spiked data submitted by USGS to compute the USGS LT-
MDL and compare it to the EPA MDL The blanks were analyzed by USGS over a period of one year and
represented a combination of 78 analytes, methods, and matrices, while the spiked sample results
represented 39 analytes, methods, and matrices. The analytes were all metals or wet chemistry parameters
such as phosphorus and nitrate/nitrite.
Condition 2: It has been subjected to peer review and publication. The LT-MDL does not appear to meet
this condition.
Information on the LT-MDL is relatively limited and EPA is not aware of additional USGS
publications beyond Open File Report 99-193 and the August 2002 presentation. EPA did not identify
any additional publications regarding the LT-MDL in its earlier literature search. The Open File Report
itself does not provide any indication that it was subject to a peer-review process.
5-28
-------
Condition 3: The error rate associated with the procedure is either known or can be estimated. The error
rate is specified by a, with a value of 0.01(1%). Therefore, the LT-MDL may meet this condition.
In its evaluation of USGS data submitted as comments (see Appendix C) EPA found that the
mean percentage of blanks results that exceeded the detection limit estimate (LT-MDL) ranged from 3.7%
to 4.4%, depending on whether the mean or median blank result was used to estimate LT-MDL. These
rates exceeded that of the EPA MDL Therefore, although EPA's evaluation found that the error rate for
the LT-MDL exceeded the theoretical error rate designed into the procedure, the error rate can be
estimated from actual data.
Condition 4: Standards exist and can be maintained to control its operatioa The LT-MDL may partially
meet this condition, in that the NWQL may have formal procedures in place that more fully describe the
LT-MDL. However, as noted above, the information in the Open File Report does not include an explicit
formula for calculation of the LT-MDL, nor are other details of the overall procedure, such as the choice
of spiking levels, provided in a clear and consistent fashion. The August 2002 presentation provides
critical information about the use of Fa that is not present the Open File Report. The LT-MDL could meet
this criterion, if the procedure were clearly documented by USGS.
Condition 5: It has attracted widespread acceptance within a relevant scientific community. The LT-
MDL does not meet this condition
EPA believes that the LT-MDL is only used at the NWQL. Several commenters, including ACIL,
suggested that EPA examine the USGS LT-MDL more closely, specifically in regards to its inclusion of
long-term variability. There is, however, no evidence in the comments that the concept has achieved a
large following among laboratories or other agencies.
5.1.7.2.2 Criterion 2: The approach should address realistic expectations of laboratory and
method performance, including routine variability.
EPA believes that the LT-MDL meets this criterion because it incorporates the variability of
responses over a long time period, and where a laboratory has multiple instruments and analysts running
the same analysis, it incorporates variability across instruments and analysts.
5.1.7.2.3 Criterion 3: The approach should be supported by a practical and affordable
procedure that a single laboratory can use to evaluate method
performance
The LT-MDL partially meets this criterion. However, the LT-MDL is not a detailed readily
available "procedure". Also, the LT-MDL requires data collected over a 12-month period. Given that
many State regulatory programs require that laboratories provide an annual demonstration of capabilities,
including demonstrating their detection limits, the use of the LT-MDL would have to be limited to those
laboratories that already have a year's worth of data available. Some other single-lab approach would
have to be used for an initial demonstration of method performance.
5.1.6.2.4 Criterion 4: The detection level approach should estimate the theoretical
concentration at which there is 99% confidence that the substance is
actually present when the analytical method is performed by experienced
staff in a well-operated laboratory
5-29
-------
According to the formula used for calculating the USGS LT-MDL, the error rate is specified by a,
and the LT-MDL is designed with a value of 0.01 (1 %). Because the method uses a nonparametric
estimate of S, it may not always yield a 1% false positive rate. EPA empirical analysis indicates false
positive rates in the range of 3.7% to 4.4%. This compares favorably with the performance of other
methods. Thus, the LT-MDL adequately meets this criterion at least in practice.
5.1.7.2.5 Criterion 6: Detection and quantitation approaches should be applicable to the
variety of decisions made under the Clean Water Act, and should support
State and local obligations to implement measurement requirements that
are at least as stringent as those set by the Federal government
The LT-MDL may meet this criterion. The LT-MDL is designed as part of a broader reporting
scheme and it is unclear that EPA, States, and local authorities would be willing or able to use results
reported according to that scheme in enforcement scenarios (e.g., "flagged" data).
5.1.8 Evaluation of the Inter-industry Analytical Group (HAG) Full-Range Validation and
Sensitivity Test
In December 2002, the Inter-Industry Analytical Group (IIAG) submitted a proposal to EPA that
recommends (1) a sensitivity test intended to "replace the MDL as a test of whether an individual
laboratory is performing adequately," and (2) an interlaboratory validation study design intended to
characterize precision and accuracy of methods used for regulatory compliance. Although their approach
was received too late for consideration prior to publication in the 2003 Assessment Document, EPA
provided notice of the approach, requested public comment on it, and agreed to evaluate the IIAG
approach in updating the 2003 assessment. Section 5.1.8.1 describes the HAG approach, and Section
5.1.8.2 describes EPA's assessment of the IIAG approach against the applicable evaluation criteria.
5.1.8.1 Description of the IIAG Approach and Procedure
Full Range Validation
IIAG has proposed that, EPA commit to performing interlaboratory method validation studies
designed to produce a "full range" of data, including precision and accuracy, from the point of instrument
detection to the upper end of the working range. IIAG has indicated that "such a full range validation will
enable EPA to consider DL/QL options in light of data quality objectives without being constrained by a
limited database." IIAG suggests that, at a minimum, EPA should commit to performing such full range
validation studies for all new methods that it develops and that all organizations submitting new methods
for EPA approval should be required to provide the full range data as well.
Sensitivity Test
IIAG also has proposed that EPA consider the use of a "sensitivity test" instead of the MDL to
demonstrate that a laboratory is capable of performing according to EPA expectations at the lower range
of a test method. IIAG's suggested process for developing this test is as follows:
• EPA would first identify the lowest concentration at which the entire analytical system gives a
recognizable signal and acceptable calibration point.
• EPA would then select a simple dilution of that concentration, and develop QC criteria based on
the test results from several laboratories performing the test at that dilution (in the same way that
QC criteria are developed by EPA for initial precision and recovery demonstrations in methods
5-30
-------
such as Method 1631).
• Laboratories could then perform an "Initial Performance Demonstration" (IPD) of their capability
to achieve the desired sensitivity by (1) analyzing several replicates of the same sample dilution
(using the full method), (2) using the results to compute the standard deviation, and (3) confirming
that the results fall within the QC criteria range. HAG emphasizes that the dilution level would
not be considered the detection level, but rather a performance level.
IIAG further suggests that this multi-replicate IPD test would be verified on an ongoing basis. To
minimize complexity, HAG suggests that the ongoing test be conducted at the same spike level as their
"Initial Performance Demonstration." IIAG did not suggest a specific frequency for conducting these
ongoing tests.
Finally, HAG suggests that EPA commit to using this IPD sensitivity test in lieu of the MDL, and
that EPA express a willingness, subject to funding availability or a third party commitment, to perform
testing as necessary to develop "sensitivity" QC criteria, and to modify the few existing Part 136 methods
that require the MDL for IPD.
Section 5.1.8.2 below discusses EPA's evaluation of the scientific elements proposed IIAG
approach.
5.1.8.2 Assessment of the IIAG Approach against the Evaluation Criteria
The following six subsections discuss the HAG approach and procedure in the context of the six
evaluation criteria. The first three criteria apply to both detection and quantitation limits, Criterion 4
applies to detection limits only, Criterion 5 applies to quantitation limits only, and Criterion 6 applies to
both. Because the IIAG full range validation and sensitivity test approach applies to both types of limits,
all 6 criteria are discussed below.
5.1.8.2.1 Criterion 1: The detection and quantitation limit approaches should be scientifically
valid.
Condition 1: It can be (and has been) tested. To EPA's knowledge, the IIAG sensitivity test approach has
not been tested by any organization, including IIAG. The HAG approach is still a rough framework, and
basic details, such as the number of replicates required and the actual spiking levels to be used, still need
to be specified. Testing of the approach in its current framework is possible but would be very expensive,
one might have to conduct tests with multiple spiking levels and with varying numbers of replicates, for
example, to be sure that the tests will reflect the final sensitivity test procedure. If the procedure were
refined to describe the exact steps and requirements, it could be tested more efficiently.
IIAG's full validation study approach can be and has been tested. For example, EPA conducted a
full interlaboratory validation study of Method 1631 prior to promulgating the method at 40 CFR 136.
That study, which involved 12 participating laboratories, yielded an overall mean percent recovery of 93
and an overall relative standard deviation of 13 percent across all samples.
IIAG has stated that "Although the full-range interlaboratory is aimed at characterizing a method's
ability to quantify rather than to detect a pollutant concentration, the study could be used to establish an
interlaboratory detection level as well" and "The best solution for performing a full-range validation to
establish detection and quantitation levels and precision and bias for promulgating nationwide standards
and compliance levels is the ASTM IDE/IQE approach."
5-31
-------
The ASTM IDE and IQE are constructed by fitting a model to variability versus concentration
data, rather than being derived from the standard deviation of replicate measurements of a single
concentration level. As discussed in Section 5.1.2 and detailed in Appendix C, EPA used data from the
Episode 6000 study to compare IDEs calculated using data from all 16 concentration levels reported to
IDEs calculated using data from only 5 of the concentrations (i.e., at 5, 10, 20, 40, and 80 times the
standard deviation of replicate measurements of a blank sample or the lowest level at which measurements
could be made). Results of the comparison are summarized in Table 9 of Appendix B to the draft TSD.
The results show that the median 16-point IDE is approximately 1.3 times greater than the median 5-point
IDE, indicating that data resulting from measurements of concentration levels in the region of detection
and quantitation in some cases may yield lower IDE's than data from a wider range of concentration data.
EPA refers readers of this document to Sections 5.1.2.2.1, 5.2.2.2.1, 5.2.2.2.2, and Appendix B
for a discussion of additional reasons why EPA believes the ambiguities and inconsistencies in IDE/IQE
procedures preclude these procedures from being the best solution for performing a full range study to
estimate detection and quantitation limits.
Condition 2: It has been subjected to peer review and publication. The HAG procedure does not meet this
criterion. EPA is not aware of any peer review or publication of the document in a peer reviewed journal.
The HAG document was submitted directly to EPA by the Petitioners, and EPA made the document
available to the public for comment.
Condition 3: The error rate associated with the procedure is either known or can be estimated. At present
the IIAG's approach consists of a proposed framework rather than a detailed procedure. It lacks key
specifics, such as how many replicates would be used in the IPD phase of the test, and what spiking levels
would be used. HAG suggests that EPA would select these levels, and suggests "probably 4 - 7" for the
number of replicates.
While IIAG suggests the dilution would be a simple dilution of the lowest calibration standard,
offering "1/3 or %, for example", they also state that "It is not absolutely necessary to reduce the spike
level below the lowest calibration point, however, and the sensitivity test could be performed with a spike
at the lowest calibrations standard instead of at a dilution of it." No guidelines are offered for which of
these levels (or other levels) should be chosen, nor are guidelines offered for the number of replicates
needed.
Given the lack of detail, the current framework would be subject to different interpretations by
different readers or users, and the error rate associated with the procedure would vary depending on how
the procedure was implemented. Because the error rate is neither known, nor can it be estimated, the
IIAG approach does not meet this condition.
The IIAG procedure is a framework with interesting aspects for further consideration by the full
scientific and regulatory community. EPA would be willing to work with IIAG and other stakeholders to
identify the details needed to augment this framework to where it would meet this condition.
Condition 4: Standards exist and can be maintained to control its operation As previously noted, the
IIAG approach consists of a proposed framework rather than a detailed procedure framework, and lacks
key details that are needed to control its operation. Given the lack of detail, the current framework does
not meet this condition.
5-32
-------
Again this procedure is a framework with interesting aspects for further consideration by the full
scientific and regulatory community. EPA would be willing to work with IIAG and other stakeholders to
identify the details needed to augment this framework to where it would meet this condition.
Condition 5: It has attracted widespread acceptance within a relevant scientific community. The IIAG
procedure does not meet this condition. It was suggested by a limited group of the relevant scientific
community (industry firms that comprise the "Inter-industry Analytical Group" and whose wastewater
discharges are regulated under the Clean Water Act), and comments on the their approach were mixed.
Excluding comments submitted by IIAG itself, EPA received comments from:
• Three electric power producers whose discharges also are regulated under CWA,
• Two publicly owned wastewater treatment systems which regulates industrial discharges to their
system under CWA and whose own discharges are subject to regulation under CWA,
• Two commercial environmental laboratories that utilize the methods and detection limit
procedures approved at 40 CFR 136 to serve their client's needs,
• One trade council, and
• One private citizen.
All three electric power firms supported the HAG approach. The two publicly owned treatment
systems offered mixed reviews. One supported the sensitivity test and offered suggestions for further
consideration; the other opposed the sensitivity test but offered limited support of the interlaboratory
validation studies, suggesting that they be limited to the relatively small group of priority pollutants whose
water quality based effluent limits are below the method reporting levels. Both of the environmental
laboratories were opposed to the IIAG approach, and the trade council suggested that it should be used "as
an alternative procedure for dischargers to implement... on a site-specific basis, at their discretion", noting
that "As an alternate method, facilities would be able to deal with this on a case-by-case basis and would
not need to utilize numerous laboratories to develop the more elaborate detection limits and quantitation
limits that the HAG proposes".
Given these comments, it would appear that acceptance may be widespread within the industrial
discharger community, but it is not widespread among the entire relevant scientific community.
5.1.8.2.2 Criterion 2: The approach should address realistic expectations of laboratory and
method performance, including routine variability.
In principle, the HAG sensitivity test meets this criterion because it is intended to provide realistic
information about laboratory and method performance, both with an initial demonstration and with
follow-up demonstrations that provide information concerning routine variability. However, and as
previously noted, the procedure is not sufficiently detailed to allow laboratories to meet this criterion. To
clearly meet this criterion, detailed specifications to allow for consistent implementation of the procedure
throughout the laboratory community need to be developed.
5.1.8.2.3 Criterion 3: The approach should be supported by a practical and affordable
procedure that a single laboratory can use to evaluate method
performance
If the IIAG framework was developed into a detailed procedure, this sensitivity approach would
meet this single laboratory criterion. This could complement the IIAG full range validation study, which
does not meet this criterion because it is an interlaboratory procedure. The sensitivity test, once detailed,
could be performed by a single laboratory and used to evaluate method performance.
5-33
-------
5.1.8.2.4 Criterion 4: The detection level approach should estimate the theoretical
concentration at which there is 99% confidence that the substance is
actually present when the analytical method is performed by experienced
staff in a well-operated laboratory
Because the spiking level to be used in HAG'S sensitivity test is not defined it is not possible to
evaluate whether that test meets or does not meet this criterion. HAG also suggests that a full-range
validation study should be used to establish an interlaboratory detection limit, and recommends use of the
ASTM IDE procedure as the best means of doing so. If this is the case, the full range validation study
would fail this criterion for the reasons given in Section 5.1.2.2.4 regarding the IDE.
5.1.8.2.6 Criterion 5: The quantitation limit should identify the concentration that gives a
recognizable signal that is consistent with the capabilities of a method
when a method is performed by experienced staff in well-operated
laboratories.
The IIAG's proposed sensitivity test requirement is likely to meet this criterion once details
regarding the procedure are specified. Depending on the spiking levels that are specified in the final
procedure, however, it is very likely that the HAG sensitivity test may not identify the lowest
concentration at which the signal is recognizable when the method is performed by experienced staff in a
well-operated laboratory.
5.1.8.2.6 Criterion 6: Detection and quantitation approaches should be applicable to the
variety of decisions made under the Clean Water Act, and should support
State and local obligations to implement measurement requirements that
are at least as stringent as those set by the Federal government
IIAG's suggested use of a full range validation study meets this criterion because such validation
studies provide useful information about the performance of the method. As noted previously, EPA
typically conducts interlaboratory validation studies at multiple concentrations ranges before
promulgating a method for nationwide use at 40 CFR part 136. However, for the reasons discussed
elsewhere in this document, EPA does not agree that data collected across the full range of the method
should be used to establish detection or quantitation levels.
In the absence of a detailed procedure that could be use to fully evaluate IIAG's, it is difficult to
determine if the HAG sensitivity test meets this criterion.
5.2 Quantitation Limit Approaches
Sections 5.2.1 through 5.2.4 describe EPA's assessment of four quantitation limit approaches.
Each discussion is divided into two major subsections. The first subsection describes the approach and,
where applicable, the procedure that supports the approach, and the second subsection details EPA's
assessment of the approach based on the five criteria established in Chapter 4 for evaluating quantitation
limit approaches. These criteria are Nos. 1 -3, 5 and 6; No. 4 only is applicable to detection limits.
5-34
-------
5.2.1 Assessment of the EPA Minimum level of Quantitation (ML)
Section 5.2.2.1 provides an overview of the ML approach and the procedures used to implement
the approach. Section 5.2.2.2 contains EPA's assessment of the ML against the five evaluation criteria
that concern quantitation limit approaches (i.e., Criteria 1-3, and Criteria 5 and 6).
5.2.1.1 Description of the ML Approach and Procedures
The definition of the ML includes a statement of the approach and the procedures used to
establish the ML. This definition states that the ML is:
"the lowest level at which the entire analytical system must give a recognizable signal
and acceptable calibration point for the analyte. It is equivalent to the concentration of
the lowest calibration standard, assuming that all method-specified sample weights,
volumes, and clean up procedures have been employed. The ML is calculated by
multiplying the MDL by 3.18 and rounding the results to the number nearest to (1, 2, or
5) x 10", where n is an integer."
The ML is designed to provide a practical embodiment of the quantification level proposed by
Currie and adopted by IUPAC. It is functionally analogous to Currie's "determination limit" (described in
Chapter 2, Section 2.1) and the American Chemical Society's Limit of Quantitation (LOQ). The LOQ is
discussed in Section 5.2.3 of this chapter. Chapter 2 (Section 2.2.2) describes the ML approach in
additional detail.
The first part of the ML definition (i.e., the lowest level at which the system gives a recognizable
signal and acceptable calibration point for the analyte) ties the quantification limit to the capabilities of
the measurement system. The second part of the ML definition provides a procedural means for
establishing the ML.
The procedural component of the definition is designed to yield an ML value that equals
approximately 10 times the standard deviation of replicate analyses used to determine the MDL. (The
exact value corresponding to 10 times the standard deviation is rounded to avoid error that would arise
from preparation of calibration standards at exact, "unrounded" concentrations.) The 3.18 multiplier is
derived by dividing 10 by the value of the t-statistic for seven replicates. Laboratories that choose to
perform MDL studies with more than the required minimum of seven replicates follow the instructions in
appendix B of 40 CFR part 136 to select the t-statistic value for the number of replicates used. Therefore,
the 3.18 multiplier for the ML calculation should be proportionally adjusted. Similarly, the Student's t-
value is adjusted when a laboratory performs the optional iterative test described in Step 7 of the MDL
procedure, or if outlier testing results in the use of less than seven replicates to establish the MDL.
5.2.1.2 Assessment of the ML against the Evaluation Criteria
The following five subsections discuss the ML approach and procedure in the context of the five
evaluation criteria that concern quantitation limit approaches (i.e., Criteria 1-3, and Criteria 5 and 6).
5.2.1.2.1 Criterion 1: The detection and quantitation limit approaches should be scientifically
valid.
Condition 1: It can be (and has been) tested. The ML meets this condition. The ML has been used
experimentally since 1979 and in the regulatory context since 1984. The ML is tested each time a
5-35
-------
laboratory calibrates an instrument because methods that include the ML require that it be included as the
lowest non-zero standard in these calibrations.
EPA also has tested the MDL and ML procedure with ten different techniques at decreasing spike
concentrations to evaluate how well the MDL and ML procedures characterized the region of interest in
preparation for this reassessment of detection and quantitation limit approaches. Results of the study
suggest that (1) although the calculated MDL and ML could vary depending on the spike level used, the
procedure was capable of reasonably estimating detection and quantitation limits when the full iterative
MDL procedure was employed, and (2) the rounding process employed to determine the ML generally
yielded consistent MLs even with slight variations in the calculated MDL. EPA recognizes that additional
guidance may be necessary on the selection of the initial spiking level and uses of the iterative procedure.
In other words, if the procedure for establishing an ML is properly implemented for a given
method, it will yield an ML value that is consistent with the approach, and this ML value can be verified
(tested) by a laboratory when it calibrates the instrument used to analyze samples by the method.
One of the stakeholders commenting on EPA's 2003 assessment suggested that the ML failed to
meet this condition because EPA should have tested it in "real world" matrices. EPA does not agree with
this suggestion for several reasons. First, it is not practical or possible to test detection limits in every real
world matrix, and there is no consensus as to which real world matrix would represent an appropriate real
world matrix for testing. Second, many real world matrices contain the target pollutant at levels well
above the detection or quantitation limit, making it impossible to characterize what can and cannot be
detected at low levels. In theory, the sample could be diluted to dilute the target pollutant, but in practice
sample dilution would also likely dilute any interferences that might be present, thereby defeating the
purpose of using a real world matrix. The current EPA approach, which exhaustively tests the ML
procedure in a reference matrix using multiple techniques and ten different concentrations that span the
entire region of interest, is more than adequate to constitute "testing" of the ML procedure. On the other
hand, where data suggests that matrix interferences may significantly affect achievable quantitation and
detection limits, this should be considered by a permit writer on a case by case basis.
Condition 2: It has been subjected to peer review and publication. The ML has not been published in a
peer reviewed journal. However, it was evaluated by four peer reviewers as part of EPA's assessment of
detection and quantitation limits. These reviewers noted that:
"The MDL and ML concepts evaluated in Section 5.1.1 and 5.2.1, respectively, are shown
in this evaluation to be technically sound and practical. " (Wait, 2002)
"With respect to the limit of quantitation concept, the EPA ML is as good as any of the
others given... " (Rocke, 2002)
"The MDL and ML have stood the test of time and provide a proven methodology which
meets evaluation criteria stated in the TSD. " (Cooke, 2002).
In addition, the definition of the ML describes the approach and the procedures used to establish
the ML. This definition is included in EPA Method 1631, which was extensively peer reviewed in
accordance with EPA policies on peer review prior to publication and promulgation. Given that EPA's
policies on peer review are as stringent as or more stringent than those used by many published journals,
the ML has met a high standard of scientific review and scrutiny, and therefore, meets the intent of this
condition.
5-36
-------
Condition 3: The error rate associated with the procedure is either known or can be estimated. If
rounding is not considered, the error can be easily estimated. The calculation is still straightforward, but
tedious, when the ML rounding procedures are employed. Given these caveats, the ML partially meets
this condition.
Condition 4: Standards exist and can be maintained to control its operatioa The ML meets this criterion.
Detailed procedures (i.e., standards) for establishing the ML are embodied in the definition.
Condition 5: It has attracted widespread acceptance within a relevant scientific community. The ML
meets this condition. The ML is analogous to the American Chemical Society's LOQ and to the
ISO/IUPAC quantification limit, which suggests widespread acceptance.
5.2.1.2.2 Criterion 2: The approach should address realistic expectations of laboratory and method
performance, including routine variability.
The ML procedure meets this criterion. It is designed to provide a means by which a laboratory
can demonstrate performance with a method under routine laboratory operating conditions. All recently
developed EPA CWA methods require that a laboratory calibrate its instrument prior to analyzing
environmental samples. The ML is defined as the lowest non-zero standard in the laboratory's calibration,
and therefore, reflects realistic expectations of laboratory performance with a given method under routine
laboratory conditions (i.e., under conditions of routine variability).
The ML is based on the standard deviation of replicate analyses used to establish the MDL. As
described in Section 5.1.1.2.2, these analyses are performed to characterize laboratory and method
performance, including routine variability, at low concentrations. When a laboratory performs an MDL
study with seven replicates and multiplies the results by 3.18, the laboratory has demonstrated that it can
achieve expected levels of performance at the ML.
Due to the variability inherent in measurement science, instrumentation, and the humans
conducting analyses, laboratories may routinely obtain limits that are lower or higher than those obtained
in another laboratory. Thus, when an ML is determined during method development, it is important to
determine that ML in more than one laboratory to ensure the ML published in the method reflects
demonstrated expectations of method performance in a community of laboratories. It is not necessary for
this community to include the entire universe of all possible laboratories that might desire to practice the
method. Rather, during the stages of method development and validation, this community only should
include well-operated laboratories with analysts who are experienced with the techniques used in the
method, and have some familiarity conducting all of the steps in the new method before generating MDLs
that will be published with the new method. See Section 5.1.1.2.2 for additional discussion of this topic.
5.2.1.2.3 Criterion 3: The approach should be supported by a practical and affordable procedure
that a single laboratory can use to evaluate method performance.
The ML meets this criterion. It is designed for use by a single laboratory. The ML can be directly
determined from the MDL, which is among the most affordable of procedures for determining detection
limits (see discussion in Section 5.1.1.2.3 for additional details), by a simple multiplication of the MDL
and a application of a rounding procedure.
5-37
-------
5.2.1.2.4 Criterion 5: The quantitation limit approach should identify the concentration that gives a
recognizable signal that is consistent with the capabilities of the method when
a method is performed by experienced staff in well-operated laboratories.
The ML meets this criterion. The ML can be verified in a laboratory each time it calibrates an
instrument. This calibration depends on identification of a recognizable signal for the analyte. In
addition, because EPA includes the ML as the low point in the calibration range, that concentration is
within the capabilities of the method, as demonstrated by either multiple single-laboratory studies or a
multi-laboratory study of the method.
Notwithstanding the preceding, analysis of Episode 6000 data (see appendices) produced
anomalous results from two methods (EPA 502.2 and 524.2) that employ instrument thresholds. For 17%
of EPA 502.2 and 49 % of EPA 524.2 analytes the calculated ML was below the concentration at which
all seven spiked replicates were detected, i.e. less than the lowest MDL spike. The Episode 6000 dataset is
not reflective of a typical compliance measurement or method development study because the range of
concentrations studied encompassed several orders of magnitude and included concentrations well below
the MDL. This atypical range was employed to push the limits of the instrumentation and the theory
underlying determination of the variability of measurements.
In a qualified operating laboratory, or during a method development study, if MLs were calculated
to be less than the concentration at which all seven spiked MDL replicates were detected, the laboratory
would take corrective measures. When a method is developed for EPA's CWA program, each laboratory
in a multi-laboratory study would consult with EPA and take corrective measures, such as calibration
adjustments so that reported MDLs are above the signal threshold. In these cases, the calculation of ML =
3.18 * MDL always yields a value greater then the MDL and meets the criterion of "recognizable signal".
5.2.1.2.5 Criterion 6: Detection and quantitation approaches should be applicable to the variety of
decisions made under the Clean Water Act, and should support State and
local obligations to implement measurement requirements that are at least as
stringent as those set by the Federal government.
The ML meets this criterion. It has been used in Clean Water Act programs since 1984.
5.2.2 Assessment of the IQE
The Interlaboratory Quantitation Estimate (IQE) was published by ASTM, International in 2000
as standard D 6512. The IDE was developed with support from members of the regulated industry in an
attempt to provide a comprehensive quantitation limit procedure that addresses the concerns of the
regulated industry, statisticians, and analysts. A brief summary of the procedure for establishing an IQE is
given in Section 5.2.2.1. Section 5.2.2.2 presents EPA's assessment of the IQE against the five criteria
established for evaluating quantitation limit approaches (i.e., Criteria 1-3, and Criteria 5 and 6).
5-38
-------
5.2.2.1 Description of the IQE Approach and Procedure
The ASTM Designation D 6512 is the Standard Practice Interlaboratory Quantitation Estimate.
As stated in the practice:
"IQEZ% is computed to be the lowest concentration for which a single measurement from a
laboratory selected from the population of qualified laboratories represented in an
interlaboratory study will have an estimated Z % relative standard deviation (Z % RSD,
based on interlaboratory standard deviation), where Z is typically an integer multiple of
10, such as 10, 20, or 30, but Z can be less than 10."
The IQE is determined and verified using a procedure containing 5 major steps with
approximately 46 substeps and conditions. The full text of the IQE procedure is available from ASTM
International. The 5 major steps and their functions are given in Section 6 of the IQE procedure and are
summarized below:
1. Overview of the procedure.
2. IQE Study Plan, Design, and Protocol - in this section, the task manager (study supervisor) chooses
the analyte, matrix, and analytical method. Details are given for the appropriate range of study
concentrations; the model of recovery vs. concentration; the study protocol (ASTM Practice D 2777 is
suggested); the instructions to be given to the participating laboratories, including reporting
requirements; the allowable sources of variation; and the number of laboratories, analysts,
measurement systems, and days over which the study will be conducted.
3. Conduct the IQE Study, Screen the Data, and Choose a Model - after the study data are collected and
screened according to ASTM Practice D 2777, the interlaboratory standard deviation (ILSD) versus
concentration data are tabulated and one of three models is fit to the data. The first attempt is at
fitting a constant model. If the attempt fails, a straight-line model is attempted. If the straight-line
model fails, a hybrid (Rocke/Lorenzato) model is fit. After fitting, the model is evaluated for
reasonableness and lack of fit. If the model fails, the study supervisor determines if a subset of the
data should be analyzed or if more data are needed.
4. Compute the IQE - the IQE is computed using the ILSD model selected in Step 3 to estimate the
relative standard deviation as a function of concentration. The first attempt is at 10% RSD (IQE10%).
If this attempt fails, IQE20o/o is tried, then IQE30o/(i. IQEs greater than 30% are not recommended.
5. Nontrivial Amount of Censored Data - this section of the IQE procedure addresses the effect of "non-
detects" or "less-than." Suggestions are given to see if uncensored data can be obtained from the
laboratories or if the study needs to be augmented with additional data. Suggestions are given for
fitting a model to data that contain less than 10% non-detects or less-than to produce an IQE.
5.2.2.2 Assessment of the IQE Against the Evaluation Criteria
The following five subsections discuss the IQE approach and procedure in the context of the five
evaluation criteria that concern detection limit approaches (i.e., Criteria 1-3, and Criteria 5 and 6).
5-39
-------
5.2.2.2.1 Criterion 1: The detection and quantitation limit approaches should be scientifically valid.
Condition 1: It can be (and has been) tested. The Electric Power Research Institute provided input into
the design of EPA Method 1631 and 1638 Validation Studies for the purpose of calculating IDEs and
IQEs. EPRI also calculated IDEs and IQEs based on these data. These two datasets include a total often
metal analytes and therefore do not cover a wide range of analytical techniques and methods. Other than
these two datasets, EPA is not aware of any organization, including ASTM, that has conducted a study to
test the IQE procedure as written (i.e., designed and implemented an interlaboratory study involving
multi-laboratory analysis of multiple concentrations of each matrix of interest). It has been tested by its
developers using simulated data sets and on interlaboratory data sets that do not adequately characterize
the low level region of interest. As part of this reassessment, EPA tested a variant of the IQE procedure
on single-laboratory data sets that were designed to characterize an analytical method in the region of
detection and quantitation. Despite the lack of comprehensive testing performed to date, the IQE
procedure can be tested if sufficient resources are invested.
Condition 2: It has been subjected to peer review and publication. Although the IQE has not been
published in the peer-reviewed scientific literature, the IQE has undergone review and ballot by members
of ASTM Committee D 19, many of whom are qualified peer reviewers. Thus, the IQE meets the intent of
this condition (i.e., submission to scrutiny of the scientific community). In addition, the IQE was
reviewed by four peer reviewers as part of EPA's assessment of detection and quantitation limit
approaches.
Condition 3: The error rate associated with the procedure is either known or can be estimated. In theory,
an expert statistician could estimate the error rate of the IQE. However, the IQE procedure is extremely
complex from an analytical chemistry and statistical perspective. As a result, it is unlikely that the error
rate could be estimated by the staff of an environmental testing laboratory. Moreover, in attempting to
follow the IQE procedure during this reassessment, EPA found the procedure to be subjective, particularly
with respect to selection of an appropriate statistical model. The subjective nature of the procedure is
likely to yield different IQEs from the same data set, depending on the staff involved in analyzing the data
and performing the calculations. (The likelihood of this problem is illustrated in appendix B to this
Assessment Document.) This subjective variability eliminates the ability to estimate the actual error
associated with the IQE. Therefore, the IQE does not meet this condition.
As discussed in Section 5.2.2.1, Condition 3, regarding the IDE, one stakeholder stated that
concerns about the complexity and subjectivity of the IQE (and IDE) procedures were unimportant, in
part, because IQEs calculated using different models were very close, and in part, because "user friendly-
software that will automatically perform the IDE and IQE calculations. EPA obtained copies of such
software from the commenter and used that software to evaluate the validity of this comment. As
described at length in Section 5.2.2.1, EPA concluded that 1) the subset of models used varies among the
software packages, 2) the software packages do not always apply the same model to the same data sets,
and 3) even if the same model is used, there is a large amount of variability between the results produced
when applying the different software packages to the same set of data. Based on these differences, EPA
concluded that the available software programs do not remove all complexity and subjectivity from the
IQE calculations. Instead, the software programs appear to introduce new issues by using steps not
included in the ASTM procedures.
Condition 4: Standards exist and can be maintained to control its operation. The IQE approach and
procedure is supported by a published procedure (standard) to control its operation. The procedure gives
the steps to be followed in determining the IQE and instructs the study supervisor how to gather the data
and compute an IQE.
5-40
-------
There are several "gray areas" in the published procedure. The most significant gray area is in
model selection. The procedure provides insufficient guidance on the use of residual plots as a basis for
selecting models and as a result, selection of the model may be very subjective, especially if the number of
concentrations is low. The discussion of what model to use after rejecting the hybrid and linear models
also is very vague. The exponential model is mentioned, as well as models with more than one
coefficient. In addition, fitting the "constant model" is never discussed in detail. Most likely, this is done
by simply calculating a mean (weighted if necessary) of the variances from the different concentrations,
however such a calculation is never explicitly stated. As discussed under Condition 4 of Section 5.1.2.2.1
(scientific validity of the IDE procedure), there appear to be inconsistencies between the IDE and IQE
that suggest conceptual conflicts between these two standards.
Based on these findings (along with those discussed under Criterion 2 below), the procedure is not
sufficient to control operation of the IQE because of the high degree of subjectivity involved in
implementing the procedure, statistical errors in the procedure, and internal inconsistencies with the IDE.
Therefore, the IQE does not meet this condition.
Condition 5: It has attracted widespread acceptance within a relevant scientific community. The IQE was
published by ASTM four years ago (2000). EPA has not found an IQE in the open literature or in an
analytical method, including an ASTM method.
5.2.2.2.2 Criterion 2: The approach should address realistic expectations of laboratory and method
performance, including routine variability.
The IQE procedure is designed to reflect expectations of interlaboratory performance, including
routine variability. The procedure contains extensive instructions for dealing with unusual conditions,
including sources of variability and outliers. Based on studies of the single-laboratory variant of the
procedure in which the model selection proved to be highly subjective, it is not clear that IQE procedure
would demonstrate realistic expectations of laboratory and method performance.
The IQE procedure suggests attempting to fit study results to a constant, linear, or hybrid model.
If all of these fail, the procedure suggests trying a different model, such as the exponential model. (The
exponential model figures more prominently in the IDE procedure, where it is one of the three main
models discussed, replacing the Rocke and Lorenzato model.) Although the exponential model may be
appropriate for the IDE (which is not tied to a fixed RSD), it yields unacceptable results when applied to
the IQE procedure. Under the exponential model, relative variability (standard deviation divided by the
true concentration) does not consistently decrease with increasing concentration (i.e., as concentration
increases, relative variability decreases down to a specific percentage, and then begins to increase). This
is not realistic of laboratory and method performance. In addition, the exponential model will often result
in having two possible values each for IQE10%, IQE20o/o, and IQE300/o.
Another concern with the IQE procedure is that use of the non-mandatory appendices in ASTM D
6512 to determine the fit of a model may produce results that differ from those that would be obtained
using the default procedures for testing model fit that are built into off-the-shelf statistical software, such
as the Excel files discussed in Condition 3.
Given the subjectivity and confusion involved in selecting the model, EPA tried using the same
data set to calculate a single-laboratory variant of the IQE with each of the available models and found
that the calculated IQEs varied widely when different models were used.
Based on the problems described above, EPA believes the IQE does not meet this criterion.
5-41
-------
5.2.2.2.3 Criterion 3: The approach should be supported by a practical and affordable procedure that
a single laboratory can use to evaluate method performance.
The IQE procedure is neither practical nor affordable in a single-laboratory context.. It is
designed for use by an ASTM study supervisor or task manager and not as a procedure that a single
laboratory can use to evaluate method performance. EPA is aware that ASTM Committee D 19 is
contemplating development of a within-laboratory quantitation estimate (WQE), but the WQE has not
been approved through an ASTM ballot, and therefore, it cannot be adequately evaluated at this time. The
WQE may meet this criterion, but the IQE does not.
Regarding affordability, EPA estimates that the cost of implementing IQE procedure would be
more than twice the cost of EPA's present implementation of the ML. The increased cost stems from the
additional low-level data required to assure that variability versus concentration is being characterized in
the region of detection and quantitation, challenges involved in applying the statistical procedures in the
IQE, and because of the anticipated reanalysis and rework required if either the procedure failed to
produce an IQE or if the resulting IQE failed to meet the specifications in the IQE procedure.
5.2.2.2.4 Criterion 5: The quantitation limit approach should identify the concentration that gives a
recognizable signal that is consistent with the capabilities of the method when a
method is performed by experienced staff in well-operated laboratories.
If the IQE were developed in an interlaboratory study that met the requirements of D 6512, the
calculated IQE would likely be achievable by experienced staff in a well-operated laboratory. Therefore,
the IQE meets this criterion.
However, similar to the discussion of criterion 5 for the ML (section 5.1.2.4) anomalous results
occur. Analysis of episode 6000, analysis of Episode 6000 data (see appendices) produced anomalous
results from two methods (EPA 502.2 and 524.2) that employ instrument thresholds. For 9% of EPA
502.2 and 59 % of EPA 524.2 analytes the calculated single-lab IQE was below the concentration at which
all seven spiked replicates were detected. These results indicate that an IQE study coordinator, after
calculating IQE from multi-labs results, would have calculated IQEs below the instrument threshold. The
IQE procedure is silent on what happens in this case. As previously noted, the Episode 6000 dataset is not
reflective of a typical compliance measurement or method development study because the range of
concentrations studied encompassed several orders of magnitude and included concentrations well below
the detection limit. And this dataset was not developed according to the procedures in D 6512 (the IQE).
5.2.2.2.5 Criterion 6: Detection and quantitation approaches should be applicable to the variety of
decisions made under the Clean Water Act, and should support State and local
obligations to implement measurement requirements that are at least as
stringent as those set by the Federal government
There is no database of IQE values for CWA analytes that were calculated according to D 6512.
These are the data with which one would compare existing CWA limits and thereby assess the effect of
using IQEs as reporting and compliance limits in CWA programs.
5.2.3 Assessment of the ACS Limit of Quantitation
The Limit of Quantitation (LOQ) was developed by the Committee on Environmental
Improvement of the American Chemical Society (ACS) and published in the same two papers as the LOD.
5-42
-------
5.2.3.1 Description of the ACS LOQ Approach and Procedure
The 1983 "Principles" define the LOQ as:
"... the level above which quantitative results may be obtained with a specified degree of
confidence. "
The same relationship used to define the LOD is used for the LOQ:
but the recommended minimal value for Kd be set at 10. Thus, the LOQ is 10o above the gross blank
signal, Sb. According to the 1983 publication, the LOQ corresponds to an uncertainty of ±30% (10o ±
3o). This uncertainty statement is based on a equal to 10% of the LOQ.
Neither the 1980 nor 1983 ACS publications provide a specific procedure for estimating the LOQ,
nor do they provide a minimum number of observations needed to estimate the gross blank signal or the
variability term ob.
5.2.3.2 Assessment of the ACS LOQ Against the Evaluation Criteria
The following five subsections discuss the ACS LOQ approach and procedure in the context of
the five evaluation criteria that concern detection limit approaches (i.e., Criteria 1-3, and Criteria 5 and 6).
5.2.3.2.1 Criterion 1: The detection and quantitation limit approaches should be scientifically valid.
Condition 1 : It can be (and has been) tested. Testing of the LOQ is hampered by the lack of a supporting
procedure for establishing an LOQ, and a conceptual dependence on the variability of blank
measurements. If the blank measurements fail to produce a response, it is impossible to calculate an LOQ
because the standard deviation of multiple zero-value results is zero. One solution for testing the
approach is to assume that the LOQ is approximately equivalent to the ML as the blank signal approaches
zero. If this is a reasonable assumption, the ML procedure is a viable means for testing the LOQ
approach, and the LOQ would meet this condition.
Condition 2: It has been subjected to peer review and publication. The ACS LOQ definition was
published in the peer-reviewed journal Analytical Chemistry in 1980 and 1983. Therefore, the ACS LOQ
meets this condition.
Condition 3: The error rate associated with the procedure is either known or can be estimated. The LOQ
meets this condition. The definition of the LOQ specifically estimates the uncertainty associated with a
concentration at the LOQ as ±30% based on 10% RSD (Kd = 10). Other choices maybe made based on
study requirements, policy judgments and/or specific results.
Condition 4: Standards exist and can be maintained to control its operatioa The ACS LOQ lacks a
clearly defined procedure for estimating the important terms required to derive it. Although it may be
possible to derive ACS LOQ values from data used to derive EPA MDL values, there is no discussion of
using replicate blanks, replicate spiked samples, or a minimum recommendation for the number of
replicates. Therefore, the ACS LOQ does not meet this condition.
5-43
-------
Condition 5: It has attracted widespread acceptance within a relevant scientific community. Because the
ACS does not develop and publish reference analytical methods, it is difficult to determine the degree of
acceptance of the LOQ. EPA has not investigated the numbers of papers published in ACS journals that
include LOQ values, but EPA's literature search for detection and quantitation approaches did not uncover
a large number of citations that promote the LOQ in particular.
5.2.3.2.2 Criterion 2: The approach should address realistic expectations of laboratory and method
performance, including routine variability
The LOQ approach is designed to address realistic expectations of laboratory and method
performance, including routine variability, and therefore, it appears to meet this criterion. Because the
ACS has not published a procedure to implement the approach, in practice the LOQ provides no direct
means for demonstrating this performance. The ACS LOQ, the only partially meets this criterion.
5.2.3.2.3 Criterion 3: The approach should be supported by a practical and affordable procedure that
a single laboratory can use to evaluate method performance.
Because the ACS LOQ approach is not supported by a clearly defined procedure for establishing
the LOQ, it does not meet this criterion.
5.2.3.2.4 Criterion 5: The quantitation limit approach should identify the concentration that gives a
recognizable signal that is consistent with the capabilities of the method when a
method is performed by experienced staff in well-operated laboratories.
Given the relationship of the ACS LOQ to the ML, EPA believes the LOQ meets this criterion for
the reasons outlined in Section 5.,2.1.2.4, which discusses EPA's assessment of the ML against Criterion 4
for evaluating quantitation limit approaches.
5.2.3.2.5 Criterion 6: Detection and quantitation approaches should be applicable to the variety of
decisions made under the Clean Water Act, and should support State and local
obligations to implement measurement requirements that are at least as
stringent as those set by the Federal government.
In the absence of a procedure for determining LOQ values, the ACS LOQ does not meet this
criterion because it cannot be used in a regulatory context. The LOQ passes this criterion only if it is
assumed to be approximately equivalent to the ML (i.e., the ML procedure is used to establish an LOQ).
5.2.4 Assessment of the IUPAC/ISO Limit of Quantitation
A similar LOQ approach was developed by IUPAC/ISO and published in the same papers as the
CRV andMDV (see Sections 5.1.4 and 5.1.5).
5.2.4.1 Description of the ISO/IUPAC LOQ Approach
The 1995 "Recommendations"defme the LOQ as:
"... the ability of a CMP [chemical measurement process] to adequately 'quantify' an
analyte. The ability to quantify is generally expressed in terms of the signal or analyte
(true) value that will produce estimates having a specified relative standard deviation
(RSD), commonly 10%."
5-44
-------
The relationship used to define the LOQ is:
LQ = KQ x OQ
The recommended value for Kg is 10. Thus, the LOQ is 10o above the blank signal, OQ.
5.2.4.2 Assessment of the IUPAC/ISOLOQ Against the Evaluation Criteria
The following five subsections discuss the IUPAC/ISO LOQ approach and procedure in the
context of the five evaluation criteria that concern detection limit approaches (i.e., Criteria 1-3, and
Criteria 5 and 6).
5.2.4.2.1 Criterion 1: The detection and quantitation limit approaches should be scientifically valid.
Condition 1: It can be (and has been) tested. Testing of the IUPAC/ISO LOQ is hampered by the lack of
a supporting procedure for establishing an LOQ, and a conceptual dependence on the variability of blank
measurements. If the blank measurements fail to produce a response, it is impossible to calculate an LOQ
because the standard deviation of zero is zero. One solution for testing the approach is to assume that the
ISO/IUPAC LOQ is approximately equivalent to the ML as the blank signal approaches zero. If this is a
reasonable assumption, the ML procedure is a viable means for testing the LOQ approach, and the
ISO/IUPAC LOQ meets this condition.
Condition 2: It has been subjected to peer review and publication. The IUPAC/ISO LOQ definition has
been published by Currie in the peer-reviewed journals Pure andAppl. Chem. in 1995; in Anal. Chim.
Acta in 1999, in Chemometrics and Intelligent Lab Systems in 1997; and in/. Radioanal. and Nuclear
Chem. in 2000. Therefore, the IUPAC/ISO LOQ meets this condition.
Condition 3: The error rate associated with the procedure is either known or can be estimated. EPA used
data generated in the Episode 6000 study to estimate the error rate associated with the LOQ. The Episode
6000 results show that the median error across all analytes and analytical techniques at 10o is
approximately ±14% with approximately 95% confidence.
Condition 4: Standards exist and can be maintained to control its operatioa The IUPAC/ISO LOQ lacks
a clearly defined procedure for estimating the important terms required to derive it. Although it may be
possible to derive IUPAC/ISO LOQ values from data used to derive EPA MDL values, there is no
discussion of using replicate blanks, replicate spiked samples, or a minimum recommendation for the
number of replicates. Therefore, the IUPAC/ISO LOQ does not meet this condition.
Condition 5: It has attracted widespread acceptance within a relevant scientific community. Acceptance
of this approach by the scientific community is currently not known. Acceptance would be indicated by
use of the LOD in ISO methods. EPA's search for detection and quantitation approaches in the open
technical literature did not uncover a large number of citations that reference the LOQ. Therefore, it is
difficult to determine if the ISO/IUPAC LOQ meets this condition.
5.4.2.2.2 Criterion 2: The approach should address realistic expectations of laboratory and method
performance, including routine variability.
The most recent publication on the IUPAC/ISO LOQ (J. Radioanal. and Nuclear Chem., op. cit.)
provides insight into this issue through measurements of 14C by accelerator mass spectrometry. Therefore,
the IUPAC/ISO LOQ passes this criterion for at least some measurement techniques.
5-45
-------
5.4.2.2.3 Criterion 3: The approach should be supported by a practical and affordable procedure that
a single laboratory can use to evaluate method performance.
The ISO/IUPAC LOQ approach is not supported by a clearly defined procedure for establishing
the LOQ. Therefore, it does not meet this criterion.
5.4.2.2.4 Criterion 5: The quantitation limit approach should identify the concentration that gives a
recognizable signal that is consistent with the capabilities of the method when a
method is performed by experienced staff in well-operated laboratories.
Assuming a relationship of the IUPAC/ISO LOQ to the ML, the LOQ satisfies this criterion for
the reasons outlined in Section 5.,2.1.2.4, which discusses EPA's assessment of the ML against Criterion 4
for evaluating quantitation limit approaches.
5.4.2.2.5 Criterion 6: Detection and quantitation approaches should be applicable to the variety of
decisions made under the Clean Water Act, and should support State and local
obligations to implement measurement requirements that are at least as
stringent as those set by the Federal government
In the absence of a procedure for determining LOQ values, the ISO/IUPAC LOQ does not meet to
meet this criterion because it cannot be used in a regulatory context. The ISO/IUPAC LOQ passes only if
the ML procedure is used to establish an LOQ.
5-46
-------
Chapter 6
Findings and Next Steps
What are EPA's findings in this revised assessment?
In this revised assessment of detection and quantitation approaches, the Agency has evaluated the
codified MDL procedure, the ML procedure that EPA proposed to codify in 2003, and several alternative
procedures. Some of these alternative procedures were submitted to EPA during the comment period on
EPA's 2003 assessment, which was detailed in the February 2003 Technical Support Document (EPA-
821-R-03-005). In today's assessment, we have:
• Identified relevant procedures to include in the assessment (Chapter 2);
• Identified issues that may be relevant to the assessment from an analytical chemistry, statistical, or
regulatory perspective (Chapter 3);
• Used six criteria to evaluate the ability of each procedure to support activities under the Clean
Water Act (Chapter 4);
• Assessed how well each procedure meets the evaluation criteria (Chapter 5);
• With real-world data and several different procedures, calculated and compared detection and
quantitation limits using, and evaluated the theoretical and practical limitations of, each procedure
(Appendices B and C).
The assessment of the theoretical and practical applications of each procedure (Appendices B and
C) suggests that different procedures produce different detection and quantitation limits. Observed
differences are largely due to different sources of variability accounted for among the procedures. The
overall assessment of each procedure against each of the evaluation criteria suggests that no single pair of
detection and quantitation limit procedures perfectly meets EPA's six evaluation criteria. Although the
MDL and ML procedures are closest to meeting these criteria, as discussed under EPA's next steps, we
recognize that this is not the end of our consideration of future improvements to EPA procedures and/or
adoption of specific alternative procedures.
In response to stakeholders who suggested that EPA clarify or revise some steps in these
procedures, we proposed modest revisions to the MDL procedure and proposed to codify an ML definition
and procedure in conjunction with the 2003 assessment. We also proposed to codify an existing option
that allows use of other detection and quantitation procedures to develop detection and quantitation limits.
Public comment on both the 2003 assessment and the proposed revisions expressed many divergent views
that conflicted with the proposed modifications to the procedures. Commenters suggested that we work
with stakeholders to discuss mutual concerns and possible solutions rather than proceed with the proposed
revisions. Some commenters submitted detailed, alternative procedures or regulatory revisions. However,
there was no agreement among these commenters as to which of the competing alternatives or revisions to
adopt, and none of them fully satisfied EPA's needs under the CWA. We have therefore decided to
withdraw the proposed revisions.
6-1
-------
What are EPA's next steps?
We believe that it is appropriate to withdraw the proposed revisions, take final action on the 2003
assessment, and explore the feasibility of using a stakeholder process to facilitate a resolution of the
technical and policy issues raised during the public comment period. It is in the best interest of all parties
to solicit additional stakeholder input through consultations. In a Federal Register notice published on
September 15, 2004 [69 FR 55547], we announced that a neutral party is studying the feasibility of a
process by which a broad group of stakeholders would work together to define and address concerns about
the way detection and quantitation limits are calculated and used to support CWA programs. This
potential stakeholder process will expand the list of interested stakeholders to include state, tribal and local
governments, environmental groups and other interested parties. We trust that this potential stakeholder
process will address the wide variety of views held by stakeholders and may lead to recommendations for
possible improvements to current EPA procedures and/or use of alternative procedures.
To facilitate open, frank and inclusive discussions, we have made every effort to ensure that this
Revised Assessment Document does not prejudge the result of the potential stakeholder process. In
particular, we recognize that the following stakeholder issues or suggestions provide a strong starting point
for a continued dialogue with stakeholders.
Assessment Evaluation Criteria Issues
The February 2003 assessment identified and discussed six criteria the Agency used to evaluate
several different approaches to detection and quantitation. The six evaluation criteria are:
Criterion 1: The detection and quantitation limit approaches should be scientifically valid.
Criterion 2: The approach should address demonstrated expectations of laboratory and method
performance, including routine variability.
Criterion 3: The approach should be supported by a practical and affordable procedure that a
single laboratory can use to evaluate method performance.
Criterion 4: The detection limit approach should identify the signal or estimated concentration at
which there is 99% confidence that the substance is actually present (i.e., a one percent false
positive rate) when the analytical method is performed by experienced staff in a we 11-operated
laboratory.
Criterion 5: The quantitation limit approach should identify the concentration that gives a
recognizable signal that is consistent with the capabilities of the method when a method is
performed by experienced staff in a well-operated laboratory.
Criterion 6: Detection and quantitation approaches should be applicable to the variety of decisions
made under the Clean Water Act, and should support state and local obligations to implement
measurement requirements that are at least as stringent as those set by the Federal government.
Stakeholders commented that these six criteria favored the MDL and ML procedures. Some
stakeholders noted instances where criterion four fails for the MDL, i.e., does not represent the limit at
which there is a 99% confidence that the observed signal is not a false positive. Stakeholders also
disagreed with EPA's reliance on only one detection and one quantitation procedure, the MDL and ML
(see criterion six discussion at 4.6 in this document) Stakeholders suggested that different detection and
6-2
-------
quantitation procedures with different levels of rigor be developed and applied to the disparate uses of
these limits in CWA programs. Uses of these limits include verification of laboratory performance,
method validation, and as a guide for reasonable bounds on values to consider for permit limits. EPA
recognizes that the complexity and statistical rigor appropriate for a detection and quantitation approach
for method development and validation would be greater than that needed for demonstrating laboratory
proficiency. Although EPA believes that the six evaluation criteria are suitable for purposes of this
assessment, they need not be the only starting point for future stakeholder evaluations of revised or
alternative detection and quantitation procedures.
Technical and Policy Issues
Some of the major comments on the MDL and ML procedures that influenced our decision to
withdraw the proposed rule, and to seek additional stakeholder input, include: (1) the MDL does not
adequately address analytical variability or systematic error (bias); (2) a need for better guidance on the
intended use of these limits in CWA programs; (3) the need for different procedures for different CWA
applications, such as method development, laboratory performance checks, and permit limits. Commenters
also asked for clearer guidance on specific steps in the MDL procedure, such as selection of initial spike
concentrations, and use of iterative and outlier procedures.
The technical issues of analytical variability and bias attributable to blanks encompass a range of
concerns. Stakeholders have suggested that detection and quantitation procedures should:
• vary in the nature and extent of statistical rigor and performance verification checks depending on
the end;
• use of a calculated limit;
account for more sources of variability, such as the variability between and within laboratories;
• require more then seven samples and collect samples over a long period of time; and
use routine blank samples collected over long periods of time to account for background signals
and temporal variability (e.g., ACIL and USGS procedures).
EPA believes these suggestions merit serious consideration, and plans to use the stakeholder
process to consider ways to address them.
Conclusion
This Revised Assessment Document addresses comments and concerns received from stakeholders
and peer reviewers. Based on this new information, EPA believes that discussion of alternatives or
improvements to current detection and quantitation concepts or procedures and uses should continue. It is
clear that there is a broad interest in improving current procedures and uses, but no consensus for a specific
procedure or procedures has emerged among the laboratory, industry, regulatory or regulated communities.
We look forward to further stakeholder participation in this process.
6-3
-------
Appendix A
Literature Search Regarding
Detection and Quantitation Limit Approaches
Introduction
Beginning in 2001, DynCorp conducted a search of published literature to identify articles that
discuss detection and quantitation limit approaches. This literature search effort was conducted under
EPA Contract No. 68-C-01-091 to support an evaluation of detection and quantitation limit approaches by
the EPA's Office of Water.
The principal goal of this literature search was to determine if any new detection or quantitation
limit approaches had been published since an earlier search conducted for EPA by Science Applications
International Corporation (SAIC) in 1997 and 1998. That search resulted in an annotated bibliography
developed by SAIC and delivered to EPA in 1998.
In August 2002, EPA included the literature search results in a draft Technical Support Document
(TSD) that was submitted for formal peer review. As part of the charge to the peer reviewers, EPA asked
them to identify any additional references. Following EPA's review of the suggested additional
references, references relevant to the TSD were added.
How the search was conducted
This search was conducted using two major techniques:
• a search of an on-line citation index (an index of articles cited by other authors), and
• a general on-line search of literature.
On-line citation index search
Because the search was intended to identify detection and quantitation limit approaches and not
specific numeric limits associated with a particular analytical method, DynCorp began by searching for
references to the major approaches known to EPA. These included the Agency's method detection limit
(MDL) and any other terms that have been suggested to the Agency as alternative detection or
quantitation limit approaches. In addition to searching for these approaches, DynCorp also searched the
citation index to identify references to the original authors of these approaches and for any other authors
who either cited the original approaches, the original papers underlying those approaches, or the authors
of those approaches. DynCorp used a similar approach to find any papers that cited the references
identified in the earlier literature search by SAIC.
DynCorp staff evaluated the full title of each identified citation to determine its relevance to
EPA's objective. Where available electronically and at no additional cost, DynCorp staff also reviewed
the abstract and/or full paper to further characterize relevance. All papers that were determined to be
relevant, or even possibly relevant, were obtained in hardcopy or electronic format for evaluation by
EPA.
After reviewing all papers determined to be relevant to EPA's objective, DynCorp examined all
of the references cited in those papers to identify additional papers of interest. These, too, were obtained
in hardcopy or electronic format for evaluation by EPA, except where noted below.
A- 1
-------
General on-line literature search
DynCorp performed an on-line direct search of published literature (e.g., a literature database of
published articles, not a citation index) using general terms such as "detection limit," "quantitation limit,"
or "calibration." As expected, this approach returned a very large numbers of papers that mention these
terms, even if the focus of the paper was on something far removed from the development or assessment
of approaches about detection and quantitation, and proved to be of limited value in serving EPA's
objectives for the search. Therefore, DynCorp discontinued this effort and narrowed the on-line literature
search to a search for additional, uncited works by authors of the approaches known to EPA or identified
through the citation index approach.
Papers determined to be relevant to EPA's objective were obtained in electronic or hardcopy
format for evaluation by EPA, except where noted below.
How the results are presented
DynCorp identified a total of 161 relevant publications using the approach described above.
Thirty-three (33) of these publications were also identified in the earlier search by SAIC. Of the 128
remaining publications, 35 were published since the SAIC search was completed.
The peer reviewers suggested additional publications covering a variety of topics, including:
quality control, analysis of mercury, and approaches to dealing with censored data. EPA reviewed the
citations from the peer reviewers and determined that 20 directly addressed detection or quantitation
approaches. In particular, EPA noted that the issue of censored data applies regardless of the specific
detection or quantitation limit associated with the data, so those citations dealing with censored data were
not included.
Each of the 181 publications identified in the search is listed in Attachment 1, which provides the
title, year of publication, authors, and source citation. The citations for the 33 papers identified in the
earlier search by SAIC are included in the attachment, and can be identified by the phrase "annotated
only" in parentheses after the title of the paper.
The final column of the attached spreadsheet is labeled "Category." All of the citations identified
in the SAIC literature search and the current search conducted by DynCorp were placed in one of the six
following categories, based on the principal characteristic of the article:
• Background - The citation discusses background information (including early works by Currie,
Kaiser, and others).
• Calibration concept - The citation primarily deals with calibration of analytical instrumentation
• Critique - The major thrust of the citation is to critique one or more approaches, as opposed to
introducing a new approach
• Multi-laboratory approach - The citation describes an approach to developing detection and/or
quantitation limits that relies on multi-laboratory measurements
• Single-laboratory approach - The citation describes an approach to developing detection and/or
quantitation limits that relies on single-laboratory measurements
• Single-laboratory, multi-level approach - The citation describes an approach to developing detection
and/or quantitation limits that relies on single-laboratory measurements but explicitly includes
multiple concentrations.
A-2
-------
Although there is some degree of overlap between categories, and some papers could probably be
classified in more than one category, each citation was classified into only one category for the purposes
of this search.
A seventh category called "Not found" was used for three papers that were identified in the
literature search, but for which copies could not readily be obtained. One paper is from a German journal
that was not available via interlibrary loan. A second article also was not available via interlibrary loan.
The third citation is an abstract by Currie, from 1983. Given that the work of Currie is well-represented
in the other citations and the fact that this citation appears to be only an abstract, additional efforts were
not expended to obtain a copy.
The 20 publications suggested by the peer reviewers were all included at the end of the list, under
an eighth category called "Suggested by a peer reviewer."
The references presented in the table were sorted by category and year of publication and are
displayed with the most recent citations in each category first.
Summary
The principal goal of this literature search effort was to determine if any new detection or
quantitation limit approaches had been published in the literature since the search by SAIC in 1997 -
1998. As anticipated, citations were identified that relate to the recent efforts of the International
Organization for Standardization (ISO), the International Union of Pure and Applied Chemists (IUPAC),
and the ASTM International. Additional articles critiquing various approaches were identified as well.
However, no previously unknown detection or quantitation limit approaches were uncovered as a result of
this effort. Likewise, the references suggested by the peer reviewers provided additional details and
applications of existing detection and quantitation approaches, but did not suggest any approaches that
had not already been identified.
A-
-------
Results of the 2001 Literature Search
Title
Oome Case Otudies of Okewed (and other ab-normal) Data Distributions Arising
in Low-Level Environmental Research
Legislative Limits Below Detection Capability
International Recommendations Offered on Analytical Detection and
Quantification Concepts and Nomenclature
Detection and Quantitation Limits! Origins and Historical Overview
1996 ASMS Fall Workshop! Limits to Confirmation, Quantitation, and Detection
Measurement precision and l/f IMoise in Analytical Instruments
Fossil- and Dio-mass Combustion! C~14 for Oource Identification, Chemical
1 racer Development, and Model Validation
Interlaboratory Comparison of Instruments Used for the Determination of
Elements in Acid Dige states of Solids
1 hrowaway Data
EPA's Office of Water Surges Toward MDL Solution
In Pursuit of Accuracy. Nomenclature, Assumptions, and Otandards
Interlaboratory Aspects of Detection Limits Used for Regulatory and vsontrol
r urposes
Noise and Detection Limits in Signal-Integrating Analytical Methods
Effects of Analytical Calibration Models on Detection Limit Estimates
Real-World Limitations to Detection
Detection Limits - A Systematic Approach to Detection Limits is Needed When
1 race Determinations are to be Performed
Lfhemometrics and Analytical Lfhemistry
Quality Control in Water Analyses
Validation of Analytical Methods
Year
2001
2000
1999
1999
1997
1996
1994
1994
1994
1994
1992
1988
1988
1988
1988
1986
1984
1983
1983
Author
L.A. Cum,
S.L.R. Ellison, VJ. BarwicK, A. Wllllam,
L.A. Currie
L.A. Currie
R. Baldwin, R.A. Bethem, R.K. Boyd, W.L.
Budde, T. Cairn,, R.D. Gibbon,, J.D.
Henion, M.A. Kaiser,
Y. Hayashl, R. Matsuda, R.B. Poe
L.A. Currie, G.A. Klouda, D.B. Klinedinst,
A.E. Sheffield, A.J.T. Jull, DJ. Donahue,
M.V. Connolly
D.E. Kimbrough, J. Wakakuwa
L.H. Keith
Larry Keith
L.A. Currie
L.D. Rogers
H.C. Omit, H. Oteigstra
K.G. Owens, C.F. Bauer, C.L. Grantr
D. Kurtz, J. Taylor, L. Sturdivan, W.
Crummett, C. Midkiff, R. Walters Jr, L.
Wood, W. Hanneman, W. Horwitz
O.A. Dorman
L.A. Currie
C. Kirchmer
J.K. Taylor
Source
Fresenius Journal of Analytical Chemistry 370: 705-718
Accreditation Quality Assurance 5: 308-313
Analytica Chimica Acta 391 ! 103
Analytica Chimica Acta 391 ! 127-134
Journal of the American Oociety for Mass Opectrometry Oi
1180-1190
Journal of Chromatography A 722: 157-167
Nuolear Instr. And Method, in Phy,lo, Re,. B 92: 404-409
Ana,,,, 11 9: 383-388
Environmental Science & Technology 28i 389A-390A
Radian
Pure & Applied Chemistry 64:455-472
ACS Symposium Series 361:94-108
ACS Symposium Series 361:126-148
ACS Symposium Series 361:194-207
ACS Symposium Series 361:288-316
Analytical Chemistry 58: A986
Chemometrlcs 56: 115-146
ES&T17:174A-181A
Analytical Chemistry 55: 600A-602A, 608A
Category
Dackground
Background
Dackground
Dackground
Dackground
Dackground
Dackground
Dackground
Dackground
Dackground
Dackground
Dackground
Dackground
Dackground
Dackground
Dackground
Dackground
Dackground
Dackground
A-4
-------
Title
1 race Analyses for Waste waters ~ /Author s response
Zur 1 heorie der Lichfunktion bei der spektrochemischen Analyse
The Reliability of Detection Limits in Analytical Chemistry
A Keview and 1 utorial Discussion of Noise and Oignal-to-Noise Katios in
Analytical Opectrometry - 1. Fundamental Principles of Oignal-to-Noise Katios
A Keview and 1 utorial Discussion of Noise and Oign-to-Noise Katios in
Analytical Opectrometry - II. Fundamental rrinciples of Oignal-to-Noise Katios
A 1 utorial rveview of Oome tlementary vsoncepts in the Otatistical tvaluation of
1 race tlement Measurements
Analysis of Lead in r o Muted vsoastal Oeawater
Multielement Analysis with an Inductively Coupled Plasma/Optical Lmission
Oystem
Interlaboratory Lead Analyses of standardized Oamples of Oeawater
Statistical and Mathematical Methods in Analytical Chemistry
Otudies of Flame and Plasma 1 orch Lmission for Oimultaneous Multi-Llement
Analysis- 1. Preliminary Investigations
Quantitative Determination! Application to Kadiochemistry
(Qualitative and (Quantitative Oensitivity in Flame r ho to me try
The Limit of Detection of Analytical Methods
A Careful Consideration of the Calibration Concept
Weighted Random-Effects Regression Models with Application to
Interlaboratory Calibration
(guidelines for Calibration in Analytical Chemistry-fart 1. Fundamentals and
Oingle Component Calibration (IUPAC recommendations 1990)
Constant-Width Calibration Intervals for Linear Kegression
Regression and Calibration with Nonconstant Lrror Variance
Year
1982
1982
1980
1978
1978
1978
1976
1976
1974
1972
1972
1968
1966
1962
2001
2001
1998
1996
1994
1990
Author
D. Foerst
V.H. Kaiser
J.D. Winefordner, J.L. Ward
C.TJ. AlKemade, W. Snelleman, G.D.
Boutilier, B.D. Pollard, J.D. Winefordner,
T.L. Chester, N. Omenetto
G.D. Bou.lier, B.D. Pollard, J.D.
Winefordner, T.L. Chester, N. Omenetto
P.W.J.M. Bouman,
C. Patterson, D. Oettle, D. Cjlover
R.M. Ajnar, P.O. Dalager, A.L. Davison
P. Brewer, N. Frew, N. Cutshall, JJ.
Wagner, R.A. Duoe, P.R. Walsn, G.L.
Hoffman, J.W.R. Dutton, W.F. Fitzgerald
L.A. Currie, JJ. Filliben, J.R. DeVoe
P.W.J.M. Bouman,, FJ. De Boer
Lloyd Currie
J. Kamirez-Munoz
J.B. Roos
S.D. Pnillips, W.T. E,,,er,T. Doiron, K.R.
Eberhardt, M.S. Levenson
R.D. Gibbons, D.K. Bhaumik
K. Danzer, L.A. Currie
R.W. Mee, K.R. Ebernard,
K.R. Ebernard,, R.W. Mee
M. Davidian, P.D. Haaland
Source
Envir. Sd. & Teon. 16: 430A - 431 A
DK 535: 309-31 9
Analytical Letter, 13: 1293-1297
Spectrocnimica Acta 33B: 383-399
Spectrocnimica Aota 33B: 401-415
Spectrocnimica Acta 33B: 625-634
Marine Cnemistry 4'. 305-319
American Laboratory 72-78
Marine Cnemistry 2'. 69-84
Ana,. Cnem. 44: 497R-512R
Spectrocnimica Acta 27B: 391-414
Anai. Cnem. 40: 586-593
Talanta 13: 87-101
Analyst 87: 832-833
Journal of Kesearch of the National Institute of Otandards
and Technology 106i 371-379
Technometrics43:192-198
Pure and Applied Chemistry 70) 993-1014
Journal of Quality Technology 26i 21-29
Chemometrics and Intelligent Laboratory Oys terns 9i Zol -
248
Category
Dackground
Dackground
Dackground
Dackground
Dackground
Dackground
Dackground
Dackground
Dackground
Dackground
Dackground
Dackground
Dackground
Dackground
Calibration
Calibration
Calibration
Calibration
Calibration
A- 5
-------
Title
Calibration with Kandomly Changing Otandard vsurves
Linear Calibration When the Coefficient of Variation is Constant
Analytical Method Comparisons by Estimates of Precision and Louver Detection
Limit
Design Considerations for Calibration
Multivariate Calibration when the Lrror Covariance Matrix is Structured
An Implementation of the Ocheffe Approach to Calibration Using Opline
Functions, Illustrated by a r ressure" V olume Calibration
Measuring and Maximizing Precision in Analyses Based on Use of Calibration
vJraphs
Calibration in Quantitative Analysis! PartZ. Confidence Regions for the Oample
Content in the Case of Linear Calibration Relations
Design Aspects of Ocheffe s Calibration 1 heory using Linear Oplines
IMonconstant Variance (degression 1 echniques for vjalibration~vjurve~Dased
Analysis
Calibration in Quantitative Analysis
Calibration Curves with IMonuniform Variance
Elimination of the Dias in the Course of Calibration
Optimal Designs for the Inverse Regression Method of Calibration
A Statistical Theory of Calibration
Un the Problem of Calibration
Statistical Processing of Calibration Data in Quantitative Analysis by Gas
Lfhromatography
tstimation of a Linear Function for a Calibration Line. Consideration of a Recent
r roposal
A Note on Regression Methods in Calibration
Classical and Inverse Regression Methods of Calibration in Lxtrapolation
Regression
Year
1989
1988
1986
1986
1985
1982
1982
1981
1980
1980
1979
1979
1978
1973
1973
1972
1970
1969
1969
1969
1968
Author
D.F. Vecchla, H.K. Iyer, P.L. Chapman
Y.C. Yao, D.F. Vecchla, H.K. Iyer
D.M. Holland, F.F. McElroy
J.P. Duonaccorsi
T. Nae,
J.A. Leonner, C.P. Reeve, C.H.
Splegelman
D.G. Mitchell, J.S. Garden
J. Agterdenbos, F.J.M.J. Maessen, J.
Balke
C.H. Spiegelman, W.J. Studden
J.S. Garden, D.G. Mitchell, W.N. Mill.
J. Agterdenbos
L. Ochwartz
L.J. Naszodi
MA Thomas, R.H.Myers
H. Scheffe
G.K. Shukla
P. BooeK, J. NovaK
J. Berkson
EJ. William.
R.G. Krutchkoff
R.L Ott,R.H. Myers
Source
Technometrics 31 ! 83-90
Probability and Statistics! Essays in Honor of Franklin A.
Grayblll, 297-309
Environmental Science & Technology 20: 1157-1161
Technometrics 28i 149-155
Technometrics 2.1'. 301-311
Technometrics 24: 229-234
Ta,anta29:921-929
Analytica Chimica Acta 1 32i 127-137
Journal of Research of the National Bureau of Otandards OO!
295-304
Ana,. Cnem. 52:2310-2315
Analytica Chimica Acta 108i 315-323
Analytical Chem. 51 ! 723-727
Technometrics 20i 201-205
Communications in Statistics 2i 419-433
The Annals of Statistics 1 ! 1 -37
Technometrics 14i 547-553
J. Chromatog. 51 ! 375-383
Technometrics 11: 649-660
Technometrics Hi 189-192
Technometrics 11: 605-608
Category
Calibration
Calibration
Calibration
Calibration
Calibration
Calibration
Calibration
Calibration
Calibration
Calibration
Calibration
Calibration
Calibration
Calibration
Calibration
Calibration
Calibration
Calibration
Calibration
Calibration
A-6
-------
Title
Classical and Inverse Regression Methods of Calibration
1 he Interpretation of Certain Regression Methods and their Use in Biological
and Industrial Research
1 he 1 hree Ks for Relevant Detection, Reliable Quantitation and Respectable
Reporting Limits
Detection and Quantification Capabilities and the t valuation of Low "Level Data!
Oome International r erspectives and Continuing Challenges
Realistic Detection Limits from Confidence Dands
Response to Comment of An Alternative Minimum Level Definition for
Analytical Quantification
Comment on An Alternative Minimum Level Definition for Analytical
Quantification
Response to Comment on An Alternative Minimum Level Definition for
Analytical Quantification
Comment on An Alternative Minimum Level Definition for Analytical
Quantification"
Response to Comment on An Alternative Mriimum Level Definition for
Analytical Quantification
A Discussion of Issues Raised by Lloyd Currie and a Cross Disciplinary View of
Detection Limits and Estimating Parameters that are Often At or Near Zero
A Mock Trial for Critical Values (Detection Limits)
Comment on An Alternative Minimum Level Definition for Analytical
Quantification
1 he Omallest Concentration
A Statistical Overview of Standard (IUPAC and ACS) and New Procedures for
Determining the Limits of Detection and Quantification! Application to
Vol tarn metric and Otripping 1 echniques (1 echnical Report)
Response to Comment on An Alternative Minimum Level Definition for
Analytical Quantification"
Oome Conceptual and Otatistical Issues in Analysis of O round water Monitoring
Data
Oome Otatistical and Conceptual Issues in the Detection of Low Level
Environmental Pollutants
Year
1967
1939
2000
2000
1999
1999
1999
1999
1998
1998
1997
1997
1997
1997
1997
1997
1996
1995
Author
R.G. Krutchkoff
C. Lisenhart
Ann Kosecrance
L.A. Currie
J.R. Burdge, D.L. MeTaggart, S.O. Farwell
Henry Kahn, William Telliard, Chuck White
H.G. Rigo
Kobert Gibbons, David Coleman, Kay
Maddalone
Henry Kahn, William Telliard, Chuck White
Robert Gibbons, David Coleman, Ray
Maddalone
C.H. Spiegelman
vs.n. Opiegelman, r . I arlow
David Kimbrough
R.F. Moran, E.N. Brown
J. Mooak, A.M. Bond, S. Meitohell, G.
Ocollary
R.D. Gibbons, D.E. Coleman, R.F.
Maddalone
R.D. Gibbons
Robert Oibbons
Source
Technometrics Q! 425-439
The Annals of Mathematical Statistics 10: 162-186
Env. Testing & Anal. 9i 13,50
Journal of Radioanalytical and Nuclear Chemistry 245i 145-
156
Journal of Chemical Education 76i 434-439
Env. Soi.&Teoh. 33:1315
Env. So, &Teoh. 33: 131 1-1 31 2
Env. So,. &Teoh. 33: 131 3-1 314
Envir. So, & Teoh 32: 2346-2348
Envir. So, & Teoh 32: 2349-2353
Chemometrics and Intelligent Laboratory Oys terns O/. lOO"
188
STATSi The Magazine for Students of Statistics 20i 13-16
Envir. So,. & Teoh. 31 1 3727-3728
Clinical Chemistry 43i 856-857
Pure and Applied Chemistry 69i 297-328
Envir. Sci. & Tech 31 1 3729-3731
Environments 7: 185-1 99
Environ. & Ecol. Statistics 2i 125-167
Category
Calibration
Calibration
Critique
Critique
Critique
Critique
Critique
Critique
Critique
Critique
Critique
Critique
Critique
Critique
Critique
Critique
Critique
Critique
A-7
-------
Title
Comment on "Method Detection Limits in Solid Waste Analysis"
Comment on "Method Detection Limits in Solid Waste Analysis"
You Cant Compute With Less" 1 hans
Limits of Detection
Conflicting Perspectives About Detection Limits and About the Censoring of
Environmental Data
Limit of Discrimination, Limit of Detection and Oensitivity in Analytical Oystems
Discussion of! A Otudy of the Precision of Lead Measurements at
Concentrations Near the Method Limit of Detection
Limits of Detection Methodologies
Method Detection Limits in Solid Waste Analysis
Denning the Limits
ft Otudy of the Precision of Lead Measurements at Concentrations Near the
Method Limit of Detection
Detection Limit Concepts! Foundations, Myths, and Utilization
Difficulties Related to Using Extreme Percentiles for Water Quality Regulations
ft Oimple rvule for Judging Oomp Nance Using highly L* en sored Oamples
Current Method for Setting Dioxin Limits in Water Requires Reexamination
Kaiser S'Sigma Criterion - A Review of the Limit of Detection
MCL Noncompliance! Is the Laboratory at Fault?
Qualitative or Quantitative Characterization of Spectrographic Methods? The
Detection and Determination Limits
False Positives, Detection Limits, and Other Laboratory Imperfections! 1 he
rvegulatory Implications
Lvaluation of Detection Limit Estimators
Chemometrics - Measurement Reliability
The Detection Limit
Year
1995
1995
1994
1994
1994
1994
1994
1993
1993
1993
1993
1992
1991
1991
1990
1990
1990
1990
1989
1988
1988
1988
Author
D.E. Coleman
Janice Wakakuwa, David Kimbrough
Ken Usbom, Ann Kosecrance
N. Cressie
M.J.R. Clark, P.M. Whitfleld
R. Ferru,, M.R. Egea
B.R. Not., R.R. Maddalone
J. Lindstedt
David Kimbrougn, Janice Wakakuwa
G. Stanko, W. Krochta, A. Stanley, T.
Dawson, K. Hillig, R. Javick, R. Obrycki, B.
Hughes, l~. Oaksa
P.M. Bertnouex
D.A. Chambers, S.S. Dubose, E.L.
Sensintaffar
P. M. Bertnouex, Ian Hau
P. M. Bertnouex, Ian Hau
J. LaKind, E. Rifkin
L.S. Oresic, V. Grdinic
Oteven Koorse
Karol hlorian
Oteven Koorse
F.C. Garner, G.L. Robertson
K. Castaneda-Mendez
P.S. Porter, R.C. Ward, H.F. Be,,
Source
Environmental Science & Technology 29i 279-280
Env,r.Sci.& Tech. 29:281-282
East Bay Municipal Utility District, Core Laboratories
Chemometrics Intelligent Laboratory Oystems £.£. lul-luo
Water Resources Bulletin 30: 1063-1079
Analytica Chimica Acta 28?! 119-145
Water Environment Research 66: 853-854
Plating and Surface Finishing 80: 81-86
Enviro. Sd. & Tech 27: 2692-2699
Environmental Lab 1: 16-20
Water Environment Research 65: 620-629
Health Phys. 63: 338-340
Research Journal WPCF 63: 873-879
Research Journal WPCF 63: 880-886
Env. Sd. & Tech 24: 963-965
Acta Pharmaceutica Jugoslavica 40: 21-61
AWWA P53-58
Chemia Analityczna 35:129-139
Environmental Law Reporter 19: 10211-10222
Lfhemometrics and Intelligent Laboratory Oystems O. OO'OJ
Clinical Chemistry 34: 2494-2498
Environmental Science & Technology 22: 856-861
Category
Critique
Critique
Critique
Critique
Critique
Critique
Critique
Critique
Critique
Critique
Critique
Critique
Critique
Critique
Critique
Critique
Critique
Critique
Critique
Critique
Critique
Critique
A- 8
-------
Title
Estimation of Detection Limits for Environmental Analytical Procedures - A
1 u to rial
Limits of Detection
Limit of Detection! A Closer Look at the IUPAC Definition
1 race Analyses for Waste waters
A
comparison of statistical and empirical detection limits
Challenges in Regulatory tnvironmetrics
Determining vjuantitation Levels for Kegulatory Purposes
Defining Detection and Quantitation Levels
Concept ^000-A Statistical Approach for Analytical Practice - Part li Limits of
' '
Statistics and Environmental Policy! Case Studies from Long- 1 erm
tnvironmental IVIonitoring Data
The Many Dimensions of Detection in Chemical Analysis
A Practical Strategy for Determining and Verifying Detection Limits
Review of the Methods of the Uo Environmental Protection Agency for Dromate
Determination and Validation of Method 317.0 for Disinfection By-Product
Anions and Low -Level Dromate
Comparison of Detection Limits in Lnvironmental Analysis - Is it Possible: An
Approach on Quality Assurance in the Lower Working Kange by Verification
Un the Assessment of Compliance with Legal Limits, Part li Signal and
Concentration Domains
Capability of Detection - Part 2
N - E - A - M ID
Quantitation Capabilities (IUPAC Recommendations 1995)
New Reporting Procedures Dased on Long- I erm Method Detection Limits and
Some Considerations for Interpretations of Water" Quality Data Provided by the
U.S. Geological Survey National Water Quality Laboratory
Year
1988
1984
1984
1983
1982
1998
1997
1996
1993
1999
1999
1983
2001
2001
2001
2001
2000
1999
1999
Author
Cliff Kirchmer
J.K. Taylor
J.P. Foley, J.G. Dorsey
Gary Long, J.D. Winefordner
U.J. Kirchmer
G CC Su
C.B. Davis
P.P. Sanders, R.L. Lippincott, A. La ton
Raymond Maddalone, James Rice, Den
Ldmondson, Dabu Nott, Judith Scott
Hadrich J et al.
vJoudey K et al.
Currle LA
T. Georgian, K.E. Osborn
D.P. Hautman, DJ. MunoM, C. Frebls,
H.P. Wagner, B.V. PeplcM
S. Gelb, J.W. Elnax
E. Desimoni, S. Mannino, B. Brunetti
ISO
LA Currie
C. J. Obinger Childress, W. T. Foreman,
B. F. Connor, and T. J. Maloney
Source
ACS Symposium Series 361: 78~93
Analytical Chemistry 56i 130A
Analytical Chem. 55: 712-724
Envir.Sci.&Tech. 16:430A
Journal of AOAC International 81' 105-110
Chemometrics Intelligent Laboratory Systems Ol. 4o~bo
Journal American Water WorK, Association 88: 104-114
Water Envlr. & TecM. Jan.93: 41-44
Deutscne Lebensmlttel-Rundscnau 1999, 95(10), 428-436
NovartFDNSym 220; 144-1 57
Abstracts of Papers of the American Chemical Society, 1 Oo
(Mar), 63-PEST
Env. Testing & Analysis 10i 13-14
Journal of Chromatography A 920i 221-229
Fresenius Journal of Analytical Chemistry 370i 673-678
Accreditation Quality Assurance 6i 452-458
ISO 11 843-2
Analytica Chimica Acta 391' 105-126
USGS Open-File Report 99-193, 19 pages.
Category
Critique
Critique
Critique
Critique
M
ultilab
Multilab
Multilab
Multilab
not found
not found
not found
Single lab
Single lab
Single lab
Single lab
Single lab
c
Single lab
A-9
-------
Title
Analyses of Poly chlorinated Diphenyls and Chlorinated Pesticides in Diota!
Method and Quality Assurance
Detection Limits of Organic Contaminants in Drinking Water
Detection! International Update, and Oome Emerging Di-lemmas Involving
Calibration, the Blank, and Multiple Detection Decisions
rxegulations ~ From an Industry P erspective or Relationships Detween Detection
Limits, Quantitation Limits, and Significant Digits
Capability of Detection - Part 1
Determination of Oite-Opecific Lf fluent Detection Limits
Multivariate Detection Limits Estimators
Nomenclature in Lvaluation of Analytical Methods including Detection and
Quantification Capabilities
Reporting Low-Level Analytical Data Third Draft (1995-11-08) " New Project of
Commission V.I., International Union of Pure and Applied Chemistry
IUPAC Recommendations for Defining and Measuring Detection and
Quantification Limits
Recommendations for the Presentation of Ke suits of Chemical Analysis
De tar chi "A Program for Detection Limits with Opecified Assurance Probabilities
and Characteristic Curves of Detection
Quality Control Level! An Alternative to Detection Levels
Multivariate Decision and Detection Limits
A Model of Measurement Precision at Low Concentrations
Robust Procedure for Calibration and Calculation of the Detection Limit of
I rimipramine by Mdsorptive Otripping V oltametry at a Carbon Paste tlectrode
Nondetects, Detection Limits, and the Probability of Detection
Detection Limits! For Linear Calibration Curves with Increasing Variance and
Multiple Future Detection Decisions
Limits of Detection in Multivariate Caibration
Estimating Detection Limits in Ultratrace Analysis. Part \'. The Variability of
Estimated Detection Limits
Year
1999
1998
1997
1997
1997
1996
1996
1995
1995
1994
1994
1994
1994
1993
1993
1993
1991
1991
1991
1991
Author
P. Cleemann, G.B. Paulson
W.M. Draper, J.S. Dhoot, J.S. Dhaliwal,
J.W. Remoy, O.K. Perera, F.J. Daumann
L.A. Currie
D. Coleman, J. Auses, N. Gram,
ISO
George Neserke, Harold Taylor
R. Doque, F.A. Rius
Lloyd Currie
William Horwitz
LA Currl,, W. Horwitz
L.A. Currie, G. Svehla
L. Sarabia, M.C. Ortiz
D.E. Kimbrough, J. Wakakuwa
A. Singh
P.M. Berthouex, D.R. Can
M.C. Ortiz, J. Aroo,, J.V. Jurarro,, J.
Lopez'Palacios, L.ft. Oarabia
D. Lambert, D. Peterson, I. I erpenning
R.D. Gibbons, F.H. Jarke, K.P. Stoub
G. Bauer, W. Wegscheider, H.M. Ortner
C.L. Stevenson, J.D. Winefordner
Source
Journal of AOAC International 82: 1175-1184
Journal of the American Water Works Association 90: 82-90
Chemometrics and Intelligent Laboratory Oys terns O/! 1b1~
181
Chemometrics and Intelligent Laboratory Oys terns O/. / I ~OU
ISO 11 843-1
WaterEnv. Re,. 66:115-119
Chemometrics and Intelligent Laboratory Oys terns O/_! 1 1 -£O
Pur, & App,. Chem. 67: 1699-1723
IUPAC
Analuses Magazine 22'. 24-26
Pur, & Applied Chemistry 66: 595-608
TRAC-Trends in Analytical Chemistry 13: 1-6
Environmental Science & Technology 28: 338-345
Analytloa Chimlca Aota 277: 205-214
Water Environment Research 65: 759-763
Analytical Chemistry 65: 678-682
JASA 86: 266-277
Waste Testing and Quality Assurance: ASTM STP 1075, D.
Friedman, Ld., American Oociety for I esting and Materials,
Philadelphia 3: 337-390
Fresenius Journal of Analytical Chemistry o4U: lob-lo.7
Applied Spectroscopy 45:1217-1224
Category
Single lab
Single lab
Single lab
Oingle lab
Oingle lab
Oingle lab
Single lab
Single lab
Single lab
Single lab
Single lab
Single lab
Single lab
Single lab
Single lab
Single lab
Single lab
Single lab
Single lab
Single lab
A- 10
-------
Title
Ke porting Low Level Data for Computerized Data Dases
Detection Limits with Specified Assurance Probabiities
Limit of Detection and Limit of Determination - Application of Different Statistical
Approaches to an Illustrative Lxample of Residue Analysis
Determining the Lowest Limit of Keiable Assay Measurement
Principles of tnviron mental Analysis
1 race Analyses for Waste waters
(guidelines for Data Acquisition and Data Quality t valuation in Environmental
Chemistry
Sensitivity and Limit of Detection in Quantitative Spectrometric Methods
Decision and Detection Limits for Linear Calibration Curves
Limits for Quantitative Detection and Quantitative Determination
A Statistical Method for Lvaluation of Limiting Detectable Jam pie
Concentrations
Initial Lvaluation of Quantitative r erf or ma nee of Chromatographic Methods
Using Replicates at Multiple Concentrations
Multivariate Detection Limits with Fixed Probabilities of Error
Lvaluation of Approximate Methods for Calculating the Limit of Detection and
Limit of Quantification
Limits of Detection, Identification and Determination! A Statistical Approach for
Practitioners
Weighted Least-Squares Approach to Calculating Limits of Detection and
Quantification by Modeling Variabiity as a Function of Concentration
Detection Limits in GO-MS Multivariate Analysis
An Alternative Minimum Level Definition for Analytical Quantification
Year
1988
1987
1987
1983
1983
1981
1980
1974
1970
1968
1967
2001
1999
1999
1998
1997
1997
1997
Author
M. Brossman, G. MoKenna, H. Kann, D.
King, R. Kleopfer, J. Taylor
C.A. Clayton, J.W. Mines, and P.O. ElKins
J. Vogelgesang
L. Oppenheimer, T.P. Capizzi, R.M.
Weppelman, H. Mehta
L.H. Keith, W. Crummett, J. Deegan Jr,
R.A. Libby, J.K. Taylor, G. Wentler
John Glaser, Denis Foerst, Gerald McKee,
Stephan Quave, William Budde
MacDougall, Daniel, Ot 3l.
J.D.IngleJr.
A. Hubaux, G. Vos
L.A. Currie
PA St. John, WJ. McCarthy, J.D.
Winefordner
M.A. Castillo, R.C. Castells
R. Boqu,, M.S. LarreoMi, F.X. Riu,
M.E. Zorn, R.D. Gibbons, W.C. Sonzogni
J. Vogelgesang, J. Hadrich
M.E. Zorn,, R.D. Gibbons, W.C. Sonzogni
Doque K et al.
Kobert Oibbons, David Coleman,
Raymond Maddalone
Source
ACS Symposium Series 361i317-327
Analytical Chemistry 59i 2506-2514
Fresenius Zeitschrift Fur Analytsche Chemie 328i 213-220
Analytical Chemistry 55i 638-643
Analytical Chemistry 55i 2210-2218
Env.Soi.&Teon 15: 1426-1 435
Ana,. Cn,m. 52: 2242-2249
Journal of Chemical Education, 51, 100-105.
Analytical Chemistry 42i 849-855
Analytical Chemistry 40i 586-593
Analytical Chem. 39i 1495-1597
Journal of Chromatography A 9^1 ! Izl-loo
Chemometrics and Intelligent Laboratory Oystems 4u! OJ/-
408
Environmental Science & Technology 33i 2291-2295
Accreditation Quality Assurance 3i 242-255
Analytical Chemistry 69i 3069-3075
Quimica Analytica, 16(2), 81-86
Environmental Science & Technology 31 ! 2071 -2077
Category
Single lab
Single lab
Single lab
Single lab
Oingle lab
Oingle lab
Single lab
Single lab
Single lab
Single lab
Single lab
Single lab -
multilevel
Single lab -
multilevel
Single lab -
multilevel
Single lab -
multilevel
Single lab -
multilevel
Single lab -
multilevel
Single lab -
multilevel
A- 11
-------
Title
A 1 wo'vjomponent Model for Measurement trror in Analytical Lfhemistry
Practical Quantitation Limits
Experimental Comparison of EPA and USATHAMA Detection and Quantitation
Capability Estimators
High Pressure Liquid Chromatography Determination of the Intermediates Oide
Reaction Products in FD&C Red No. 2 and FD&C Yellow No. 5i Statistical
Analysis of Instrument Response
Method Detection Limits! Application to Organic Environmental Chemistry Data
Detection limits
Statistical Approaches to Estimating Mean Water Quality Concentration with
Detection Limits
Detection Limit of Isotope Dilution Mass Opectrometry
ISO 17025 Requirements! How to Evaluate Uncertainty for Dioxin Analysis in
Food and Feed from Validation Data?
Are Target Dioxin Levels in Animal Feed rig stuffs Achievable for Laboratories in
I erms of Analytical Requirements . Kesults of an Interlaboratory study
Environmental Statistics with O'PLUo
Quantifying Uncertainty in Analytical Chemistry
Development and Harmonisation of Measurement Uncertainty Principles.
Protocol for Uncertainty Evaluation from Validation Data
Statistical Procedures for Analysis of Environmental Monitoring Data & Risk
Assessment
Year
1995
1992
1991
1978
2002
2002
2002
2002
2002
2002
2001
2000
2000
1998
Author
L/avid rvocke, Otefan Lorenzato
R.D. Gibbons, N.E. Grams, F.H. Jarke,
K.P. Stoub
C.L. Grant, A.D. Hewitt, T.F. Jenkins
C.J. Bayley, E.A. Cox, J.A. Springer
Wad,, T. L, J. L. Serlcano, Y. Qlan, G.
Wolff, and G. Denoux
van der Voet, H.
O hum way, K., K. Azari, and M. Kayhanian
Yu, L. L., J. D. Fasse,,, and W. F. Gutnrle
O. Eppe, and E. De Pauw
\J. Eppe, and E. L/e r auw
Millard, S. P., and Neerchal, N. K.
Ellison, S. L. R., M. Roslein, and A.
Williams, eds.
Barwick V.J., Ellison S.L.R.
McBean, E. A., and Rovers, F. A.
Source
Technometrics37:176-184
Chemometrics and Intelligent Laboratory Oy stems \L.\ £.1*3-
235.
American Laboratory Zo! lb~OO
J. Assoo. Off. Ana,. Chem 61 1 1404-1414.
Presentation at ACo Oymposium, Boston, MA, August ^00^
Encyclopedia of Environme tries, 1, El~Ohaarawi, A. H. and
Plegorsch, W. W. (ed».), 504-515. CnicMester: John Wiley &
Oons
Environmental Ocience and I echnology OU. OOT-O~OOOO
Analytical Chemistry. 74i 3887-3891
Proceedings of the ZZnd International Oymposium on
Halogenated Environmental Organic Pollutants and POPs,
Barcelona, Spain, August 12-18, 2002, Vol. 59, pp. 403-
406, 2002
Proceedings of the ££nd International Oymposium on
Malogenated Environmental Organic r o Mutants and r Or s,
Barcelona, Spain, August 12-18, 2002, Vol. 59, pp. 407-
410,2002
Chapman & Hall/CRC, Boca Raton, FL
EURACHEM/CITAC Guide 2000
VAM Project 3.2.1
Prentice Hal, PTR, Upper Saddle River, NJ
Category
Single lab -
multilevel
Single lab -
multilevel
Single lab -
multilevel
Single lab -
multilevel
Ouggested by a
peer reviewer
Ouggested by a
peer reviewer
Ouggested by a
peer reviewer
Ouggested by a
peer reviewer
Ouggested by a
peer reviewer
Ouggested by a
peer reviewer
Ouggested by a
peer reviewer
Ouggested by a
peer reviewer
Ouggested by a
peer reviewer
Ouggested by a
peer reviewer
A- 12
-------
Title
Foundations and future of detection and quantification limits
Statistical Methods for Ground water Monitoring
Opatial chemostati sties
Hypothesis testing with values below detection limit in environmental studies
A new approach for accommodation of below detection limit data in trend
analysis of water quality
Lrrors and detection limits
Otatistical inference from multiply censored environmental data
Less than obvious! Otatistical treatment of data below the detection limit
Environmental tests! Are they valid:
Detection in Analytical Chemistry! Importance, 1 heory, and Practice
Year
1996
1994
1994
1994
1994
1992
1991
1990
1990
1988
Author
Currie, L. A.
Gibbons, R. D.
Cressie, N.
Slyman, D. J., de Peyster, A., and
Lsonohoe, l\. I\
NagaraJ, N. K., and Brunenmeister, S. L
Adams, M. J.
El-Snaarawi, A. H., and Naderi, A
Helsel, D. R
Maynard, A. W.
Currie, L. A.
Source
Proceedings of the American Otatistical Association, Oection
on Otatistics and the Lnvironment, 1 -8
John Wiley & Sons, New York
Lnvironmental Otatistics, Assessment, and Forecasting,
Cothern, C. R. and Ross, N. P. (ed,.), 131-146. Boca Raton,
PL! Lewis Publishers
Environmental Science & Technology 28: 898-90
tnvironmental Otatistics, Assessment, and Forecasting,
Cothern, C. R. and Ross, N. P. (ed,.), 113-127. Boca Raton,
FL: Lewis Publishers
In Methods of Environmental Data Analysis, Hewitt, C. N.
(ed.), 181 -212. Amsterdam! Elsevier Applied Science
Environmental Monitoring and Assessment 17: 339-347
Environmental Science & Technology 24 ! 1766-1774
Chemical Technology 20i 151-155.
American Chemical Oociety, New York
Category
Ouggested by a
peer reviewer
Ouggested by a
peer reviewer
Ouggested by a
peer reviewer
Ouggested by a
peer reviewer
Ouggested by a
peer reviewer
Ouggested by a
peer reviewer
Ouggested by a
peer reviewer
Ouggested by a
peer reviewer
Ouggested by a
peer reviewer
Ouggested by a
peer
reviewer
A- i:
-------
Appendix B
Computation of Detection and Quantitation Limits
INTRODUCTION
This appendix supports the Revised Assessment Document (RAD) for EPA's assessment of
detection and quantitation approaches. It presumes that the reader has read chapters three through five
of the RAD.
We have compared detection and quantitation limits computed from data gathered by EPA or
submitted to EPA by stakeholders commenting on EPA's February 2003 (EPA-821-R-03-005)
assessment. The comparison shows that, in general, detection limits derived from a single concentration
level such as EPA's MDL are, on average, approximately the same as detection limits derived from
similar approaches such as the ACS LOD and LOQ and ISO/IUPAC CRV and MDV, and are
approximately three times lower than a single-laboratory variant of ASTM's IDE; and that all quantitation
limit approaches, such as EPA's ML, the ACS and ISO/IUPAC LOQ, and a single-laboratory variant of
ASTM's IQE, produce quantitation limits that are generally only slightly different
EPA's Approach to Establishing Detection and Quantitation Limits in Analytical Methods
The Engineering and Analysis Division (BAD) within EPA's Office of Science and Technology
develops analytical methods for use in EPA's Clean Water Act (CWA) programs. In developing these
methods, BAD first conducts a single-laboratory study in which an MDL and ML are determined
followed by multiple single-laboratory studies in which the MDL and ML are either verified or if
necessary, revised. If resources, time, and applications of the method warrant, an interlaboratory study is
conducted in which the MDL and ML are further verified or, if necessary, revised.
To set an MDL, which is both conservative and achievable by qualified laboratories, we generally
select the highest MDL from among the MDLs determined or verified by laboratories in the various
studies. For example, EPA determined the MDL in Method 1631 (mercury by cold-vapor atomic
fluorescence) as 0.05 ng/L in a single laboratory and revised this MDL to 0.2 ng/L based on multiple
single-laboratory studies. All laboratories verified the MDL of 0.2 ng/L in an interlaboratory study.
Unlike a single-lab MDL and ML computed in a laboratory quality-control setting, the interlaboratory
MDL established during method development is set as a high-biased estimate of Currie's Lc. Thus, the
single-lab MDL and resulting ML, when scaled up with the interlaboratory MDL data, are very
conservative. This interlaboratory scaling up protects against unrealistically low values, and responds to
concerns that the MDL is a single-laboratory approach that produces unrealistically low MDLs.
DETECTION AND QUANTITATION LIMITS ASSESSED
EPA used several datasets to evaluate various approaches to determining detection and
quantitation values. These data are described in the Data section of this Appendix.
In the original Assessment Document (EPA, February 2003), four different detection and three
different quantitation limits were evaluated and compared. The detection limits were the EPA method
B- 1
-------
detection limit (MDL), the International Standards Organization/International Union of Pure and Applied
Chemistry (ISO/IUPAC) critical value (CRV) and minimum detectable value (MDV), and a single-
laboratory variant of the ASTM interlaboratory detection estimate (IDE). The quantitation limits were the
EPA minimum level of quantitation (ML), the ISO limit of quantitation (LOQ), and a single-laboratory
variant of the ASTM interlaboratory quantitation estimate (IQE).
Several stakeholders commenting on EPA's assessment of data expressed difficulty in replicating
EPA's calculations supporting these evaluations. Based on these comments, EPA reviewed the computer
programs used to calculate the various limits, and compared results obtained using these programs to
calculation results and software packages submitted by commenters. EPA concluded that many of the
discrepancies between EPA and commenter calculations were due to differences in the datasets and
software used (see Software Comparison, later in this appendix). As a result of this review, EPA did,
however, find some discrepancies which have been resolved in this document. Revisions are listed below:
• In calculating the single-laboratory IDE (SL-IDE) and single-laboratory IQE (SL-IQE) based on
the Exponential model using the Episode 6000 and Method 1631 and 1638 validation study data,
incorrect weights were used when modeling recovery. Because the majority of the SL-IDEs
were calculated using this model, most of the SL-IDEs presented in Tables 2, 6, 7 and 8 have
changed. Because the SL-IQEs were not calculated based on the exponential models, these
values did not change.
• When calculating MLs based on the Episode 6000 data, the resulting ML was incorrectly rounded
up for many analytes. This has been corrected, and many of the calculated MLs in Tables 4 and
5 have changed.
• In the 2003 assessment, blank results were included in the calculations of the ISO/IUPAC CRV,
MDV and LOQ. Upon further review, it was decided that it was invalid to use blank results
included in the Episode 6000 study, because these blanks were used to assess carry-over, and
would not be representative of routine blank analyses. Therefore, the ISO/IUPAC limits were
re-calculated using the lowest spike concentration in place of blank results.
• For two analytes in the Episode 6000 data (uranium and thallium by Method 20 0.8), incorrect
formatting caused multiple spiking levels to be combined improperly. This affected the calculation
of all limits for these analytes. This calculation has been fixed, and the limits have changed
slightly for these two analytes.
• After completion of the Original Assessment Document, a new version of the IDE procedure
(D6091-03) was published by ASTM. This procedure included the use of a standard deviation
bias correction factor which was not included in the prior version (D6091-97). Therefore, all
IDEs calculated using the Episode 6000 and Methods 1631 and 1638 validation study data were
re-calculated using this correction factor. For the majority of analytes, the resulting IDEs
increased slightly (by approximately 4%).
The effect of these changes on the analyses are discussed in the Results of Computations section
of this Appendix. To better explain how calculations were run, Appendix C gives example calculations of
the SL-IDE, SL-IQE, MDL and ML for one analyte.
B-2
-------
Along with comments on EPA's assessment, both the American Council of Independent
Laboratories (ACIL) and USGS submitted data and procedures. ACIL submitted a procedure for
calculating a critical level (CRV) and Long-Term MDL (LTMDL). USGS submitted its procedure for
calculating a long-term MDL (USGS LT-MDL). Both the ACIL critical level and USGS LT-MDL are
estimates of Currie's Lc, and are therefore comparable to the EPA MDL. Both the ACIL and USGS
procedures, however, are based on results collected over a long period of time. The ACIL critical level is
based on blank results, and the USGS LT-MDL is based on spiked results. The formula for the ACIL
critical level is identical to that of EPA's MDL, except that the mean of the blanks is added to the product
of the standard deviation and t-statistic. The USGS procedure does not use a sample standard deviation,
but instead uses a non-parametric estimate of variability that is based on the interquartile range. The
USGS LT-MDL procedure also allows addition of the mean or median of blank results to the LT-MDL.
ACIL also suggested a separate CRV procedure (ACIL "Case 2") for calculating estimates for
those methods for which analysis of blank samples does not produce a signal. For thes e metho ds, ACIL
suggested an iterative procedure that first determines the lowest level at which all 7 replicates are
detected, and then estimates the CRV as the lowest of the observed results of 7 spikes. The analogue of
Currie's Ld is estimated as this lowest spike level EPA finds merit in the idea of dividing the methods
into two groups (depending on the presence or absence of a signal from analysis of blank samples) and in
the idea of estimating the detection level of the instrument, and plans to further investigate the ACIL
approach. However, the particular implementation of the ACIL Case 2 procedure has some conceptual
problems that precluded it from evaluation at this time. These problems are described later in this
Appendix (see "Episode 6000 Data").
EPA provides further discussion of these approaches and the Agency's reasons for selecting
them in Chapters 1 and 2 of the RAD.
Commonality of Approaches
The EPA, ACS, and ISO/IUPAC approaches are all multiples of the standard deviation of either
replicate measurements of a blank or of the lowest spike concentration that produces positive (non-zero)
results for all 7 replicates. Similarly, the ACIL and USGS approaches are based on multiples of a
parametric or nonparametric estimate of variability of replicate measurements, with the difference that
the given estimate includes greater sources of variability than those of the other single-concentration
approaches.
Other subtle distinctions are that (1) ISO/IUPAC suggest a false positive rate of 5 % (a = 0.05)
for the CRV and MDV, whereas EPA specifies a falsepositive rate of 1 % (a = 0.01) forthe MDL and
(2) the EPA MDL was calculated by pooling data from two concentration levels after determining that
the variabilities of the two concentration levels are not significantly different (as provided as an option in
step 7 of the MDL procedure), thereby increasing the degrees of freedom to 12 from the 6 used in
computation of the ISO/IUPAC CRV and ACS LOD. The consequence of distinction (1) is that an
approach with a higher allowed false positive rate (a = 0.05) will produce a lower detection limit than an
approach with a lower false positive rate (a = 0.01). The consequence of distinction (2) is that a
detection limit resulting from pooling at two levels will be more stable and likely somewhat lower than a
detection limit at a single level (given the same variability at each level) because the degrees of freedom
are increased in the t statistic.
B-3
-------
The AC S and IS O/IUPAC approaches specify replicate measurements of blank samples. In
computing detection and quantitation limits from the Episode 6000 data, blank results were not used, as
blanks analyzed in this study included carry-over effects, and were therefore not representative of routine
blank results. Therefore, the lowest spike concentration (or, in the case of the MDL, two lowest spike
concentrations) that produced a non-zero result was used for computation of all approaches. This
simplification condensed the EPA MDL and the ACS LOD to a single approach subsequently termed the
EPA/ACS DL. Similarly, the EPA ML and ACS LOQ were condensed to a single approach, termed the
EPA/ACS QL.
The remaining single-concentration approaches are the ISO/IUPAC CRV, MDV, and LOQ, the
ACIL critical level and the USGS LT-MDL. The ISO/IUPAC CRV differs from the EPA/ACS DL
because of its suggested use of a false positive rate of 5% (a = 0.05) versus use of a false positive rate of
1% (a = .01) in the EPA/ACS DL. The ISO/IUPAC MDV also differs from the EPA/ACS DL because
of (1) its suggested use of a false positive rate of 5% (a = 0.05), (2) its stated false negative rate of 5 %
(P=0.05), and (3) recovery correction (estimated using a linearregression). Therefore, the ISO/IUPAC
CRV and MDV were each treated separately (were not combined with the EPA or ACS approaches)
from the other detection limit approaches in the data analysis. The ISO/IUPAC LOQ is also different
from the other quantitation limit approaches and was treated separately from these approaches. The
ACIL critical level differs from the EPA/ACS DL in its inclusion of long-term variability and the addition
of the mean blank result to the limit. The USGS LT-MDL differs from the EPA/ACS DL in its inclusion
of long-term variability, the addition of the median or mean blank result to the limit, and the use of a
nonparametric estimate of variability in place of the sample standard deviation. Because of the lack of
long-term variability and representative blank results in the Episode 6000 data, the ACIL critical level and
USGS LT-MDL could not be calculated using the Episode 6000 data. Assessments of these approaches
in comparison to the EPA/ACS DL were done using blank and spiked sample data that were submitted to
the Agency by ACIL and USGS.
The ASTM IDE and IQE were treated separately because they are constructed by fitting a
model to variability versus concentration data, rather than being derived from the standard deviation of
replicate measurements of a single concentration, (as are the EPA, ACS, ISO/IUPAC and ACIL
approaches). Similar to some of the ISO/IUPAC approaches, the ASTM IDE and IQE include
"protection" against false negatives and recovery correction (see section 3.3 of the Revised Assessment
Document for a discussion on EPA's concerns about false negative protection). The IQE, but not IDE,
also includes an added correction for the bias associated with an estimate of the true standard deviation at
each concentration. In the context of the IQE, the word "bias" means the amount by which the estimated
sample standard deviation is low compared to the true population standard deviation, and should not be
confused with common use of the word "bias" in an analytical measurement.
Single-laboratory Variants of Interlaboratory Approaches
Because the EPA, ACS, and ISO/IUPAC approaches are single-laboratory approaches, and the
ASTM IDE and IQE are interlaboratory approaches, the ASTM approaches could not be computed using
the single-laboratory data in the Episode 6000 studies. To solve this problem, single-laboratory variants of
the IDE and IQE were used. These single-laboratory variants were termed the SL-IDE and SL-IQE for
"single-laboratory IDE" and "single-laboratory IQE," respectively. The SL-IDEs and SL-IQEs were
constructed using the overall standard deviation within a single laboratory at each concentration rather
than the overall standard deviation across all laboratories at each concentration.
B-4
-------
Attempted Application to Intel-laboratory Data
EPA attempted to apply the various approaches to interlaboratory study data in response to a
request by the Petitioners to the Settlement Agreement and so that detection and quantitation limits could
be compared. However, because the EPA, ACS, and ISO/IUPAC approaches are single-laboratory
approaches, whereas, the ASTM approaches are interlaboratory approaches, it was not possible to
compute directly comparable detection and quantitation limits from the same data.
It was possible, however, to compare the detection and quantitation limits produced by EPA and
the Electric Power Research Institute (EPRJ) from the EPA Method 1631 and EPA Method 1638
interlaboratory study data. Although the resulting detection and quantitation limits are either single-
laboratory (EPA) or interlaboratory (ASTM), as appropriate to the particular approach, a comparison of
the resulting limits can be informative. The EPRI detection and quantitation limits are presented in EPRI
reports of the results of the Method 1631 and Method 1638 studies.
DATA
Datasets Evaluated
The datasets used to evaluate the detection and quantitation approaches discussed above are
described in this section. EPA computed EPA/ACS DLs and QLs; ISO/IUPAC CRVs, MDVs and
LOQs; and single-laboratory variants of ASTM IDEs (SL-IDEs) and IQEs (SL-IQEs) using the Episode
6000 data. EPA also computed IDEs and IQEs for the Method 1631 and 1638 interlaboratory study data.
EPA computed ACIL's critical level, USGS's LT-MDL and EPA's MDL based on a combination of
blank and spiked data submitted by USGS, and performed an assessment of the effect of long-term
variability based on blank data submitted by ACIL.
EPA's Variability versus Concentration Studies ("Episode 6000")
In 1997 and 1998, EPA conducted a study of variability vs. concentration for a number of
analytical methods. Six laboratories were employed for the analyses; each analyte and method
combination was tested by one of these laboratories. For nearly all of the technologies, the studies were
conducted by spiking reagent (i.e., blank, presumably "clean") water at 16 different concentrations per
analyte, ranging from 100 times an initial estimate of the MDL to 0.1 times the initial estimate. A total of
198 analytes were measured, generally with seven replicates analyzed at each concentration. Details of
the study design are described in EPA's Study Plan for Characterizing Variability as a Function of
Concentration for a Variety of Analytical Techniques (July 1998), and in Appendix C of the February
2003 Assessment document. Based on the sampling episode number assigned to the study by the EPA
Sample Control Center, the study and results have become known as the Episode 6000 study and data.
The analytes and analytical techniques studied were:
• Total suspended solids (TSS) by gravimetry
• Metals by graphite furnace atomic absorption spectroscopy (GFAA)
• Metals by inductively-coupled plasma atomic emission spectrometry (ICP/AES)
• Hardness by ethylene diamine tetraacetic acid (EDTA) titration
• Phosphorus by colorimetry
B-5
-------
• Ammonia by ion-selective electrode
• Volatile organic compounds bypurge-and-trap capillary column gas chromatography with a
photoionization detector (GC/PID) and electrolytic conductivity detector (GC/ELCD) in series
• Volatile organic compounds by gas chromatography with a mass spectrometer (GC/MS)
• Available cyanide by flow-injection/ligand exchange/amperometric detection
• Metals by inductively-coupled plasma spectrometry with a mass spectrometer (ICP/MS)
EPA's 2003 assessment of detection and quantitation examined a dataset populated with the results of
this study, the object of which was to characterize analytical variability as a function of concentration over
a wide range of concentrations, analytes, and analytical methods. Data from this study, including many
tables and plots, were provided in the record supporting EPA's original assessment and discussed in
EPA's "Technical Support Document for the Assessment of Detection and Quantitation Approaches,"
EPA 821-R-03-005, February 2003. The database developed contains a total of approximately 22,000
data points. This study was conducted in contract laboratories. EPA performed a contract compliance
review of these studies at the time the studies were conducted, but not a point-by-point review of each of
the tens of thousands of data points.
In the study, an initial (range finding) MDL was determined for each combination of analyte and
analytical technique using a revised draft of the MDL procedure. The revised draft had three significant
changes: (1) the definition was more closely conformed to the MDL procedure; (2) optional iterative step
7 of the MDL procedure was made mandatory; and (3) the spike concentration to MDL was reduced
from 5 to 3 in an attempt to narrow the resulting MDL. During data gathering, two laboratories
complained that the reduction in spike to determined MDL ratio (from 5 to 3) caused a large number of
iterations and stated that 5 was more reasonable. Subsequently, EPA returned to the spike to MDL ratio
of 5 published in the 40 CFR 136, Appendix B procedure.
After determining the initial MDL, each laboratory analyzed 7 replicates of samples spiked at
concentrations of 100,50, 20, 10, 7.5, 5.0, 3.5,2.0, 1.5, 1.0, 0.75, 0.50,0.35, 0.20, 0.15, and 0.10 times the
initial MDL. In a few instances, laboratories analyzed more than 7 replicates. Results associated with
the replicate analyses at each concentration level were obtained, as often as possible, using the same
calibration that was used in determining the initial MDL. Where laboratory reports indicated that multiple
calibrations were conducted, the association between each result and its calibration was used in the data
analysis.
Spiked aqueous solutions were analyzed in order from the highest concentration (100 times the MDL)
to the concentration at which 3 or more non-detects (zeros) were encountered among the 7 replicates, or
the lowest concentration specified (0.1 times the MDL), whichever occurred first. This analysis order (1)
minimized carryover that could occur in some methods if a low-concentration sample had followed a high-
concentration sample (as may happen when samples are analyzed in random order), and (2) prevented
collection of a large number of zeros if the signal disappeared.
A variant of the iterative MDL procedure was used for organic compounds determined by
chromatographic methods. Methods for organics normally list many (15 to 100) analytes, and the
response for each analyte is different. Therefore, to determine an MDL for each analyte, the
concentration of the spike would need to be inversely proportional to the response. Making a spiking
solution with 15 to 100 different concentrations is cumbersome and error prone. The approach used in the
study was to run 7 replicates at decreasing concentrations until signal extinction, then select the
B-6
-------
concentration(s) appropriate for the MDL for each analyte according to the MDL procedure. In some
cases the laboratories selected the concentrations, in others cases, EPA did. This approach was generally
applied for organics analysis. However, laboratories also had the option of using some combination of
monotonically decreasing concentrations described above and a few selected concentrations to achieve
the desired spiking levels.
Some commenters on the 2003 assessment noted possible errors. EPA reviewed these comments
and examined the individual data values and other aspects of the assessment that commenters thought
were in error. Commenters commented most frequently on measurements of organic compounds by
EPA Methods 502.2 (halogenated and aromatic volatilesby GC with photoionization and electrolytic
conductivity detectors in series) and 524.2 (volatilesby GC/MS) that were included in the Episode 6000
dataset. EPA performed a more comprehensive review of these data points, and found that the
calculated recoveries of some of the compounds are higher or lower than would be expected for the
analytical technologies employed. There also appear to be low background concentrations of some
compounds in the reagent (blank) into which the analytes were spiked. Backgrounds are commonly
observed in determinations of metals, radionuclides, and some volatiles.
Without the raw data for the analyses in question, it is not possible to unequivocally determine the root
cause(s) of the high or low recoveries and possible backgrounds. However, atypical recoveries may have
been the result of (1) laboratories making measurements at levels as much as 50 times below the lowest
level to which they would normally calibrate to establish MDLs and MLs at as low a level as could be
measured, and (2) EPA's request that the laboratories use a single calibration (rather than multiple) to
prevent discontinuities in the variability vs concentration trends that were the object of these studies.
Another possible explanation for the low apparent recoveries is the setting of thresholds in the GC and
GC/MS analyses. If a small constant area of a GC response peak is removed by thresholding, the relative
amount of area that is removed will increase as the concentration is reduced, resulting in lower apparent
recoveries at the lower concentrations. This effect would be consistent with observations in some of the
data.
As for possible backgrounds for volatiles or metals, these backgrounds likely were either present in
the reagent (blank) water used by the laboratories for the MDL determinations, or by carry-over from
one sample to another. To test for carry-over, some laboratories analyzed one or more blank sample
between spike levels after verification of calibration. Instances in which multiple blanks were analyzed
often show decreasing small concentrations for some of the analytes. However, these backgrounds
resulting from carry-over mean that blank results should not be used to assess false positive rates of the
different limits calculated using the Episode 6000 data.
Interlaboratory Study Data
EPA used data from two interlaboratory method validation studies to calculate IDEs and IQEs for a
total of 10 metal analytes. These studies were conducted by EPA to evaluate performance of EPA
Methods 1631 and 1638, and to gather data to evaluate existing performance specifications, including
detection and quantitation limits. To expand the scope of the study, the Electric Power Research Institute
(EPRI) funded the distribution of additional samples to study participants. Each study included multiple
participant laboratories: twelve for Method 1631 and eight for Method 1638.
B-7
-------
The two studies were designed so that each participating laboratory would analyze sample pairs of
each matrix at concentrations that would span the analytical range of the method. Each laboratory was
provided with multiple sample pairs, including samples measured in filtered effluent, unfiltered effluent,
marine water, filtered freshwater, and spiked reagent water. Each laboratory analyzed reagent water
sample pairs for each analyte at five different concentration levels. The results of the reagent water
analyses were used to fit variability functions and calculate IDEs and IQEs.
Data from these studies also are discussed in Chapter 1 of this document.
Data Submitted by Stakeholders
EPA also used datasets containing results from analysis of blank samples provided by two
stakeholders. Blanks analyzed over a period of three months for five analytes using Method 200.7 were
provided by the American Council of Independent Laboratories (ACIL), while blanks analyzed over a
period of one year representing 78 analytes were provided by the US Geological Survey (USGS). In
addition to these blank results, USGS sent results of the analyse of spiked samples representing 39
analytes. Because spiked samples were analyzed only at a single concentration level, many of the
different detection and quantitation limits, such as the SL-IDE and SL-IQE, cannot be calculated using
these data. However, a comparison of the critical level suggested by the ACIL, the LT-MDL suggested
by USGS, and the EPA MDL was performed using the blank and spiked results.
The data submitted by ACIL and USGS also are discussed in Chapter 1 of this document.
Datasets Not Evaluated
The Petitioners and Intervenor to the Settlement Agreement provided the list of datasets shown in
Table 1 and suggested that EPA evaluate detection/quantitation limit approaches using the datasets on the
list. However, in reviewing the datasets suggested, EPA determined that many were developed for
characterizing the behavior of an analyte or analytes across the analytical range of a method, rather than
in the region of detection and quantitation. For example, any dataset developed prior to the advent of the
IDE and IQE would be inappropriate because there could not have been an estimate of IDE0 or IQE0
(i.e., an initial estimate of the given limit; see Section 6.2.2.1 of D6091 and D6512). This eliminates all
datasets in Table 1 except the EPA/EPRI Method 1631, the EPA/EPRI Method 1638 dataset, and the
MMA 2001-2 dataset. It is possible that some spike level in one or more of the datasets developed prior
to the advent of the IDE and IQE would fortuitously meet the IDE/IQE criteria. But the IDE and IQE
can be circular; i.e., once developed from a given dataset, there may be a spike level in the dataset that
can be construed to meet the criteria. Datasets developed without following the IDE and IQE
procedures, particularly without making an a priori estimate of IDE0 or IQE0, do not meet the
requirements of the IDE and IQE procedures, regardless of whether the data in them can be construed to
have met those requirements after the fact.
In addition, these datasets do not lend themselves to the comparisons used in this report because the
developers of these datasets did not apply the measurements needed to establish an MDL and ML.
Therefore, MDLs and MLs could not be determined for comparisons (see the section titled "EPA's
Approach to Establishing Detection and Quantitation Limits in Analytical Methods").
B-
-------
The EPA 6000 dataset is comprehensive in coverage of analytes, analytical techniques, and a
concentration range from 0.1 to 100 times the MDL, whereas the datasets suggested by Petitioners focus
on metals, two Aroclors, and concentrations across the analytical range of the method. The range of data
used for construction of an IDE or IQE is particularly important. As detailed in the discussion of the
"Effect of number and spacing of concentrations for determination of the SL-IDE and SL-IQE" below,
including data across the analytical range in calculation of an SL-IDE significantly raises the SL-IDE.
After EPA published the February 2003 Assessment Document for comment, three commenters
offered to provide EPA with additional data that would enhance EPA's assessment. EPA requested the
data offered by each of these organizations, but received a response from only two of the three (an
analytical laboratory and USGS). After evaluating these data, EPA determined that the data from the
analytical laboratory were not useful because they were limited to calibration data and did not include the
data from extraction that is needed to compare detection/quantitation approaches.
Michigan Manufacturers Associatio n (MMA) Dataset
In March of 2002, John Phillips of Ford Motor Company submitted a report of results from a study of
two Aroclors (PCBs) by the Michigan Manufacturers Association (MMA) for EPA's consideration in
evaluating detection and quantitation limit approaches. EPA did not use this dataset because of problems,
such as the dataset was limited to a maximum number of five analytical results per spike level, which is
inconsistent with the minimum number of seven analytical results per spike level required for determining
an MDL, and other values that are determined using non-ASTM approaches. In comments on EPA's
evaluation, Hunton and Williams (a law firm representing the Inter-industry Analytical Group), stated that
EPA should not have excluded the MMA dataset from its assessment of detection and quantitation
approaches. EPA notes, however, that because of the insufficient number of analytical results,
comparison of various detection and quantitation approaches is not possible with this dataset, and has not
included the dataset in this evaluation. In addition, MMA samples spiked with low levels of PCBs as
Aroclors produced an average recovery on the order of 500% at the lowest spike concentration whereas
PCBs are recovered at approximately 80% from water in this concentration range (seethe recovery data
in EPA Methods 608 and 1668A). A logical explanation for the 500% recoveries in the MMA study is
that the samples were contaminated by the sample preparation laboratory, by many of the participant
laboratories, or both. A single and simple test, which was not conducted in the MMA study, of an aliquot
of the prepared water samples using a method, such as EPA Method 1668A, would have demonstrated
that the samples were free from contamination and contained the stated spike concentrations at the time
that the samples were prepared
COMPUTATIONS
All computations were carried out using Statistical Analysis System (SAS) version 8.01. The
equations for all approaches were programmed into the SAS software by a senior statistician, with
assistance from senior analysts. There is some ambiguity in the IUPAC/ISO and ASTM detection and
quantitation limit approaches and in interpretation of results from the ASTM approaches. Several
formulas are given in the IUP AC/ISO documentation, but none are defined to be the official ISO/IUPAC
detection and quantitation limit approaches. Therefore, calculations for the CRV, MDV, and LOQ were
chosen because they were most representative of Lloyd Currie's definitions of a critical value, detection
B-9
-------
limit and quantitation limit. Ambiguity in results from the ASTM approaches is attributable to the
subjective nature of interpreting residual plots for each analyte. To resolve this issue, IDE and IQE
models were chosen using significance tests for slope and curvature.
References used for the IUPAC/ISO approaches were those published by Currie in Pure and
Applied Chemistry 67:10, 1699-1723 (1995) as updated by Analytica Chimica Acta 391 105-126 (1999).
Where needed, the ASTM approaches were programmed as single-laboratory variants of the Practices D
6091 (IDE) and D 6512 (IQE). EPA has included the SAS program code on the CD-ROM that supports
this document.
To assess stakeholder comments about calculations of the IDE and IQE that were performed and
summarized in the original assessment document, EPA requested additional software packages offered by
commenters who use the software to determine these limits. On April 20, 2004, EPA received copies of
two software packages written for the purpose of determining the IDE and IQE from a representative of
Ford Motor Company. The first of these is Qcalc (version 1.0), a DOS-based program. The second of
these is an Excel spreadsheet which utilizes Excel functions, macros and an add-in function to determine
IDEs and IQEs. These two programs were compared to the SAS programs used by EPA by calculating
IDEs and IQEs based on a subset of the Episode 6000 dataset. The results of this comparison are
described later in this Appendix (see section titled "Comparison of IDE/IQEs Calculated Using Different
Software Packages").
Calculation of the ACIL CRV, USGS LTMDL, and EPA MDL was done using analytical results of
blank and spiked samples submitted by USGS. Specific details of these calculations are described in the
section titled "USGS Blank and Spiked Metals and Nutrient Data" later in this Appendix.
RESULTS OF COMPUTATIONS
Detection and quantitation limits are presented in a set of tables corresponding to the Episode 6000
study, a single table corresponding to the Method 1631 and Method 1638 studies, and a single table
summarizing limits calculated using data submitted by USGS. Within the Episode 6000 dataset, results for
detection limits are compared followed by results for quantitation limits. Within the comparison of limits
(detection or quantitation), the first table compares the actual limits followed by a table of percent
differences between limits.
Episode 6000 data
Table 2 compares detection limits produced by four approaches (EPA/ACS DL; ISO/IUPAC CRV;
ISO/IUPAC MDV; and ASTM SL-IDE) and Table 3 presents the percent difference between these
approaches, using the formula given below:
(Lim-DL}
% difference = — —*10 0%
(Lim + DL)!2
where: DL is the EPA/ACS DL for the given analyte, and
Lim is the corresponding limit (CR V, MD V, or SL-IDE) being compared to the DL.
B- 10
-------
The median percent difference between the EPA/ACS DL and each of the other three limits was
compared to 0% using two significance tests: the sign test and Wilcoxon rank-sum test. The sign test
evaluates whether the given limit exceeds the EPA/ACS DL 50% of the time. The Wilcoxon rank-sum
test is a more powerful test which, unlike the sign test, takes into account the magnitude of the difference
between the two limits by ranking the percentage differences presented in Table 3.
The ISO/IUPAC CRV was less than the corresponding EPA/ACS DL for 97% of the analytes and
methods, with a median percent difference of -35.7%. The median percent difference of ISO/IUPAC
CRV to EPA/ACS DL was significantly less than 0% based on both the sign and Wilcoxon tests with a =
0.05 (p<0.0001 for both tests). The major reason for this difference is most likely the different Type I
error rate for the two approaches (a = 0.01 for the EPA/ACS DL and a = 0.05 for the ISO/IUPAC
CRV).
The median percent difference between the ISO/IUPAC MDV and the EPA/ACS DL is 8.8% with
the MDV exceeding the DL for 53% of the analytes. The median percent difference between the
ISO/IUPAC MDV and EPA/ACS DL did not differ significantly from 0% based on the sign test
(p=0.523) orthe Wilcoxon rank-sum test (p=0.164) with a = 0.05. The likelyreason that the two
approaches do not yield significantly different results is that the correction for false negatives and
recovery correction in the MDV (p = 0.05) are counteracted by the smaller Type I error rate for the
EPA/ACS DL.
The median percent difference between the ASTM SL-IDE and the EPA/ACS DL is 108.7%; i.e.,
the single-laboratory variant of the ASTM IDE is, on average, three times as large as that of the EPA and
ACS approaches. The SL-IDE was greater than the corresponding EPA/ACS DL for 92% of the
analytes and methods. The median ratio differed significantly from 1, based on both the sign and
Wilcoxon tests with a = 0.05 (p<0.0001 for both tests). The median ratio and percent of SL-IDEs
exceeding the corresponding EPA/ACS DL both increased slightly compared to the calculations
presented in the original assessment document, due to the correction of the exponential model calculations
for the SL-IDE and the use of the standard deviation bias correction. It is not surprising that the SL-IDE
results were generally greater than the EPA/ACS DL, as the SL-IDE is an estimate of Currie's LD,
whereas the EPA/ACS DL is an estimate of Currie's Lc. In addition, the use of two tolerance interval
limits in the IDE calculation likely also led to the large difference between the SL-IDE and EPA/ACS
DLs.
Table 4 compares quantitation limits produced by the three approaches (EPA/ACS QL; ISO LOQ;
and ASTM SL-IQE) and Table 5 compares the percent difference between these approaches taking the
EPA/ACS QL as reference. Similarly to the detection limit approaches, the median percent difference
was compared to 0% using the sign and Wilcoxon tests. The median percent difference between the
ISO/IUPAC LOQ and the EPA/ACS QL is -4.2%, and the median percent difference between the
ASTM SL-IQE and the EPA/ACS QL is 19.6%. The ISO LOQ and ASTM SL-IQE are greater than
the corresponding EPA/ACS QL for 47% and 62% of the analytes and methods, respectively. The
median ratio between the LOQ and QL did not differ significantly from 0% based on the sign test
(p=0.390), but did based on the Wilcoxon test (p=0.043) at a=0.05. The median ratio between the SL-
IQE and QL differed significantly from 0% based on both the sign test (p=0.001) and the Wilcoxon test
(pO.OOOl).
B- 11
-------
For the SL-IQE comparisons, this result is different from those presented in the original assessment
document, due to the fixed rounding issue in the ML calculations (see discussion under Detection and
Quantitation Limits Assessed). Because the EPA/ACS QL and the SL-IQE are both estimates of
Currie's LQ, the reason for this difference is not clear. One possible reason for this significant difference
is that the SL-IQE does not assume that variability at the quantitation limit is equal to variability of the
blank, whereas the EPA/ACS QL does. However, it is worth noting that the difference seems to be
strongly affected by which model was used to calculate the SL-IQE. The median percent difference
between the QL and SL-IQE is -7.7% when the hybrid model is used to calculate the SL-IQE compared
to 67.9% and 179.6% for the linear and constant models, respectively. While use of the constant model
assumes that the variability is constant between the blank and quantitation limit, this model type is
generally chosen only when there are unusually high results at one or more of the lower spike levels for a
given analyte. Therefore, the SL-IQEs calculated for these analytes are likely somewhat biased high.
Although the Episode 6000 dataset is not ideal for evaluating the ACIL Case 2 iterative approach for
those methods/instruments for which analysis of blank samples does not produce a signal, EPA estimated
the ACIL Case 2 CRV using the lowest concentration level at which all 7 replicates were observed to
test if the conceptual problem with ACIL's implementation of Case 2 occurs in practice. EPA noticed
that, because the estimate of Currie's Lc is based on measured values and the estimate of Currie LD is
based on spike level, the estimate of Lj, could theoretically fall below Lc for methods with recovery that
systematically exceeds 100% or for data with some contamination. Looking at Episode 6000 data, EPA
confirmed that this problem may occur in practice. In fact, it occurred for 35 of the 146 analytes (24%)
measured using methods that do not always result in signals from analysis of blank samples.
EPA/EPRI Method 1631 and 1638 Interlaboratory Method Validation Study Data
Table 6 compares detection and quantitation limits computed from data generated in the Method 1631
and Method 1638 interlaboratory studies. MDLsand MLs are those listed in EPA Methods 1631 and
1638. EPA computed IDEs and IQEs for the purpose of preparing this assessment. IDEs and IQEs
computed by EPRI are from the EPRI reports on the Method 1631 and Method 1638 interlaboratory
studies.
In reviewing these data, it must be recognized that the EPA MDLs and MLs are the result of
selecting the highest MDL in EPA's single-laboratory studies or interlaboratory study, whereas the IDEs
and IQEs are the result of a statistical process that includes recovery correction, correction for bias in the
sample standard deviation (IQE only), allowance for prediction and tolerance intervals, interlaboratory
variability, and model selection. The most significant reason for the instances of a large disparity between
the EPA-determined IDEs/IQEs and the EPRI-determined IDEs/IQEs is model selection. EPA selected
the model based on a strict application of the IDE and IQE procedures by a senior statistician. For those
instances in which EPA and EPRI selected the same model, the IDEs and IQEs are nearly the same.
Table 7 compares IDEs and IQEs resulting from the four main model types described in the ASTM
IDE and IQE procedures. IDEs and IQEs resulting from the constant model were the highest for all
analytes. IDEs and IQEs resulting from the other three models were almost equal for some analytes
(lead, for example), and differed by more than an order of magnitude for others (mercury, for example).
For two analytes, the IDE and IQE estimated using the linear model were negative. This was due to a
negative intercept estimate in the precision model The ASTM IDE and IQE procedures dictate that the
linear model should not be used in this situation.
B- 12
-------
Table 7 also includes RSDs between the IDEs and IQEs calculated using the different model types.
This was done based on commenter statements that the choice of model had only a minimal effect on the
resulting IDE or IQE. This analysis is discussed later in this Appendix (see "Comparison of IDE and
IQEs calculated using Different Models").
USGS Blank and Spiked Metals and Nutrient Data
USGS supplied EPA with blank data collected over a period of one year for 78 metals and nutrient
analytes and spiked data collected over a period of one year for 39 metals and nutrient analytes. These
results were used to calculate both the USGS LT-MDL and ACIL critical level. The ACIL critical level
was calculated using the blank results for the given analyte and method. The USGS LT-MDL was
calculated based on the spike results for the given analyte and method. In addition, the LT-MDL was
calculated in two ways: by adding the mean of the blank results for the given analyte and method, and by
adding the median of the blank results for the given analyte and method.
The EPA MDL also was calculated for each analyte/method using the spiked sample results provided
by USGS. Because MDLs are typically calculated using fewer replicates than the 15 to 24 analyzed by
USGS, EPA calculated the MDL by simulating different subsets of 7 replicates. Subsets were created by
taking each set of 7 consecutive spiked results, i.e., the first 7 samples analyzed would be one subset, the
2nd through 8th samples analyzed would be another subset, etc. This yielded a total of n-6 subsets,
where n is the number of total samples for that analyte. The MDL was then determined by randomly
choosing one of the n-6 subset MDLs. While the use of only seven replicates run consecutively in each
subset minimized the effect of long-term variability, it is worth noting that the amount of temporal
variability in each subset is still greater than that typically included in the EPA MDL (i.e., MDL datasets
typically are generated in a single day); the time interval between the first and last replicate analyzed
within a subset ranged from 30 to 48 days. Therefore, the calculated MDLs are likely somewhat higher
than those that would be calculated using results generated over a single day.
After calculation of these limits, the percentage of blank results included in the dataset that exceed
each limit for each analyte was calculated. Because all limits were calculated at the 99% confidence
level, it would be expected that the average percent of blanks exceeding each limit would be
approximately 1% when the blank results follow a Normal distribution centered at 0. Limits based on
each of the calculations are presented in Table 10.
Generally, the percentage of blanks exceeding the ACIL critical level was lower than the percentage
exceeding the other limits (see summary table following Table 10). The percentages of blanks exceeding
the EPA MDL were slightly higher compared to the percentages exceeding the ACIL critical level, due to
a small subset of analytes with notable blank bias. The USGS LT-MDL had higher rates of blank
exceedance than either the ACIL or EPA limits, regardless of whether the mean or median was added to
the limit. This suggests that the effect of blank bias was smaller than the effect of the method of
estimating variability. USGS uses the nonparametric calculation to lessen the effect of outliers on the
estimate of variability. Because those blanks that exceed a given limit are likely to be outliers themselves,
this can lead to inflated exceedance rates. However, it is worth noting that, for the majority of analytes
where blanks exceeded the calculated USGS limits, multiple blank results were greater than the
associated limit. This suggests that some non-outlying blank results also are exceeding the USGS limits
for some analytes.
B- 13
-------
DISCUSSION
Negative detection limits for the ISO/IDPAC MDV
The calculated ISO/IUPAC MDV was negative for 29 analytes in the Episode 6000 data. Negative
MDVs are attributable to the use of a regression model to estimate recovery at each concentration. The
standard errors and correlation of the regression parameters are included in the calculation of the MDV.
Analytes for which the MDV was negative seemed to coincide with an unusually large standard error of
the regression intercept, which generally occurred when the estimated intercept was strongly negative.
The large standard error of the intercept was likely due to extrapolating the recovery model to zero
concentration; the error around a regression line is greatest for concentrations furthest away from the
mean spike level. The effect of this extrapolation also may be seen in the Episode 6000 data. No
negative results were used in the MDV and LOQ calculations, yet the median recovery intercept for the
analytes analyzed in the Episode 6000 dataset was equal to -0.11. The standard errors of the intercept
and slope estimates were generally high (interceptmedian^ 0.27, slope median=0.011), and therefore the
estimated intercept and slope terms were frequently not significantly different from 0 and 1, respectively
(intercept: not different from zero for 167 analytes/methods; slope not significantly different from 1 for
106 analytes; both intercept and slope not significantly different for 79 analytes). Because the recovery
model parameters are not significantly different from 0 or 1 for the majority of analytes, and both the
estimated slope and the standard errors of the slope and intercept are included in the calculation of the
MDV and LOQ, the inclusion of the recovery model estimates may bias the calculated limits, to the point
that the resulting MDV can be negative.
Effect of number and spacing of concentrations for determination of the SL-IDE and SL-
IQE
Tests in the Episode 6000 studies were conducted at 16 concentration levels. The IDE procedure
suggests using at least 5 concentration levels. Based on statistical theory we would expect the number
and spacing of concenfration levels to affect the outcome, with a larger number of concentrations
producing a more reliable estimate. EPA used the Episode 6000 dataset to test this hypothesis.
The IDE procedure suggests spike concentrations at 0.5, 1.0, 2, 4, and 8 times an initial estimate of
the IDE (IDE0). IDE0 is estimated at 10 times the standard deviation of analytical results of blanks or
replicates of the lowest level that can be measured. EPA's Episode 6000 database contain results of
analysis of at least 7 replicates at each of at least 16 concentration levels from 0.1 to 100 times the initial
estimate of the MDL (a factor of 1000). Between 0.1 and 10 times the MDL, the data are spaced a
factor of approximately 1.5 apart. Above 10 times the MDL, the data are spaced at 10,20, 50 and 100
times the MDL. The reason for the narrow spacing between 0.1 to 10 times the MDL was to attempt to
allow more precise characterization of variability in the region of the MDL.
The SL-IDEs and SL-IQEs in Tables 2 and 4, respectively, were computed and reported using all 16
concentration levels because data were available at all of these levels. However, to determine the effect
of the IDE procedure, a separate data analysis was performed. In this separate analysis, concentration
levels were limited to a total of 5, and the 5 levels were selected to be as consistent as possible with the
levels specified in the IDE procedure; i.e., at 5, 10, 20, 40, and 80 times the standard deviation of
replicate measurements of a blank or the lowest level at which measurements could be made. The
statement "lowest level at which measurements can be made" was interpreted to mean inclusion or
B- 14
-------
exclusion of results containing zeros and/or negative numbers. For purposes of this evaluation,
concentrations that produced results containing a zero or negative number were excluded; i.e., the lowest
concentration that contained no zeros or negative numbers was chosen as the concentration at which the
standard deviation would be calculated for the purpose of estimating IDE0 and IQE0. Zeros and negative
numbers were used in all of the other steps in calculating SL-IDEs and SL-IQEs.
The SL-IDE was calculated after selecting the levels based on IDE0, and the results were compared
to results produced when all 16 levels were included in calculating the SL-IDE. Results are summarized
in Table 8. This table shows that the median percent difference between the 6-point IDE and the 16-point
IDE is approximately -24.9% (where negative percent differences indicate that the 5-point IDE is less
than the 16-point IDE). For those instances in which the same model was chosen (108 out of 198), the
median percent difference was -35.6%, which was significantly different from 0%based on both the
Wilcoxon rank-sum test and the sign test (p < 0.0001 for both tests). For those instances in which a
different model was chosen (90 out of 198), the median percent difference was 1.3%, which was not
significantly different from 0% based on either test (Wilcoxon: p=0.85; sign test: pX).99). Because the
choice of model can have a confounding effect on any differences between 16-point and 5-point SL-
IDEs, the focus should be on the instances in which the same model was chosen. For these instances, the
results indicate that only data in the region of detection and quantitation should be used to establish a
detection or quantitation limit.
A similar comparison was performed between SL-IQEs (10%) calculated using all concentration
levels to SL-IQEs (10%) calculated using only 5 concentration levels. Results of this comparisons are
summarized in Table 9. While differences between the two calculations were not significant based on
either the sign test (p=0.567) or the Wilcoxon test (p=0.345), the differences were larger than those
between SL-IDEs, as seen by the larger median percent difference of -194.6%. Unlike the IDE
comparison, a different model was used to calculate the 5-point SL-IQE than was used to calculate the
16-point SL-IQE for most analytes. For these 145 analytes, the percent differences were large (median
percent difference = 613.9%) but not systematically positive or negative (sign test: p=0.507, Wilcoxon:
p=0.606). For the 50 analytes for which the same model was used to calculate the 5-point and 16-point
SL-IQEs, the percent differences were strongly negative (median percent difference = -2,442.7%) and
significantly less than 0 (sign test: p=0.015, Wilcoxon: p=0.0007).
The reason for the use of 5 versus 16 concentration levels yielded significantly different results for the
SL-IDE, but not for the SL-IQE, was likely due to the different model types that are recommended in the
ASTM IDE and IQE procedures. Systematic differences in the calculated limit appear to occur when the
same model type is applied to the 5-point and 16-point datasets. Because the exponential model is chosen
based on the significance tests for most analytes in the IDE procedure, the model type used rarely differs
between the two sets. There was less consistency in selecting models in the IQE procedure, and the
choice of model differed between the 5-point and 16-point SL-IQE for approximately 75% of the
analytes. Some of these differences, such as using the constant model instead of the hybrid model for the
5-point SL-IQE calculation, appeared to result in higher SL-IQEs, while others, such as using the linear of
hybrid model in place of the constant model for the 5-point calculation, appeared to yield lower SL-IQEs.
Therefore, while differences in the selected model resulted in large percent differences, these differences
were not consistently positive or negative.
B- 15
-------
Relative Standard Deviation at the ML and SL-IQE in the Episode 6000 Study
The minimum level of quantitation (ML) is directed at the level at which 10% relative standard
deviation (RSD) is attained. However, because the ML is not established at exactly 10% RSD, but is
determined by multiplying the standard deviation that is obtained in determination of an MDL by 10 (as
recommended by both ACS and Currie for ACS and ISO/IUPAC LOQs), the resulting RSD may not be
10%. The Episode 6000 data provided the opportunity to determine the actual value of the RSD at the
ML. For analytes that did not have a spike concentration at the ML, the RSD was determined by linear
interpolation between spike levels. Results of the determination showed that the overall median RSD at
the ML across all analytes in the Episode 6000 study was 9%, and the median RSD for the 10 analytical
techniques ranged between 4 and 16 percent. For 29 analytes, no RSD could be calculated because
signals were not generated for samples spiked at the ML. This was likely due to limitations with this
dataset that are discussed earlier in this Appendix (see "EPA's Variability versus Concentration Studies").
For 114 of the 169 remaining analytes, the RSD fell between 5% and 15%. Among the analytes that fell
outside this range, 28 had RSDs below 5% and 27 had RSDs greater than 15%.
Because IQEs target a specified RSD, RSDs were also calculated based on the SL-IQEs calculated
for the Episode 6000 data Unlike the ML, the SL-IQE procedure does not contain a rounding step and,
therefore, the calculated value never corresponded to one of the spike levels used in the study. For this
reason, interpolation was required to calculate RSDs at the given SL-IQE value. The overall median
RSD based on the SL-IQEs was 7%, with method-specific median RSDs ranging from 6% to 11%. No
RSD could be calculated for 9 analytes because signals were not generated for samples spiked
immediately above or below the SL-IQE. Similarly to the ML, this was likely due to issues with this
dataset that are discussed earlier in this Appendix.
Effect of Outliers on Detection/Quantitation Calculations
The detection and quantitation limits based on the Episode 6000 dataset presented in Tables 2 through
5 were calculated without removing any outlying results. This decision was made based on several
reasons. There were generally only 7 results per spike level for each analyte, which is a very small
dataset for which to apply outlier tests and removal. In addition, MDL and ML procedures do not include
outlier removal steps and, therefore, removing outliers for any of the other procedures would hinder
comparisons of the calculated limits themselves. However, based on stakeholder comments, an
assessment of the effect of outlier removal procedures on the different detection and quantitation limits
was added to this Appendix.
Table 11 shows MDLs and SL-IDEs calculated after Grubbs outlier test (Grubbs F.E. "Procedures for
Detecting Outlying Observations in Samples," Technometrics, vol. 11 No. 1 1969) was applied to the
data. Grubbs test was run at the 5% significance level, and a maximum of one result per spike level was
removed based on the results of the test. The choice of outlier test and the associated significance level
follows instructions in ASTM-D2777. However, a significance level of 1% is more appropriate for outlier
removal tests, as a small sample size coupled with the significance level of 5% can lead to inappropriate
removal of outliers. This is true especially for studies evaluating multiple concentrations. For example, in
the Episode 6000 study, there were 16 concentrations and 149 of the 198 analytes considered had an
outlier present at one or more concentrations based on application of Grubbs test with 5% significance
level.
B- 16
-------
For each analyte, the percent difference of the SL-IDE or MDL calculated using all data compared to
the SL-IDE or MDL (calculated using the data after outlier removal) was determined. Summary
statistics of these ratios are presented in Table 11. Analytes without outliers are not included in the table
or the analyses discussed in this section.
Generally, SL-IDEs decreased slightly when outliers were removed. This is not surprising, as the
removal of an outlying result decreases the variability at that spike level. The decrease in the SL-IDEs
was not large, however, as the median percent difference comparing SL-IDEs calculated with and
without outlier removal was 14.3%, where a positive percent difference indicates thatthe SL-IDE
calculated without outlier removal was greater than the SL-IDE calculated after outlier removal. For a
few analytes, removing outliers led to a change in the choice of model used to calculate the SL-IDE. In
these cases, the presence of the outliers generally forced the constant model to be used; when outliers
were removed, the exponential model was used. Therefore, the change in the calculated SL-IDE for
these analytes was greater (median percent difference = 114.'
Removal of outliers only changed the MDL results if outlier removal changed the choice of spike
levels used to calculate the MDL, or occurred at one of the spike levels from which the original MDL
was calculated. This occurred for 60 of the 149 analytes for which any outliers were removed. In these
cases, the decrease in the MDL was slightly larger than the change in the SL-IDEs (median percent
difference = 30.1
For a small subset of analytes, either the SL-IDE or MDL increased after outlier removal.
Generally, these increases were very small, and were likely due to increased tolerance factors or
decreased mean recoveries for the SL-IDE, or to increased t-statistics for the MDL.
SL-IQEs and MLs calculated with and without outlier removal are presented in Table 12. The effect
of outlier removal on calculated SL-IQEs and MLs was generally similar to that on the SL-IDEs and
MDLs. For the SL-IQE, the choice of model changed more frequently than for the SL-IDE (31 analytes
compared to 8 for the SL-IDE). However, the median percent difference was almost equal to that for
the SL-IDE (16.3%). The calculated ML changed based on outlier removal for only 31 analytes,
compared to 60 for the MDL. This number was smaller than for the MDL because the ML rounding
frequently overshadowed the effect of outliers. However, for the ML, the changes that did occur were
greater (median percent difference = 66.'
Evaluation of IDE/IQE Procedures
Comparison of IDE and IQEs calculated using Different Models
In the February 2003 Assessment Document, EPA expressed concern about the large amount of
variability between calculated IDEs and IQEs resulting from the four different model types, and the
subjectivity involved in selecting the most appropriate model. One stakeholder commented that this
concern was not valid, and that IDEs calculated using different models were generally very close. To test
this statement, EPA calculated SL-IDEs and SL-IQEs using each of the four major model types, and
calculated RSDs between the different values for each analyte ("cross-model RSDs"). The resulting
SL-IDEs are presented in Table 13. Median RSDs calculated for all analytes are presented at the bottom
of the table. For several analytes, the calculated SL-IDE based on the linear model was negative due to
the negative intercept of the fitted model. Because the ASTM procedure for calculating the IDE states
B- 17
-------
that the linear model should not be used in these instances, the SL-IDE based on the linear model was not
included in these RSD calculations.
There is a large amount of variability between RSDs calculated with these data using the different
models. Generally, SL-IDEs calculated using the constant model were much greater than those
calculated using the other models. The hybrid model yielded the lowest SL-IDEs, excluding cases where
the linear model SL-IDE was negative. The SL-IDEs calculated using the hybrid and exponential models
were quite similar for some analytes, but quite different for others. When examined separately by
method, the variability between models was generally smaller for metals methods than organics methods.
However, there was a large difference in cross-model RSDs between the two metals methods, (i.e., IDEs
across models in Method 1620 had a median RSD of 27%, whereas IDEs across models in Method 200.8
had a median RSD of 88%).
RSDs between SL-IQEs calculated using the different models are included in Table 14. The
variability between the different model estimates was similar to that of the SL-IDEs, with a median RSD
of 136% between SL-IQEs (10%). Method-specific median cross-model RSDs among SL-IQEs (10%)
ranged from 24% for Method 1620 to 166% for Method 524.2.
To assess the effect of interlaboratory variability on the differences between estimates calculated
from different models, cross-model RSDs were determined between the different IDEs and IQEs
calculated based on the interlaboratory validation studies ofMethods 1631 and 1638. These RSDs are
presented in Table 7. Based on these data, the variability between model estimates appears to increase
when the variability between laboratories is included. Cross-model RSDs between the IDEs calculated
from the different model types ranged between 61% and 162%, with a median of 123%. These RSDs
are greater than those calculated using the single-laboratory metals data in Episode 6000. Variability
between IQEs was smaller than the variability between IDEs. Cross-model RSDs between IQEs ranged
between 50% and 190%, with a median of 99%.
Comparison of IDE/IQEs Calculated Using Different Software Packages
A stakeholder commenting on EPA's February 2003 data assessment stated that the Agency's
concerns about the complexity and subjectivity in the IDE and IQE procedures were unimportant due in
part to the availability of software that will automatically perform the IDE and IQE calculations. EPA
obtained two software packages from this stakeholder (see the section titled "Computations") to aid in
responding to this and other comments regarding the calculation of IDEs and IQEs in the February 2003
TSD.
EPA compared these two software programs using a random subset of 20 analytes from the Episode
6000 dataset. To ensure that differences between results were due to the programs themselves, the same
data were used for each program. Table 15 presents a comparison of the IDE and IQE10 (IQE at 10%
RSD) results based on the two software packages, along with limits calculated using SAS programs (the
latter limits match those presented in Tables 2 and 4). In addition, summary statistics of this comparison
are presented in Table 16. Comparisons between IDEs and IQEs calculated using QCalc and the Excel
software could not be done for all models, because QCalc only performs each calculation using two of the
four models (exponential and hybrid for the IDE calculation, and linear and hybrid for the IQE
calculation).
B- 18
-------
Generally, IDEs and IQEs calculated using SAS programs were very close to those determined using
QCalc based on the same model type. The median ratio of the IDE or IQE calculated using SAS
compared to the IDE or IQE calculated using QCalc equaled 0.99 or 1.00 for all model types. For two
analytes (1,1,-dichloroethene and selenium by Method 1620) the hybrid IDEs and IQEs differed greatly
between QCalc and the SAS programs. This appeared to be because the intercept term estimated by
QCalc was negative for these analytes (resulting in negative IDEs and IQEs), whereas the intercept term
estimated by SAS was approximately the positive absolute value of this estimate (resulting in positive
IDEs and IQEs).
IDEs and IQEs calculated using the Excel file were generally comparable to those calculated using
the SAS programs and QCalc for the constant, linear, and exponential models. The differences between
the values calculated using the Excel file and other packages, however, were much greater for the hybrid
model. As seen by the median ratios, the estimated IDEs and IQEs determined based on the hybrid
model using Excel were slightly higher than those determined using SAS, and approximately twice those
determined using QCalc. Part of this difference is due to the negative values calculated by QCalc for two
analytes. However, the calculated values differed greatly, as the resulting IQEs calculated by Excel
using the hybrid model ranged from less than 0 to more than 6 times greater than that calculated using the
SAS programs. These differences seem to be due to how the hybrid model is fit using Excel. The Solver
add-in function used by Excel does not seem to follow the same Newton's Non-Linear Least Squares
algorithm described in the ASTM procedures and followed by EPA's SAS programs and QCalc.
In addition to differences in calculated limits based on the same model type, the different programs
may yield different IDEs or IQEs based on which model type is indicated as most appropriate by a
particular software package. QCalc and the Excel file both automatically suggest the same model type
for the IDE and IQE. However, EPA often used a different model type for calculating the IDE and IQE.
This was done because the ASTM IDE procedure lists constant, linear, and exponential as the three
major model types to be considered, whereas the ASTM IQE procedure lists the constant, linear, and
hybrid as the three major model types. Therefore, while the exponential model was used by EPA to
calculate most IDEs, it was not used to calculate any of the IQEs. Because of this, while EPA and
QCalc selected the same model type to calculate the IDE for only one analyte, the same model type was
selected to calculate the IQE for 17 of the 20 analytes.
The Excel file frequently chose a different model type than QCalc and the EPA SAS programs to
calculate the IDE and IQE. The Excel file selected a different model type than QCalc for 14 of the 20
analytes, and selected a different model than EPA's SAS program to calculate the IDE and IQE for 19
and 17 analytes, respectively. The reason for this appears to be that the Excel file suggests that the
appropriate decision be based on which model has the smallest sum of squared residuals. This is different
from the statistical tests of slope and curvature used by QCalc and the SAS programs and also described
in the ASTM procedures. While both QCalc and the Excel file also include graphs to aid in model
selection, and could potentially yield more consistent model selection through these graphs, it is likely that
many users would prefer the clearer answer provided by statistical tests or comparisons of sums of
squared residuals.
Based on these differences in selecting and fitting models, it does not appear that the two available
software programs remove all complexity and subjectivity from the calculation of IDEs and IQEs.
Instead, they appear to introduce new issues by using steps not included in the ASTM procedures. While
QCalc appears to follow the ASTM procedures more closely than the Excel file, it does not perform
B- 19
-------
calculations for all model types and, therefore, may introduce greater subjectivity by only providing
calculated limits based on inappropriate models.
Effect Of Long-Term Variability
Several stakeholders commenting on EPA's assessment expressed concern about the lack of
long-term variability included in the MDL procedure. Commenters state that the lack of long-term
variability leads to underestimates of Currie's critical value (Lc). In addition, ACIL included datasets
containing results of blank samples analyzed over three months for 5 analytes to show this effect. These
commenters pointed to the ACIL procedures for calculating the critical level (CRV) and long-term MDL
(ACIL LT-MDL) and the USGS procedure for calculating the long-term MDL (LT-MDL), which include
the collection of blanks over a long period of time.
EPA assessed the effect of long-term variability on calculated limits by simulating multiple 7-replicate
subsets from the full dataset, and comparing these short-term CRVs to the CRV calculated using the full
dataset. Subsets were created by taking each set of 7 consecutive blanks, (i.e., the first 7 blanks analyzed
would be one subset, the 2nd through 8th blanks analyzed would be another subset, etc.). This yielded a
total of n-6 subsets, where n is the number of total blanks for that analyte. Because a blank will be used
in as many as 7 subsets, the variability of the short-term CRVs was lower than what would be expected;
however, the approach was used to yield the greatest number of simulated subsets. The CRV was then
calculated for each subset:
where CRV;, X . , and s; are the critical value, the mean, and the standard deviation for the ith simulated
subset of blank results, respectively. The overall CRV was calculated using the same formula, using the
mean and standard deviation based on all blank results and a lower t-statistic based on the greater number
of blank replicates. Table 17 shows the results of the comparison of calculated short-term and long-term
CRVs for the five analytes.
While the range of days from which sets of 7 replicates were simulated varied from between one
week to greater than 3 weeks, graphical analyses did not show any effect of the number of days on the
resulting CRV. The total number of blanks also did not seem to have an effect on the percentage of
short-term CRVs that exceeded the overall CRV. The mean short-term CRV was generally very close
to the overall CRV for each analyte. However, for three of the five analytes, the majority of the
short-term CRVs exceeded were lower than the overall CRV, indicating that long-term variability did
have an effect on the resulting limit. For the other analytes, the effect of any added variability was
counteracted by the smaller t-statistic used in the calculation. These t-statistics ranged between 2.4 and
2.5 between analytes, well below the 3.14 used when only 7 replicates are available.
One possible reason for the number of short-term CRVs falling below the overall CRV was the
presence of outliers. The ACIL procedure permits the use of an outlier procedure to remove outlying
high or low blanks. EPA used Grubbs test and identified 3 blank results for silver, and 1 blank result each
for barium and chromium, as outliers. After removal of these results, the overall and short-term CRVs
were re-calculated for these 3 analytes. The results of these calculations are given in Table 18.
B-20
-------
Because an outlying result is used in the calculation of the overall CRV (but only for a maximum of 7
of the short-term CRVs), the effect of outlier removal was greater for the overall CRV than on the
short-term CRVs. For all 3 analytes, the majority of the short-term CRVs were above the overall CRV,
and the mean short-term CRV was slightly higher than the overall CRV. This was consistent with the
results of cadmium and copper shown in Table 17, for which no outliers were detected. Because no
information was available about why these results could have been outlying, it is not known if they were
the result of a known error, or were in fact the result of the long-term variability included in the study.
However, it appears that the effect of long-term variability is generally not large when compared to the
effect of using more replicates on the t-statistic multiplier.
As stated in Section 3.3.3, a greater number of replicates will yield improved estimates of standard
deviation and, therefore, better estimates of Currie's Lc. Based on this, although, EPA does not feel
estimations of Lc based on 7 replicates are biased low, these estimates may be less precise than those
based on greater replicates. The large variability of the 7-replicate CRVs can be seen in the large ranges
of short-term CRVs calculated with and without outlier removal. The use of the higher t-statistic also
seems to counteract the added long-term variability. The ACIL procedure suggests 7-replicate CRVs are
underestimates, and should therefore be multiplied by a factor of 2. The short-term CRV calculated in the
ACIL procedure is based on blanks analyzed in a single batch and, therefore, are not comparable to the
short-term CRVs simulated by EPA. However, such a multiplier is not necessary in calculating the MDL,
even if long-term variability is not included in the analyses.
SUMMARY
Public comment on the February 2003 Assessment Document and the proposed regulatory revisions
expressed many divergent views about the merits and usefulness of EPA's 2003 assessment and
proposed regulatory revisions. We recognize that there is a broad interest in improving current
procedures and uses, but no consensus for a specific procedure or procedures has emerged among the
laboratory, industry, regulatory or regulated communities. Thus, we have withdrawn the March 2003
proposed revisions and, to meet the terms of the settlement agreement that is described in chapter 1, are
taking final action on the 2003 Assessment Document in this Revised Assessment Document. This is not
the end of our efforts to work together, as stakeholders have suggested, to discuss mutual concerns and
possible solutions. We look forward to continued stakeholder participation in an ongoing dialog about the
development and use of detection and quantitation limits in CWA programs.
In this appendix, we have compared detection and quantitation limits computed from data gathered by
EPA or submitted to EPA. This comparison shows that, in general, detection limits derived from a single
concentration level such as EPA's MDL are, on average, approximately the same as detection limits
derived from similar approaches such as the ACS LOD and LOQ and ISCMUPAC CRV and MDV, and
are approximately three times lower than a single-laboratory variant of ASTM's IDE; and that all
quantitation limit approaches, such as EPA's ML, the ACS and ISO/IUPAC LOQ, and a single-laboratory
variant of ASTM's IQE, produce quantitation limits that are generally only slightly different. In addition,
the following are general statements about the datasets and/or analyses described in this appendix.
1. Variability of Results
Comparisons of detection and quantitation limits show high variability among the limits calculated
using the different approaches, even with data containing 7 replicates at 16 concentration levels (see
B-21
-------
the summary statistics at the end of Tables 3, 5, and 7). The net effect is that the systematic
differences among detection and quantitation limits produced by the various approaches are
overwhelmed by variability, i.e., there is a small systematic difference among the approaches but
great variability in the detection and quantitation limits fora given analyte. This result is not surprising
given the variability of data in the region of detection and quantitation. However, it is difficult to
postulate a solution to the problem. Gathering more data in the region of detection and quantitation
would appear to be a solution, but 91 data points were gathered for each analyte in the region
between 0.1 and 10 times the MDL in the Episode 6000 studies, and it is unlikely that any organization
could afford to gather even this amount of data for determination of a detection limit. Given the high
degree of variability of the data, EPA's approach of conducting a single-laboratory study to gain a
first estimate, followed by multiple single-laboratory studies to verify or revise the estimate, and an
interlaboratory study, where warranted, to further verify and revise the estimate, is a reasonable
means of establishing detection and quantitation limits because of the checks and balances that occur
at each step.
2. Regression Analysis
Using a regression line to estimate a recovery correction at zero concentration causes great swings in
the resulting detection and quantitation limits such as thelSO/IUPAC MDV and LOQ. The
estimated regression parameters for the recovery models were often not significant, and the inclusion
of the estimated slope and the standard errors of the slope and intercept will, therefore, unnecessarily
bias the calculated MDV and LOQ, such that the calculated MDVs may be negative (see Discussion
section "Negative detection limits for the ISO/IUPAC MDV, and Table 2 for instances of negative
detection limits"). The estimated recovery model used in calculating the IDE and IQE is also strongly
affected by the chosen model of variability vs. concentration (see Tables 13 and 14). Even though a
linear regression is used to model recovery in each case, the weights used in the model are calculated
based on the variability model, and can vary greatly when the number of concentrations used is low.
For the Episode 6000 data, the median RSD of the recovery slopes from the four different models
used in the IDE calculations for a given analyte and method was 5%. In addition, for 77 of the
analytes and methods (39%), at least one estimated recovery slope was greater than 1, and at least
one was less than 1. This suggests that the method could be considered to be high biased (and the
final IDE and IQE would be decreased by the recovery correction) and low biased (and the final IDE
and IQE increased) for the analyte, depending on the chosen precision model. For many analytes the
slopes were not significantly different from 1, suggesting that a recovery correction may not be
appropriate at all. This is in addition to the philosophical issue as to whether recovery correction is
warranted. If there is to be a correction for recovery, it maybe better to use some average or
median value than a regression, or use a measured value near the region of interest.
3. IDE and IQE
Additional development of the ASTM IDE and IQE is needed before they can be used routinely, not
only because of the complexity of the procedures, but also because of the ambiguity in determining
that the correct model has been selected. While different software packages are available that
perform most of the calculations, there are many inconsistencies between these programs, and
between the programs and the ASTM procedures, that add another area of subjectivity to the
determination of IDE and IQEs. (For the consequences of model selection, compare the IDEs and
IQEs determined by EPA and EPRI in Table 6, and the IDEs and IQEs calculated from the different
B-22
-------
model types in Table 7. Some differ considerably as a result of model selection in application of the
IDE and IQE procedures by different statisticians. In addition, the use of different software may lead
to the selection of different models, as seen in Table 15.)
4. Quantitation Limit Appro aches
Quantitation limit approaches such as EPA's ML and the ACS and ISO/IUPAC LOQ that are
directed 10% RSD actually produce RSDs that are in the range of the 10% intended (see the
discussion in the Section titled "RSD at the ML in the Episode 6000 Study"). The median RSDs for
each method in the Episode 6000 dataset ranged from 6% to 16%, and 58% of the individual analyte
RSDs fell between 5% and 15%.
Commenters on our February 2003 Assessment Document suggested that procedures submitted by a
laboratory association (ACIL) and the U.S. Geological Survey as alternatives to the MDL and ML should
be considered. We agree, have evaluated these procedures in this Revised Assessment Document, and
believe they provide a starting point for continued stakeholder discussions.
Regarding these two procedures, we note the ACIL CRV generally yielded lower false positive rates
than the USGS LT-MDL. This likely was due to the nonparametric estimate of variability used in the
USGS procedure. False positive rates for the EPA MDL, which uses a parametric variability but does
not include the mean blank result, were lower than the USGS LT-MDL, which does include the mean
blank result. The ACIL procedure states that calculated CRVs are based on fewer replicates and/or
short-term variability are biased low, and includes optional alternate calculations to use in these situations.
However, comparison of CRVs calculated with full set of long-term blanks to those calculated with
subsets of 7 blanks suggest that the absence of long-term variability is counteracted by the larger t-
statistic used when the number of blank results is smaller.
ACIL also included a separate procedure for methods for which analysis of blank samples does not
always produce a signal. The idea of dividing methods into two groups has merit. However, the current
ACIL procedure for these methods often generates estimates of Currie's Lc that are above the estimate
of Currie's LD when contamination is present.
B-23
-------
TABLES
Table 1. Datasets Suggested by Petitioners
Dataset and year
AAMA 1996-7
AAMA 1996-7
AAMA 1996-7
MMA 2000-1
EPA/EPR1 1997-8
EPA/EPR1 1997-8
EPRI1987
EPRI1990
EPRI1994
EPRI1996
Analyte and technology
IVU,a,s by ICP/AES (200.7)
Mercury by CVAA (2452)
PCB, by GC/ECD (608.2)
PCB 1216 and 1260 by GC/ECD
IVUroury by CVAF (1631)
Metal, by ICPA/IS (1638)
M,,a,,byGFAA(EPA200)
Metal, by ICP/AES (EPA 200.7)
Ai, B,,T, byGFAA(EPA200)
Cd, As, CrbyGFAA(EPA200)
B-24
-------
Table 2. Comparison of Detection Limits (H9/L
except where footnoted) for the Episode 6000 Dataset
Analyte
1 ,1 ,1 ,Znetrachloroethane
1 ,1 ,1 ,£"tetrachloroethane
1,1,1 "Irichloroetnane
1,1,1 "Irichloroetnane
1,1,2,2-toe+1,2,3-top
1 ,1 ,£,£-tetrachloroethane
1 ,1 ,£-1richloroe»iane
1 ,1 -dichloroethane
1 ,1 -dichloroethane
1 ,1 "dichloroethene
1 ,1 -dichloroethene
1 , 1 "dichloropropa none
1 , 1 "dichloro pro pene
1 ,£,O-1richlorobenzene
1 ,£,O-1richlorobenzene
1 ,£,O-1richloro benzene
1 ,£,O ~*"i chloropro pa ne
1 ,Z,4~1richlorobenzene
1 ,Z,4~1richlorobenzene
1 ,£,4 -trim ethyl benzene
1 ,Z,4-rimethylbenzene
1 ,Z-dibromo-O-chloropropane
1 ,£-dibromoetiane
1 ,£ ~di brom oetnane
1 , £ ~di chloroben zene
1 , £ ~di chloroben zene
1 ,£~dichlorobenzene
1 ,/_-dichloroethane
1 ,/_-dichloroethane
Method
502.2
524.2
502.2
524.2
502.2
524.2
502 2
524.2
502.2
524.2
502.2
524.2
524.2
524.2
502.2
502.2
524.2
524.2
502 2
502.2
524.2
502.2
524.2
524.2
502.2
524.2
502.2
502.2
524.2
502.2
524.2
Procedure
ELCD
ELCD
ELCD
ELCD
ELCD
ELCD
ELCD
PID
ELCD
PID
PID
ELCD
ELCD
PID
ELCD
EPA/
ACSDL
0.041
0.052
0.012
0.055
0.064
0.132
0 024
0.075
0.010
0.033
0.038
0.054
5.184
0.045
0.048
0.057
0.070
7.328
0 022
0.070
0.053
0.095
0.012
1.457
0.096
0.127
0.035
0.033
0.030
0.017
0.039
ISO
CRV
0.005
0.039
0.009
0.021
0.047
0.131
0 004
0.043
0.007
0.020
0.030
0.035
3.146
0.012
0.034
0.042
0.040
0.046
0 014
0.038
0.050
0.053
0.009
0.391
0.007
0.117
0.031
0.024
0.023
0.003
0.024
ISO
MDV
0.009
-0.047
0.017
0.003
0.086
0.128
0 006
0.040
0.014
0.016
0.073
-0.037
5.635
-0.030
0.065
0.088
0.031
0.033
0 030
0.080
0.052
0.119
0.017
0.701
0.013
0.170
0.061
0.054
-0.016
0.005
0.013
ASTM
SL-IDE
0.034
0.244
0.041
0.308
0.179
0.436
0 032
0.319
0.083
0.229
0.234
0.335
6.372
0.287
0.134
0.115
0.275
1.263
0 088
0.124
0.224
0.125
0.144
1.749
0.164
0.326
0.065
0.148
0.130
0.042
0.258
B-25
-------
Table 2. Comparison of Detection Limits (H9/L
except where footnoted) for the Episode 6000 Dataset
Analyte
1 r£~di chloro pro pane
1 r£~dichk>ropropane
1 ,3,5-1mb + 4-chlorotoluene
1 ,O,b-rimethylbenzene
1 ,O~dichlorobenzene
1 ,O~dichlorobenzene
1 ,O~dichlorobenzene
1 ,O~dichloropropane
1 ,O~dichloropropane
1 ,4-dichtorobenzene
1 ,4-dichtorobenzene
I "chlorobutane
£r£-dichtoro pro pane
2 b t
£ -chloro toluene
Z -chloro toluene
£~chlorotoluene
Z'hexanone
Z-nitropropane
4 "chloro toluene
4 "chloro toluene
T"isopropyltoluene
4 -methyl"^ -pen tan one
Acetone
Acrylonitrile
Ally! Chloride
Aluminum
Aluminum
Ammonia as Nitrogen
Antimony
Antimony
Arsenic
Method
502.2
524.2
502.2
524.2
502.2
502.2
524.2
502.2
524.2
502.2
524.2
524.2
524.2
524 2
502.2
502.2
524.2
524.2
524.2
502.2
524.2
524.2
524.2
524.2
524.2
524.2
1620
200.8
350.3
1620
200.8
1620
Procedure
ELCD
PID
ELCD
PID
ELCD
ELCD
ELCD
PID
ELCD
EPA/
ACSDL
0.023
0.056
0.067
0.011
0.035
0.093
0.023
0.016
0.038
0.026
0.023
0.020
2.376
0 417
0.108
0.238
0.016
1.316
0.901
0.110
0.010
0.010
0.812
0.859
0.863
0.032
29.555
19.145
0.010
1.552
0.178
1.065
ISO
CRV
0.014
0.030
0.045
0.008
0.005
0.077
0.016
0.008
0.024
0.005
0.017
0.016
0.103
0 297
0.029
0.135
0.009
0.148
0.275
0.027
0.008
0.008
0.480
0.440
0.444
0.026
15.043
1.690
0.007
0.801
0.003
0.917
ISO
MDV
0.029
0.026
0.100
0.008
0.010
0.170
-0.014
0.015
-0.015
0.009
-0.044
0.018
-0.159
0 511
0.056
0.302
0.002
0.231
0.452
0.050
0.007
0.003
0.733
0.804
0.653
0.005
28.666
3.547
0.014
1.754
0.007
1.375
ASTM
SL-IDE
0.043
0.247
0.114
0.135
0.118
0.126
0.143
0.047
0.202
0.061
0.140
0.220
0.691
0 833
0.175
0.230
0.136
0.902
1.082
0.149
0.123
0.117
1.195
2.120
1.333
0.229
206.975
12.747
0.014
4.260
0.019
1.410
B-26
-------
Table 2. Comparison of Detection Limits (H9/L
except where footnoted) for the Episode 6000 Dataset
Analyte
Arsonio
Barium
Barium
Denzene
Denzene
Beryllium
Beryllium
Doron
Dromo benzene
Dromobenzene
Bremob.nMn.
Dromochloromethane
Dromochloromethane
Drom odichlorom etnane
Drom odichlorom etnane
Dromoform
Bromoform
Dromom ethane
Dromom ethane
Cadmium
Cadmium
Calcium
Carbon Disulfide
vsarbon 1 etrachloride
Carbon tet+1 ,1 "dcp
Lfhloroace tonitrile
Lfhloro benzene
Lfhloro benzene
Lfhlorobenzene
Chi oroe thane
Chi oroe thane
Chloroform
Method
200.8
1620
200.8
502.2
524.2
1620
200.8
1620
502.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
1620
200.8
1620
524.2
524.2
502.2
524.2
502.2
502.2
524.2
502.2
524.2
502.2
Procedure
PID
ELCD
PID
ELCD
ELCD
ELCD
ELCD
ELCD
ELCD
PID
ELCD
ELCD
EPA/
ACSDL
0.226
1.702
0.033
0.030
0.014
0.528
0.007
15.387
0.131
0.012
0.044
0.013
0.125
0.004
0.043
0.006
0.123
0.267
0.068
0.127
0.004
36.726
0.027
0.038
0.029
0.919
0.011
0.030
0.025
0.108
0.066
0.043
ISO
CRM
0.137
1.337
0.029
0.029
0.014
0.339
0.004
10.356
0.093
0.009
0.036
0.012
0.113
0.003
0.026
0.003
0.065
0.219
0.055
0.079
0.007
35.822
0.015
0.027
0.008
0.773
0.010
0.025
0.022
0.008
0.041
0.006
ISO
MDV
0.272
1.831
0.061
0.067
0.026
0.408
0.006
17.792
0.186
0.019
-0.060
0.024
0.159
0.005
0.019
0.001
0.031
0.358
0.056
0.134
0.012
72.397
-0.040
-0.040
0.016
1.527
0.022
0.055
0.012
0.009
0.038
0.009
ASTM
SL-IDE
0.366
1.837
0.084
0.079
0.125
0.448
0.024
21.161
0.765
0.050
0.211
0.482
0.345
0.075
0.205
1.513
0.400
7.293
0.280
0.191
0.022
41.358
0.239
0.314
0.072
1.569
0.460
0.064
0.133
2.598
0.395
0.032
B-27
-------
Table 2. Comparison of Detection Limits (H9/L
except where footnoted) for the Episode 6000 Dataset
Analyte
Ch,oro,orm
Chlorom ethane
Chlorom ethane
Chromium
Chromium
Cis-1 ,2-dce+2,2-dcp
Cis-1 ,£~dichloroethene
Cis-1 ,O~dichloropropene
Cis-1 ,O~dichloropropene
Cis~ 1 ,0 "dichl oro pro pene
Cobalt
Cobalt
Copper
Copper
D b m fri
Uibromochloromelhane
D,bromomethan,
Uibromom ethane
n
n
Diethyl Ether
Ethyl Me thacryla te
t thy (benzene
tthyl benzene
M 1
Hardness
Mexachl oro butadiene
Mexachl oro butadiene
Mexachl oro ethane
Mexchl obutadiene + na phtha lene
Iron
I so propyl benzene
I so pro pyl benzene
Method
524.2
502.2
524.2
1620
200.8
502.2
524.2
502.2
502.2
524.2
1620
200.8
1620
200.8
502 2
524.2
502.2
524.2
502 2
524 2
524.2
524.2
502.2
524.2
130.2
502.2
524.2
524.2
502.2
1620
502.2
524.2
Procedure
ELCD
ELCD
ELCD
PID
ELCD
ELCD
ELCD
PID
ELCD
PID
PID
EPA/
ACSDL
0.036
0.070
0.045
0.310
0.073
0.013
0.040
0.007
0.057
0.038
9.820
0.001
6.046
0.037
0 009
0.051
0.007
0.102
0 009
0 083
0.120
0.045
0.021
0.033
0.828
0.043
0.068
0.056
0.649
90.409
0.020
0.011
ISO
CRV
0.027
0.049
0.036
0.254
0.062
0.009
0.033
0.002
0.048
0.024
4.017
0.001
4.990
0.027
0 006
0.031
0.005
0.082
0 003
0 054
0.114
0.031
0.015
0.028
0.554
0.010
0.035
0.049
0.143
270.433
0.015
0.010
ISO
MDV
0.021
0.130
0.065
0.386
0.125
0.016
-0.023
0.004
0.099
-0.004
8.094
-0.067
10.512
0.053
0 011
0.004
0.010
0.112
-0 020
0 037
0.163
0.013
0.035
-0.024
1.152
0.021
-0.031
0.038
0.321
472.249
0.035
0.010
ASTM
SL-IDE
0.225
0.250
0.253
0.496
0.408
0.055
0.234
0.074
0.082
0.173
16.463
0.074
21.189
0.798
0 436
0.287
0.460
0.388
0 240
0 560
0.376
0.273
0.078
0.198
2.258
0.094
0.308
0.288
0.597
373.590
0.060
0.120
B-28
-------
Table 2. Comparison of Detection Limits (H9/L
except where footnoted) for the Episode 6000 Dataset
Analyte
Lead
Lead
M+p Xylene
M+p Xylene
Magnesium
Manganese
Manganese
Mercury
Methacrylon itrile
Methyl Iodide
Methyl 1 erfbutyl Lther
Methyla crylate
Methylene Chloride
Methylene Chloride
Methylm ethacryl ate
Molybdenum
Molybdenum
N- butyl benzene
N- butyl benzene
N ~ pro pyl benzene
IM-propyl benzene
IMa phthalene
Nickel
Nickel
o'xylene
o~xylene+ styrene
r -isoproptol+1 ,4'dcb
r entachloroetiane
Oec~ butyl benzene
Oec-butyl benzene
Oelenium
Oelenium
Method
1620
200.8
502.2
524.2
1620
1620
200.8
200.8
524.2
524.2
524.2
524.2
502.2
524.2
524.2
1620
200.8
502.2
524.2
502.2
524.2
524.2
1620
200.8
524.2
502.2
502.2
524.2
502.2
524.2
1620
200.8
Procedure
PID
ELCD
PID
PID
PID
PID
PID
EPA/
ACSDL
1.647
0.655
0.090
0.013
103.033
6.856
0.031
0.004
0.356
0.025
0.026
0.220
0.128
0.082
0.225
2.455
0.004
0.030
0.016
0.040
0.038
0.048
20.219
0.146
0.018
0.059
0.073
0.553
0.055
0.014
0.849
0.192
ISO
CRV
1.186
0.061
0.012
0.008
88.729
1.081
0.030
0.003
0.228
0.023
0.016
0.202
1.835
0.072
0.085
1.714
0.003
0.023
0.014
0.022
0.026
0.040
13.262
0.058
0.015
0.037
0.056
0.019
0.032
0.011
0.619
0.156
ISO
MDV
1.965
0.120
0.026
0.004
175.316
2.591
0.049
-0.018
0.362
-0.013
-0.033
0.353
4.917
0.093
0.117
3.787
0.000
0.049
0.026
0.049
-0.053
0.044
25.697
0.107
-0.032
0.082
0.123
-0.100
0.075
-0.012
1.493
0.302
ASTM
SL-IDE
2.423
0.204
0.121
0.142
105.998
6.808
0.109
0.027
0.718
0.193
0.225
0.601
2.841
0.314
0.535
3.034
0.271
0.141
0.152
0.092
0.284
0.186
25.560
0.083
0.198
0.116
0.159
0.408
0.081
0.140
1.975
0.416
B-29
-------
Table 2. Comparison of Detection Limits (H9/L
except where footnoted) for the Episode 6000 Dataset
Analyte
Silver
Silver
Sodlum
Otyrene
1 ert~butytoenzene
1 erfbutyfoenzene
1 etrachloroetiene
1 etracriloroetiene
1 etracriloroetiene
Thallium
Thallium
Thorium
Tin
Titanium
1 oluene
1 oluene
T D 1
1 otal r hosphorus
1 otal Ouspended Oolids
1 rans'l ,Z~dichloroethene
1 rans'l ,Z~dichloroethene
1 rans'l ,O~dichloropropene
1 rans" lrO~dichloropropene
1 rans" lrO~dichloropropene
Trans-1 ,4~dichloro-2-butene
I richloroetiene
I richloroetiene
I richloroethene
Tii
Trichloroiuoromethane
Uranium
Vanadium
Vanadium
Method
1620
200.8
1620
524.2
502.2
524.2
502.2
502.2
524.2
1620
200.8
200.8
1620
1620
502.2
524.2
365.2
160.2
502.2
524.2
502.2
502.2
524.2
524.2
502.2
502.2
524.2
502 2
524 2
200.8
1620
200.8
Procedure
PID
ELCD
PID
PID
ELCD
ELCD
PID
ELCD
PID
ELCD
EPA/
ACSDL
4.907
0.004
69.530
0.014
0.029
0.022
0.018
0.062
0.085
0.512
0.000
0.001
3.670
4.777
0.070
0.020
0.006
1.170
0.041
0.038
0.012
0.058
0.051
0.512
0.012
0.027
0.061
0 108
0 087
0.000
7.344
0.555
ISO
CRV
3.588
0.002
49.595
0.011
0.020
0.012
0.014
0.040
0.084
0.651
0.000
0.001
2.019
4.453
0.028
0.006
0.005
0.948
0.041
0.032
0.003
0.045
0.025
0.348
0.001
0.018
0.058
0 249
0 075
0.000
4.207
0.512
ISO
MDV
6.495
0.004
97.649
0.010
0.047
0.023
0.029
0.094
0.047
1.406
0.001
-0.005
3.143
8.050
0.063
-0.004
0.009
1.945
0.090
-0.016
0.005
0.095
-0.007
0.576
0.003
0.042
0.056
0 612
0 038
0.000
8.359
0.994
ASTM
SL-IDE
10.668
0.012
138.768
0.141
0.074
0.186
0.061
0.156
0.469
1.153
0.001
0.001
3.932
5.376
0.064
0.146
0.013
3.005
0.081
0.300
0.098
0.092
0.223
1.250
0.059
0.097
0.332
2 079
0 384
0.000
10.630
0.864
B-30
-------
Table 2. Comparison of Detection Limits (H9/L
except where footnoted) for the Episode 6000 Dataset
Analyte
Vinyl Chloride
Vinyl Chloride
WAD Cyanic,,
Aylene ( 1 o tal)
Y.trlum
Zinc
Zinc
Method
502.2
524.2
1677
524.2
1620
1620
200.8
Procedure
ELCD
EPA/
ACSDL
0.270
0.043
0.572
0.009
1.923
2.597
0.900
ISO
CRV
0.039
0.031
0.169
0.005
1.370
2.301
0.461
ISO
MDV
0.077
-0.007
0.319
0.007
2.518
3.697
0.806
ASTM
SL-IDE
3.672
0.365
0.701
0.128
3.247
4.500
1.598
Ke suits reported as mg/L
Note! LLLfU or rlU in the Procedure column indicates the pho to~ioniza tion detector (rlU) or electrolytic
cond uctivity
or (ELCD) in EPA
502.2
Table 3. Percent Differences of
Detection Limits to the EPA/ACS DL for the Episode 6000 Dataset
Analyte
1,1,1 ,2-tetrachloroethane
1,1,1 ,2-tetrachloroetnane
1,1,2,2-,0,+1,2,3-top
1 ,1 ,£,£-tetrachloroethane
1 ,1 ,£-richloroetiane
1 ,1 ,Z-trichloroetnane
1 ,1 -dichloroethane
1 ,1 -dichloroethane
1 ,1 -dichloroethene
1 ,1 -dichloroethene
1 , 1 "dichloro pro pa none
1 , 1 'dichloropropene
1 r£,O~1richlorobenzene
1 r£,O~1richlorobenzene
1 r£,O~1richlorobenzene
1 r£,O~1richloropropane
Method
502.2
524.2
502 2
524 2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
524.2
524.2
502.2
502.2
524.2
524.2
Procedure
ELCD
ELCD
ELCD
ELCD
ELCD
ELCD
ELCD
PID
ISO CRV/
MDL
-159.2%
-28.9%
-34 4%
-89 4%
-29.7%
-0.6%
-146.2%
-53.2%
-40.1%
-50.5%
-25.4%
-42.8%
-48.9%
-117.1%
-34.9%
-29.4%
-53.5%
-197.5%
ISO MDV/
MDL
-131.0%
-4142.5%
32 1%
-177 7%
29.9%
-3.4%
-116.9%
-60.4%
31.0%
-70.3%
61.8%
-1080.2%
8.3%
-1021.1%
30.2%
42.0%
-76.9%
-198.2%
SL-IDE/
MDL
-20.3%
129.8%
108 8%
139 1%
94.7%
107.0%
27.6%
124.0%
156.8%
150.0%
143.5%
144.1%
20.6%
146.2%
94.9%
67.0%
119.2%
-141.2%
B-31
-------
Table 3. Percent Differences of
Detection Limits to the EPA/ACS DL for the Episode 6000 Dataset
Analyte
1 ,£,4-fichlorobenzene
1 ,Z,4 "trim ethyl benzene
1 ,2, 4 -trim ethyl benzene
1 ,£-dibromo-O-chloro pro pane
1 ,£-dibromoetiane
1 ,£-dibromoetiane
1 ,£-dichlorobenzene
1 r£-dichlorobenzene
1 r£-dichlorobenzene
1 ,£-dichloroethane
1 ,£-dichloroethane
1 r£-dichloropropane
1 r£-dichloropropane
1 ,3,5-1mb + 4-chlorotoluene
1 ,3, 5 -trim ethyl benzene
1 3-d hlo b
'
1 ,0~dichloro benzene
1 ,0~dichloro benzene
1 ,0~dichloro pro pane
1 ,O "dichloro pro pa ne
1 ,4-dichtorobenzene
1 ,4-dichlorobenzene
I "chlorobutane
£r£-dichloropropane
2 b t
£-chlorotoluene
£~Ghloro toluene
Z'chloro toluene
Z'hexanone
Z-nitro pro pane
Method
502.2
502 2
524 2
502.2
524.2
524.2
502.2
524.2
502.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
502 2
502.2
524.2
502.2
524.2
502.2
524.2
524.2
524.2
524 2
502.2
502.2
524.2
524.2
524.2
Procedure
ELCD
PID
PID
ELCD
ELCD
PID
ELCD
ELCD
PID
ELCD
PID
ELCD
ELCD
ELCD
PID
ISO CRV/
MDL
-39.7%
-59 9%
-5 0%
-55.5%
-25.8%
-115.4%
-172.1%
-8.6%
-12.4%
-30.7%
-28.0%
-140.1%
-48.6%
-45.0%
-59.7%
-39.6%
-33.9%
-151 2%
-19.1%
-35.5%
-63.5%
-45.7%
-136.9%
-33.3%
-24.0%
-183.3%
-33 5%
-116.2%
-55.5%
-54.7%
-159.6%
-106.6%
ISO MDV/
MDL
31.4%
13 5%
-1 3%
23.0%
33.0%
-70.1%
-150.8%
28.7%
53.6%
48.9%
-655.2%
-106.3%
-98.0%
22.4%
-75.2%
39.4%
-28.8%
-1122%
58.3%
-754.8%
-2.1%
-457.8%
-94.1%
654.4%
-11.7%
-228.6%
20 2%
-64.0%
23.6%
-165.4%
-140.3%
-66.3%
SL-IDE/
MDL
121.2%
55 5%
123 6%
28.0%
168.6%
18.2%
52.9%
87.8%
59.5%
127.6%
125.1%
83.9%
147.5%
61.1%
125.7%
51.0%
169.3%
108 7%
30.0%
144.1%
100.1%
136.4%
80.6%
142.5%
166.8%
-109.9%
66 6%
47.7%
-3.6%
158.1%
-37.3%
18.2%
B-32
-------
Table 3. Percent Differences of
Detection Limits to the EPA/ACS DL for the Episode 6000 Dataset
Analyte
4 -chloro toluene
4 ~ ch 1 or o toluene
4~isopropyltoluene
4-m e thy |-£ -pen ta none
Acetone
Acrylonitrile
Ally! Chloride
Aluminum
Aluminum
Ammon,a a, N,,roO.n
Antimony
Antimony
Arsenic
Arsenic
Barium
Darium
Denzene
Denzene
Deryllium
Beryllium
boron
Bremob.n«n.
Bromob.n«n.
Bromob.n«n.
Dromochloromethane
n
Drom odi chl orom ethane
Dromodichlorometiane
Bromo,orm
bromoform
Drom om ethane
Drom om ethane
Method
502.2
524.2
524.2
524.2
524.2
524.2
524.2
1620
200.8
350.3
1620
200.8
1620
200.8
1620
200.8
502.2
524.2
1620
200.8
1620
502.2
502.2
524.2
502.2
524 2
502.2
524.2
502.2
524.2
502.2
524.2
Procedure
ELCD
PID
ELCD
PID
ELCD
ELCD
ELCD
ELCD
ISO CRV/
MDL
-119.9%
-21.8%
-18.2%
-51.4%
-64.5%
-64.0%
-19.8%
-65.1%
-167.6%
-39.8%
-63.8%
-193.1%
-14.9%
-49.1%
-24.0%
-12.2%
-2.5%
-1.9%
-43.8%
-55.8%
-39.1%
-33.8%
-31.6%
-18.1%
-11.8%
-10 3%
-35.9%
-47.8%
-64.7%
-62.6%
-19.7%
-21.0%
ISO MDV/
MDL
-74.8%
-26.2%
-95.8%
-10.3%
-6.6%
-27.7%
-150.4%
-3.1%
-137.5%
30.4%
12.2%
-185.9%
25.4%
18.7%
7.3%
59.9%
76.2%
57.8%
-25.6%
-16.7%
14.5%
34.8%
44.4%
1274.8%
55.9%
23 8%
27.1%
-76.2%
-129.0%
-120.5%
29.2%
-19.6%
SL-IDE/
MDL
30.5%
170.8%
169.2%
38.1%
84.7%
42.9%
150.6%
150.0%
-40.1%
31.7%
93.2%
-161.5%
27.9%
47.5%
7.6%
87.9%
89.5%
158.7%
-16.5%
109.7%
31.6%
141.6%
121.7%
131.5%
189.2%
93 6%
178.8%
130.5%
198.4%
105.6%
185.9%
122.1%
B-33
-------
Table 3. Percent Differences of
Detection Limits to the EPA/ACS DL for the Episode 6000 Dataset
Analyte
Cadmium
Cadmium
Calcium
Carbon Uisulfide
L*arbon 1 etrachloride
Carbontet+1 ,1 "dcp
Chloroace tonitrile
Lfhloro benzene
Chloro benzene
Lfhlorobenzene
Lf hi or o ethane
Lf hi oroe thane
L*hloroform
L*hloroform
Lf hi or o methane
Chlorom ethane
Chrom,um
Chromium
Cis-1 ,2-dce + 2,2-dcp
Cis~l ,£~dichloroethene
Cis-1 ,O~dichloropropene
L*is~ 1 , O'dichloropropene
Lfis~ 1 , 0'dichloropropene
Cobalt
Cobalt
Lf opper
Lf opper
D b m n
D b hi ih
Uibromome thane
Uibromome thane
Uichlorodifluorome thane
Method
1620
200.8
1620
524.2
524.2
502.2
524.2
502.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
1620
200.8
502.2
524.2
502.2
502.2
524.2
1620
200.8
1620
200.8
502 2
524 2
502.2
524.2
502.2
Procedure
ELCD
ELCD
PID
ELCD
ELCD
ELCD
ELCD
ELCD
PID
ELCD
ELCD
ELCD
ISO CRVI
MDL
-47.1%
55.5%
-2.5%
-52.8%
-33.8%
-110.8%
-17.3%
-11.3%
-19.2%
-12.7%
-171.0%
-47.0%
-150.5%
-29.2%
-34.7%
-21.8%
-20.0%
-16.5%
-39.4%
-19.1%
-101.0%
-17.5%
-47.6%
-83.9%
-23.4%
-19.1%
-33.0%
-46 7%
-49 9%
-21.1%
-21.5%
-91.4%
ISO MDV/
MDL
5.5%
99.6%
65.4%
990.7%
10302.8%
-55.2%
49.7%
61.5%
58.8%
-66.3%
-169.4%
-53.1%
-129.4%
-51.9%
60.2%
37.1%
21.9%
52.5%
21.8%
-760.6%
-61.1%
54.1%
-251.5%
-19.3%
206.3%
53.9%
35.3%
17 2%
-168 4%
38.8%
9.2%
511.1%
SL-IDE/
MDL
40.1%
138.8%
11.9%
160.0%
156.6%
85.6%
52.3%
190.3%
71.4%
137.5%
184.1%
142.6%
-27.3%
144.5%
112.8%
139.8%
46.3%
139.3%
124.0%
141.9%
164.6%
36.0%
127.2%
50.6%
194.5%
111.2%
182.2%
191 8%
139 6%
194.4%
116.8%
185.7%
B-34
-------
Table 3. Percent Differences of
Detection Limits to the EPA/ACS DL for the Episode 6000 Dataset
Analyte
Uichlorodi luorom ethane
Diethyl Ether
Lthyl Me thacryla te
t thy 1 benzene
Lthyl benzene
Hardness
H exa chl or o butadiene
|_i
Hexachloroe thane
rlexchlobutadiene~l~naphthalene
Iron
1 sopropyl benzene
1 sopropyl benzene
Lead
Lead
M+p Xylene
M+p Xylene
Magnesium
Manganese
Manganese
Mercury
Methacrylon itrile
Methyl Iodide
Methyl 1 ert "butyl Lther
Methyla crylate
Methylene Chloride
Methylene Chloride
Methylm etna cry 1 ate
Molybdenum
Molybdenum
N "butyl benzene
N "butyl benzene
Method
524.2
524.2
524.2
502.2
524.2
130.2
502.2
524 2
524.2
502.2
1620
502.2
524.2
1620
200.8
502.2
524.2
1620
1620
200.8
200.8
524.2
524.2
524.2
524.2
502.2
524.2
524.2
1620
200.8
502.2
524.2
Procedure
PID
ELCD
PID
PID
PID
ELCD
PID
ISO CRV/
MDL
-42.3%
-4.9%
-37.9%
-35.2%
-18.3%
-39.6%
-123.8%
-63 3%
-12.4%
-127.7%
99.8%
-30.2%
-8.3%
-32.6%
-165.8%
-154.5%
-51.9%
-14.9%
-145.5%
-2.7%
-22.3%
-43.7%
-7.9%
-45.4%
-8.6%
173.9%
-13.4%
-90.7%
-35.5%
-25.1%
-26.9%
-11.7%
ISO MDV/
MDL
-76.4%
30.4%
-108.3%
46.8%
-1245.1%
32.7%
-69.6%
-528 0%
-38.6%
-67.8%
135.7%
53.0%
-3.3%
17.6%
-138.0%
-109.6%
-100.0%
51.9%
-90.3%
45.4%
331.3%
1.8%
-613.8%
1591.3%
46.5%
189.8%
12.6%
-63.2%
42.7%
-195.0%
49.2%
50.0%
SL-IDE/
MDL
148.1%
103.3%
143.1%
113.8%
142.3%
92.6%
74.3%
127 6%
134.9%
-8.4 %
122.1%
98.7%
167.1%
38.1%
-105.1%
28.6%
166.8%
2.8 %
-0.7%
112.6%
145.0%
67.4%
153.7%
158.7%
92.9%
182.7%
117.2%
81.6%
21.1%
194.5%
130.0%
162.5%
B-35
-------
Table 3. Percent Differences of
Detection Limits to the EPA/ACS DL for the Episode 6000 Dataset
Analyte
IM'propylbenzene
IM'propyl benzene
Naphthalene
Nickel
Nickel
o-xylene
o-xylene+styrene
r -isoproptol + 1 ,4'dcb
re nta chl or o ethane
Oec~ butyl benzene
Oec~ butyl benzene
Selenium
Selenium
Silver
Silver
Oodium
Otyrene
1 erfbutybenzene
1 erfbutybenzene
1 etrachloroetiene
1 etrachloroetiene
1 etrachloroetiene
Thallium
Thallium
Thorium
Tin
Titanium
1 oluene
1 oluene
1 otal Phosphorus
1 otal Ouspended Oolkjs
T 12
'
Method
502.2
524.2
524.2
1620
200.8
524.2
502.2
502.2
524.2
502.2
524.2
1620
200.8
1620
200.8
1620
524.2
502.2
524.2
502.2
502.2
524.2
1620
200.8
200.8
1620
1620
502.2
524.2
365.2
160.2
5022
Procedure
PID
PID
PID
PID
PID
ELCD
PID
PID
ELCD
ISO CRV/
MDL
-58.0%
-38.7%
-19.7%
-41.6%
-86.4%
-22.0%
-46.2%
-25.1%
-186.5%
-52.4%
-27.1%
-31.3%
-20.4%
-31.1%
-77.6%
-33.5%
-22.6%
-36.4%
-60.7%
-26.2%
-42.6%
-0.3%
24.0%
-18.1%
-17.9%
-58.1%
-7.0%
-85.5%
-112.6%
-25.1%
-21.0%
1 2%
ISO MDV/
MDL
20.6%
1215.0%
-8.6%
23.9%
-30.4%
735.4%
32.4%
51.8%
-288.7%
29.7%
2196.0%
55.0%
44.8%
27.9%
-5.4%
33.6%
-31.1%
48.6%
5.5%
47.3%
41.5%
-57.5%
93.3%
44.5%
270.2%
-15.5%
51.0%
-11.0%
-290.2%
44.5%
49.7%
75 2%
SL-IDE/
MDL
77.9%
152.9%
117.7%
23.3%
-55.2%
166.0%
65.1%
74.3%
-30.2%
37.9%
163.9%
79.8%
73.8%
74.0%
102.6%
66.5%
163.6%
88.6%
157.8%
109.0%
86.4%
138.9%
77.0%
67.0%
50.1%
6.9%
11.8%
-8.1%
152.2%
77.5%
87.9%
66 8%
B-36
-------
Table 3. Percent Differences of
Detection Limits to the EPA/ACS DL for the Episode 6000 Dataset
Analyte
1 rans'l ,£~dichloroethene
1 rans" l,O~dichloropropene
1 rans" l,O~dichloropropene
1 rans'l ,O~dichloropropene
Trans-1 ,4~dichloro-2-butene
I richloroethene
I richloroethene
I richloroethene
T
1 richlorofluoromethane
Uranium
Vanadium
Vanadium
Vinyl Chloride
Vinyl Chloride
WAD Cyanide
Xylene ( 1 o tal)
Yttrium
Zinc
Zinc
Method
524.2
502.2
502.2
524.2
524.2
502.2
502.2
524.2
502 2
524.2
200.8
1620
200.8
502.2
524.2
1677
524.2
1620
1620
200.8
Procedure
ELCD
PID
ELCD
PID
ELCD
ELCD
ISO CRV/
MDL
-18.1%
-117.4%
-26.6%
-69.2%
-38.0%
-156.0%
-38.3%
-4.9%
78 9%
-15.3%
-75.4%
-54.3%
-8.0%
-149.6%
-32.6%
-108.6%
-54.0%
-33.6%
-12.1%
-64.6%
ISO MDV/
MDL
-495.6%
-79.8%
47.3%
-260.7%
11.8%
-127.8%
42.8%
-9.7%
140 1%
-78.4%
-32.9%
12.9%
56.7%
-111.4%
-274.6%
-56.8%
-20.8%
26.8%
34.9%
-11.0%
SL-IDE/
MDL
154.9%
157.3%
44.8%
125.8%
83.8%
133.2%
112.7%
137.6%
180 3%
125.9%
27.6%
36.6%
43.6%
172.7%
157.7%
20.2%
174.0%
51.2%
53.6%
55.8%
Note! LLLfU or rlU in the Procedure column indicates the pho to'ioniza tion detector (rlU) or electrolytic conductivity
de.eo.or (ELCD) In EPA Me.nod 502.2
B-37
-------
Summary Statistics for Table 3
Minimum
25th percentile
Median
75th percentile
Maximum
ISO CRV/
EPA/ACS DL
% Difference
-197.5%
-60.5%
-35.7%
-19.9%
173.9%
ISO MDV/
EPA/ACS DL
% Difference
-4142.5%
-76.4%
8.8%
44.5%
10302.8%
SL-IDE/
EPA/ACS DL
% Difference
-161.5%
51.0%
108.7%
144.1%
198.4%
CRVvs. DL
MDVvs. DL
SL-IDEvs. DL
Median %
Difference
-35.7%
8.8%
1087%
p-valuefor%
difference=0
<0.0001
0.164
<0.0001
Table 4. Comparison Quantitation Limits for the Episode 6000 Dataset
(ng/L except where footnoted)
Analyte
1 ,1 ,1 ,Znetrachloroethane
1 ,1 ,1 ,Znetrachloroethane
1 ,1 ,1 "Irichloroelhane
1 ,1 ,1 -Irichloroelhane
1,1,2,2-,0,+1,2,3-top
1 ,1 ,£,£-tetrachloroethane
1 ,1 ,Z-trichloroetnane
1 ,1 ,^-trichloroetnane
1 ,1 -dichloroethane
1 ,1 -dichloroethane
1 ,1 -dichloroethene
1 ,1 -dichloroethene
1 ,1 -dichloropropanone
1 ,1 -dichloropropene
1 r£rO~1richlorobenzene
1 ,£,O-1richlorobenzene
Method
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
524.2
524.2
502.2
502.2
Procedure
ELCD
ELCD
ELCD
ELCD
ELCD
ELCD
ELCD
PID
EPA/
ACSQL
0.2
0.2
0.05
0.2
0.2
0.5
0.1
0.2
0.05
0.1
0.1
0.2
20
0.2
0.2
0.2
ISO/
IUPAC LOQ
0.023
0.183
0.044
0.102
0.227
0.597
0.018
0.212
0.037
0.099
0.191
0.159
15.409
0.057
0.168
0.226
ASTM
SL-IQE
0.030
0.181
0.830
0.240
5.514
0.569
0.060
0.290
0.527
0.115
3.796
0.129
12.705
0.180
0.851
0.248
B-38
-------
Table 4. Comparison Quantitation Limits for the Episode 6000 Dataset
(H9/L except where footnoted)
Analyte
1 ,£ ,O ~ 1r i chil or o benzene
1 ,£ ,O~1richloro pro pane
1 ,Z,4~1richlorobenzene
1 ,Z,4~1richlorobenzene
1 ,Z,4-trichlorobenzene
1 ,Z,4-trimethyl benzene
1 , 2, 4-trimethyl benzene
1 2 d b 3
I ,£~dibromoetnane
1 ,£~dibromoetnane
1 ,£ ~di chloroben zene
1 ,£ ~di chloroben zene
1 ,2-dichtorobenzene
1 ,£-dichloroetriane
1 ,/_-dichloroetriane
1 ,/_-dichloropropane
1 ,/_-dichloropropane
1 ,3,5-tmb + 4-chlorotoluene
1 ,O,b -trim ethyl benzene
1 ,O-dichtorobenzene
1 ,O ~di chloroben zene
1 ,O ~di chloroben zene
1 ,0 ~di chloropropane
1 ,0 ~di chloropropane
1 ,4-dichtorobenzene
1 ,4-dichtorobenzene
1 "chlorobutane
/_,/_-dichloropropane
Z'butanone
L. ~ ch 1 or o toluene
L. ~ ch 1 or o toluene
L. ~ ch 1 or o toluene
Method
524.2
524.2
502.2
502.2
524.2
502.2
524.2
5242
502.2
524.2
502.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
502.2
524.2
502.2
524.2
502.2
524.2
524.2
524.2
524.2
502.2
502.2
524.2
Procedure
ELCD
PID
PID
ELCD
ELCD
PID
ELCD
ELCD
PID
ELCD
PID
ELCD
ELCD
ELCD
PID
EPA/
ACSQL
0.2
20
0.1
0.2
0.2
0.5
0.05
5
0.5
0.5
0.1
0.1
0.1
0.05
0.1
0.1
0.2
0.2
0.05
0.1
0.2
0.1
0.05
0.1
0.1
0.1
0.05
10
2
0.5
1
0.05
ISO/
IUPAC LOQ
0.192
0.268
0.078
0.208
0.231
0.307
0.050
1 842
0.037
0.560
0.158
0.139
0.101
0.015
0.122
0.075
0.148
0.259
0.044
0.027
0.438
0.080
0.040
0.114
0.025
0.069
0.082
0.572
1.416
0.145
0.781
0.046
ASTM
SL-IQE
0.216
11.316
0.401
0.439
0.141
0.653
20.896
71 1826
0.592
0.417
0.183
0.346
0.085
0.065
0.222
0.102
0.196
0.189
23.744
0.936
0.465
0.076
0.054
0.139
0.101
0.078
29.943
38.009
0.893
0.493
0.849
0.053
B-39
-------
Table 4. Comparison Quantitation Limits for the Episode 6000 Dataset
(H9/L except where footnoted)
Analyte
£~hexanone
£~nitropropane
4 - chl or o toluene
4 ~ chl or o toluene
4-isopropyltoluene
4-methy|-£-pentanone
Acetone
Meryl onitrile
Ally! Chloride
Aluminum
Aluminum
Ammonia as Nitrogen
Antimony
Antimony
Arsenic
Arsenic
Darium
Darium
Denzene
Denzene
Beryllium
Beryllium
Doron
Dromo benzene
Dromo benzene
Dromo benzene
D
D
Dromodichlorom ethane
Dromodichlorom ethane
Bromo,orm
Dromoform
Method
524.2
524.2
502.2
524.2
524.2
524.2
524.2
524.2
524.2
1620
200.8
350.3
1620
200.8
1620
200.8
1620
200.8
502.2
524.2
1620
200.8
1620
502.2
502.2
524.2
5022
5242
502.2
524.2
502.2
524.2
Procedure
ELCD
PID
ELCD
PID
ELCD
ELCD
ELCD
EPA/
ACSQL
5
2
0.5
0.05
0.05
2
2
2
0.1
100
50
0.05
5
0.5
5
1
5
0.1
0.1
0.05
2
0.02
50
0.5
0.05
0.2
0 05
0 5
0.02
0.2
0.02
0.5
ISO/
IUPAC LOQ
0.669
1.280
0.132
0.037
0.043
2.066
2.114
1.816
0.129
76.242
9.418
0.037
4.784
0.017
3.684
0.720
4.722
0.161
0.173
0.075
1.055
0.018
46.040
0.599
0.050
0.167
0 065
0 549
0.015
0.135
0.018
0.287
ASTM
SL-IQE
0.442
0.590
0.142 1
23.810
0.016
1.785
2.741
28.056
29.674
464.069
29.684
0.035
9.551
0.034
3.097
0.798
4.118
0.211
0.182
0.044
0.980
0.044
51.134
3.529
0.100
0.140
1 598
0 368
0.424
0.128
3.393
0.482
B-40
-------
Table 4. Comparison Quantitation Limits for the Episode 6000 Dataset
(H9/L except where footnoted)
Analyte
Bromome,nane
Bromometnane
Cadmium
Cadmium
Calcium
Carbon Disulfide
Carbon 1 etrachloride
Carbontet+1 ,\ "dcp
Chloroace tonitrile
Chloro benzene
Chloro benzene
Chloro benzene
Chloro ethane
Chloro ethane
Chloroform
Chloroform
Chloro me thane
Chloro me thane
Chromium
Chromium
Cis-1 ,2-dce+2,2-dcp
Cis~l ,£~dichloroethene
Cis~ 1 rO~dichloropropene
Cis~ 1 rO~dichloropropene
Cis~ 1 rO~dichloropropene
Cobalt
p
Copper
Copper
Uibromochloro me thane
D-b hi 1h
Uibromom ethane
Method
502.2
524.2
1620
200.8
1620
524.2
524.2
502.2
524.2
502.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
1620
200.8
502.2
524.2
502.2
502.2
524.2
1620
2008
1620
200.8
502.2
524 2
502.2
Procedure
ELCD
ELCD
ELCD
PID
ELCD
ELCD
ELCD
ELCD
ELCD
PID
ELCD
ELCD
EPA/
ACSQL
1
0.2
0.5
0.02
100
0.1
0.1
0.1
2
0.05
0.1
0.1
0.5
0.2
0.2
0.1
0.2
0.2
1
0.2
0.05
0.1
0.02
0.2
0.1
50
0 005
20
0.1
0.02
0 2
0.02
ISO/
IUPAC LOQ
3
undefined
0.252
0.346
0.046
186.530
0.077
0.127
0.046
4.170
0.058
0.143
0.108
0.053
0.185
0.029
0.138
0.342
0.181
0.993
0.331
0.045
0.154
0.013
0.254
0.117
20.916
3
27.513
0.142
0.030
0 149
0.028
ASTM
SL-IQE
16.351
0.226
0.410
0.063
99.975
0.101
0.140
0.069
3.310
1.766
0.119
0.059
5.826
0.255
0.025
0.121
1.734
0.141
1.259
1.028
0.039
0.144
0.415
0.017 1
0.141
40.837
4
47.509
1.825
1.252
0 288
1.395
B-41
-------
Table 4. Comparison Quantitation Limits for the Episode 6000 Dataset
(H9/L except where footnoted)
Analyte
D^omometnane
L/ichlorodifluorom ethane
Uichlorodifluorom ethane
Diethyl Ether
Ethyl Methacryla te
t thy 1 benzene
t thy (benzene
Hardness
Mexachloro butadiene
Mexachloro butadiene
Mexachloro ethane
M exchlo buta diene + na phthalene
Iron
1 sopropyl benzene
1 sopropyl benzene
Lead
Lead
M + p Xylene
M + p Xylene
Magnesium
Manganese
Manganese
Mercury
Methacrylon itrile
Methyl Iodide
Methyl 1 erfbutyl Lther
Methyla crylate
Methylene Chloride
Methylene Chloride
Methylm ethacryl ate
Molybdenum
Molybdenum
Method
524.2
502.2
524.2
524.2
524.2
502.2
524.2
130.2
502.2
524.2
524.2
502.2
1620
502.2
524.2
1620
200.8
502.2
524.2
1620
1620
200.8
200.8
524.2
524.2
524.2
524.2
502.2
524.2
524.2
1620
200.8
Procedure
ELCD
PID
ELCD
PID
PID
PID
ELCD
EPA/
ACSQL
0.5
0.02
0.2
0.5
0.2
0.1
0.1
2
0.2
0.2
0.2
2
200
0.1
0.05
5
2
0.2
0.05
500
20
0.1
0.02
1
0.1
0.1
1
0.5
0.2
1
10
0.01
ISO/
IUPAC LOQ
0.400
0.012
0.290
0.563
0.139
0.089
0.123
2.973
0.054
0.160
0.232
0.834
1490.589
0.090
0.056
5.062
0.318
0.068
0.042
454.043
7.948
0.133
0.056
1.066
0.108
0.073
0.966
3
undefined
0.354
0.381
9.752
0.052
ASTM
SL-IQE
0.460
1.091 5
0.480
0.404
0.183
0.157
0.077
5.465
0.243
0.228
0.167
1.542
996.565 5
0.129
25.592
5.698
0.685
0.222
24.651
267.199
15.264
0.245
0.039
19.062
0.083
0.122
0.727
6.033
0.433
20.773
7.597
0.608
B-42
-------
Table 4. Comparison Quantitation Limits for the Episode 6000 Dataset
(H9/L except where footnoted)
Analyte
IM~ butyl benzene
IM~ butyl benzene
N- pro pyl benzene
N- pro pyl benzene
Naphthalene
Nickel
Nickel
o'xylene
o'xylene + styrene
r-isoproptol + 1 ,4'dcb
r entachloroetiane
Oec~ butyl benzene
Oec~ butyl benzene
Selenium
Oelenium
Silver
Silver
Oodium
Otyrene
1 erfbutylbenzene
1 ert'butylbenzene
1 etrachloroetiene
1 etrachloroetiene
1 etrachloroetiene
Thallium
Thallium
1 horium
Tin
1 Itanium
1 oluene
1 oluene
T D 2
1 otal r hosphorus
Method
502.2
524.2
502.2
524.2
524.2
1620
200.8
524.2
502.2
502.2
524.2
502.2
524.2
1620
200.8
1620
200.8
1620
524.2
502.2
524.2
502.2
502.2
524.2
1620
200.8
200.8
1620
1620
502.2
524.2
365.2
Procedure
PID
PID
PID
PID
PID
PID
ELCD
PID
PID
EPA/
ACSQL
0.1
0.05
0.2
0.1
0.2
100
0.5
0.05
0.2
0.2
2
0.2
0.05
2
0.5
20
0.02
200
0.05
0.1
0.1
0.05
0.2
0.2
2
0.002
0.002
10
20
0.2
0.05
0.02
ISO/
IUPAC LOQ
0.128
0.077
0.128
0.110
0.184
66.486
0.287
0.062
0.210
0.318
0.086
0.193
0.063
3.859
0.805
16.734
0.011
251.546
0.054
0.121
0.063
0.076
0.244
0.378
3.748
0.002
0.005
9.237
20.807
0.162
0.028
0.024
ASTM
SL-IQE
0.745
0.067
0.186
29.878
0.108
67.206
0.183
0.040
0.181
0.456
0.551
0.157
0.047
5.235
1.045
25.842
0.056
337.755
0.041
0.203
0.073
0.122
0.750
30. 554 6
2.799
0.002
0.004
9.406
14.236
0.194
0.046
0.030
B-43
-------
Table 4. Comparison Quantitation Limits for the Episode 6000 Dataset
(H9/L except where footnoted)
Analyte
1 otal Ouspended Oolids
1 rans'l ,£~dichloroethene
1 rans'l ,£~dichloroethene
1 rans'l ,O~dichloropropene
1 rans'l ,O~dichloropropene
1 rans'l ,O~dichloropropene
1 rans'l ,4~dichloro-Z-butene
1 richloroethene
1 richloroethene
1 richloroethene
T
T hi 1
Uranium
Vanadium
Vanadium
Vinyl Chloride
Vinyl Chloride
WAD Cyanide
Aylene ( 1 o tal)
Yttrium
Zlno
Zinc
Method
160.2
502.2
524.2
502.2
502.2
524.2
524.2
502.2
502.2
524.2
502 2
524 2
200.8
1620
200.8
502.2
524.2
1677
524.2
1620
1620
200.8
Procedure
ELCD
ELCD
PID
ELCD
PID
ELCD
ELCD
EPA/
ACSQL
5
0.2
0.1
0.05
0.2
0.2
2
0.05
0.1
0.2
0 5
0 2
0.001
20
2
1
0.2
2
0.02
5
10
2
ISO/
IUPAC LOQ
5.011
0.234
0.141
0.016
0.244
0.121
1.803
0.008
0.108
0.284
1 612
0 279
0.001
21.586
2.627
0.264
0.139
0.852
0.027
6.571
9.575
2.147
ASTM
SL-IQE
6.729
0.191
0.153
0.729
0.175
0.218
30.108
3.169
0.401
0.167
4 662
42 490 6
0.001
24.338
1.933
8.234
0.219
1.624
23.520
8.962
10.452
7.024
node! (linear)
1|QE 10% undefined, IQE 20% reported
Results reported as mg/L
" IQE 10%, IQE 20% and IQE 30% all negative based on ono
5 IQE 10% and IQE 20% both negative, IQE 30% reported
Hybrid model selected but did not converge, Ivj t IU/0 based on constant model instead
Note j LLCL) or PIL) in the Procedure column indicates the photo~ioniza tion detector (rlD) or electrolytic
conductivity detector (ELCD) in EPA Method 502.2
B-44
-------
Table 5. Percent Differences of Quantitation Limits to the EPA/ACS QL
for the Episode 6000 Dataset
Analyte
1 ,1 ,1 ,£"tetrachloroethane
1112
, , , tetrac roet ane
1,1,1 "Irichloroetnane
1,1,1 "Irichloroetnane
1,1,2,2-,o,+1,2,3-,op
1 ,1 ,£,£-tetrachloroethane
1 ,1 -dichloroethane
1 ,1 -dichloroethane
1 ,1 "dichloroethene
1 ,1 -dichloroethene
1 , 1 "dichloropropanone
1 ,1 "dichloropropene
1 ,£,O-1richlorobenzene
1 ,£,O-1richlorobenzene
1 ,£,O-1richlorobenzene
1 ,£,O-1richloropropane
1 ,Z,4-1richlorobenzene
1 ,2,4-trichlorobenzene
1 ,£,4-1richlorobenzene
1 , 2, 4-trimethyl benzene
1 , 2, 4-trimethyl benzene
1 ,Z-dibromo-O-chloropropane
1 ,£-dibromoe1hane
1 ,£-dibromoethane
1 ,£-dichk>robenzene
1 ,£-dichk>robenzene
1 ,£-dichk>robenzene
1 ,£-dichk>roethane
1 ,£-dichk>roethane
1 ,£ "dichloro pro pa ne
Method
502.2
524 2
502.2
524.2
502.2
524.2
502 2
524 2
502.2
524.2
502.2
524.2
524.2
524.2
502.2
502.2
524.2
524.2
502.2
502.2
524.2
502.2
524.2
524.2
502.2
524.2
502.2
502.2
524.2
502.2
524.2
502.2
Procedure
ELCD
ELCD
ELCD
ELCD
ELCD
ELCD
ELCD
PID
ELCD
PID
PID
ELCD
ELCD
PID
ELCD
ELCD
ISO
LOQ/ML
-158.7%
-8 7%
-11.8%
-65.1%
12.8%
17.6%
-138 2%
5 9%
-28.6%
-0.7%
62.5%
-22.8%
-25.9%
-111.1%
-17.6%
12.2%
-4.2%
-194.7%
-25.2%
3.8%
14.5%
-47.8%
0.5%
-92.3%
-172.7%
11.3%
45.1%
32.9%
0.6%
-108.1%
19.7%
-28.5%
SL-IQE/ML
-147.3%
-9 8%
177.3%
18.0%
186.0%
12.9%
-49 6%
36 6%
165.3%
13.7%
189.7%
-43.3%
-44.6%
-10.5%
123.9%
21.3%
7.7%
-55.5%
120.2%
74.9%
-34.9%
26.5%
199.0%
173.7%
16.9%
-18.1%
58.8%
110.2%
-16.5%
26.0%
75.6%
2.3%
B-45
-------
Table 5. Percent Differences of Quantitation Limits to the EPA/ACS QL
for the Episode 6000 Dataset
Analyte
1 r£~dichloropropane
1 ,O,b-1mb+4-chlorotoluene
1 , 3, 5-trimethyl benzene
1 ,O~dichlorobenzene
1 ,O~dichlorobenzene
1 ,O~dichlorobenzene
1 ,O~dichloropropane
1 ,O~dichloropropane
1 ,4'dichlorobenzene
1 ,4-dichtorobenzene
I 'chlorobutane
£r£~dichk>ropropane
2 b t
£~chlorotoluene
L. ~ chl or o toluene
L. ~ ch 1 or o toluene
£~hexanone
Z'nitro pro pane
4-chlorotoluene
4 ~ chl or o toluene
4-isopropyltoluene
4-m e thy \~£. -pen ta none
Acetone
Acrylonitrile
Ally! chloride
Aluminum
Aluminum
Ammon,a a, n,trog,n
Antimony
Antimony
Arsenic
Arsenic
Method
524.2
502.2
524.2
502.2
502.2
524.2
502.2
524.2
502.2
524.2
524.2
524.2
524 2
502.2
502.2
524.2
524.2
524.2
502.2
524.2
524.2
524.2
524.2
524.2
524.2
1620
200.8
350.3
1620
200.8
1620
200.8
Procedure
PID
ELCD
PID
ELCD
ELCD
ELCD
PID
ELCD
ICP/MS
ICP/MS
ICP/MS
ISO
LOQ/ML
-29.8%
25.7%
-13.6%
-114.2%
74.5%
-22.2%
-22.9%
13.0%
-120.8%
-37.2%
48.8%
-178.4%
-34 2%
-109.9%
-24.6%
-7.6%
-152.8%
-43.9%
-116.3%
-29.1%
-15.4%
3.2%
5.5%
-9.7%
25.5%
-27.0%
-136.6%
-30.9%
-4.4%
-186.6%
-30.3%
-32.5%
SL-IQE/ML
-1.9%
-5.5%
199.2%
161.4%
79.7%
-27.3%
7.5%
32.7%
1.0%
-24.2%
199.3%
116.7%
-76 6%
-1.4%
-16.4%
6.3%
-167.5%
-108.9%
-111.5%
199.2%
-101.7%
-11.3%
31.3%
173.4%
198.7%
129.1%
-51.0%
-34.1%
62.6%
-174.7%
-47.0%
-22.5%
B-46
-------
Table 5. Percent Differences of Quantitation Limits to the EPA/ACS QL
for the Episode 6000 Dataset
Analyte
Darium
Darium
Denzene
Denzene
Beryllium
Beryllium
Doron
Dromo benzene
Bromob.nMn.
Dromo benzene
Dromochlorom ethane
Dromochlorom ethane
Dromodichlorom ethane
Dromodichlorom ethane
Dromoform
Bromo,orm
Dromom ethane
Bromom,tflan,
Cadmium
Cadmium
Calcium
Carbon disulfide
Q
Carbontet+1 ,1 "dcp
Chloroace tonitrile
Chloro benzene
Chloro benzene
Chloro benzene
Chloroethane
Lf hi or o ethane
L*hloroform
L*hloroform
Method
1620
200.8
502.2
524.2
1620
200.8
1620
502.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
1620
200.8
1620
524.2
5242
502.2
524.2
502.2
502.2
524.2
502.2
524.2
502.2
524.2
Procedure
ICP/MS
PID
ICP/MS
ELCD
PID
ELCD
ELCD
ELCD
ELCD
ICP/MS
ELCD
ELCD
PID
ELCD
ELCD
ISO
LOQ/ML
-5.7%
46.6%
53.7%
40.5%
-61.9%
-9.9%
-8.2%
18.0%
-0.9%
-18.1%
25.8%
9.3%
-25.6%
-38.7%
-12.0%
-54.2%
N/A
23.2%
-36.4%
79.2%
60.4%
-26.2%
23 9%
-74.3%
70.3%
15.7%
35.2%
7.4%
-161.8%
-8.0%
-149.4%
31.7%
SL-IQE/ML
-19.3%
71.5%
58.1%
-13.1%
-68.5%
75.0%
2.2%
150.4%
67.0%
-35.4%
187.9%
-30.3%
182.0%
-43.9%
197.7%
-3.7%
176.9%
12.2%
-19.7%
103.6%
-0.0%
1.3%
33 4%
-37.3%
49.3%
189.0%
17.4%
-50.8%
168.4%
24.2%
-155.3%
19.2%
B-47
-------
Table 5. Percent Differences of Quantitation Limits to the EPA/ACS QL
for the Episode 6000 Dataset
Analyte
C hi or o methane
C hi or o methane
Chromium
C,romlum
Cis-1 ,2-dce+2,2-dcp
Cis-1 ,£-dichloroethene
Cis~ 1 ,O~dichloropropene
Cis~ 1 ,O~dichloropropene
Cis~ 1 ,O "dichl oropropene
Cobalt
Cobalt
Copper
Copper
Uibromochlorom ethane
Uibromochlorom ethane
Uibromom ethane
Uibromom ethane
L*i chlorodifluorom ethane
L*i chlorodifluorom ethane
Uiethyl ether
tthyl methacrylate
t thy 1 benzene
tthyl benzene
Hardness
|_|
|_|
|_|
H e xchl o b uta d i en e + naphthalene
Iron
1 sopropyl benzene
1 sopropyl benzene
Lead
Method
502.2
524.2
1620
200.8
502.2
524.2
502.2
502.2
524.2
1620
200.8
1620
200.8
502.2
524.2
502.2
524.2
502.2
524.2
524.2
524.2
502.2
524.2
130.2
502 2
524 2
524 2
502.2
1620
502.2
524.2
1620
Procedure
ELCD
ICP/MS
ELCD
ELCD
PID
ICP/MS
ICP/MS
ELCD
ELCD
ELCD
PID
ELCD
PID
PID
ISO
LOQ/ML
52.5%
-9.8%
-0.7%
49.3%
-9.5%
42.4%
-45.8%
23.8%
15.3%
-82.0%
N/A
31.6%
34.6%
41.1%
-29.0%
34.7%
-22.2%
-53.2%
36.7%
11.9%
-35.7%
-11.5%
20.4%
39.1%
-1144%
-22 3%
148%
-82.3%
152.7%
-10.4%
11.9%
1.2%
SL-IQE/ML
158.6%
-34.9%
22.9%
134.9%
-24.7%
36.1%
181.6%
-168.7%
34.2%
-20.2%
N/A
81.5%
179.2%
193.7%
36.0%
194.3%
-8.3%
192.8%
82.3%
-21.3%
-8.9%
44.6%
-25.4%
92.8%
194%
133%
-17 7%
-25.9%
133.1%
25.3%
199.2%
13.1%
B-48
-------
Table 5. Percent Differences of Quantitation Limits to the EPA/ACS QL
for the Episode 6000 Dataset
Analyte
Lead
M+p xylene
M+p xylene
Magnesium
Manganese
Manganese
Mercury
Methacrylon itrile
Methyl iodide
Methyl terr butyl ether
Methyla crylate
Methylene chloride
Methylene chloride
Methylm ethacryl ate
Molybdenum
Molybdenum
N "butyl benzene
N "butyl benzene
IM-propyl benzene
N ~ pro pyl benzene
Naphthalene
Nickel
Nickel
0-xylene
Q
r -isoproptol + 1 ,4'dcb
r entachl or o ethane
Oec'butyl benzene
Oec~ butyl benzene
Oelenium
Oelenium
Silver
Method
200.8
502.2
524.2
1620
1620
200.8
200.8
524.2
524.2
524.2
524.2
502.2
524.2
524.2
1620
200.8
502.2
524.2
502.2
524.2
524.2
1620
200.8
524.2
502 2
502.2
524.2
502.2
524.2
1620
200.8
1620
Procedure
ICP/MS
PID
ICP/MS
ICP/MS
ELCD
ICP/MS
PID
PID
ICP/MS
PID
PID
PID
ICP/MS
ISO
LOQ/ML
-145.2%
-98.3%
-17.7%
-9.6%
-86.2%
28.0%
94.2%
6.4%
7.3%
-31.2%
-3.5%
N/A
55.6%
-89.5%
-2.5%
135.3%
24.5%
42.2%
-44.1%
9.9%
-8.2%
-40.3%
-54.2%
21.3%
4 9%
45.6%
-183.5%
-3.4%
22.8%
63.5%
46.8%
-17.8%
SL-IQE/ML
-98.0%
10.6%
199.2%
-60.7%
-26.9%
84.1%
63.6%
180.1%
-18.1%
20.1%
-31.7%
169.4%
73.6%
181.6%
-27.3%
193.5%
152.7%
29.5%
-7.1%
198.7%
-59.5%
-39.2%
-92.9%
-21.4%
-10 0%
78.1%
-113.6%
-24.2%
-5.2%
89.4%
70.6%
25.5%
B-49
-------
Table 5. Percent Differences of Quantitation Limits to the EPA/ACS QL
for the Episode 6000 Dataset
Analyte
Silver
O odium
Otyrene
1 ert'butybenzene
1 erfbutytoenzene
1 etracriloroetiene
1 etracriloroetiene
1 etracriloroetiene
Thallium
1 hallium
1 horium
Tin
1 Itanium
1 oluene
1 oluene
1 otal phosphorus
T
P
1 rans-1 ,2-djchloroetnene
1 rans-1 ,2~dicnioroetnene
1 rans- 1 rO~dicmoropropene
1 rans~ 1 rO~dicmoropropene
1 rans-1 ,3~dicnioropropene
1 richloroetiene
1 richloroetiene
1 richloroelhene
T
T
Uranium
Vanadium
Vanadium
,,
Method
200.8
1620
524.2
502.2
524.2
502.2
502.2
524.2
1620
200.8
200.8
1620
1620
502.2
524.2
365.2
160 2
502.2
524.2
502.2
502.2
524.2
5242
502.2
502.2
524.2
5022
5242
200.8
1620
200.8
5022
Procedure
ICP/MS
PID
ELCD
PID
ICP/MS
ICP/MS
PID
ELCD
ELCD
PID
ELCD
PID
ELCD
ICP/MS
ICP/MS
ELCD
ISO
LOQ/ML
-59.2%
22.8%
6.9%
19.2%
-44.8%
40.9%
19.6%
61.5%
60.8%
3.8%
90.3%
-7.9%
4.0%
-21.1%
-57.9%
17.2%
0 2%
15.7%
33.7%
-101.5%
19.8%
-49.5%
-10 4%
-144.3%
7.8%
34.6%
105 3%
33 0%
-33.2%
7.6%
27.1%
-116 4%
SL-IQE/ML
94.7%
51.2%
-20.6%
67.9%
-30.6%
83.5%
115.8%
197.4%
33.3%
16.3%
74.9%
-6.1%
-33.7%
-3.0%
-9.1%
39.9%
29 5%
-4.9%
41.7%
174.3%
-13.4%
8.7%
175 1%
193.8%
120.2%
-17.8%
161 3%
198 1%
2.6%
19.6%
-3.4%
156 7%
B-50
-------
Table 5. Percent Differences of Quantitation Limits to the EPA/ACS QL
for the Episode 6000 Dataset
Analyte
v
Wad cyanide
Xylene (total)
Yttrium
Zlno
Zinc
Method
5242
1677
524.2
1620
1620
200.8
Procedure
WADCN
ICP/MS
ISO
LOQ/ML
-35 7%
-80.5%
28.4%
27.2%
-4.3%
7.1%
SL-IQE/ML
9 3%
-20.8%
199.7%
56.8%
4.4%
111.4%
Note! LLLfU or r ID in the Procedure column indicates the photo~ioniza tion detector (rlU)
conductivity detector (ELC D) in EPA Method 502.2
or electrolytic
Summary Statistics for Table 5
Minimum
25th percen tile
Median
75th percen tile
Maximum
ISO LOQA3L
-194.7%
-35.0%
-4.2%
23.0%
152.7%
SL-K3E/QL
-174.7%
-18.1%
19.6%
111.4%
199.7%
Comparison
LOQv,. QL
SL-IQEv,.QL
Sign Test
p-value
0.390
0.0001
Wilcoxon
p-value
0.043
<0.0001
Comparison
SL-IQEv,. QL
(constant model used for SL'IQE)
SL-IQEvs. QL
(Linear model used for SL'IQE)
SL-IQEvs. QL
(Hybrid model used tor SI_-|QE)
#
analytes
32
65
100
Median %
Difference
179.6%
67.9%
-7.7%
Sign Test
p-value
<0.0001
<0.0001
0.533
Wilcoxon
p-value
<0.0001
<0.0001
0.160
B-51
-------
Table 6. Detection and Quantitation Limits for EPA Methods 1631 and 1638
as Computed by EPA and by EPRI (ng/L)
Elementi
Antimony
Cadmium
vsopper
Lead
Mercury
Nickel
Selenium
Silver
Thallium
Zinc
Ambient
WQC2
14000
370
2400
540
12
8200
5000
320
1700
32000
Detection limits
MDLin
Method
9.7
25
87
15
0.2
330
450
29
7.9
140
IDE computed by
EPA
170
160
800
140
0.81
230
810
440
28
1800
EPRI
110
150
770
160
0.43
130
600
—
20
2100
Quantitation limits
ML in
Method
20
100
200
50
0.5
1000
1000
100
20
500
IQE computed by
EPA
270
540
3800
420
0.55
15000
630
5500
88
21000
EPRI
270
380
3000
370
1.6
330
720
—
50
26100
Mercury determined by EPA Method 1631; al others by EPA Metiod 1638
B-52
-------
Table 7. Comparison of IDEs and IQEs resulting from all model types for EPA Methods 1631 and 1638
Calculated IDEs
Analyte
Antimony
Cadmium
Copper
Lead
Mercury
Nickel
Selenium
Silver
Thallium
Zinc
IDE, Based on Given Model
Constant
2500
1200
2700
400
8.3
7000
4500
2500
230
10,000
Linear
-80 1
130
1000
150
0.058
-48 1
720
710
22
1600
Exponential
170
160
800
140
0.81
230
810
440
28
1800
Hybrid
100
150
720
150
0.52
120
530
650
17
1700
RSD (%)
148%
129%
72%
61%
162%
161%
117%
89%
140%
110%
Calculated IQEs (10%)
Analyte
An-non,
Cadmium
vsopper
Lead
Mercury
Nickel
Oelenium
Silver
Thallium
Zlno
IQE, Based on Given Model
Constant
5400
2600
5900
860
18
15,000
9600
5500
500
22,000
Linear
-570 1
540
3800
420
0.55
-1601
7600
1500"
88
21,000
Exponential
380
380
2100
340
2.1
500
2200
1500
67
4800
Hybrid
270
380
2300
330
1.6
270
630 3
undefined
47
6700
RSD(%)
145%
112%
50%
52%
150%
190%
86%
82%
124%
67%
1N
egative due to negative intercept estimate in precision model.
IQE
undefined, IQE 2.Q /O reported
" IQE 10% negative, IQE 20% reported
B-53
-------
Table 8. Comparison of 16-point and 5-point
Single-laboratory IDEs (SL-IDEs) for the Episode 6000 Dataset
(ng/L except where footnoted)
Analyte
1112
' ' '
1112
' ' '
1 1 1-lricmoroehane
1 1 1-ricmoroehane
1,1,2,2-,oe + 1,2,3-,op
1 ,1 ,Z,£-tetrachloroethane
1 ,1 ,£-1richloroe»iane
1 ,1 ,£-1richloroe»iane
1 ,1 -dichloroethane
1 ,1 -dichloroethane
1 ,1 "dichloroethene
1 ,1 -dichloroethene
1 ,1 -dichloropropanone
1 ,1 "dichloropropene
1 ,£,O-1richloro benzene
1 ,£,O-1richloro benzene
1 ,/_,O-1richloro benzene
1 ,£,O-1richloro pro pane
124
, , trichlorobenzene
1 , £., 4 "trichloro benzene
1 , £., 4 "trichloro benzene
124
, , trimethylbenzene
1 ,2, 4 -trim ethyl benzene
I ,£"di bromo-O-chloro pro pane
1 ,£-dibromoetiane
1 ,£-dibromoetiane
1 2-d hlo b
'
1 2-d hio b
'
1 2-d hk> b
'
1 ,/_-dichloroethane
Method
502 2
524 2
502 2
524 2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
524.2
524.2
502.2
502.2
524.2
524.2
502 2
502.2
524.2
502 2
524.2
524.2
502.2
524.2
502 2
502 2
524 2
502.2
Procedure
ELCD
ELCD
ELCD
ELCD
ELCD
ELCD
ELCD
PID
ELCD
PID
PID
ELCD
ELCD
PID
ELCD
SL-IDE
(16)
0 034
0 244
0 041
0 308
0.179
0.436
0.032
0.319
0.083
0.229
0.234
0.335
6.372
0.287
0.134
0.115
0.275
1.263
0 088
0.124
0.224
0 125
0.144
1.749
0.164
0.326
0 065
0 148
0 130
0.042
SL-IDE (5)
0 011
0 170
0 044
0 035
3.548
0.538
0.013
0.229
0.036
0.084
0.120
0.080
8.941
4.435
0.169
0.069
0.150
16.238
0 100
0.075
0.115
0 143
0.059
0.432
0.025
0.316
0 057
0 077
0 069
0.026
Percent
Difference
-99 6%
-35 8%
6 2%
-159 4%
180.8%
20.8%
-86.7%
-32.8%
-78.8%
-92.7%
-64.0%
-122.6%
33.6%
175.7%
23.1%
-49.9%
-59.2%
171.1%
13 1%
-48.9%
-64.6%
128%
-84.6%
-120.8%
-147.8%
-3.1%
-13 4%
-62 5%
-61 3%
-48.3%
SL-IDE 16
Model
r
xponen la
r
xponen la
r
xponen la
r
xponen la
Lxponential
Lxponential
Lxponential
Lxponential
Lxponential
Lxponential
Lxponential
Lxponential
Lxponential
Lxponential
Lxponential
Lxponential
Lxponential
Lxponential
E
xponentia
Lxponential
Lxponential
E
xponentia
Lxponential
Lxponential
Lxponential
Lxponential
E
xponen la
E
xponen la
E
xponen la
Lxponential
SL-IDE 5
Model
Linear
r
xponen la
r
xponen la
H
y
Constant
txponential
Linear
Lxponential
Lxponential
Lxponential
txponential
Hybrid
txponential
r 1
vsonsta nt
Constant
txponential
txponential
r 1
vsonsta nt
Lxponential
Lxponential
txponential
Hybrid
Linear
txponential
Linear
E
xponen la
E
xponen la
txponential
B-54
-------
Table 8. Comparison of 16-point and 5-point
Single-laboratory IDEs (SL-IDEs) for the Episode 6000 Dataset
(ng/L except where footnoted)
Analyte
1 2-dichtoroethane
'
I r£~dichloro pro pane
1 2-dichtoro
' P P
1 ,3,5-1mb + 4-chlorotoluene
1 ,3,5-lrim ethyl benzene
1 3
, dichlorobenzene
1 ,3-dichtorobenzene
1 ,3-dichtorobenzene
1 ,3-dichtoropropane
1 ,3-dichtoropropane
1 ,4-dichtorobenzene
1 ,4-dichtorobenzene
1 -chlorobutane
£,£-dichtoropropane
2 b t
L- ~ chl or o toluene
^
chlorotoluene
L- ~ ohl or o toluene
2-hexanone
2-nitropropane
4 -chlorotoluene
4 -chlorotoluene
4 "I so pro py (toluene
4-methy|-2-pentanone
Acetone
Acrylonitrile
Ally! Chloride
Al
Al
Ammonia as Nitrogen2
Method
5242
502.2
5242
502.2
524.2
502 2
502.2
524.2
502.2
524.2
502.2
524.2
524.2
524.2
524 2
502.2
502 2
524.2
524.2
524.2
502.2
524.2
524.2
524.2
524.2
524.2
524.2
1620
200 8
350 3
Procedure
ELCD
PID
ELCD
PID
ELCD
ELCD
ELCD
PID
ELCD
SL-IDE
(16)
0 258
0.043
0 247
0.114
0.135
0 118
0.126
0.143
0.047
0.202
0.061
0.140
0.220
0.691
0 833
0.175
0 230
0.136
0.902
1.082
0.149
0.123
0.117
1.195
2.120
1.333
0.229
206 975
12 747
0 014
SL-IDE (5)
0 211
0.087
0 221
0.141
0.049
0 615
0.197
0.038
0.020
0.122
0.040
0.051
0.061
0.122
1 441
0.117
0 409
0.039
0.904
9.354
0.145
0.038
0.038
1.088
30.183
1.077
0.073
73 421
22 654
0 040
Percent
Difference
-19 9%
67.5%
-11 1%
21.4%
-94.1%
135 5%
43.9%
-116.4%
-81.3%
-49.2%
-40.5%
-93.7%
-113.5%
-139.9%
53 5%
-40.2%
56 2%
-111.2%
0.3%
158.5%
-3.2%
-105.5%
-101.3%
-9.3%
173.8%
-21.3%
-103.6%
-95 3%
56 0%
94 0%
SL-IDE 16
Model
Exponential
Exponential
Exponential
r
xponentia
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
r
xponentia
Exponential
r
xponentia
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
E
xponentia
E
xponential
SL-IDE 5
Model
Constant
Constant
Exponential
Constant
Exponential
Exponential
Exponential
Linear
Exponential
Linear
Hybrid
r
xponentia
Lxponential
Lxponential
Exponential
Constant
Linear
Exponential
Lxponential
Exponential
Constant
Exponential
Hybrid
B-55
-------
Table 8. Comparison of 16-point and 5-point
Single-laboratory IDEs (SL-IDEs) for the Episode 6000 Dataset
(ng/L except where footnoted)
Analyte
Antimony
Antimony
Arsenic
Arsenic
Barium
Barium
Denzene
Denzene
Deryllium
Deryllium
Doron
Dromobenzene
Bromobenzene
Bromobenzene
Drom ochlorom ethane
Drom ochlorom ethane
Dromodichlorometiane
Dromodichlorometiane
Dromoform
Dromoform
Drom om ethane
Drom om ethane
Cadmium
Cadmium
Calcium
Carbon Disulfide
L*arbon I etrachloride
Carbontet+1 ,1 "dcp
Lfhloroace tonitrile
Lfhloro benzene
Method
1620
200.8
1620
2008
1620
200 8
502.2
524.2
1620
200.8
1620
502.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
1620
200.8
1620
524.2
524.2
502.2
524.2
502.2
Procedure
PID
ELCD
PID
ELCD
ELCD
ELCD
ELCD
ELCD
ELCD
SL-IDE
(16)
4.260
0.019
1.410
0 366
1 837
0 084
0.079
0.125
0.448
0.024
21.161
0.765
0.050
0.211
0.482
0.345
0.075
0.205
1.513
0.400
7.293
0.280
0.191
0.022
41 358
0.239
0.314
0.072
1.569
0.460
SL-IDE (5)
6.467
0.304
2.268
0 374
1 624
0 073
0.061
0.030
0.438
0.017
22.333
0.348
0.025
0.165
0.044
0.507
0.026
0.088
0.025
0.336
0.760
0.154
0.211
0.016
53 375
0.087
0.174
0.061
2.079
0.064
Percent
Difference
41.2%
176.5%
46.6%
2 1%
-12 3%
-13 7%
-25.0%
-122.6%
-2.2%
-34.2%
5.4%
-75.0%
-65.4%
-24.1%
-166.9%
38.1%
-95.5%
-79.7%
-193.5%
-17.4%
-162.3%
-57.8%
9.8%
-33.8%
25 4%
-93.6%
-57.3%
-15.5%
28.0%
-151.5%
SL-IDE 16
Model
Constant
Exponential
Exponential
r
xponentia
txponential
txponential
txponential
txponential
txponential
Linear
txponential
txponential
tinear
txponential
txponential
txponential
Constant
txponential
Constant
txponential
txponential
txponential
L_
txponential
txponential
txponential
txponential
tinear
SL-IDE 5
Model
tinear
Constant
Constant
txponential
txponential
txponential
Constant
txponential
txponential
txponential
txponential
txponential
txponential
txponential
txponential
Linear
txponential
txponential
Linear
txponential
Constant
tinear
tinear
txponential
txponential
txponential
B-56
-------
Table 8. Comparison of 16-point and 5-point
Single-laboratory IDEs (SL-IDEs) for the Episode 6000 Dataset
(ng/L except where footnoted)
Analyte
P
P
C h loro ethane
p
Chloroform
Chloroform
Chloro me thane
Chloro me thane
Chromium
Chromium
Cis-1 ,2-dce+2,2-dcp
Cis-1 ,2-dichloroethene
Cis" 1 rO~dichloropropene
Cis" 1 rO~dichloropropene
Cis" 1 rO~dichloropropene
Cobalt
Cobalt
P
opper
Copper
Uibromochlorometiane
Uibromochlorometiane
Uibromome thane
Lsibrom om ethane
Uichlorodifuorome thane
Lsichlorodifuorom ethane
Diethyl Ether
Lthyl Me thacryla te
Lthylbenzene
Lthylbenzene
H 2
ardness
Method
5022
5242
502.2
5242
502.2
524.2
502.2
524.2
1620
200.8
502.2
524.2
502.2
502.2
524.2
1620
200.8
1620
200.8
502.2
524.2
502.2
524.2
502.2
524.2
524.2
524.2
502.2
524.2
130 2
Procedure
PID
ELCD
ELCD
ELCD
ELCD
ELCD
PID
ELCD
ELCD
ELCD
PID
SL-IDE
(16)
0 064
0 133
2.598
0 395
0.032
0.225
0.250
0.253
0.496
0.408
0.055
0.234
0.074
0.082
0.173
16.463
0.074
21 189
0.798
0.436
0.287
0.460
0.388
0.240
0.560
0.376
0.273
0.078
0.198
2 258
SL-IDE (5)
0 059
0 034
0.096
0 303
0.008
0.104
0.520
0.150
0.759
0.491
0.039
0.201
0.024
0.111
0.119
12.267
0.001
15 897
0.905
0.394
0.203
0.298
0.439
1.225
0.591
0.330
0.259
0.050
0.107
4 886
Percent
Difference
-7 8%
-118 1%
-185.7%
-26 3%
-117.3%
-73.4%
70.3%
-51.2%
41.8%
18.5%
-35.0%
-15.2%
-102.4%
30.2%
-37.1%
-29.2%
-195.2%
-28 5%
12.6%
-10.1%
-34.3%
-42.8%
12.5%
134.5%
5.4%
-12.9%
-5.2%
-44.2%
-59.5%
73 6%
SL-IDE 16
Model
Constant
Exponential
Exponential
txponential
txponential
txponential
Linear
txponential
txponential
txponential
txponential
txponential
txponential
Constant
Constant
Linear
txponential
Linear
txponential
txponential
txponential
txponential
txponential
txponential
txponential
E
xponential
SL-IDE 5
Model
tinear
tinear
txponential
Constant
txponential
Constant
Constant
txponential
txponential
txponential
txponential
txponential
txponential
txponential
Constant
Constant
txponential
Constant
txponential
Constant
txponential
txponential
txponential
txponential
txponential
B-57
-------
Table 8. Comparison of 16-point and 5-point
Single-laboratory IDEs (SL-IDEs) for the Episode 6000 Dataset
(ng/L except where footnoted)
Analyte
Hexachloro butadiene
M
M
M e xchl o b uta d i en e + naphthalene
Iron
I sopropyl benzene
I sop ropyl benzene
Lead
Lead
M + p Xylene
M + p Xylene
Magnesium
M
anganese
M
anganese
IVIercury
Methacrylon itrile
Methyl Iodide
Methyl I ert-butyl Lther
Methyla crylate
Methylene Chloride
Methylene Chloride
Methylm ethacryl ate
M
olybdenum
Molybdenum
N- butyl benzene
N
butylbenzene
N
propyl benzene
N
propyl benzene
IMa phthalene
Nickel
Method
502.2
5242
5242
502.2
1620
502.2
524.2
1620
200.8
502.2
524.2
1620
1620
200 8
200.8
524.2
524.2
524.2
524.2
502.2
524.2
524.2
1620
200.8
502.2
524 2
502 2
524 2
524.2
1620
Procedure
ELCD
PID
PID
PID
ELCD
PID
PID
SL-IDE
(16)
0.094
0 308
0 288
0.597
373 590
0.060
0.120
2.423
0.204
0.121
0.142
105.998
6 808
0 109
0.027
0.718
0.193
0.225
0.601
2.841
0.314
0.535
3 034
0.271
0.152
0 092
25 560
0 083
0.141
0.284
SL-IDE (5)
0.073
0 237
0 260
0.592
1064 987
0.041
0.037
2.951
2.872
0.119
0.031
184.221
4 548
0 077
0.014
0.552
0.109
0.173
0.569
-1.381
0.158
0.382
6 028
0.006
0.056
0 105
41 908
0 070
0.052
0.052
Percent
Difference
-24.8%
-26 0%
-10 1%
-1.0%
96 1%
-37.0%
-104.7%
19.6%
173.5%
-1.2%
-127.3%
53.9%
-39 8%
-34 7%
-63.8%
-26.2%
-55.5%
-26.3%
-5.5%
-578.5%
-66.1%
-33.3%
66 1%
-191.8%
-93.0%
139%
48 5%
-16 1%
-91.4%
-137.6%
SL-IDE 16
Model
Exponential
Exponential
.
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
Constant
Exponential
Exponential
r
xponentia
Constant
Lxponential
E
xponentia
E
xponentia
E
xponentia
Lxponential
Lxponential
SL-IDE 5
Model
Linear
Constant
Exponential
Exponential
Constant
Constant
Constant
Exponential
Constant
Hybrid
Hybrid
Exponential
Exponential
Exponential
Constant
Exponential
Linear
Constant
Lxponential
Linear
Hybrid
B-58
-------
Table 8. Comparison of 16-point and 5-point
Single-laboratory IDEs (SL-IDEs) for the Episode 6000 Dataset
(ng/L except where footnoted)
Analyte
y
o'xylene + styrene
r -isoproptol + 1 ,4'dcb
p
entachloroefiane
Oeo~ butyl benzene
Oec~ butyl benzene
Oelenium
Selenium
Silver
Silver
Oodium
Otyrene
1 erfbutybenzene
1 ert'butybenzene
1 etrachloroetiene
1 etrachloroetiene
1 etrachloroetiene
Thallium
Thallium
1 horium
Tin
1 Itanium
1 oluene
1 oluene
1 otal Phosphorus
1 otal Ouspended Oolids
T 12
'
I rans~ I r£~dichloroethene
I rans'l ,O~dichloropropene
Method
2008
5242
502.2
502.2
524 2
502.2
524.2
1620
200.8
1620
200 8
1620
524.2
502.2
524.2
502.2
502.2
524.2
1620
200.8
200.8
1620
1620
502.2
524.2
365.2
160.2
5022
524.2
502.2
Procedure
PID
PID
PID
PID
ELCD
PID
PID
ELCD
ELCD
SL-IDE
(16)
0 186
0 198
0.116
0.408
0 159
0.081
0.140
1.975
0.416
10.668
0 012
138.768
0.141
0.074
0.186
0.061
0.156
0.469
1.153
0.001
0.001
3.932
5.376
0.064
0.146
0.013
3.005
0 081
0.300
0.098
SL-IDE (5)
0 194
0 082
0.151
0.437
0 150
0.057
0.040
1.801
0.342
11.589
-0 084
140.860
0.048
0.051
0.057
0.054
0.103
0.550
1.249
0.000
0.000
4.651
20.828
0.064
0.558
0.011
2.370
0 066
0.075
0.033
Percent
Difference
4 1%
-82 9%
26.8%
7.0%
-5 8%
-35.3%
-111.6%
-9.2%
-19.5%
8.3%
269 8%
1.5%
-98.2%
-35.9%
-106.6%
-11.0%
-40.6%
15.9%
8.0%
-76.1%
-93.4%
16.8%
117.9%
-1.3%
117.1%
-18.1%
-23.6%
-21 7%
-119.7%
-98.9%
SL-IDE 16
Model
Exponential
Exponential
r
xponentia
Exponential
Exponential
Exponential
Exponential
Exponential
r
xponentia
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
Exponential
E
xponen la
Exponential
Exponential
SL-IDE 5
Model
Constant
Linear
Exponential
Exponential
Exponential
Exponential
Constant
Constant1
Exponential
Exponential
Exponential
Exponential
Exponential
Linear
Linear
Linear
Lxponential
Constant
Exponential
Constant
Constant
r 1
Ifonsta nt
Exponential
Exponential
Linear
Hybrid
Exponential
B-59
-------
Table 8. Comparison of 16-point and 5-point
Single-laboratory IDEs (SL-IDEs) for the Episode 6000 Dataset
(ng/L except where footnoted)
Analyte
Trans-1 3'dichloro
' P P
Trans-1 3'dichloro
' P P
Tricnioroethene
I richloroethene
1 richloroethene
T
nc oro uoromet ane
-p
nc oro uoromet ane
Uranium
Vanadium
Vanadium
Vinyl Chloride
Vinyl Chloride
WAD Cyanide
Xylene (1 o tal)
Yttrium
Zinc
Zinc
Method
5022
5242
5242
5022
502.2
524.2
5022
5242
200.8
1620
200.8
502.2
524.2
1677
524.2
1620
1620
200 8
Procedure
PID
ELCD
PID
ELCD
ELCD
SL-IDE
(16)
0 092
0 223
1 250
0 059
0.097
0.332
2 079
0 384
0.000
10.630
0.864
3.672
0.365
0 701
0.128
3 247
4 500
1 598
SL-IDE (5)
0 116
0 132
1 448
0 020
0.089
0.344
0 688
0 384
0.000
9.082
1.023
0.387
0.188
1 296
0.029
13 972
6 943
5 245
Percent
Difference
22 7%
-51 1%
147%
-99 6%
-8.5%
3.6%
-100 5%
0 1%
-70.8%
-15.7%
16.9%
-161.9%
-63.8%
59 6%
-126.9%
124 6%
42 7%
106 6%
SL-IDE 16
Model
Exponential
Exponential
p
onstant
r
xponential
txponential
txponential
txponential
Constant
txponential
•
txponential
r
xponentia
r
xponentia
E
xponentia
SL-IDE 5
Model
txponential
tinear
p
onstant
r
xponential
txponential
txponential
tinear
Linear
Linear
Lxponential
Note! ELCD or PID In the Procedure column ir
(ELCD) m EPA Me,nod 502.2
Original model picked was Hybrid, but failed to
Results reported as mg/t
r(PID).
converge
B-60
-------
Summary Statistics for Table 8
Number of Analytes
Minimum:
25th percentile:
Median:
75th percentile:
Maximum:
SL-IDE(16)vs.
SL-IDE (5)
(all analytes)
198
-578.5%
-79.5%
-24.9%
12.8%
269.8%
SL-IDE(16)vs.SL-IDE(5)
(same model used)
108
-578.5%
-80.1%
-35.6%
-9.3%
53.5%
SL-IDE(16)vs.SL-IDE(5)
(all analytes)
SL-IDE(16)vs.SL-IDE(5)
(same model used)
SL-IDE(16)vs.SL-IDE(5)
(different models used)
Number of
analytes
198
108
90
Median %
Difference
-24.9%
-35.6%
1.3%
Sign Test p-
value
<0.0001
<0.0001
>0.999
SL-IDE(16)vs.
SL-IDE (5)
(different models
used)
90
-195.2%
-72.2%
1.3%
55.5%
269.8%
Wilcoxon
p-value
<0.0001
<0.0001
0.847
B-61
-------
Table 9. Comparison of 16-point and 5-point
Single-laboratory IQEs at 10% RSD (SL-IQEs 10%) for the Episode 6000 Dataset
(M9/L except where footnoted)
Analyte
1 ,1 ,1 ,£netrachloroethane
1 ,1 ,1 ,£netrachloroethane
1 ,1 ,1 "Irichloroelhane
1 ,1 ,1 -Irichloroelhane
1,1,2,2-,0,+1,2,3-top
1 ,1 ,£,£netrachloroethane
1 ,1 ,£~1richloroe1hane
, ~dichk> roe thane
, ~dichk> roe thane
, -dichloroethene
, -dichloroethene
, -dichloropropanone
, -dichloropropene
, £., O ~ 1r i ch 1 or o benzene
, £., O ~ 1r i ch 1 or o benzene
1 ,£,O-1richlorobenzene
1 ,£,O-1richloropropane
1 ,Z,4-1richlorobenzene
1 ,2,4-trimethylbenzene
194
,£-,** trimethylbenzene
19 "3
' Jll-»l «JIH«J O <-lllroethane
1 ,£~dichloropropane
1 ,£~dichloropropane
IOC ,A
I,O,
-------
Table 9. Comparison of 16-point and 5-point
Single-laboratory IQEs at 10% RSD (SL-IQEs 10%) for the Episode 6000 Dataset
(M9/L except where footnoted)
Analyte
£~chloro toluene
L. ~ ch 1 or o toluene
£~hexanone
£~nitropropane
T1 ~ ch 1 or o toluene
T1 ~ ch 1 or o toluene
4-isopropyltoluene
4-m e thy \~£. -pen ta none
Acetone
Acrylonitrile
Ally! Chloride
Aluminum
Aluminum
Ammonia as Nitrogen
Antimony
Tntimony
Trsenic
Trsenic
arium
arium
enzene
enzene
eryllium
eryllium
oron
romobenzene
romobenzene
romobenzene
romochlorom ethane
romochlorom ethane
romodichlorometiane
romodichlorometiane
romoform
romoform
romom ethane
romom ethane
Cadmium
p . .
i-faamium
Calcium
Carbon Disulfide
Isarbon 1 etrachloride
P -1-1 1
Uarbontet , dcp
p
Lfhloroace tonitrile
Lfhlorobenzene
If hi oro benzene
If hi oro benzene
Method
502.2
524.2
524.2
524.2
502.2
524.2
524.2
524.2
524.2
524.2
524.2
1620
200.8
350.3
1620
200.8
1620
200.8
1620
200.8
502.2
524.2
1620
200.8
1620
502.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
1620
200 8
1620
524.2
524.2
502 2
524 2
502.2
502.2
524.2
Procedure
PID
ELCD
ICP/MS
ICP/MS
ICP/MS
ICP/MS
PID
ICP/MS
ELCD
PID
ELCD
ELCD
ELCD
ELCD
ICP/MS
ELCD
ELCD
PID
SL-
IQE10%
(16)
0.849
0.053
0.442
0.590
0.1421
23.810
0.016
1.785
2.741
28.056
29.674
464.069
29.684
0.035
9.551
0.034
3.097
0.798
4.118
0.211
0.182
0.044
0.980
0.044
51.134
3.529
0.100
0.140
1.598
0.368
0.424
0.128
3.393
0.482
16.351
0.226
0.410
0 063
99.975
0.101
0.140
0 069
3 310
1.766
0.119
0.059
SL-
IQE10%
(5)
0.806
0.044
61.796
17.783
0.485
0.837
1.194
14.514
59.415
19.275
0.164
144.530
47.196
0.082
8.364s
0.633
4.656
0.847
3.334
0.153
0.130
0.029
0.985
0.036
46.392
29.488
0.057
0.187
0.057
0.592
0.465
0.111
0.068
0.406
2.195
0.412
0.400
0 033
109.600
0.268
0.520
1 553
31 753
1.558
0.034J
0.831
Percent
Difference
-5.2%
-19.1%
197.2%
187.2%
109.4%
-186.4%
194.6%
156.2%
182.4%
-37.1%
-197.8%
-105.0%
45.6%
78.8%
-3.6%
179.8%
40.2%
6.1%
-21.1%
-32.1%
-33.2%
-41.0%
0.6%
-19.9%
-9.7%
157.2%
-55.4%
28.7%
-186.1%
46.5%
9.1%
-13.8%
-192.1%
-17.1%
-152.7%
58.4%
-2.6%
-63 4%
9.2%
90.3%
115.1%
183 1%
162 2%
-12.5%
-110.6%
173.3%
SL-IQE
Model (16)
Hybrid
Hybrid
Hybrid
Hybrid
Hybrid
Constant
Hybrid
Hybrid
Hybrid
Constant
Constant
Constant
Hybrid
Hybrid
Constant
Hybrid
Hybrid
Hybrid
Constant
Linear
Linear
Hybrid
Hybrid
Hybrid
Linear
Linear
Linear
Hybrid
Linear
Hybrid
Linear
Hybrid
Constant
Hybrid
Constant
Hybrid
Hybrid
|_|
Hybrid
Linear
Hybrid
Hybrid
|_|
y n
|_|
y n
Linear
Hybrid
Hybrid
SL-IQE
Model (5)
Constant
Linear
Constant
Constant
Linear
Constant
Constant
Constant
Lf onstant
Constant
Hybrid
Lf onstant
Lf onstant
Lf onstant
Lf onstant
Lf onstant
Lf onstant
Hybrid
Constant
Constant
Linear
Linear
Linear
Constant
Hybrid
Linear
Hybrid
Hybrid
Hybrid
Hybrid
Constant
Linear
Linear
Hybrid
Hybrid
Linear
Linear
P t .
Constant
Linear
Linear
Constant
Linear
Constant
B-63
-------
Table 9. Comparison of 16-point and 5-point
Single-laboratory IQEs at 10% RSD (SL-IQEs 10%) for the Episode 6000 Dataset
(M9/L except where footnoted)
Analyte
H
L* hi oroe thane
Lf hi oroe thane
Lfhloroform
Lfhloroform
Lf hi or o methane
Lf hi or o methane
Lhromium
Chromium
Cis-1 ,2-dce + 2,2-dcp
Cis'l ,£~dichloroethene
Cis'l ,O~dichloropropene
Cis'l ,O~dichloropropene
Cis'l ,O~dichloropropene
Cobalt
Cobalt
Lop per
Lfopper
L/ibromochlorom ethane
L/ibromochlorom ethane
L/i bromom ethane
L/i bromom ethane
L/i chl or o di fl u or o methane
L/i chl or o di fl u or o methane
L)i ethyl ether
Lthyl methacrylate
Lthyl benzene
Lthyl benzene
Hardness
U
U
rlexachlo roe thane
Iron
1 sopropyl benzene
1 so propyl benzene
Lead
Lead
M + p xylene
M + p xylene
Magnesium
Manganese
IVlanganese
Mercury
Methacrylon itrile
Methyl Iodide
Methyl terf butyl ether
Method
502.2
524.2
502.2
524.2
502.2
524.2
1620
200.8
502.2
524.2
502.2
502.2
524.2
1620
200.8
1620
200.8
502.2
524.2
502.2
524.2
502.2
524.2
524.2
524.2
502.2
524.2
130.2
5022
5242
524.2
RO? ?
1620
502.2
524.2
1620
200.8
502.2
524.2
1620
1620
200.8
200.8
524.2
524.2
524.2
Procedure
ELCD
ELCD
ELCD
ICP/MS
ELCD
ELCD
PID
ICP/MS
ICP/MS
ELCD
ELCD
ELCD
PID
ELCD
pin
PID
ICP/MS
PID
ICP/MS
ICP/MS
SL-
IQE10%
(16)
5.826
0.255
0.025
0.121
1.734
0.141
1.259
1.028
0.039
0.144
0.415
0.0171
0.141
40.837
N/A"
47.509
1.825
1.252
0.288
1.395
0.460
1.091b
0.480
0.404
0.183
0.157
0.077
5.465
0 243
0 228
0.167
1 54?
996.565 b
0.129
25.592
5.698
0.685
0.222
24.651
267.199
15.264
0.245
0.039
19.062
0.083
0.122
SL-
IQE10%
(5)
0.644
0.207
0.033
0.092
1.049
0.191
1.558
1.022
1.055
0.151
0.4476
0.226
0.085
25.933
0.001
32.643
1.885
0.809
0.167
0.587
0.498
2.470
0.442
0.525
0.141
0.007J
0.064
10.032
0 582
0 232
0.386
1 193
2186.832
0.032
1.157
6.059
5.983
0.240
0.034
378.277
9.339
0.160
0.017 '
1.111
3.681
15.132b
Percent
Difference
-160.2%
-20.8%
26.1%
-27.7%
-49.2%
30.4%
21.2%
-0.6%
185.7%
4.9%
7.4%
172.0%
-49.3%
-44.6%
0.0%
-37.1%
3.2%
-43.0%
-53.2%
-81.6%
7.9%
77.4%
-8.1%
26.0%
-26.0%
-182.9%
-19.2%
58.9%
82 2%
1 7%
78.9%
-25 6%
74.8%
-120.6%
-182.7%
6.1%
158.9%
7.6%
-199.4%
34.4%
-48.2%
-41.8%
-79.4%
-178.0%
191.1%
196.8%
SL-IQE
Model (16)
Constant
Hybrid
Linear
Hybrid
Linear
Hybrid
Linear
Linear
Hybrid
Hybrid
Linear
Hybrid
Hybrid
Linear
Linear
Constant
Constant
Linear
Hybrid
Linear
Hybrid
Linear
Hybrid
Hybrid
Hybrid
Hybrid
Hybrid
Linear
H
Hybrid
H
Hybrid
Hybrid
Linear
Linear
Constant
Linear
Linear
Hybrid
Constant
Linear
Constant
Constant
Hybrid
Constant
Hybrid
Hybrid
SL-IQE
Model (5)
Linear
Hybrid
Linear
Linear
Constant
Linear
Constant
Constant
Lf onstant
Hybrid
Lf onstant
Llnea,
Llnea,
Llnea,
Hybrid
Lf onstant
Constant
Constant
Hybrid
Constant
Hybrid
Constant
Hybrid
Hybrid
Llnea,
Linear
Llnea,
Lf onstant
Linoa,
Linoa,
Linear
Constant
Linear
Lf onstant
Lf onstant
Lf onstant
Lf onstant
Hybrid
Lf onstant
Constant
Lf onstant
Hybrid
Hybrid
Constant
Constant
B-64
-------
Table 9. Comparison of 16-point and 5-point
Single-laboratory IQEs at 10% RSD (SL-IQEs 10%) for the Episode 6000 Dataset
(M9/L except where footnoted)
Analyte
VI ethyl a crylate
Methylene Chloride
Methylene Chloride
Methylm ethacryl ate
Molybdenum
Molybdenum
IM "butyl benzene
IM "butyl benzene
IM-propyl benzene
IM'propyl benzene
Naphthalene
Nickel
Nickel
U'xylene
L)-xylene + styrene
P-isoproptol + 1 ,4-dcb
r entachloroetiane
Oec~ butyl benzene
Oec~ butyl benzene
Selenium
Selenium
Silver
Silver
O odium
Otyrene
I erfbutylbenzene
I ert'butylbenzene
I etrachloroetiene
I etrachloroetiene
T-
1 etrachloroetiene
Thallium
Thallium
1 horium
Tin
1 Itanium
1 oluene
1 oluene
r D z
1 otal r hosphorus
r 12
r 12
I rans~1 ,O~dichloropropene
1 rans~1 ,O~dichloropropene
1 rans~1 ,O~dichloropropene
frans-1 ,4~dichloro-2-butene
I richloroethene
Method
524.2
502.2
524.2
524.2
1620
200.8
502.2
524.2
502.2
524.2
524.2
1620
200.8
524.2
502.2
502.2
524.2
502.2
524.2
1620
200.8
1620
200.8
1620
524.2
502.2
524.2
502.2
502.2
524 2
1620
200.8
200.8
1620
1620
502.2
524.2
365.2
160.2
502.2
5242
502.2
502.2
524.2
524.2
502.2
Procedure
ELCD
ICP/MS
PID
PID
ICP/MS
PID
PID
PID
ICP/MS
ICP/MS
PID
ELCD
PID
ICP/MS
ICP/MS
PID
ELCD
ELCD
PID
ELCD
SL-
IQE10%
(16)
0.727
6.033
0.433
20.773
7.597
0.608
0.745
0.067
0.186
29.878
0.108
67.206
0.183
0.040
0.181
0.456
0.551
0.157
0.047
5.235
1.045
25.842
0.056
337.755
0.041
0.203
0.073
0.122
0.750
30 554 B
2.799
0.002
0.004
9.406
14.236
0.194
0.046
0.030
6.729
0.191
0 153
0.729
0.175
0.218
30.108
3.169
SL-
IQE10%
(5)
0.853
N/A"
0.293
0.873
11.866
0.012
0.586
1.287
0.212
0.118
0.256
86.054
0.147
0.016
0.305
0.302
1.036
0.754
1.266
4.076
0.707
22.813
N/A"
333.796
0.067
0.111
0.074
0.182
0.385
1 643
2.745
0.001
0.001
9.772
42.768
0.131
1.145b
0.026
6.929
0.081 b
0 171
0.485
0.238
0.101
1.768
1.010
Percent
Difference
16.0%
N/A
-38.5%
-183.9%
43.9%
-192.4%
-24.0%
180.1%
13.0%
-198.4%
81.1%
24.6%
-21.9%
-85.5%
51.0%
-40.8%
61.1%
131.1%
185.5%
-24.9%
-38.6%
-12.5%
N/A
-1.2%
49.3%
-58.9%
1.1%
39.7%
-64.4%
-179 6%
-1.9%
-76.8%
-134.2%
3.8%
100.1%
-39.1%
184.7%
-15.8%
2.9%
-80.6%
113%
-40.2%
30.7%
-73.5%
-177.8%
-103.3%
SL-IQE
Model (16)
Hybrid
Constant
Hybrid
Constant
Linear
Constant
Linear
Hybrid
Hybrid
Constant
Hybrid
Linear
Hybrid
Hybrid
Linear
Linear
Hybrid
Hybrid
Hybrid
Linear
Linear
Linear
Linear
Linear
Hybrid
Linear
Hybrid
Hybrid
Linear
Linear
Linear
Linear
Linear
Linear
Linear
Hybrid
Hybrid
Hybrid
Hybrid
H
y
Linear
Hybrid
Hybrid
Constant
Linear
SL-IQE
Model (5)
Linear
Constant
Linear
Linear
Constant
Constant
Llnea,
Constant
Constant
Hybrid
Hybrid
Constant
Constant
Llnea,
Constant
Constant
Linear
Constant
Constant
Linear
Hybrid
Constant
Linear
Linear
Llnea,
Hybrid
Llnea,
Llnea,
Linear
1
Lineal
Linear
Linear
Constant
Llnea,
Constant
Constant
Constant
Llnea,
Llnea,
Llnea,
H
y
Constant
Llnea,
Hybrid
Hybrid
Constant
B-65
-------
Table 9. Comparison of 16-point and 5-point
Single-laboratory IQEs at 10% RSD (SL-IQEs 10%) for the Episode 6000 Dataset
(M9/L except where footnoted)
Analyte
I richloroetiene
1 richloroetiene
Triomoroluorometnane
Triomoroluorometnane
Uranium
Vanadium
vVnadium
Vinyl Chloride
Vinyl Chloride
Wad Cyanide
XI \
ylene ^totalj
Yttrium
Zino
Zino
Method
502.2
524.2
502.2
524.2
200.8
1620
200.8
502.2
524.2
1677
524.2
1620
1620
200.8
Procedure
PID
ELCD
ICP/MS
ICP/MS
ELCD
WADCN
ICP/MS
SL-
IQE10%
(16)
0.401
0.167
4.662
42.490 6
0.001
24.338
1.933
8.234
0.219
1.624
23.520
8.962
10.452
7.024
SL-
IQE10%
(5)
0.079
1.068
1.355
0.301
0.000
17.798
2.225
3.258
0.652
2.661
0.017
28.689
14.257
10.927
Percent
Difference
-134.4%
145.8%
-109.9%
-197.2%
-69.1%
-31.0%
14.1%
-86.6%
99.2%
48.4%
-199.7%
104.8%
30.8%
43.5%
SL-IQE
Model (16)
Linear
Hybrid
Constant
Constant
Linear
Hybrid
Hybrid
Constant
Hybrid
Linear
Constant
Linear
Hybrid
Linear
SL-IQE
Model (5)
Linear
Linear
Constant
Hybrid
Linear
Linear
Linear
Linear
Linear
Constant
Hybrid
Constant
Constant
Constant
1 IQE 10% undefined, IQE 20% reported
Ke suits reported as mg/L
3 IQE 10% negative, IQE 20% reported
" IQE 10%, IQE 20%, IQE30% aii negativ.
5 IQE 10% and IQE 20% botn nega.ve, IQE 30% reported
Hy
1 U /O
B-66
-------
Summary Statistics for Table 9
Number of Analytes
Minimum:
25th percentile:
Median:
75th percentile:
Maximum:
SL-IQE10
(16)vs.SL-
IQE10(5)
(all
analytes)
195
-19,971.5%
-6,115.2%
-194.6%
4,562.6%
19,715.8%
SL-IQE10(16)vs. SL-
IQE10 (5)
(same model used)
50
-19,237.7%
-7,243.8%
-2,442.7%
576.4%
15724.6%
SL-IQE10(16)vs. SL-
IQE10(5)
(all analytes)
SL-IQE10(16)vs. SL-
IQE10(5)
(same model used)
SL-IQE10(16)vs. SL-
IQE10 (5)
(different models used)
Number of
analytes
195
50
145
Median %
Difference
-194.600
-2,442.7%
613.9%
Sign Test p-
value
0.567
0.015
0.507
SL-IQE10(16)vs.
SL-IQE10(5)
(different models
used)
145
-19,971.5%
-4,927.0%
613.9%
6109.3%
19,715.8%
Wilcoxon
p-value
0.345
0.001
0.606
B-67
-------
Table 10. Comparison of ACIL, USGS and EPA Limits Calculating using USGS Blank and Spiked data
Analyte
Ammonia (FCA)
Ammonia (rL*L*)
Ammonia Low Level (FCC)
Arsenic, Dissolved
Arsenic, Total
Cadmium, Dissolved byGFAA
Cadmium, Total by GFAA
Chromium, Dissolved by GFAA
Chromium, Total byGFAA
Cobalt, Dissolved by GFAA
Cobalt, Total by GFAA
Copper, Dissolved by GFAA
Copper, Total byGFAA
Lead, Dissolved by GFAA
Lead, Total by GFAA
Molybdenum (Wastewater) by GFAA
Nickel, Dissolved by GFAA
Nickel, Total by GFAA
NltrateMtrlte (FCA)
# blanks
52
52
52
26
26
26
26
26
26
26
26
26
26
26
26
25
26
26
26
52
# spikes
24
24
15
24
24
24
24
24
24
24
24
24
24
24
24
24
24
24
24
24
ACIL CRV
Limit
0.022
0.023
0.006
1.068
1.493
0.082
0.075
0.441
0.316
1.287
1.639
0.536
0.684
0.609
0.774
0.906
0.862
0.991
1.765
0.018
% exceeding
0.0%
1.9%
1.9%
3.8%
3.8%
0.0%
0.0%
3.8%
3.8%
0.0%
3.8%
3.8%
0.0%
3.8%
3.8%
0.0%
3.8%
0.0%
0.0%
0.0%
USGS LT-MDL
(adding median)
Limit
0.021
0.011
0.003
1.005
0.829
0.099
0.084
0.466
0.341
1.911
1.053
0.408
0.766
0.861
0.780
0.779
1.098
1.014
0.936
0.009
% exceeding
0.0%
11.5%
21.2%
3.8%
7.7%
0.0%
0.0%
0.0%
3.8%
0.0%
3.8%
3.8%
0.0%
0.0%
3.8%
0.0%
0.0%
0.0%
19.2%
21.2%
USGS LT-MDL (adding
mean)
Limit
0.021
0.011
0.004
1.071
0.825
0.095
0.089
0.475
0.340
1.847
1.093
0.421
0.764
0.857
0.736
0.778
1.082
0.909
1.167
0.010
% exceeding
0.0%
11.5%
7.7%
3.8%
7.7%
0.0%
0.0%
0.0%
3.8%
0.0%
3.8%
3.8%
0.0%
0.0%
3.8%
0.0%
3.8%
0.0%
11.5%
17.3%
EPAMDL
(Randomly selected from
simulated
7-replicate MDLs)
Limit
0.062
0.012
0.006
0.895
1.298
0.121
0.130
0.473
3.540
1.451
1.076
1.102
26.384
0.860
0.678
0.639
1.190
2.568
2.076
0.009
% exceeding
0.0%
9.6%
1.9%
3.8%
3.8%
0.0%
0.0%
0.0%
0.0%
0.0%
3.8%
0.0%
0.0%
0.0%
3.8%
4.0%
0.0%
0.0%
0.0%
21.2%
B-68
-------
Table 10. Comparison of ACIL, USGS and EPA Limits Calculating using USGS Blank and Spiked data
Analyte
NitrateM trite (FCC)
Nitrate/Nitrite Low L»v»i (FCC)
Ni,ri,e (FCC)
Nitrite Low Level (FCC)
Ortnophospnate (I~CC)
Ortnophosphiate Low Level (FCC)
phosphorus, Low Level l~ iltered
P L L F
r hosphorus, Low Level in
Waste water
Oelenium, Dissolved
Oelenium, I otal
Silver Dissolved by GFAA
Sllverjotal by GFAA
TKN/ Ammonia (FCA)
TKN/ Ammonia (FCC)
TKN/ Ammonia (WCA)
Total Phosphorus (FCA)
I otal Phosphorus (i~L*L*)
Total Phosphorus (WCA)
# blanks
52
52
52
52
52
52
52
52
52
26
26
26
26
52
52
52
52
52
52
# spikes
15
24
15
24
24
24
24
24
24
24
24
24
24
24
24
24
24
24
24
ACIL CRV
Limit
0.023
0.007
0.003
0.001
0.022
0.002
0.003
0.003
0.003
1.174
2.123
0.088
0.140
0.070
0.083
0.483
0.021
0.026
0.027
% exceeding
3.8%
0.0%
0.0%
0.0%
3.8%
0.0%
1.9%
0.0%
3.8%
0.0%
3.8%
3.8%
3.8%
0.0%
1.9%
1.9%
3.8%
0.0%
1.9%
USGS LT-MDL
(adding median)
Limit
0.025
0.008
0.002
0.002
0.008
0.000
0.003
0.003
0.004
1.434
1.211
0.159
0.125
0.092
0.056
0.081
0.026
0.025
0.023
% exceeding
1.9%
0.0%
1.9%
0.0%
19.2%
26.9%
0.0%
0.0%
1.9%
0.0%
7.7%
0.0%
3.8%
0.0%
3.8%
1.9%
0.0%
0.0%
1.9%
USGS LT-MDL (adding
mean)
Limit
0.026
0.008
0.002
0.002
0.010
0.000
0.003
0.003
0.004
1.410
1.324
0.158
0.131
0.091
0.059
0.104
0.026
0.025
0.023
% exceeding
1.9%
0.0%
1.9%
0.0%
15.4%
26.9%
0.0%
0.0%
1.9%
0.0%
7.7%
0.0%
3.8%
0.0%
3.8%
1.9%
0.0%
0.0%
1.9%
EPAMDL
(Randomly selected from
simulated
7-replicate MDLs)
Limit
0.019
0.006
0.003
0.002
0.010
0.001
0.003
0.004
0.009
1.334
1.130
0.122
0.196
0.071
0.049
0.071
0.022
0.023
0.021
% exceeding
5.8%
11.5%
0.0%
0.0%
15.4%
0.0%
0.0%
0.0%
0.0%
0.0%
11.5%
0.0%
0.0%
0.0%
7.7%
1.9%
1.9%
0.0%
3.8%
B-69
-------
Summary Statistics for Table 10.
Limit Type
ACIL CRV
USGS LT-
MDL (adding
media nj
USGS LT-
MDL (adding
meanj
EPAMDL
% of Blanks Exceeding Limit for Dataset
Mean
1.9%
4.4%
3.7%
2.9%
Standard Error
0.3%
1.2%
0.9%
0.8%
B-70
-------
Table 11. Comparison of SL-IDEs and MDLs calculated With and Without Outlier Removal,
Episode 6000 Data
(ng/L except where footnoted)
Analyte
, , I ,£~tetrachloroethane
, , I ,£~tetrachloroethane
, ,2,2-tce + 1,2,3-tcp
2 2
, ,£-1richloroetiane
1 ,1 -dichloroethane
1 ,1 -dichloroethane
1 ,1 "dichloroethene
1 ,1 "dichloropropene
1 ,£,O-1richlorobenzene
1 ,£,O-1richlorobenzene
1 ,£,O-1richlorobenzene
1 ,£ ,O-1richloro pro pane
1 ,Z,4-1richlorobenzene
1 ,2,4-lrichlorobenzene
1 ,2, 4-lrimethyl benzene
1 , 2, 4-trimethyl benzene
1 2 d b 3 hi
I r£-dibromoetiane
I r£-dibromoetiane
1 ,/_ -dichloro ben zene
1 ,/_ -dichloro ben zene
1 ,/_-dichloroethane
1 ,/_-dichloroethane
1 ,£ -dichloro pro pane
1 ,£ -dichloro pro pane
1 ,0,b~ trim ethyl benzene
,O -dichloro ben zene
,O -dichloro benzene
,O -dichloro pro pane
,O -dichloro pro pane
,4 -dichloro ben zene
, 4 "dichloro ben zene
C-tC- "dichloro pro pane
9
L. ~ chl or o toluene
L. ~ chl or o toluene
£"hexanone
4 ~ chl or o toluene
4 ~ chl or o toluene
Ally! Chloride
Aluminum
Aluminum
Ammonia as Nitrogen
Method
502.2
524.2
502 2
524 2
502.2
524.2
502.2
502.2
524.2
502.2
524.2
502.2
502.2
524.2
524.2
502.2
502.2
502.2
524.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
524.2
502.2
502.2
502.2
524.2
502.2
524.2
524.2
524 2
502.2
502.2
524.2
502.2
524.2
524.2
1620
200.8
350.3
Procedure
ELCD
ELCD
ELCD
ELCD
ELCD
ELCD
ELCD
PID
ELCD
PID
PID
ELCD
ELCD
ELCD
ELCD
ELCD
PID
ELCD
ELCD
ELCD
PID
ELCD
ICP/MS
Outliers
Kept
0.034
0.244
0 041
0 308
0.179
0.436
0.032
0.083
0.229
0.234
0.287
0.134
0.115
0.275
1.263
0.088
0.124
0.125
0.144
1.749
0.164
0.326
0.065
0.130
0.042
0.258
0.043
0.247
0.135
0.118
0.126
0.047
0.202
0.061
0.140
0.691
0 833
0.175
0.230
0.902
0.149
0.123
0.229
206.975
12.747
0.014
SL-ID
Outliers
Dropped
0.024
0.211
0 038
0 311
0.123
0.296
0.026
0.060
0.187
0.165
0.294
0.066
0.095
0.256
1.046
0.076
0.117
0.107
0.134
1.368
0.146
0.290
0.061
0.133
0.029
0.237
0.031
0.175
0.127
0.073
0.106
0.037
0.182
0.053
0.130
0.630
0 696
0.161
0.143
0.753
0.134
0.114
0.213
47.299
9.371
0.013
E
Model Used
(Kept/Dropped)
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
C/E
E/E
E/E
M
Outliers
Kept
0.041
0.052
0 012
0 055
0.064
0.132
0.024
0.010
0.033
0.038
0.045
0.048
0.057
0.070
7.328
0.022
0.070
0.095
0.012
1.457
0.096
0.127
0.035
0.030
0.017
0.039
0.023
0.056
0.011
0.035
0.093
0.016
0.038
0.026
0.023
2.376
0 417
0.108
0.238
1.316
0.110
0.010
0.032
29.555
19.145
0.010
DL
Outliers
dropped
0.006
0.052
0 012
0 055
0.064
0.132
0.018
0.014
0.033
0.028
0.045
0.021
0.057
0.070
4.014
0.022
0.070
0.095
0.026
1.457
0.095
0.127
0.035
0.025
0.017
0.059
0.029
0.026
0.011
0.014
0.067
0.014
0.038
0.026
0.023
2.376
0 874
0.108
0.086
0.426
0.083
0.010
0.029
19.524
0.839
0.01C
B-71
-------
Table 11. Comparison of SL-IDEs and MDLs calculated With and Without Outlier Removal,
Episode 6000 Data
(ng/L except where footnoted)
Analyte
Antimony
Arsonio
arium
arium
enzene
eryllium
ery Ilium
Dromobenzene
Drom ob enzene
Dromobenzene
Drom ochlorom ethane
Drom odichlorom ethane
Drom odichlorom ethane
Dromoform
Dromoform
Drom om ethane
Cadmium
Cadmium
Calcium
Lsarbon I etrachloride
Uarbontet+1 ,1 "dcp
Lfhlorobenzene
L h 1 or o benzene
L h 1 or o ethane
L h 1 or o ethane
Uhloroform
Lfhlorom ethane
Lfhlorom ethane
Chromium
Chromium
Cis-1 ,2-dce+2,2-dcp
L*is~l ,O~dichloropropene
L*is~l ,O~dichloropropene
L*is~l ,O~dichloropropene
Uobalt
Uobalt
Lf opper
Lf opper
L/ibromochloromethane
L/ibromochloromethane
Uibromom ethane
Dibromometnan,
Uichlorodifuorom ethane
Diethyl Ether
Ethyl Me thacryla te
L thy (benzene
Method
200.8
200.8
1620
200.8
502.2
1620
200.8
502.2
502.2
524.2
502.2
502.2
524.2
502.2
524.2
502.2
1620
200.8
1620
524.2
502.2
502.2
502.2
502.2
524.2
502.2
502.2
524.2
1620
200.8
502.2
502.2
502.2
524.2
1620
200.8
1620
200.8
502.2
524.2
502.2
524.2
502.2
524.2
524.2
502.2
Procedure
ICP/MS
ICP/MS
ICP/MS
PID
ICP/MS
ELCD
PID
ELCD
ELCD
ELCD
ELCD
ICP/MS
ELCD
ELCD
PID
ELCD
ELCD
ELCD
ICP/MS
ELCD
ELCD
PID
ICP/MS
ICP/MS
ELCD
ELCD
ELCD
PID
SL-IDE
Outliers
Kept
0.019
0.366
1.837
0.084
0.079
0.448
0.024
0.765
0.050
0.211
0.482
0.075
0.205
1.513
0.400
7.293
0.191
0.022
41.358
0.314
0.072
0.460
0.064
2.598
0.395
0.032
0.250
0.253
0.496
0.408
0.055
0.074
0.082
0.173
16.463
0.074
21.189
0.798
0.436
0.287
0.460
0.388
0.240
0.376
0.273
0.078
Outliers
Dropped
0.014
0.347
1.441
0.068
0.074
0.430
0.021
0.242
0.046
0.195
0.390
0.065
0.190
1.504
0.363
7.427
0.159
0.022
36.054
0.288
0.068
0.378
0.055
2.357
0.362
0.026
0.150
0.302
0.464
0.207
0.052
0.062
0.138
0.145
15.625
0.074
14.718
0.160
0.413
0.210
0.344
0.319
0.069
0.301
0.246
0.073
Model Used
(Kept/Dropped)
E/E
E/E
C/C
E/E
E/E
E/E
E/E
L/E
E/E
E/E
L/L
E/E
E/E
C/C
E/E
C/C
E/E
E/E
L/L
E/E
E/E
L/L
E/E
C/C
E/E
E/E
E/E
E/E
E/E
L/E
E/E
E/E
E/E
E/E
E/E
C/C
C/C
C/E
L/L
E/E
L/L
E/E
E/E
E/E
E/E
E/E
MDL
Outliers
Kept
0.178
0.226
1.702
0.033
0.030
0.528
0.007
0.131
0.012
0.044
0.013
0.004
0.043
0.006
0.123
0.267
0.127
0.004
36.726
0.038
0.029
0.011
0.030
0.108
0.066
0.043
0.070
0.045
0.310
0.073
0.013
0.007
0.057
0.038
9.820
0.001
6.046
0.037
0.009
0.051
0.007
0.102
0.009
0.120
0.045
0.021
Outliers
dropped
0.008
0.226
1.702
0.018
0.030
0.528
0.007
0.131
0.012
0.044
0.013
0.004
0.043
0.006
0.123
0.477
0.127
0.004
36.726
0.038
0.029
0.011
0.026
0.011
0.048
0.043
0.070
0.045
0.310
0.073
0.013
0.007
0.057
0.036
9.820
0.001
6.046
0.037
0.006
0.051
0.007
0.102
0.071
0.120
0.035
0.021
B-72
-------
Table 11. Comparison of SL-IDEs and MDLs calculated With and Without Outlier Removal,
Episode 6000 Data
(ng/L except where footnoted)
Analyte
L thy 1 benzene
PI exa chl or o butadiene
PI exchlo butadiene+ na phthalene
ron
1 sopropyl benzene
1 sopropyl benzene
Lead
Lead
M+p xylene
Magnesium
Manganese
Manganese
Mercury
Methacrylon itrile
Methyla crylate
Methylene Chloride
Methylm ethacryl ate
Molybdenum
Molybdenum
IM- butyl benzene
IM ~ pro pyl benzene
Naphthalene
Nickel
Nickel
L)-xylene + styrene
P-isoproptol + 1 ,4-dcb
r entachloroetiane
Oec~ butyl benzene
Selenium
Silver
Silver
1 erfbutybenzene
1 etrachloroetiene
1 etrachloroetiene
1 etrachloroetiene
Thallium
Thorium
fln
Titanium
Toluene
1 oluene
1 otal Ouspended Oolids
1 rans'l ,£~dichloroethene
1 rans~ 1 rO~dichloropropene
1 rans'l ,O~dichloropropene
1 rans'l ,O~dichloropropene
Method
524.2
502.2
502.2
1620
502.2
524.2
1620
200.8
502.2
1620
1620
200.8
200.8
524.2
524.2
524.2
524.2
1620
200.8
502.2
502.2
524.2
1620
200.8
502.2
502.2
524.2
502.2
200.8
1620
200.8
502.2
502.2
502.2
524.2
200.8
200.8
1620
1620
502.2
524.2
160.2
502.2
502.2
502.2
524.2
Procedure
ELCD
PID
PID
ICP/MS
PID
ICP/MS
ICP/MS
ICP/MS
PID
PID
ICP/MS
PID
PID
PID
ICP/MS
ICP/MS
PID
ELCD
PID
ICP/MS
ICP/MS
PID
ELCD
ELCD
PID
SL-IDE
Outliers
Kept
0.198
0.094
0.597
373.590
0.060
0.120
2.423
0.204
0.121
105.998
6.808
0.109
0.027
0.718
0.601
0.314
0.535
3.034
0.271
0.141
0.092
0.186
25.560
0.083
0.116
0.159
0.408
0.081
0.416
10.668
0.012
0.074
0.061
0.156
0.469
0.001
0.001
3.932
5.376
0.064
0.146
3.005
0.081
0.098
0.092
0.223
Outliers
Dropped
0.184
0.081
0.490
42.840
0.047
0.107
1.855
0.133
0.114
100.489
2.183
0.018
0.024
0.492
0.477
0.279
0.480
2.683
0.027 1
0.105
0.071
0.219
23.853
0.057
0.087
0.131
0.351
0.068
0.324
10.718
0.010
0.082
0.054
0.131
0.393
0.001
0.001
3.700
4.732
0.056
0.136
3.060
0.073
0.083
0.088
0.188
Model Used
(Kept/Dropped)
E/E
E/E
E/E
L/E
E/E
E/E
E/E
E/E
E/E
E/E
C/E
C/E
E/E
E/E
E/E
E/E
E/E
E/E
C/C
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/L
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
E/E
MDL
Outliers
Kept
0.033
0.043
0.649
90.409
0.020
0.011
1.647
0.655
0.090
103.033
6.856
0.031
0.004
0.356
0.220
0.082
0.225
2.455
0.004
0.030
0.040
0.048
20.219
0.146
0.059
0.073
0.553
0.055
0.192
4.907
0.004
0.029
0.018
0.062
0.085
0.000
0.001
3.670
4.777
0.070
0.020
1.170
0.041
0.012
0.058
0.051
Outliers
dropped
0.023
0.043
0.649
19.188
0.020
0.010
1.288
0.131
0.090
103.033
1.176
0.012
0.004
0.336
0.220
0.082
0.225
2.455
0.002
0.083
0.040
0.048
20.219
0.075
0.043
0.054
0.207
0.036
0.192
4.250
0.004
0.029
0.018
0.062
0.027
0.000
0.001
3.670
4.663
0.071
0.018
0.980
0.041
0.012
0.058
0.051
B-73
-------
Table 11. Comparison of SL-IDEs and MDLs calculated With and Without Outlier Removal,
Episode 6000 Data
(ng/L except where footnoted)
Analyte
I richloroetiene
1 richloroetiene
1 richloroetiene
Triomoroluoromethan,
Triomoroluoromethan,
Uranium
Vinyl Chloride
Wad Cyanide
Y.trlum
Zlno
Zlno
1 Q
rvesults reported as mg/L
Method
502.2
502.2
524.2
502.2
524.2
200.8
502.2
1677
1620
1620
200.8
IDE did nc
Procedure
ELCD
PID
ELCD
ICP/MS
ELCD
WADCN
ICP/MS
Outliers
Kept
0.059
0.097
0.332
2.079
0.384
0.000
3.672
0.701
3.247
4.500
1.598
SL-ID
Outliers
Dropped
0.049
0.078
0.333
1.762
0.528
0.000
3.577
0.665
3.078
4.135
1.016
/p
E
Model Used
(Kept/Dropped)
E/E
E/E
E/E
C/C
E/E
E/E
C/C
L/L
E/E
E/E
E/E
\
M
Outliers
Kept
0.012
0.027
0.061
0.108
0.087
0.000
0.270
0.572
1.923
2.597
0.900
DL
Outliers
dropped
0.012
0.027
0.061
0.012
0.087
0.000
0.270
0.550
1.923
2.597
0.585
Summary Statistics for Table 11.
Percent Difference
(Positive if limit with
outliers kept>limit with
outliers removed)
SL-IDE (a,,)
SL-IDE (same model used)
SL-IDE (different model
u..a)
MDL
# Analytes
149
141
8
60
Minimum
-51.6%
-51.6%
-0.5%
-115.4%
25th
Percentile
7.1%
6.9%
93.4%
4.4%
Median
14.3%
13.7%
114.7%
30.2%
75th
Percentile
24.4%
22.2%
135.9%
75.6%
Maximum
164.2%
164.2%
158.9%
183.7%
B-74
-------
Table 12. Comparison of SL-IQEs and MLs calculated With and Without Outlier Removal, Episode 6000
Data (|jg/L exceptwhere footnoted)
Analyte
1 ,1 ,1 ,Znetrachloroethane
1 ,1 ,1 ,Znetrachloroethane
1,1,1 "Irichloroetnane
1,1,1 "Irichloroetnane
1,1,2,2-toe+1,2,3-top
1 ,1 ,£,£-tetrachloroethane
1 ,1 -dichloroethane
1 ,1 -dichloroethane
1 ,1 "dichloroethene
1 ,1 "dichloropropene
1 ,£,O-1richlorobenzene
O 0
O 0
, £,O ~1ri chloropro pa ne
,£,4 ~1ri chloroben zene
, £., 4 ~ trichl or o benzene
1 , 2, 4-trimethyl benzene
1 ,2, 4 -trim ethyl benzene
I ,^~dibromo~O~chloropropane
I ,£~dibromoetiane
1 ,£~dibromoethane
1 ,2-dichlorobenzene
,£~dichlorobenzene
,2-dichtoroe thane
,2-dichtoroe thane
,/_-dichloro pro pane
,/_-dichloro pro pane
1 ,0,b -trim ethyl benzene
1 3-d hlo
I ,O ~di chloroben zene
I ,O~dichloropropa ne
I ,O~dichloropropa ne
1 ,4-dichtorobenzene
1 , 4 ~di chloroben zene
/_,/_-dichloropropane
Z-butanone
L. ~ ch 1 or o toluene
£~chloro toluene
£~hexanone
4-chlorotoluene
4 - chl or o toluene
Ally! Chloride
Aluminum
Aluminum
Ammonia as Nitrogen
Antimony
Method
502.2
524.2
502.2
524.2
502.2
524.2
502 2
502.2
524.2
502.2
524.2
502.2
502 2
524 2
524.2
502.2
502.2
502.2
524.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
524.2
502 2
502.2
502.2
524.2
502.2
524.2
524.2
524.2
502.2
502.2
524.2
502.2
524.2
524.2
1620
200.8
350.3
200.8
Procedure
ELCD
ELCD
ELCD
ELCD
ELCD
ELCD
ELCD
PID
ELCD
PID
PID
ELCD
ELCD
ELCD
ELCD
ELCD
PID
ELCD
ELCD
ELCD
PID
ELCD
ICP/MS
ICP/MS
Outliers
Kept
0.030
0.181
0.830
0.240
5.514
0.569
0 060
0.527
0.115
3.796
0.180
0.851
0 248
0 216
11.316
0.401
0.439
0.653
20.896
71.182b
0.592
0.417
0.183
0.085
0.065
0.222
0.102
0.196
23.744
0 936
0.465
0.054
0.139
0.101
0.078
38.009
0.893
0.493
0.849
0.442
0.142
23.810
29.674
464.069
29.684
0.035
0.034
SL-K3E (
Outliers
Dropped
0.023
0.142
2.207
0.157
5.290 5
0.318
0 030
0.311
25. 620 5
3.827
0.090
0.117
0 190
0 217
5.134
0.226
0.429
0.621
21.013
72.198 b
0.560
0.418
0.114
0.067
0.031
0.168
0.038
0.085
23.877
0 463
0.401
0.059
0.151
0.079
0.077
38.29S
0.534
0.43S
0.77C
0.51£
0.517
23.941
29.866
156.043
31.466
0.032
0.02C
10%)
Model Used
(Kept/Dropped)
H/H
H/H
L/C
H/H
C/C
H/H
L/H
III
H/C
L/L
H/H
L/L
H/H
H/H
L/L
L/L
L/L
L/L
C/C
C/C
L/L
H/H
L/H
H/H
H/H
H/H
L/H
H/H
C/C
L/L
L/L
L/H
H/H
H/H
H/H
C/C
H/H
H/H
H/L
H/H
H/H
C/C
C/C
C/L
H/L
H/H
H/H
IV
Outliers
Kept
0.2
0.2
0.05
0.2
0.2
0.5
0 1
0.05
0.1
0.1
0.2
0.2
02
02
20
0.1
0.2
0.5
0.05
5
0.5
0.5
0.1
0.1
0.05
0.1
0.1
0.2
0.05
0 1
0.2
0.05
0.1
0.1
0.1
10
2
0.5
1
5
0.5
0.05
0.1
100
50
0.05
0.5
IL
Outliers
Dropped
0.02
0.2
0.05
0.2
0.2
0.5
0 05
0.05
0.1
0.1
0.2
0.1
02
02
1C
0.1
0.2
0.5
0.1
t
0.2
0.5
0.1
0.1
0.0!
0.2
0.1
0.1
0.0!
0 0'
0.2
0.0!
0.'
0.1
0.1
1(
I
0.!
o.;
i
o.;
0.0!
0.'
5(
I
0.0!
o.o;
B-75
-------
Table 12. Comparison of SL-IQEs and MLs calculated With and Without Outlier Removal, Episode 6000
Data (|jg/L exceptwhere footnoted)
Analyte
Arsenic
arium
arium
enzene
eryllium
eryllium
Q
Dromo benzene
Dromo benzene
D r om o chl or o methane
D r om o dichl or om ethane
D r om o di chl or om ethane
Bromoform
Bromoform
Bromomethane
Cadmium
Cadmium
Calcium
vsarbon I etrachloride
Carbontet+1 ,1 "dcp
Lfhlorobenzene
Lfhlorobenzene
Lf hi or o ethane
Lf hi or o ethane
L*hloroform
Lfhlorom ethane
C hi or o methane
Chromium
Chromium
Cis-1 ,2-dce+2,2-dcp
Lfis~ I rO~dichloropropene
Lfis~ I rO~dichloropropene
Lfis~ I rO~dichloropropene
Cobalt
Cobalt
Copper
Copper
Uibromochlorom ethane
L/ibromochloromethane
Dibromomethan,
Dibromomethan,
L/i chl or o di fl u or o methane
Diethyl Ether
Lthyl Me thacryla te
L thy (benzene
Lthyl benzene
Hexaohlorobutadten,
Method
200.8
1620
200.8
502.2
1620
200.8
502.2
502.2
524.2
502.2
502.2
524.2
502.2
524.2
502.2
1620
200.8
1620
524.2
502.2
502.2
502.2
502.2
524.2
502.2
502.2
524.2
1620
200.8
502.2
502.2
502.2
524.2
1620
200.8
1620
200.8
502.2
524.2
502.2
524.2
502.2
524.2
524.2
502.2
524.2
502.2
Procedure
ICP/MS
ICP/MS
PID
ICP/MS
ELCD
PID
ELCD
ELCD
ELCD
ELCD
ICP/MS
ELCD
ELCD
PID
ELCD
ELCD
ELCD
ICP/MS
ELCD
ELCD
PID
ICP/MS
ICP/MS
ELCD
ELCD
ELCD
PID
ELCD
Outliers
Kept
0.798
4.118
0.211
0.182
0.980
0.044
3.529
0.100
0.140
1.598
0.424
0.128
3.393
0.482
16.351
0.410
0.063
99.975
0.140
0.069
1.766
0.119
5.826
0.255
0.025
1.734
0.141
1.259
1.028
0.039
0.415
0.017 n
0.141
40.837
N/A3
47.509
1.825
1.252
0.288
1.395
0.460
1.091"
0.404
0.183
0.157
0.077
0.243
SL-K3E (
Outliers
Dropped
0.747
3.231
0.191
0.149
0.975
0.038
0.594
0.022
0.143
1.344
0.323
0.131
3.350
0.484
16.541
0.422
0.068
88.075
0.061
4.481
1.514
0.100
5.285
0.202
0.006
0.766
0.187
1.072
0.636
0.038
0.131
0.262
0.070
39.614
N/A3
33.000
1.706
1.189
0.177
1.099
0.473
5.023
0.400
0.109
0.149
0.047
0.194
10%)
Model Used
(Kept/Dropped)
H/H
C/C
L/L
L/H
H/H
H/H
L/H
L/L
H/H
L/L
L/L
H/H
C/C
H/H
C/C
H/L
H/H
L/L
H/H
H/C
L/L
H/H
C/C
H/H
L/H
L/L
H/H
L/L
L/L
H/H
L/H
H/H
H/H
L/L
N/A
C/C
C/C
L/L
H/H
L/L
H/H
L/C
H/H
H/H
H/H
H/H
H/H
IV
Outliers
Kept
1
5
0.1
0.1
2
0.02
0.5
0.05
0.2
0.05
0.02
0.2
0.02
0.5
1
0.5
0.02
100
0.1
0.1
0.05
0.1
0.5
0.2
0.2
0.2
0.2
1
0.2
0.05
0.02
0.2
0.1
50
0.005
20
0.1
0.02
0.2
0.02
0.5
0.02
0.5
0.2
0.1
0.1
0.2
IL
Outliers
Dropped
1
5
0.05
0.1
2
0.02
0.5
0.05
0.2
0.05
0.02
0.2
0.02
0.5
2
0.5
0.02
100
0.1
0.1
0.05
0.1
0.05
0.2
0.2
0.2
0.2
0.2
0.05
0.02
0.2
0.1
5C
0.00!
2C
0.1
o.o;
0.2
o.o;
0.5
0.2
0.5
0.1
0.1
0.1
o.;
B-76
-------
Table 12. Comparison of SL-IQEs and MLs calculated With and Without Outlier Removal, Episode 6000
Data (|jg/L exceptwhere footnoted)
Analyte
H +
ron
sopropyl benzene
sopropyl benzene
Lead
Lead
M + p xylene
Magnesium
Manganese
Manganese
Mercury
Methacrylon itrile
Methyla cry late
Methylene Chloride
Methylm etna cry 1 ate
Molybdenum
Molybdenum
N "butyl benzene
IM ~ pro pyl benzene
Naphthalene
Nickel
Nickel
r -isoproptol+1 ,4 ~dcb
r entachloroetiane
Oec~ butyl benzene
Selenium
Silver
Silver
1 erfbutybenzene
1 etrachloroetiene
1 etrachloroetiene
1 etrachloroetiene
Thallium
Thorium
Tin
1 Itanium
1 oluene
1 oluene
1 otal Ousp ended Oolids
1 rans'l ,£~dichloroethene
1 rans'l ,O~dichloropropene
1 rans'l ,O~dichloropropene
1 rans'l ,O~dichloropropene
1 richloroethene
1 richloroethene
1 richloroetiene
Method
502.2
1620
502.2
524.2
1620
200.8
502.2
1620
1620
200.8
200.8
524.2
524.2
524.2
524.2
1620
200.8
502.2
502.2
524.2
1620
200.8
502.2
502.2
524.2
502.2
200.8
1620
200.8
502.2
502.2
502.2
524.2
200.8
200.8
1620
1620
502.2
524.2
160.2
502.2
502.2
502.2
524.2
502.2
502.2
524.2
Procedure
PID
PID
ICP/MS
PID
ICP/MS
ICP/MS
ICP/MS
PID
PID
ICP/MS
PID
PID
PID
ICP/MS
ICP/MS
PID
ELCD
PID
ICP/MS
ICP/MS
PID
ELCD
ELCD
PID
ELCD
PID
Outliers
Kept
1.542
996.565"
0.129
25.592
5.698
0.685
0.222
267.199
15.264
0.245
0.039
19.062
0.727
0.433
20.773
7.597
0.608
0.745
0.186
0.108
67.206
0.183
0.181
0.456
0.551
0.157
1.045
25.842
0.056
0.203
0.122
0.750
30.554 b
0.002
0.004
9.406
14.236
0.194
0.046
6.729
0.191
0.729
0.175
0.218
3.169
0.401
0.167
SL-K3E (
Outliers
Dropped
1.216
151.265
1.928
25.726
4.449
0.281
0.217
259.424
5.629
0.071
0.033
19.451
0.586
0.390
20.951
6.737
0.011
0.397
0.128
0.166
58.049
0.116
0.140
0.330
0.406
0.101
0.607
25.005
0.027
0.121
0.092
0.664
0.275
0.002
0.001
8.651
13.166
0.084
0.039
7.441
0.159
0.610
0.173
0.124
0.041 '
0.332
0.237
10%)
Model Used
(Kept/Dropped)
H/H
L/H
L/C
C/C
III
L/H
H/H
III
Cll
Cll
H/H
C/C
H/H
H/H
C/C
III
C/H
III
H/H
H/H
III
H/H
L/H
III
H/H
H/H
L/H
III
III
III
H/H
III
C/H
III
L/H
III
III
III
H/H
H/L
H/H
III
H/H
H/H
III
III
H/H
IV
Outliers
Kept
2
200
0.1
0.05
5
2
0.2
500
20
0.1
0.02
1
1
0.2
1
10
0.01
0.1
0.2
0.2
100
0.5
0.2
0.2
2
0.2
0.5
20
0.02
0.1
0.05
0.2
0.2
0.002
0.002
10
20
0.2
0.05
5
0.2
0.05
0.2
0.2
0.05
0.1
0.2
IL
Outliers
Dropped
2
50
0.1
0.05
5
0.5
0.2
500
5
0.05
0.02
1
1
0.2
1
10
0.005
0.2
0.2
0.2
100
0.2
0.2
0.2
1
0.1
0.5
2C
0.02
0.1
0.05
0.2
0.1
o.oo;
o.oo;
1C
2C
0.2
0.0!
I
0.2
0.0!
0.2
0.2
0.0!
0.1
0.2
B-77
-------
Table 12. Comparison of SL-IQEs and MLs calculated With and Without Outlier Removal, Episode 6000
Data (|jg/L exceptwhere footnoted)
Analyte
T
T
Uranium
Vinyl Chloride
Wad Cyanide
Yttrium
Zinc
Zinc
Method
502.2
524.2
200.8
502.2
1677
1620
1620
200.8
Procedure
ELCD
ICP/MS
ELCD
WADCN
ICP/MS
Outliers
Kept
4.662
42.490 5
0.001
8.234
1.624
8.962
10.452
7.024
SL-K3E (
Outliers
Dropped
3.950
0.228
0.001
8.020
1.543
8.501
11.630
2.291
10%)
Model Used
(Kept/Dropped)
C/C
C/H
L/H
C/C
III
III
H/L
L/H
IV
Outliers
Kept
0.5
0.2
0.001
1
2
5
10
2
IL
Outliers
Dropped
0.05
0.2
0.001
1
2
5
10
2
1|QE 10% undefined, IQE 20% reported
2 p /i
3 IQE7o°/0?FQEe20% and IQE 30% all negative based on or
" IQE 10% and IQE 20% botn nega.ve, IQE 30% reported
Hybrid model selected
otconverge, lv< t I U/O based on constant modelinstead
Summary Statistics for Table 12
Percent Difference (Positive if
limit with outliers kept> limit
with outliers removed)
SL-IQE (a,,)
SL-IQE (same model used)
SL-IQE (different model used)
ML
# Analytes
148
117
31
31
Minimum
-198.2%
-176.3%
-198.2%
-163.6%
25th Percentile
1.0%
0.0%
-7.7%
66.7%
Median
16.3%
2.8%
53.1%
66.7%
75th Percentile
50.2%
23.7%
107.1%
120.0%
Maximum
197.9%
194.9%
197.9%
184.6%
B-78
-------
Table 13. Comparison of SL-IDEs calculated
(ng/L except
using different Model Types, Episode 6000 Data
where footnoted)
Analyte
1 ,1 ,1 ,Znetrachtoroethane
1 ,1 ,1 ,Znetrachtoroethane
1,1,1 -irichloroetnane
1,1,1 -irichloroetnane
1,1,2,2-,0,+1,2,3-top
1 ,1 ,£,£netrachtoroethane
1 ,1 ,£~1richloroe»iane
1 ,1 ,£~1richloroe»iane
1 ,1 -dichloroethane
1 ,1 -dichloroethane
1 ,1 "dichloroethene
1 ,1 -dichloroethene
1 , 1 "dichloro pro pa none
1 , 1 "dichloro pro pene
1 ,£ ,O"1richloro ben zene
1 ,£ ,O"1richloro ben zene
1 ,£,O-1richlorobenzene
1 ,^ ,O~1richloro pro pane
1 ,Z,4~1richlorobenzene
1 ,2,4-trichlorobenzene
1 ,Z,4~1richlorobenzene
1 ,Z,4-1rimethyl benzene
1 ,Z,4-1rimethyl benzene
1 2 d b 3 hi
1 ,£-dibromoetiane
1 ,£-dibromoetiane
1 ,£-dichtorobenzene
1 ,£-dichtorobenzene
1 ,£ -dichloro ben zene
1 , ^-dichloroethane
1 , ^-dichloroethane
1 ,£ "dichloro pro pa ne
1 ,£-dichtoropropane
1 ,3,5-1mb + 4-chlorotoluene
1 ,O,b-1rimethyl benzene
1 ,O-dichtorobenzene
1 ,O-dichtorobenzene
1 ,O-dichtorobenzene
1 ,O-dichtoropropane
1 ,O-dichtoropropane
1 ,4-dichtorobenzene
1 ,4-dichtorobenzene
1 "chlorobutane
£,/_ -dichloro pro pane
O
C- ~ chl or o toluene
C- ~ chl or o toluene
C- ~ chl or o toluene
Method
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
524.2
524.2
502.2
502.2
524.2
524.2
502.2
502.2
524.2
502.2
524.2
524.2
502.2
524.2
502.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
502.2
524.2
502.2
524.2
502.2
524.2
524.2
524.2
524.2
502.2
502.2
524.2
Procedure
ELCD
ELCD
ELCD
ELCD
ELCD
ELCD
ELCD
PID
ELCD
PID
PID
ELCD
ELCD
PID
ELCD
ELCD
PID
ELCD
PID
ELCD
ELCD
ELCD
PID
SL-IDE, Based on Given Model
Constant
0.687
11.051
0.985
14.141
2.597
12.456
0.476
7.245
0.801
11.355
1.167
18.473
15.292
13.573
0.942
0.640
18.047
12.464
0.739
0.688
14.387
0.889
9.319
34.167
0.543
8.173
0.653
0.895
12.369
0.951
7.061
0.733
9.388
1.526
10.590
0.775
0.773
12.273
0.578
6.432
0.654
11.443
13.444
17.294
14.170
1.533
0.977
11.146
Linear
0.000
-1.234
0.016
-0.836
-0.222
-1.517
0.016
-0.407
0.083
-0.642
0.305
-2.042
4.713
-0.554
0.117
0.134
-1.759
3.599
0.082
0.113
-1.058
0.125
-0.074
-7.305
0.184
-0.811
0.037
0.136
-1.392
-0.041
-0.485
0.015
-0.729
0.084
-0.059
0.230
0.102
-1.099
0.015
-0.320
0.050
-1.116
-0.406
-0.134
-1.296
0.051
0.272
-0.639
Exponential
0.034
0.244
0.041
0.308
0.179
0.436
0.032
0.319
0.083
0.229
0.234
0.335
6.372
0.287
0.134
0.115
0.275
1.263
0.088
0.124
0.224
0.125
0.144
1.749
0.164
0.326
0.065
0.148
0.130
0.042
0.258
0.043
0.247
0.114
0.135
0.118
0.126
0.143
0.047
0.202
0.061
0.140
0.220
0.691
0.833
0.175
0.230
0.136
Hybrid
0.010
0.078
0.010
0.098
N/A1
0.248
0.016
0.127
0.067
0.049
0.213
0.050
6.513
0.073
0.117
0.083
0.090
0.041
0.069
0.100
0.059
0.108
0.020
N/A1
0.160
0.184
0.045
0.121
0.036
0.022
0.097
0.024
0.085
0.073
0.016
0.103
0.099
0.033
0.028
0.061
0.033
0.034
0.024
0.152
0.384
0.166
0.187
0.023
RSD
184%
166%
183%
166%
123%
160%
169%
158%
140%
167%
96%
168%
58%
167%
125%
109%
168%
129%
135%
112%
168%
123%
169%
128%
71%
158%
151%
117%
170%
157%
161%
173%
164%
160%
170%
103%
121%
170%
164%
163%
152%
169%
169%
161%
153%
146%
90%
170%
B-79
-------
Table 13. Comparison of SL-IDEs calculated
(ng/L except
using different Model Types, Episode 6000 Data
where footnoted)
Analyte
£~hexanone
2 t
4 - chl or o toluene
T1 ~ ohl or o toluene
T~isopropyltoluene
4-methy|-£-pentanone
Acetone
Acrylonitrile
Ally! Chloride
Aluminum
Aluminum
Ammonia as Nitrogen
Tntimony
Tntimony
Arsenic
Arsenic
arium
Barium
enzene
enzene
eryllium
eryllium
oron
romo benzene
romo benzene
romo benzene
O
g
romodichlorometiane
romodichlorometiane
Q
Q
Q
romom ethane
La dm Sum
Ladmium
Lalcium
Carbon Disulfide
Larbon 1 etrachloride
Larbontet+1 ,1 "dcp
Lhloroace tonitrile
Lhlo rob enzene
Lhlo rob enzene
Lhlo rob enzene
Lhlo roe thane
Lhlo roe thane
Lhloroform
Lhloroform
Method
524.2
524.2
502.2
524.2
524.2
524.2
524.2
524.2
524.2
1620
200.8
350.3
1620
200.8
1620
200.8
1620
200.8
502.2
524.2
1620
200.8
1620
502.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
1620
200.8
1620
524.2
524.2
502.2
524.2
502.2
502.2
524.2
502.2
524.2
502.2
524.2
Procedure
ELCD
ICP/MS
ICP/MS
ICP/MS
ICP/MS
PID
ICP/MS
ELCD
PID
ELCD
ELCD
ELCD
ELCD
ICP/MS
ELCD
ELCD
PID
ELCD
ELCD
SL-IDE, Based on Given Model
Constant
22.744
18.337
1.792
10.619
9.108
20.121
22.659
13.467
13.324
206.975
41.919
0.078
4.260
0.229
2.131
2.023
1.837
0.257
0.802
8.619
1.587
0.170
38.617
1.685
0.569
12.851
0.939
8.929
0.617
8.020
1.513
10.207
7.293
12.379
0.364
0.040
54.321
14.835
15.266
1.998
11.548
0.982
0.749
10.276
2.598
14.465
0.732
9.385
Linear
-5.136
-3.854
-0.022
-0.329
0.162
-5.006
-1.723
-1.190
-0.815
88.830
12.689
0.009
3.728
0.027
1.510
0.257
1.522
0.085
0.036
-0.122
0.365
0.013
20.625
0.765
0.028
-1.691
0.482
-0.807
0.111
-0.455
1.161
-1.309
5.796
-1.072
0.208
0.022
41.358
-1.181
-1.197
-0.007
-0.814
0.460
0.020
-0.665
2.161
-0.836
0.006
-0.399
Exponential
0.902
1.082
0.149
0.123
0.117
1.195
2.120
1.333
0.229
51.697
12.747
0.014
3.562
0.019
1.410
0.366
1.300
0.084
0.079
0.125
0.448
0.024
21.161
0.499
0.050
0.211
0.162
0.345
0.075
0.205
0.381
0.400
4.313
0.280
0.191
0.022
37.020
0.239
0.314
0.072
1.569
0.189
0.064
0.133
1.091
0.395
0.032
0.225
Hybrid
0.188
0.254
0.112
0.013
0.007
0.773
1.092
0.715
0.051
N/A1
12.961
0.013
3.596
0.015
1.390
0.345
1.306
0.079
0.060
0.019
0.431
0.018
20.805
0.515
0.032
0.060
0.157
0.161
0.060
0.056
0.381
0.211
N/A'
0.096
0.180
0.026
37.410
0.040
0.056
0.020
1.453
0.183
0.048
0.026
1.053
0.104
0.004
0.051
RSD
161%
156%
140%
170%
192%
150%
141%
139%
168%
70%
73%
114%
9%
144%
22%
114%
17%
69%
152%
169%
83%
134%
35%
65%
157%
168%
85%
159%
125%
165%
66%
159%
26%
166%
37%
31%
19%
168%
167%
162%
119%
83%
160%
169%
45%
165%
185%
166%
B-80
-------
Table 13. Comparison of SL-IDEs calculated
(ng/L except
using different Model Types, Episode 6000 Data
where footnoted)
Analyte
Lfhlorom ethane
Lfhlorom ethane
Chromium
Chromium
Cis-1 ,2-dce+2,2-dcp
Uis-1 ,£~dichloroethene
Lfis~ 1 ,O~dichloropropene
Lfis~ 1 rO~dichloropropene
Lfis~ 1 rO~dichloropropene
IfObalt
IfObalt
Lopper
Lopper
r\
r\
Dibromomethan,
Uibromom ethane
L/ichlorodifluorom ethane
L/ichlorodifluorom ethane
Diethyl Ether
Lthyl Me thacryla te
t thy (benzene
t thy (benzene
Hardness
~lexachloro butadiene
|_|
_|
_| +
ron
sopropyl benzene
sopropyl benzene
_ead
_ead
Vl + p xylene
Vl + p xylene
Vlagnesium
Vlanganese
Vlanganese
Vlercury
Vlethacrylon itrile
VI ethyl Iodide
VI ethyl terf butyl ether
VI ethyl a crylate
Vlethylene Chloride
Vlethylene Chloride
Vlethylm ethacryl ate
Molybdenum
Vlolybdenum
Method
502.2
524.2
1620
200.8
502.2
524.2
502.2
502.2
524.2
1620
200.8
1620
200.8
502.2
524.2
502.2
524.2
502.2
524.2
524.2
524.2
502.2
524.2
130.2
502.2
524.2
524.2
502.2
1620
502.2
524.2
1620
200.8
502.2
524.2
1620
1620
200.8
200.8
524.2
524.2
524.2
524.2
502.2
524.2
524.2
1620
200.8
Procedure
ELCD
ICP/MS
ELCD
ELCD
PID
ICP/MS
ICP/MS
ELCD
ELCD
ELCD
PID
ELCD
PID
PID
ICP/MS
PID
ICP/MS
ICP/MS
ELCD
ICP/MS
SL-IDE, Based on Given Model
Constant
1.130
19.617
1.090
0.672
1.893
11.249
0.716
0.933
7.072
30.100
0.074
21.189
0.798
0.784
8.159
0.836
7.135
2.194
24.275
12.008
10.053
0.888
11.939
3.658
0.997
17.734
18.095
1.442
486.971
0.856
11.414
3.976
1.007
1.701
10.994
145.717
6.808
0.109
0.827
8.883
12.103
10.845
13.820
2.841
8.787
9.597
4.908
0.271
Linear
0.453
-2.484
0.528
0.408
-0.048
-0.960
0.083
0.039
-0.454
16.339
-0.012
16.989
0.404
0.436
-0.667
0.460
-0.585
0.348
-4.798
-1.243
-0.957
0.020
-0.776
2.362
0.105
-2.203
-2.155
0.793
373.590
0.025
-0.141
2.396
0.265
0.005
-0.206
112.074
4.201
0.065
0.006
-0.181
-0.866
-1.117
-1.522
1.822
-0.455
-0.342
3.163
0.096
Exponential
0.250
0.253
0.496
0.284
0.055
0.234
0.074
0.082
0.173
16.463
-0.004
14.754
0.205
0.144
0.287
0.192
0.388
0.240
0.560
0.376
0.273
0.078
0.198
2.258
0.094
0.308
0.288
0.597
125.364
0.060
0.120
2.423
0.204
0.121
0.142
105.998
2.993
0.034
0.027
0.718
0.193
0.225
0.601
-3.178
0.314
0.535
3.034
0.180
Hybrid
0.233
0.056
0.471
0.290
0.012
0.062
0.061
0.013
0.062
16.102
-0.001
14.861
0.207
0.141
0.126
0.184
0.203
0.153
0.183
0.175
0.079
0.060
0.032
2.385
0.065
0.092
0.069
0.523
124.648
0.033
0.012
2.437
0.200
0.088
0.016
106.575
3.033
0.034
0.016
0.356
0.035
0.053
0.315
N/A'
0.188
0.244
3.042
-0.007
RSD
82%
169%
46%
44%
164%
167%
138%
167%
165%
35%
192%
18%
69%
81%
161%
73%
153%
133%
166%
162%
164%
160%
168%
25%
144%
167%
168%
50%
66%
168%
170%
28%
94%
170%
170%
16%
42%
59%
185%
145%
168%
167%
157%
651%
159%
154%
26%
88%
B-81
-------
Table 13. Comparison of SL-IDEs calculated
(ng/L except
using different Model Types, Episode 6000 Data
where footnoted)
Analyte
Naphthalene
N- butyl benzene
N- butyl benzene
Nickel
Nickel
IM'propyl benzene
IM'propyl benzene
o-xylene
o~xylene + styrene
r entachloroetiane
r-isoproptol + 1 ,4'dcb
Oec~ butyl benzene
Oec~ butyl benzene
Oelenium
Oelenium
Sl.ver
Silver
Sodium
Otyrene
1 erf butytoenzene
1 erf butytoenzene
1 etrachloroetiene
1 etrachloroetiene
1 etrachloroetiene
rhallium
1 hallium
1 horium
Tin
1 itanium
Toluene
Toluene
1 otal r hosphorus
1 otal Ouspended Oolids
trans' 1 ^'dichloroetiene
trans' 1 ^'dichloroetiene
trans-13-dieh.oropropene
trans-13-dichloropropene
trans-13-dichloropropene
I richloroethene
I richloroethene
I richloroethene
r
r
Uranium
Vanadium
Vanadium
Vinyl Chloride
Method
524.2
502.2
524.2
1620
200.8
502.2
524.2
524.2
502.2
524.2
502.2
502.2
524.2
1620
200.8
1620
200.8
1620
524.2
502.2
524.2
502.2
502.2
524.2
1620
200.8
200.8
1620
1620
502.2
524.2
365.2
160.2
502.2
524.2
502.2
502.2
524.2
524.2
502.2
502.2
524.2
502.2
524.2
200.8
1620
200.8
502.2
Procedure
PID
ICP/MS
PID
PID
PID
PID
ICP/MS
ICP/MS
PID
ELCD
PID
ICP/MS
ICP/MS
PID
ELCD
ELCD
PID
ELCD
PID
ELCD
ICP/MS
ICP/MS
ELCD
SL-IDE, Based on Given Model
Constant
14.829
0.714
10.237
50.587
1.023
0.785
13.415
11.622
1.372
11.186
1.583
0.942
11.240
4.161
2.090
13.219
0.048
169.136
10.516
0.854
11.706
0.927
1.027
13.627
1.726
0.003
0.032
5.755
8.500
0.731
9.778
0.018
4.317
0.922
13.734
0.666
0.650
6.714
14.301
1.006
0.914
12.510
2.079
19.248
0.002
22.721
2.762
3.672
Linear
-0.891
0.215
-0.145
26.333
0.176
0.075
-0.751
-0.802
0.043
-0.793
0.091
0.053
0.080
2.054
0.406
11.098
0.020
141.290
-0.600
0.038
-0.323
0.029
0.114
-0.451
1.185
0.001
0.002
3.991
6.012
0.044
-0.303
0.014
3.195
0.067
-0.953
0.201
0.052
-0.432
-1.059
0.035
0.066
-0.619
1.656
-2.147
0.000
9.967
0.730
3.036
Exponential
0.186
0.141
0.152
25.560
0.083
0.092
0.284
0.198
0.116
0.408
0.159
0.081
0.140
1.975
0.416
10.668
0.012
138.768
0.141
0.074
0.186
0.061
0.156
0.469
1.153
0.001
0.001
3.932
5.376
0.064
0.146
0.013
3.005
0.081
0.300
0.098
0.092
0.223
1.250
0.059
0.097
0.332
1.107
0.384
0.000
10.630
0.864
1.756
Hybrid
0.044
0.135
0.028
24.898
0.072
0.066
0.061
0.017
0.082
0.237
0.118
0.052
0.020
1.971
0.364
10.801
0.010
140.811
0.017
0.050
0.030
0.031
0.127
N/A'
1.161
0.001
0.000
3.986
5.419
0.051
0.019
0.013
2.977
0.060
0.062
0.087
0.068
0.096
0.782
0.038
0.069
0.065
1.076
N/A'
0.000
10.693
0.840
1.690
RSD
169%
92%
169%
39%
136%
139%
167%
168%
160%
159%
150%
156%
194%
43%
104%
10%
77%
10%
169%
158%
169%
169%
126%
132%
21%
73%
176%
20%
23%
152%
169%
16%
19%
151%
167%
104%
135%
161%
141%
169%
146%
165%
32%
136%
116%
46%
75%
39%
B-82
-------
Table 13. Comparison of SL-IDEs calculated using different Model Types, Episode 6000 Data
(M9/L except where footnoted)
Analyte
Vinyl Chloride
Wad Cyanide
Xylene (total)
Y.trlum
Zlno
Zlno
Method
524.2
1677
524.2
1620
1620
200.8
Procedure
WADCN
ICP/MS
SL-IDE, Based on Given Model
Constant
22.292
1.023
10.490
4.569
14.628
7.561
Linear
-3.345
0.701
-0.264
3.520
3.804
2.537
Exponential
0.365
0.620
0.128
3.247
4.500
1.598
Hybrid
0.083
0.638
0.008
3.279
4.425
1.610
RSD
168%
25%
170%
17%
76%
86%
Hybrid
Results
d to converge
s mg/L
Summary Statistics for Table 13
Method
A..
502.2
524.2
1620
200.8
#Analytes
198
65
81
26
21
Minimum
8.5%
25.7%
58.2%
8.5%
31.0%
25th Percentile
81.8%
103.5%
159.2%
18.1%
72.5%
Median
151.1%
140.1%
166.0%
26.8%
88.0%
75th Percentile
166.7%
159.9%
168.5%
42.4%
134.5%
Maximum
650.6%
650.6%
194.5%
83.0%
191.6%
B-83
-------
Table 14. Comparison of SL-IQEs calculated
(ng/L except
using different Model Types, Episode 6000 Data
where footnoted)
Analyte
112
,1,1 , £. te tr a ch tor o ethane
,1,1 -trichloroetiane
, 1, 1 "Irichloroetnane
,1,2,2-tce+1,2,3-lcp
,1 ,£,£-tetrachk>roethane
, 1 ,£~1richloroetiane
, 1 ,£~1richloroetiane
,1 -dichloroethane
,1 -dichloroethane
,1 -dichloroethene
,1 -dichloroethene
, 1 "dichloro pro pa none
,1 -dichloropropene
, £., O - trichl or o benzene
, £., O - trichl or o benzene
, £., O - trie hi or o benzene
,Z,O-1richloropropane
, £., 4 - trie hi or o benzene
, £., 4 - trichl or o benzene
, £., 4 - trichl or o benzene
,^,4-tri methyl benzene
,^,4-tri methyl benzene
,£~dibromo~O~chloropropane
,£~dibromoethane
,£~dibromoethane
,£~dichk>robenzene
,Z~dichloro benzene
,Z~dichloro benzene
,£~dichloroethane
,£~dichloroethane
,Z~dichloro pro pane
,Z~dichloro pro pane
,O,t)-1m b+4~chloro toluene
,O,t)~1rirn ethyl benzene
,O~dichk>robenzene
,O~dichk>robenzene
,O~dichloro benzene
,O~dichk>ropropane
,O~dichloro pro pane
,4-dichtoro benzene
,4-dichtoro benzene
-chlorobutane
Z,£-dichtoro pro pane
Z'butanone
Z-chlorotoluene
Z-chlorotoluene
£~chlorotoluene
Method
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
524.2
524.2
502.2
502.2
524.2
524.2
502.2
502.2
524.2
502.2
524.2
524.2
502.2
524.2
502.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
502.2
524.2
502.2
524.2
502.2
524.2
524.2
524.2
524.2
502.2
502.2
524.2
Procedure
ELCD
ELCD
ELCD
ELCD
ELCD
ELCD
ELCD
PID
ELCD
PID
PID
ELCD
ELCD
PID
ELCD
ELCD
PID
ELCD
PID
ELCD
ELCD
ELCD
PID
SL-IQE 10%, Based on Given Model
Constant
1.541
24.612
2.208
31.494
5.514
27.377
1.067
15.923
1.795
25.290
2.617
41.142
30.102
30.229
2.113
1.435
40.193
27.394
1.658
1.544
32.041
1.993
20.896
71.182
1.218
17.963
1.465
1.992
27.734
2.132
15.586
1.643
20.909
3.422
23.744
1.738
1.732
27.518
1.287
14.324
1.467
25.657
29.943
38.009
30.407
3.438
2.176
24.990
Linear
0.000
-4.974
0.830
-4.112
-1.416
-5.971
0.060
-1.175
0.527
-2.390
3.796
-28.559
12.705
-2.582
0.851
0.482
-12.045
11.316
0.401
0.439
-5.251
0.653
-0.243
-145.715
0.592
-2.444
0.183
0.638
-6.758
0.266
-1.407
0.102
-2.433
0.396
-0.208
0.936
0.465
-4.866
0.054
-0.934
0.218
-5.226
-1.682
-15.752
-4.569
1.364
1.249
-2.436
Exponential
0.078
0.556
0.096
0.704
0.430
1.001
0.075
0.726
0.200
0.521
0.627
0.767
15.558
0.655
0.334
0.279
0.628
2.981
0.212
0.303
0.510
0.309
0.326
4.217
0.401
0.743
0.154
0.367
0.294
0.100
0.585
0.101
0.562
0.268
0.305
0.289
0.309
0.324
0.110
0.458
0.144
0.316
0.499
1.607
1.934
0.452
0.597
0.308
Hybrid
0.030
0.181
0.058
0.240
N/A2
0.569
0.040
0.290
0.178
0.115
0.886
0.129
15.041
0.180
0.341
0.248
0.216
0.166
0.186
0.276
0.141
0.291
0.048
N/A'
0.381
0.417
0.121
0.346
0.085
0.065
0.222
0.065
0.196
0.189
0.037
0.267
0.288
0.076
0.067
0.139
0.101
0.078
0.060
0.464
0.893
0.493
0.849
0.053
RSDi
182.6%
165.7%
126.0%
165.7%
120.9%
159.1%
162.6%
157.7%
113.2%
166.8%
75.6%
167.7%
43.2%
166.2%
92.1%
91.5%
167.9%
117.0%
114.4%
94.7%
168.0%
99.2%
168.6%
125.6%
60.5%
157.5%
136.6%
93.6%
169.7%
155.8%
160.5%
162.6%
164.1%
147.0%
169.5%
85.9%
99.3%
169.5%
159.5%
162.8%
136.4%
169.3%
168.5%
159.8%
151.2%
97.4%
56.9%
169.5%
B-84
-------
Table 14. Comparison of SL-IQEs calculated
(ng/L except
using different Model Types, Episode 6000 Data
where footnoted)
Analyte
£~hexanone
2 t
4 "chloro toluene
T1 "chloro toluene
4~i sopropyltoluene
4-methy|-£-pentanone
Acetone
Acrylonitrile
Ally! Chloride
Aluminum
Aluminum
Ammonia as Nitrogen4
Antimony
Antimony
Arsenic
Arsenic
Darium
Darium
Denzene
Denzene
Beryllium
Beryllium
Doron
Bromobenzen,
Bromobenzen,
Dromobenzene
Dromochlorom ethane
Dromochlorom ethane
Dromodichlorometiane
Dromodichlorometiane
Bromoform
Bromoform
Bromomethane
Bromomethane
Cadmium
Cadmium
Calcium
Carbon Uisulfide
Carbon 1 etrachloride
Carbontet+1 ,1 "dcp
Chloroace tonitrile
Chloro benzene
Chloro benzene
Lfhlorobenzene
Lf hi or o ethane
Lf hi or o ethane
L*hloroform
L*hloroform
Method
524.2
524.2
502.2
524.2
524.2
524.2
524.2
524.2
524.2
1620
200.8
350.3
1620
200.8
1620
200.8
1620
200.8
502.2
524.2
1620
200.8
1620
502.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
502.2
524.2
1620
200.8
1620
524.2
524.2
502.2
524.2
502.2
502.2
524.2
502.2
524.2
502.2
524.2
Procedure
ELCD
ICP/MS
ICP/MS
ICP/MS
ICP/MS
PID
ICP/MS
ELCD
PID
ELCD
ELCD
ELCD
ELCD
ICP/MS
ELCD
ELCD
PID
ELCD
ELCD
SL-IQE 10%, Based on Given Model
Constant
47.881
38.203
4.017
23.810
20.421
41.919
47.703
28.056
29.674
464.069
93.989
0.175
9.551
0.525
4.705
4.629
4.118
0.589
1.798
19.325
3.559
0.382
86.584
3.704
1.277
28.621
2.106
19.625
1.384
17.863
3.393
22.334
16.351
27.570
0.816
0.090
121.796
33.263
34.000
4.480
24.059
2.202
1.679
23.041
5.826
31.932
1.640
20.902
Linear
-30.174
-16.221
0.161
-1.231
0.528
-23.810
-8.481
-3.845
-3.694
255.899
37.673
0.052
8.719
0.073
3.542
0.692
3.475
0.211
0.182
-0.385
0.964
0.041
51.134
3.529
0.100
-7.963
1.598
-2.531
0.424
-1.404
2.540
-4.327
5.779
-5.134
0.505
0.065
99.975
-7.679
-7.521
0.105
-2.331
1.766
0.092
-2.418
4.368
-4.186
0.025
-1.329
Exponential
2.102
2.531
0.383
0.278
0.265
2.804
5.137
3.129
0.521
130.746
30.404
0.035
8.275
0.044
3.240
0.859
2.973
0.197
0.189
0.284
1.044
0.057
49.514
1.408
0.118
0.479
0.399
0.787
0.178
0.465
0.922
0.914
N/AJ
0.637
0.445
0.054
86.815
0.545
0.718
0.167
3.679
0.477
0.151
0.300
2.730
0.907
0.075
0.511
Hybrid
0.442
0.590
N/A3
0.032
0.016
1.785
2.741
1.651
0.121
N/A2
29.684
0.035
8.104
0.034
3.097
0.798
2.934
0.183
0.155
0.044
0.980
0.044
47.266
1.417
0.079
0.140
0.379
0.368
0.148
0.128
0.877
0.482
N/A'
0.226
0.410
0.063
84.600
0.101
0.140
0.069
3.310
0.458
0.119
0.059
2.613
0.255
0.011
0.121
RSDi
160.2%
153.7%
142.4%
169.9%
189.9%
147.6%
136.5%
135.6%
167.7%
59.4%
64.5%
90.3%
7.5%
140.8%
20.0%
110.3%
16.4%
66.6%
139.7%
168.9%
78.3%
127.8%
31.9%
50.7%
149.8%
167.7%
77.6%
158.8%
108.8%
164.9%
64.3%
157.9%
67.6%
165.3%
34.1%
23.1%
17.4%
168.3%
166.8%
181.2%
114.7%
72.9%
152.8%
169.2%
39.2%
164.1%
183.1%
165.6%
B- 85
-------
Table 14. Comparison of SL-IQEs calculated
(ng/L except
using different Model Types, Episode 6000 Data
where footnoted)
Analyte
L* hi or o methane
L* hi or o methane
Chromium
Chromium
Cis-1 ,2-dce+2,2-dcp
Cis'l ,£~dichloroethene
Lfis~ 1 ,0~dichloropropene
Lfis~ 1 rO~dichloropropene
Lfis~ 1 rO~dichloropropene
Cobalt
Cobalt
Lf opper
Lf opper
Uibromochlorom ethane
Uibromochlorom ethane
Uibromom ethane
Uibromom ethane
Uichlorodifuorom ethane
Uichlorodifuorom ethane
Diethyl Ether
Lthyl Me thacryla te
L thy (benzene
Lthyl benzene
Hardness
rl exa chl or o butadiene
rl exa chl or o butadiene
n exa chl oroe thane
H +
Iron
1 so propyl benzene
1 so propyl benzene
Lead
Lead
M+p xylene
M+p xylene
Magnesium
Manganese
Manganese
Mercury
Methacrylon itrile
Methyl Iodide
Methyl tert-butyl ether
Methyla crylate
Methylene Chloride
Methylene Chloride
Methylm etna cry 1 ate
Molybdenum
Molybdenum
Method
502.2
524.2
1620
200.8
502.2
524.2
502.2
502.2
524.2
1620
200.8
1620
200.8
502.2
524.2
502.2
524.2
502.2
524.2
524.2
524.2
502.2
524.2
130.2
502.2
524.2
524.2
502.2
1620
502.2
524.2
1620
200.8
502.2
524.2
1620
1620
200.8
200.8
524.2
524.2
524.2
524.2
502.2
524.2
524.2
1620
200.8
Procedure
ELCD
ICP/MS
ELCD
ELCD
PID
ICP/MS
ICP/MS
ELCD
ELCD
ELCD
PID
ELCD
PID
PID
ICP/MS
PID
ICP/MS
ICP/MS
ELCD
ICP/MS
SL-IQE 10%, Based on Given Model
Constant
2.533
43.690
2.444
1.538
4.244
25.054
1.604
2.077
15.751
67.490
0.166
47.509
1.825
1.757
18.012
1.874
15.614
4.918
53.352
26.391
22.094
1.991
26.591
8.005
2.236
39.496
40.301
3.234
1091.863
1.919
25.592
8.914
2.305
3.813
24.651
326.719
15.264
0.245
1.854
19.062
26.956
23.940
29.913
6.033
19.701
20.773
11.003
0.608
Linear
1.734
-89.292
1.259
1.028
0.218
-3.865
0.415
0.222
-1.358
40.837
-0.022
39.683
0.984
1.252
-2.066
1.395
-1.663
-0.244
30.938
-4.619
-3.192
0.128
-3.326
5.465
0.753
-21.961
-19.924
2.358
-281.500
0.129
-0.498
5.698
0.685
0.031
-0.743
267.199
10.195
0.156
0.019
-0.518
-3.833
-4.171
-5.560
5.201
-1.528
-1.043
7.597
0.260
Exponential
0.650
0.577
1.141
0.681
0.127
0.532
0.177
0.196
0.391
38.691
-0.009
34.348
0.487
0.349
0.653
0.475
0.885
0.732
1.297
0.860
0.621
0.188
0.450
5.109
0.228
0.703
0.657
1.524
N/AJ
0.141
0.270
5.587
0.478
0.285
0.321
247.396
7.113
0.079
0.063
1.655
0.439
0.511
1.386
-4.095
0.717
1.228
7.049
N/AJ
Hybrid
0.678
0.141
1.062
0.669
0.039
0.144
0.151
N/A3
0.141
36.682
0.002
33.546
0.477
0.330
0.288
0.447
0.460
0.654
0.480
0.404
0.183
0.157
0.077
5.258
0.243
0.228
0.167
1.542
N/AJ
0.088
0.029
5.489
0.462
0.222
0.037
240.982
6.899
0.076
0.039
0.815
0.083
0.122
0.727
N/A'
0.433
0.561
6.869
0.026
RSDi
65.0%
169.0%
44.0%
41.7%
178.0%
166.4%
117.3%
129.7%
164.7%
31.5%
138.6%
16.6%
67.2%
76.3%
160.3%
67.3%
152.6%
116.1%
118.6%
161.4%
164.1%
148.8%
168.2%
23.0%
109.3%
167.2%
168.0%
37.5%
N/A
158.1%
170.2%
25.9%
90.4%
167.3%
169.5%
14.4%
39.5%
57.3%
183.8%
143.5%
168.3%
166.5%
156.1%
10.5%
158.9%
152.7%
23.9%
98.3%
B-86
-------
Table 14. Comparison of SL-IQEs calculated
(ng/L except
using different Model Types, Episode 6000 Data
where footnoted)
Analyte
N -butyl benzene
N -butyl benzene
IM'propyl benzene
IM-propyl benzene
Napthalene
Nickel
Nickel
0-xylene
P -isoproptol + 1 ,4'dcb
r entachloroetiane
Oec~ butyl benzene
Oec~ butyl benzene
Oelenium
Oelenium
Silver
Silver
Sodium
Otyrene
1 ert'butylbenzene
1 ert'butylbenzene
1 etrachloroetiene
1 etrachloroetiene
1 etrachloroetiene
Thallium
Thallium
1 horium
Tin
1 itanium
1 oluene
1 oluene
1 otal r hosphorus
1 otal Ouspended Oolids
1 rans~ 1 r£~dichloroethene
1 rans'l ,£~dichloroethene
1 rans'l ,O~dichloropropene
1 rans'l ,O~dichloropropene
1 rans'l ,O~dichloropropene
1 richloroethene
1 richloroethene
1 richloroethene
1 richlorofuoromethane
T
Uranium
Vanadium
Vanadium
Vinyl Chloride
Method
502.2
524.2
502.2
524.2
524.2
1620
200.8
524.2
502.2
502.2
524.2
502.2
524.2
1620
200.8
1620
200.8
1620
524.2
502.2
524.2
502.2
502.2
524.2
1620
200.8
200.8
1620
1620
502.2
524.2
365.2
160.2
502.2
524.2
502.2
502.2
524.2
524.2
502.2
502.2
524.2
502.2
524.2
200.8
1620
200.8
502.2
Procedure
PID
PID
ICP/MS
PID
PID
PID
ICP/MS
ICP/MS
PID
ELCD
PID
ICP/MS
ICP/MS
PID
ELCD
ELCD
PID
ELCD
PID
ELCD
ICP/MS
ICP/MS
ELCD
SL-IQE 10%, Based on Given Model
Constant
1.601
22.952
1.759
29.878
33.249
113.424
2.341
25.884
3.077
3.550
24.914
2.112
25.203
9.268
4.686
29.640
0.107
379.229
23.420
1.916
26.246
2.078
2.303
30.554
3.870
0.007
0.074
12.904
19.058
1.640
21.925
0.040
9.679
2.068
30.588
1.492
1.457
14.821
30.108
2.256
2.049
27.861
4.662
42.490
0.005
50.943
6.320
8.234
Linear
0.745
-0.521
0.351
-3.650
-4.704
67.206
0.800
-3.313
0.181
0.456
-3.372
0.346
0.279
5.235
1.045
25.842
0.056
337.755
-2.180
0.203
-1.197
0.415
0.750
-2.553
2.799
0.002
0.004
9.406
14.236
0.194
-1.050
0.032
7.570
0.795
-4.773
0.729
0.206
-1.254
-3.685
3.169
0.401
-2.666
5.166
-50.543
0.001
26.049
1.828
4.775
Exponential
0.343
0.345
0.221
0.647
0.422
60.455
0.202
0.450
0.272
0.380
0.934
0.196
0.316
4.657
0.957
24.547
0.030
323.935
0.318
0.177
0.423
0.145
0.392
1.080
2.661
0.002
0.003
9.064
12.443
0.153
0.330
0.030
6.985
0.197
0.684
0.237
0.221
0.506
2.938
0.141
0.235
0.759
3.222
0.881
0.001
25.112
2.022
3.544
Hybrid
0.325
0.067
0.186
0.148
0.108
57.072
0.183
0.040
0.202
0.312
0.551
0.157
0.047
4.474
0.829
24.294
0.034
317.747
0.041
0.135
0.073
0.122
0.400
N/A'
2.614
0.002
0.001
8.971
12.213
0.124
0.046
0.030
6.729
0.191
0.153
0.212
0.175
0.218
1.819
0.120
0.209
0.167
3.308
N/A'
0.001
24.338
1.933
3.828
RSDi
79.3%
168.6%
120.2%
166.5%
169.1%
35.2%
115.1%
168.4%
153.2%
134.9%
158.6%
134.2%
193.4%
38.3%
99.7%
9.5%
62.6%
8.1%
169.3%
143.6%
168.4%
135.5%
94.7%
131.8%
19.9%
70.9%
174.7%
18.7%
21.9%
140.6%
168.8%
14.1%
17.3%
108.7%
166.3%
89.8%
122.1%
161.1%
137.8%
108.1%
122.7%
164.9%
23.8%
135.7%
112.1%
40.8%
72.6%
42.3%
B- 87
-------
Table 14. Comparison of SL-IQEs calculated using different Model Types, Episode 6000 Data
(ng/L except where footnoted)
Analyte
Vinyl Chloride
Wad Cyanide
Xylene (total)
Y.trlum
Zlno
Zlno
Method
524.2
1677
524.2
1620
1620
200.8
Procedure
WADCN
ICP/MS
SL-IQE 10%, Based on Given Model
Constant
49.647
2.277
23.520
10.244
32.799
17.301
Linear
49.158
1.624
-0.952
8.962
12.850
7.024
Exponential
0.837
1.414
0.290
7.839
10.999
3.817
Hybrid
0.219
1.424
0.019
7.516
10.452
3.741
RSDi
113.0%
24.2%
169.8%
14.3%
64.0%
80.4%
Isalculation ii
Cjiven model
3IQE10%
4 p
(\esults repoi
lated based on given model
Summary Statistics for Table 14
Method
A..
502.2
524.2
1620
200.8
# analytes
197
65
81
25
21
Minimum
7.5%
10.5%
43.2%
7.5%
23.1%
25th Percentile
72.6%
79.3%
157.9%
16.6%
66.6%
Median
135.6%
114.4%
165.7%
23.9%
90.4%
75th Percentile
165.3%
142.4%
168.4%
38.3%
115.1%
Maximum
193.4%
183.1%
193.4%
78.3%
183.8%
B-
-------
Table 15. Comparison of SL-IDEs and SL-IQEs Calculated Using Different Software
1 ,3,5-trimethylbenzene (524.2)
B-89
-------
Table 15. Comparison of SL-IDEs and SL-IQEs Calculated Using Different Software
Cis-1,3-dichloropropene (502.2 ELCD)
B-90
-------
Table 15. Comparison of SL-IDEs and SL-IQEs Calculated Using Different Software
B-91
-------
Table 15. Comparison of SL-IDEs and SL-IQEs Calculated Using Different Software
Analyte
, (524.2)
Vanadium (1620)
Vlny, Cn.oNc,, (524.2)
,(1620)
1 Calculated using SAS programs written by EPA to run IDE and IQE calculations. Results are the same as those
presented in Tables 2 and 4.
2 Limits in bold indicate the calculated IDE or IQE based on the model suggested as most appropriate based on the
given software.
3 No value could be calculated due to model not converging.
4 Based on statistical tests, QCalc determined that the constant model should be used to calculate the IDE and IQE.
However, determination of the IDE and IQE using the constant model is not run by this program.
B-92
-------
Table 16. Summary Statistics of Ratios Comparing IDEs/IQEs using different Software Packages
Comparison
Ratio
QCa.c/ SAS
Exoe,/ SAS
QCalc/ Excel
Model Type
Hybrid
exponential
Linear
Hybrid
txponential
Linear
Constant
Hybrid
Lxponential
Linear
Limit
IDE
IQE10
IDE
IQE10
IDE
IQE10
IDE
IQE10
IDE
IQE10
IDE
IQE10
IDE
IQE10
IDE
IQE10
Minimum
-0.17
-1.00
0.98
0.97
-0.11
-0.65
1.00
0.99
1.00
0.97
0.99
0.98
-365,000
-225,000
0.96
0.99
25th
Percentile
0.99
0.99
0.99
0.99
-0.000003
-0.000009
1.01
1.00
1.01
1.00
1.00
0.99
-12.85
-2.07
0.98
1.00
Median
0.99
1.00
0.99
1.00
1.10
1.06
1.01
1.00
1.01
1.00
1.00
0.99
0.54
0.52
0.99
1.00
75th
Percentile
1.00
1.00
1.00
1.00
1.32
1.35
1.02
1.00
1.02
1.00
1.01
1.00
0.93
0.91
0.99
1.00
Maximum
1.03
1.07
1.03
1.00
3.22
3.27
1.05
1.00
1.06
1.00
1.02
1.00
1.01
1.01
0.99
1.00
B-93
-------
Table 17. Comparison of Simulated 7-replicate ACIL CRVs to Overall CRV, ACIL Blanks
Analyte
Barium
Cadmium
Chromium
Copper
Silver
# Blanks*
26
33
55
52
45
Overall
CRV
0.0039
0.0012
0.0048
0.0035
0.0105
# simulated
7-replicate
CRVs
20
27
49
46
39
Mean of
Simulated 7-
replicate
CRVs
0.0039
0.0014
0.0051
0.0039
0.0100
Range of
Simulated
7-replicate
CRVs
0.0011 ,0
0.0083
0.00044,0
0.0019
0.0014 ,o
0.0117
0.0010 ,o
0.0059
0.0019 ,o
0.0326
Range of
Days between
1st and Last
of 7
consecutive
replicates
7,o 26
11,o 24
7,o 20
7,o 20
7,o 20
% short-term
CRVs
exceeding
Overall CRV
30
67
29
78
28
Analyzed over a period of 3 months
Table 18. Comparison of Simulated 7-replicate ACIL CRVs to Overall CRV, ACIL Blanks
After Outlier Removal
Analyte
Barium
Chromium
Silver
# Blanks*
25
54
42
Overall
CRV
0.0020
0.0040
0.0031
# simulated
7-replicate
CRVs
19
48
36
Mean of
Simulated 7-
replicate CRVs
0.0021
0.0044
0.0038
Range of
Simulated
7-replicate
CRVs
0.0011 ,0
0.0029
0.0014 ,o
0.0080
0.0019 ,o
0.0058
Range of
Days
between 1st
and Last of 7
consecutive
replicates
11 ,o26
7,o 20
8,0 21
% short-term
CRVs
exceeding
Overall CRV
74
56
72
Analyzed over a period of 3 months
B-94
-------
Appendix C
Example Calculations
This Appendix is included to support Appendices B of this Assessment Document, by providing
example calculations of the single-laboratory variants of the Interlaboratory Detection Estimate (SL-IDE) and
Interlaboratory Quantitation Estimate (SL-IQE) as described in ASTM D6091 and ASTM D6512, respectively.
Example calculations of the method detection limit (MDL) and minimum level of quantitation (ML) also are
included. The example calculations provided in this Appendix were used in the data analyses presented in
Appendix B.
All abbreviations and symbols used in the SL-IDE and SL-IQE calculations match those given in the
ASTM procedures. The linear and exponential standard deviation models and all recovery models were fit
using the PROC REG procedure in SAS Version 8.1. The hybrid standard deviation model was fit using
Newton's Non-Linear Least Squares procedure as described in ASTM D6512, programmed using SAS Version
8.1. The dataset used in these examples is that included for 1,1,1,2- tetrachloroethane in EPA's Episode 6000
(see Chapter 1 and Appendix B of this document for descriptions of datasets).
Single-Laboratory IDE (SL-IDE)
The procedure for calculating the IDE that is described in ASTM D6091 stipulates use of data from
multiple laboratories. However, because analytes in the Episode 6000 dataset were only measured by a single
laboratory, EPA calculated a variant of the IDE which was called the single-laboratory IDE (SL-IDE). The SL-
IDE and the analyses performed using the SL-IDE are described in greater detail in Appendix B of this
Assessment document.
In order to calculate the SL-IDE, means and standard deviations are needed for each spike level. The
means and standard deviations for 1,1,1,2-tetrachloroethane are listed in Table 1.
Table 1. Mean and Standard Deviation Calculated at each Spike Level
Spike (ug/L)
0.01
0.015
0.02
0.035
0.05
0.075
0.1
0.15
0.20
N
7
7
7
7
7
7
6
7
7
Mean (ug/L)
0.0016
0.001
0.0007
0.0057
0.0081
0.0263
0.0295
0.0536
0.0991
SD (ug/L)
0.0018
0.0017
0.0010
0.0036
0.0024
0.0202
0.0039
0.0046
0.0158
C- 14
-------
Spike (ug/L)
0.35
0.50
0.75
1.0
2.0
5.0
10.0
N
7
7
6
8
7
8
7
Mean (ug/L)
0.235
0.3744
0.6193
0.8368
1.9560
5.0994
10.4453
SD (ug/L)
0.0078
0.0257
0.0262
0.0814
0.0980
0.2382
0.5469
In order to choose the appropriate model to calculate the IDE, significance tests were used.
The fitted unweighted linear model was:
S = 0.000039515 + 0.05326 * T, where T corresponds to spike concentration
The slope of this model was significantly greater than 0, and therefore the constant model was rejected.
The fitted unweighted exponential model (fit by natural log-transforming standard deviations) was:
Log(S)= -5.02407 + 0.54851 * T
The slope of this model was significantly greater than 0, thus, the linear model was rejected.
Based on this assessment, the exponential model was used in Appendix B to calculate the IDE for this analyte.
While the exponential model was chosen as the most appropriate model for this analyte, the calculation of the
SL-IDE using all four model types is presented in this Appendix. This was done to provide a step-by-step
example for the calculation of the SL-IDE using all of the different model types.
Constant model: The pooled within-spike variance was first calculated using the equation below:
16
y.io, -I)"*,'1
16
where: s; is the standard deviation of the results for spike level i, and
n; is the number of replicates for spike level i.
The calculated pooled within-spike variance (g2) is 0.024, and the square root of this value, g, equals 0.155.
C-15
-------
A linear regression model was then fit for the mean results for the 16 spike levels. The estimates of slope and
intercept for this model are: a = -0.089 and b=1.0478, respectively.
Based on these results:
YC = (kl * g) + a = (0.155 * kl) - 0.089 = (0.155 * 2.6) - 0.089 = 0.3137
where: YC = the recovery critical value as defined in ASTM D6091, and
kl = 2.6 (a conservative number based on the total n of 112)
LC = (YC - a)/b = (0.3137 + 0.089) /1.0478 = 0.3848
where: LC = the true concentration critical value as defined in ASTM D6091.
IDE = LC + (k2 * g)/b = 0.3848 + (1.86 * 0.155)71.0478 = 0.660
where: k2 = 1.86 (a conservative number based on the total n of 112).
Linear Model:
An unweighted linear regression model was fit, predicting standard deviation based on concentration, using
PROC REG in SAS Version 8.1. The estimated parameters are: g = 0.0000392 and h = 0.05326. Based on these
parameters, weights for the recovery model were calculated for each spike value. For each concentration, the
weight was calculated as:
weight =
, for each true concentration T;.
The calculated weights are given in Table 2.
Table 2. Calculated Weights based on Linear Model
Spike (ug/L)
0.01
0.015
0.02
0.035
0.05
0.075
0.1
0.15
Est. SD (ug/L)
0.00057
0.00084
0.00110
0.00190
0.00270
0.00403
0.00537
0.00803
Weight
3,058,709
1,423,673
819,854
276,031
136,940
61,454
34,736
15,514
C-16
-------
Spike (ug/L)
0.20
0.35
0.50
0.75
1.0
2.0
5.0
10.0
Est. SD (ug/L)
0.01069
0.01868
0.02667
0.03999
0.05330
0.10657
0.26635
0.53267
Weight
8,748
2,865
1,406
625.4
352.0
88.1
14.1
3.52
Using these weights, the fitted recovery model estimates were a = -0.00898 and b = 0.6860. Based on these
results:
YC = (kl * g) + a = (0.0000392 * 2.6) - 0.00898 = -0.00888, and
LC = (YC - a)/b = (-0.00888 + 0.00898) / 0.6860 = 0.00015
For the linear model, the SL-IDE must be calculated recursively. The initial estimate of the SL-IDE, LD0, was:
LD0 = LC + (k2*s(0)) / b = 0.00025.
Each following estimate was calculated using the recursive formula:
LDi+1 = [k, *t(0) + ki*(g + h*LDi)]/b
Results of the recursive LD calculations are given in Table 3.
Table 3. Recursive SL-IDE Calculations, Linear Model
LD estimate
run
0
1
2
3
LD estimate
0.000255
0.000291
0.000297
0.000297
The recursive estimates of LD converge to 6 decimal places by the third iteration. Therefore, the linear model
estimate of the IDE = 0.000297 ug/L.
C-17
-------
Exponential Model:
An unweighted linear regression model was fit, predicting natural log-transformed standard deviation based on
concentration. The estimated parameters are: g = 0.00658 and h = 0.54851. Based on these parameters, weights
for the recovery model were calculated for each spike value. For each concentration, the weight was calculated
as:
wei&ht = = , for each true concentration T,.
R. 9^iCi*T's^
S J Q :<: P \
1 \O /
The calculated weights are given in Table 4.
Table 4. Calculated Weights based on Exponential Model
Spike (ug/L)
0.01
0.015
0.02
0.035
0.05
0.075
0.1
0.15
0.20
0.35
0.50
0.75
1.0
2.0
5.0
10.0
Est. SD (ug/L)
0.00661
0.00663
0.00665
0.00671
0.00676
0.00685
0.00695
0.00714
0.00734
0.00797
0.00865
0.00993
0.01138
0.01970
0.10213
1.58566
Weight
22,861
22,736
22,611
22,242
21,879
21,287
20,711
19,606
18,560
15,744
13,355
10,152
7,717
2,576
96
0.40
C-18
-------
Using these weights, the fitted recovery model estimates were a = -0.04585, and b = 0.91696. Based on these
results:
YC = (kl * g) + a = (0.00658 * 2.6) - 0.04585 = -0.0287, and
LC = (YC - a)/b = (-0.0287 + 0.04585) / 0.91696 = 0.0187
For the Exponential model, the SL-IDE must be calculated recursively. The initial estimate of the SL-IDE, LD0,
was:
LD 0 = LC + (k2*s(0)) / b = 0.03199.
Each following estimate was calculate using the recursive formula:
Results of the recursive LD calculation are given in Table 5, below.
Table 5. Recursive SL-IDE Calculations, Exponential Model
LD estimate run
0
i
2
LD estimate
0.031993
0.032229
0.032231
The recursive estimates of LD converge to 6 decimal places by the second iteration. Therefore, the exponential
model estimate of the IDE = 0.032231 ug/L.
C-19
-------
Hybrid Model:
The Hybrid model was fit using Newton's Method for Non-linear Least Squares. Summary statistics from this fit of the hybrid model are presented
in Table 6, using the same notation as shown in ASTM D6512-00.
Table 6. Summary Statistics from Newton's Non-Linear Least Squares
Run
0
i
2
g
0.00095
0.00143
0.00148
h
0.05465
0.05228
0.05184
U
1,254330
981,892
958,193
V
4,285
4,275
4,309
C
19,889
15,368
15,092
d
2xlO-10
3xlO-10
3xlO-10
P
555.95
41.83
4.47
q
-0.592
-1.132
-0.123
Ag
0.00048
0.00005
5 x 10 "6
Ah
-0.00237
-0.00044
-0.00005
dg%
50.5
3.45
0.37
dh%
43.4
8.5
0.9
Because dg% (the percent difference between the last 2 estimates of g) and dh% (the percent difference between the last 2 estimates of h) were both
less than 1% in run 2, the model converged, and the estimated parameters of the hybrid model were:
g = g run2 +A g mn2 = 0.00148 + 0.000005 = 0.00149
h = h mn2+ A h mn2 = 0.05184 - 0.00005 = 0.05179
Using these fitted parameters, the weights for the recovery model were calculated as shown in Table 7.
C-20
-------
Table 7. Calculated Weights, Hybrid Model
Spike (ug/L)
0.01
0.015
0.02
0.035
0.05
0.075
0.1
0.15
0.20
0.35
0.50
0.75
1.0
2.0
5.0
10.0
Est. SD (ug/L)
0.00158
0.00168
0.00181
0.00234
0.00299
0.00416
0.00539
0.00791
0.01046
0.01819
0.02594
0.03887
0.05181
0.10358
0.25893
0.51786
Weight
403,037
355,066
304,351
181,881
112,141
57,811
34,447
15,987
9,134
3,024
1,487
662
373
93.2
14.9
3.73
Using these weights, the fitted recovery model estimates were a = -0.01471, and b = 0.74338.
these results:
YC = (kl * g) + a = (0.00149 * 2.6) - 0.01471 = -0.01085, and
LC = (YC - a)/b = (-0.01085 + 0.01471) / 0.74338 = 0.00520
LD had to be calculated recursively. The initial estimate of LD was:
LD 0 = LC + (k2*s(0)) / b = 0.00893.
Each following estimate was calculated using the recursive formula:
1 =[*!*
Based on
C-21
-------
Results of the recursive LD calculation are given in Table 8.
Table 8. Recursive SL-IDE Calculations, Hybrid model
LD estimate run
0
i
2
•-t
3
LD estimate
0.008925
0.009101
0.009108
0.009108
The recursive estimates of LD converge to 6 decimal places by the third iteration. Therefore, the hybrid model
estimate of the IDE = 0.009108 ug/L.
Single-Laboratory IQE (SL-IQE)
The procedure for the IQE described in ASTM D6512 stipulates use of data from multiple laboratories.
However, because analytes in the Episode 6000 dataset were only measured by a single laboratory, EPA
calculated a variant of the IQE which was called the single-laboratory IDE (SL-IQE). The SL-IQE and the
analyses performed using the SL-IQE are described in greater detail in Appendix B of this Assessment
document.
Fitting and selection of models in the IQE calculation process are identical to the IDE calculation process
except:
• The Hybrid model was considered in model selection instead of the Exponential model, based on
significance tests for curvature as described in 6.3.3.2 (g) - (i) of ASTM D6512.
• A bias-correction adjustment factor is applied to calculated standard deviations prior to modeling as
described in 6.3.3.2 (b) of ASTM D6512.
Therefore, the example calculation begins with the fitted model parameters for each model type, and
demonstrates the calculation of each IQE value.
C-22
-------
Constant model:
Using the same steps for fitting the constant model as described in the SL-IDE example, the fitted precision and
recovery model parameters are determined to be:
g = 0.1615
a =-0.0894, and b = 1.0478.
The IQE (10%) was calculated as: IQE (10%) = (g/b)*(100/10) = 1.541
The IQE (20%) was calculated as: IQE (20%) = (g/b)*( 100/20) = 0.770
The IQE (30%) was calculated as: IQE (30%) = (g/b)*(100/30) = 0.514
Linear model:
Using the same steps for fitting the linear model as described in the SL-IDE example, the fitted precision and
recovery model parameters are determined to be:
g = 4.2xlQ-7, h = 0.0555
a =-0.0087, b = 0.6810
The IQE (10%) was calculated as: IQE (10%) = g/(b*(10/100)-h) = 3.3 x 10 '5
The IQE (20%) was calculated as: IQE (20%) = g/(b*(20/100)-h) = 5.2 x 10 '6
The IQE (30%) was calculated as: IQE (30%) = g/(b*(30/100)-h) = 2.8 x 10 '6
Hybrid model:
Using the same steps for fitting the hybrid model as described in the SL-IDE example, the fitted precision and
recovery model parameters are determined to be:
g = 0.00155, h = 0.0540
a =-0.0147, b = 0.7434
The IQE (10%) was calculated as:
1QE(1 0%> , g = = 0.0304
^^
V 100
C-23
-------
The IQE (20%) was calculated as:
IQE(20%}= g = 0.0112
'
V ' 100
The IQE (30%) was calculated as:
g
= 0.0072
100
Exponential model:
Using the same steps for fitting the constant model as described in the SL-IDE example, the fitted precision and
recovery model parameters are determined to be:
g = 0.0069, h = 0.5482
a = -0.0459, b = 0.9170
For the Exponential model, the IQE must be solved recursively. The initial estimate of the IQE was set to the
IDE (re-calculated using bias-corrected standard deviations, and therefore not matching the IDE presented in
the example above). The IQE was then re-calculated using the estimate from the prior round, based on the
equation below:
Zb
where: Z i = 10, 20 or 30, depending on the IQE being calculated.
C-24
-------
Results of the recursive calculations for the IQEs are given in Table 9.
Table 9. Recursive SL-IDE Calculations, Exponential model
Run
0
i
2
•-t
3
4
IQE (10%)
0.0355
0.0763
0.0780
0.0781
0.0781
IQE (20%)
0.0355
0.0381
0.0382
0.0382
0.0382
IQE (30%)
0.0355
0.0254
0.0253
0.0253
0.0253
MDL/ML
This section gives an example calculation of the MDL and ML determined using the Episode 6000 data, and
presented in Appendix B. Due to the nature of the study design, MDLs could not be determined following the
MDL procedure directly. Therefore, the MDL was calculated based on the results of the two lowest spike
levels with all positive results for which the standard deviations were not significantly different.
The lowest two spike levels with all positive, non-zero results are 0.050 |-ig/L and 0.075 |-ig/L. From Table 1,
the standard deviations at these concentrations are 0.0024 |_ig/L and 0.0202 |_ig/L, respectively. The F test was
then run on the variances at these two spike levels:
F =
(0.0202)' 0.0004
(0.0024)- 0.000006
= 70.385
The critical value for the F test at cc=0.10, where both variances are based on 7 results, is 3.05. Because 70.385
> 3.05, the variance at the higher concentration is significantly greater than the variance at the lower
concentration, and these two concentrations cannot be used to calculate the MDL.
The next lowest spike level (0.10 i-ig/L) has only 6 results, but all results are greater than 0. Therefore, an F test
was run comparing variances at 0.075 |-ig/L and 0.10 |-ig/L. From Table 1, the standard deviation at 0.10 |-ig/L is
0.0039 |j,g/L. The results of the F test are:
F =
(0.0039)' 0.00002
(0.0202)2 ~ 0.0004
= 0,037
C-25
-------
The critical value for this F test is 3.11, slightly higher than for the prior comparison due to the fewer number of
results at the higher spike level. Because 0.037 < 3.11, the variance at the higher spike level is not significantly
greater than the variance at the lower spike level. Therefore, the MDL is calculated based on these two spike
levels:
:|: ,
( 0.99 ,7 +6 -2)
= 0,041
The ML is determined by first multiplying the pooled standard deviation (0.015 |-ig/L from the calculation
above) by 10. This yields a result of 0.15 |-ig/L. Based on the ML rounding scheme, this becomes 0.2 |_ig/L.
C-26
------- |