Statistical Issues in Farmworker Studies
Referencing: A Survey of Laboratory and Statistical Issues Related to Farmworker Exposure Studies
Barr et al. (2006) surveyed statistical issues related
to farmworker exposure studies. However, they made several factual and
conceptual errors that need to be called to the readers' attention.
First, Barr et al. (2006) claimed incorrectly that
"representativeness" is optional and not a necessary condition
for a well-designed investigation. For convenience samples,
[T]he results only pertain to the sample itself,
and should not be used to make quantitative statements about any population
– including the population from which the sample was selected."
[U.S. Environmental Protection Agency (EPA) 2003]
Barr et al. (2006) stated that "because responses
from convenience samples are likely to be better than that for a representative
sample, they may actually be more 'representative.'"
The fallacy of this statement is shown by a hypothetical CNN call-in
response to a question from 100% of its viewers that perfectly represents
all CNN viewers. In this illustrative example, the 100% response would
not represent the entire population of the United States as well as
a probability-based survey of the U.S. population that included non-CNN
viewers that achieved an 80% response rate.
In their article, Barr et al. (2006) claimed that
"perfectly random sampling across all relevant factors is therefore
almost universally impractical." Acquavella et al. (2004) monitored
a probability sample of pesticide applicators; U.S. EPA provided several
TEAM (Total Exposure Assessment Methodology) studies using a scientific
probability design (Thomas 1993; Wallace 1991; Wallace et al. 1987),
as did the World Health Organization, U.S. EPA, and Harvard University
for the government of Kuwait during the 1991 oil fires (Mage DT, Wallace
LA, Kollander M, personal communication). The Centers for Disease Control
and Prevention's (CDC) National Health and Nutrition Examination
Survey study (CDC 2003) is another excellent example of proper probability-based
sample selection.
According to Barr et al. (2006), it is possible
to identify and "sample known or anticipated 'hot spots'
of [pesticide] exposure." There are only two categories of applicators
expected to be at high risk of a high pesticide exposure event: the
inexperienced applicators who are still learning how to apply pesticides
safely, and those applicators who do not follow the mandatory manufacturer's
label requirements in violation of federal law (Mage et al. 2002). Whereas
the former cohort might be identified by a screening question about
prior numbers of applications, there is no certain way to identify the
latter group, who will likely not admit to taking shortcuts or refusing
to use required personal protective equipment, because they might be
incriminating themselves. Finally, such an applicator may succumb to
the Hawthorne effect [not mentioned by Barr et al. (2006) as a caveat],
defined by Last (1988) as "the effect of being under study upon
the persons being studied."
Barr et al. (2006) claimed that "some form
of convenience sampling is typically adopted in practice." Unfortunately,
this claim is true; some of these authors did use convenience sampling
in previous studies (Curwin et al. 2002, 2005) in which subjects were
recruited by "word of mouth." A friend or neighbor recruited
by an enrolled subject might not be "an independent sample"
if he or she has some similar characteristics (e.g., crops grown, acreage,
age, race, education, sex) as the recruiter. This haphazard practice
of using volunteers for convenience, or even subjects based on expert
choice (Hoppin et al. 2006), limits the validity of the study, as theoretical
confidence intervals and significance p-values become meaningless.
The weakness of all nonprobability sampling is its
subjectivity that precludes the development of a theoretical framework
for it. (Kalton 1983)
Finally, as former U.S. EPA scientists who pioneered
agency exposure science, we are disappointed that this article was cleared
for publication by the U.S. EPA because it is not in accordance with
U.S. EPA (and other agency) requirements to follow the Office of Management
and Budget's (OMB) data collection policies (OMB 2006) that require
"selecting samples using generally accepted statistical methods
(e.g., probabilistic methods that can provide estimates of sampling
error)." The U.S. EPA (2003) stated:
Probability sampling must be used at each stage
of respondent selection. You may encounter difficulties in clearing
the survey through OMB if you do not insist that probability selection
methods be used.
Recent samples of high-risk subpopulations and their
exposures to particles were undertaken by the U.S. EPA using doctor-identified
subjects, and these were therefore not probability-based samples. The
OMB allowed these studies but required that a statement be made in all
resulting publications that the results could be applied only to the
participants, even if chosen in this case by expert judgment, and must
not be extrapolated to larger populations. We believe a similar statement
should be made in all publications of studies using alternatives to
probability-based sampling.
In summary, Barr et al. (2006) attempted to review
survey design practices, but they do not seem to understand that the
convenience samples they advocate apply only to the subjects selected
and not to the larger populations from which they are taken.
The authors declare they have no competing financial
interests.
David T. Mage
Temple University (retired)
Newark, Delaware
Lance A. Wallace
U.S. Environmental Protection
Agency (retired)
Reston, Virginia
Mel Kollander
Temple University (retired)
Alexandria, Virginia
Wayne R. Ott
Stanford University
Stanford, California
References
Acquavella JF, Alexander BH, Mandel JS, Gustin C,
Baker B, Chapman P, et al. 2004. Glyphosate biomonitoring for farmers
and their families: results from the Farm Family Exposure Study. Environ
Health Perspect 112: 321–326.
Barr DB, Landsittel D, Nishioka M, Thomas K, Curwin
B, Raymer J, et al. 2006. A survey of laboratory and statistical issues
related to farmworker exposure studies. Environ Health Perspect 114:961–968.
CDC (Centers for Disease Control and Prevention).
2003. NHANES 1999-2000 Public Data Release File Documentation. Available:
http://www.cdc.gov/nchs/data/nhanes/gendoc.pdf [accessed 26 October
2006].
Curwin BD, Hein MJ, Sanderson WT, Barr DB, Heederik
D, Reynolds SJ, et al. 2005. Urinary and hand wipe pesticide levels
among farmers and nonfarmers in Iowa. J Expo Anal Environ Epidemiol
15:500–508.
Curwin B, Sanderson W, Reynolds S, Hein M, Alavanja
M. 2002. Pesticide use and practices in an Iowa farm family pesticide
exposure study. J Agric Saf Health 8: 423–433.
Hoppin JA, Adgate JL, Eberhart M, Nishioka MG, Ryan
PB. 2006. Environmental exposure assessment of pesticides in farmworker's
homes. Environ Health Perspect 114:929–935.
Kalton G. 1983. Introduction to Survey Sampling.
Newbury Park, CA:Sage Publications.
Last JM. 1988. A Dictionary of Epidemiology. New
York:Oxford University Press.
Mage DT, Alavanja MC, Sandler DP, McDonnell CJ,
Kross B, Rowland A, et al. 2000. A model for predicting the frequency
of high pesticide exposure events in the Agricultural Health Study.
Environ Res 83:67–71.
OMB (Office of Management and Budget). 2006. Standards
and Guidelines for Statistical Surveys, September 2006. Available: http://www.whitehouse.gov/omb/inforeg/statpolicy/standards_stat_surveys.pdf [accessed 26 October 2006].
Thomas KW, Pellizzari ED, Clayton CA, Whitaker DA,
Shores RC, Spengler J, et al. 1993. Particle Total Exposure Assessment
Methodology (PTEAM) 1990 study: method performance and data quality
for personal, indoor, and outdoor monitoring. J Expo Anal Environ Epidemiol
3:203–226.
U.S. EPA. 2003. Survey Management Handbook Volume
1: Guidelines for Planning and Managing a Statistical Survey. EPA-260-B-03-003.
Washington, DC:U.S. Environmental Protection Agency, Office of Policy
Planning and Evaluation. Available: http://www.epa.gov/oamcinc1/0510667/handbook.pdf
[accessed 24 October 2006].
Wallace LA. 1991. Personal exposure to 25 volatile
organic compounds. EPA's 1987 team study in Los Angeles, California.
Toxicol Ind Health 7:203–208.
Wallace LA, Pellizzari ED, Hartwell TD, Sparacino
C, Whitmore R, Sheldon L, et al. 1987. The TEAM (Total Exposure Assessment
Methodology) Study: personal exposures to toxic substances in air, drinking
water, and breath of 400 residents of New Jersey, North Carolina, and
North Dakota. Environ Res 43:290–307.
Statistical Issues: Barr et al. Respond
Mage et al. criticize our article (Barr et al. 2006),
stressing six "… factual and conceptual errors that need
to be called to the readers' attention." We appreciate their
careful reading of our work, and they raise several important points
regarding survey design. However, we take issue with some of their statements.
Many investigations are designed to generalize the
results of the research performed within a sample population to a larger
population. In these types of investigations, enrollment of a representative
sample is a necessary condition for making inferences to the larger
population through known selection probabilities that are then used
for applying sampling weights to study results. However, in order to
generalize the results, these studies must have an adequate sample size,
high response rate, and, importantly, a preliminary assessment of whether
the factors for probability selection and weighting will be relevant
to the condition being measured. Although representative samples are
desirable and have been achieved in many studies, some studies in rare
or difficult-to-reach populations cannot practically meet the criteria
mentioned above.
Farmworkers are often not the ones applying pesticides
(i.e., they are not applicators); quite often these farmworkers are
unaware of the actual pesticides being applied or when they are applied.
The potential for undue exposure may be more likely if farmworkers are
not properly informed of the application or reentry times or if they
do not understand the potential exposure scenarios.
Research investigations involving farmworker exposures
can present particular difficulties in selecting a representative sample
of the population. Data for developing relevant sampling frames and
selection probabilities are often limited by demographic and work factors.
Obtaining high response rates for farmworkers can also be challenging
because of problems associated with access, high mobility, geographic
dispersion, trust, and cultural practices (Arcury et al. 2006). However,
these populations remain important and potentially vulnerable populations
that should be included in research investigations, even if the conditions
for using a probability sample cannot always practically be met.
In any particular situation, the decision to use
probability sampling will depend on the hypothesis. However, important
and relevant research questions can be investigated without selecting
a representative sample of the population, as noted by Mage et al. regarding
particle exposure studies in high-risk subpopulations. Nonprobability
samples can provide useful information on particular hard-to-reach populations,
for intensive examination of conditions and factors related to exposures,
or for hypothesis generation. By forcing all studies to conform to the
same design, we may not be able to answer specific research questions.
The studies cited by Mage et al. were designed as
probability samples, but each had differential drop-out rates during
selection and sampling. Potential drop-out nonrepresentativeness can
be accounted for if the effect on exposure or the outcome variable is
known. However, in some populations, the details of the accounting are
not easily accomplished. Mage et al. cited the National Health and
Nutrition Examination Survey (NHANES) as "another excellent example
of proper probability-based sample selection." However, even NHANES
III (1988–1994) used a nonprobability sample for the environmental
subset (Hill et al. 1995). These data provided an invaluable first look
at U.S. population exposures and served as a basis to add statistical
sampling for environmental chemicals to the current NHANES series. These
data have also been used to estimate doses in the U.S. population for
comparison to the U.S. Environmental Protection Agency's reference
doses (Mage et al. 2004). In fact, it is often difficult to design a
population-based study without preliminary data.
We have participated in the design and implementation
of studies using both probability and nonprobability sampling that have
added invaluable information on various population exposures. We recognize
the practical difficulties and challenges for meeting the criteria for
representative sampling in farmworker populations and also the important
information that such studies can provide. Although we agree that without
a probability sample, the results should only apply to individuals in
studies and should not be generalized to a population, we disagree with
the contention made by Mage et al. that no useful information can come
from studies using samples that do not fulfill their criteria.
The authors declare they have no competing financial
interests.
Dana B. Barr
National Center for Environmental Health
Centers for Disease Control and Prevention
Atlanta, Georgia
Doug Landsittel
Department of Mathematics and Computer Science
Duquesne University
Pittsburgh, Pennsylvania
Marcia Nishioka
Battelle Memorial Institute
Columbus, Ohio
Kent Thomas
National Exposure Research Laboratory
Office of Research and Development
U.S. Environmental Protection Agency
Research Triangle Park, North Carolina
Brian Curwin
National Institute for Occupational
Safety and Health
Centers for Disease Control and Prevention
Cincinnati, Ohio
James Raymer
RTI International
Research Triangle Park, North Carolina
Kirby C. Donnelly
Texas A&M University System
Health Science Center
College Station, Texas
Linda McCauley
School of Human Environmental Sciences
University of Pennsylvania
Philadelphia, Pennsylvania
P. Barry Ryan
Rollins School of Public Health
Emory University
Atlanta, Georgia
References
Arcury TA, Quandt SA, Barr DB, Hoppin JA, McCauley
L, Grzywacz JG, et al. 2006. Farmworker exposure to pesticides: methodologic
issues for the collection of comparable data. Environ Health Perspect
114:923–928.
Barr DB, Landsittel D, Nishioka M, Thomas K, Curwin
B, Raymer J, et al. 2006. A survey of laboratory and statistical issues
related to farmworker exposure studies. Environ Health Perspect 114:961–968
Hill RH Jr, Head SL, Baker S, Gregg M, Shealy DB,
Bailey SL, et al. 1995. Pesticide residues in urine of adults living
in the United States: reference range concentrations. Environ Res 71:99–108.
Mage DT, Allen RH, Gondy G, Smith W, Barr DB, Needham
LL. 2004. Estimating pesticide dose from urinary pesticide concentration
data by creatinine correction in the Third National Health and Nutrition
Examination Survey (NHANES-III). J Expo Anal Environ Epidemiol 14:457–465.
|