V. SPECIAL TOPICS

A. Subject Attrition and Retention

191. Biglan, A., Hood, D., Brozovsky, P., Ochs, L., Ary, D., & Black, C. Subject attrition in prevention research. In: Leukefeld, C.G., & Bukoski, W.J., eds. Drug Abuse Prevention Intervention Research: Methodological Issues. National Institute on Drug Abuse Research Monograph 107. DHHS Pub. No. (ADM)91-1761. Washington, DC: Supt. of Docs., U.S. Govt. Print. Off., 1991, pp. 213-234.

The chapter discusses the role of subject attrition in substance abuse prevention research. Attrition may affect the validity of experimental comparisons and may limit the extent to which findings can be generalized to adolescents at highest risk. The authors examine concerns about subject attrition, present methods for analyzing attrition in evaluations of prevention programs, and make recommendations for minimizing the extent and impact of attrition in such evaluations.

192. Ellickson, P.L., Bianca, D., & Shoeff, D.C. Containing attrition in school-based research: An innovative approach. Eval Rev 12:331-351, 1988.

This article describes a successful approach for tracking a highly mobile group of junior high school transferees and thereby minimizing attrition in a longitudinal study of adolescent behavior. Students were tracked through the home or the new school. When students were located through the latter route, the authors sent the surveys directly to the school itself instead of asking for a home mailing address. This approach avoided asking school officials to give out personal information and enhanced the likelihood of the survey being delivered. Overall, the tracking effort cut nonresponse attributable to between-school mobility by two-thirds and reduced the attrition rate by one-half. The new-school strategy, which was a particularly effective technique for finding student transferees, accounted for a significant proportion of that improvement.

193. Hansen, W., Tobler, N., & Graham, J. Attrition in substance abuse prevention research: A meta-analysis of 85 longitudinally followed cohorts. Eval Rev 14(6):677-685, 1990.

This meta-analysis of substance abuse prevention studies reveals that the mean proportion of subjects retained dropped from 81.4 percent at 3-month followup to 67.5 percent at 3-year followup. Time from pretest alone accounted for less than 5 percent of the variance. Other available predictors of retention were not significant. Researchers are encouraged to interpret their results in light of these normative data and to adopt second-effort strategies to reduce attrition.

194. Willett, J.B., & Singer, J.D. From whether to when: New methods for studying student dropout and teacher attrition. Rev Educ Res 61(4):407-450, 1991.

The authors shows how the methods of survival analysis (also known as event history analysis) lend themselves naturally to the study of the timing of educational events. Drawing examples from the literature on teacher attrition and student dropout and graduation, the authors introduce a panoply of survival methods useful for describing the timing of educational transitions, e.g., student dropout and teacher attrition, and for building statistical models of the risk of event occurrence over time.

195. Streissguth, A.P., & Giunta, C.T. Subject recruitment and retention for longitudinal research: Practical considerations for a nonintervention model. In: Kilbey, M.M., & Asghar, K., eds. Methodological Issues in Epidemiological, Prevention, and Treatment Research on Drug-Exposed Women and Their Children. National Institute on Drug Abuse Research Monograph 117. DHHS Pub. No. (ADM)92-1881. Washington, DC: Supt. of Docs., U.S. Govt. Print. Off., 1992, pp. 137-154.

The chapter discusses strategies for subject recruitment and retention for longitudinal research. It examines issues that need to be considered in the planning of a study, factors that affect the success of the recruitment effort, and special techniques for tracing subjects and retaining subjects. It also discusses issues in staff recruitment, training, and inspiration, as well as budgeting.

Back to Top

B. Statistical Analysis with Missing Data

196. Allison, P.D. Estimation of linear models with incomplete data. In: Clogg, C., ed. Sociological Methodology 1987. San Francisco: Jossey-Bass, 1987, pp. 71-103.

The chapter describes the uses of ML to handle missing data problems in LISREL type of models. After describing the various missing data mechanisms first described by Rubin (1976), the author describes the assumptions underlying the method he proposed. The core of the paper describes how to use LISREL to get ML estimates of linear models with incomplete data. The method capitalizes on the ability of LISREL to estimate simultaneously the same model for two or more samples. For incomplete data problems, the sample is divided into subsamples, each having a different set of variables present. The model is then estimated simultaneously for all subsamples, constraining corresponding parameters to be equal across subsamples. The author further discusses the special techniques that are needed in such estimation. There is also an appendix that contains proofs.

197. Little, R.J.A., & Rubin, D.B. Statistical Analysis With Missing Data. New York: Wiley, 1987.

The book surveys methodology for handling missing-data problems and presents a likelihood-based theory for analysis with missing data that systematizes these methods. Part I of the book discusses ad hoc approaches to missing-value problems in three important areas of statistics: analysis of variance of planned experiments, survey sampling, and multivariate analysis. Part II presents a systematic approach to the analysis of data with missing values, where inferences are based on likelihoods derived from formal statistical models for the data and the missing-data mechanisms. Applications of the approach are presented in a variety of contexts, including regression, factor analysis, contingency table analysis, time series, and sample survey inference.

198. Little, R.J.A., & Schenker, N. Missing Data. In: Arminger, G., Clogg, C.C., & Sobel, M.E., eds. A Handbook for Statistical Modeling in the Social and Behavioral Sciences. New York: Plenum, 1992, pp. 39-75.

In this chapter, Little and Schenker discuss modern methods for dealing with missing data. It covers important concepts underlying missing data procedures, differences between naive approaches, and more principled approaches. Then the chapter discusses various missing data procedures, such as weighting adjustment for unit nonresponse, maximum likelihood approaches, nonignorable nonresponse models, and multiple imputation. It also covers other Bayesian simulation methods.

199. Rindskopf, D. A general approach to categorical data analysis with missing data, using generalized linear models with composite links. Psychometrika 57:29-42, 1992.

Rindskopf describes a general approach for analyzing categorical data when there are missing data. The method is based on generalized linear models with composite links. The approach can be used (among other applications) to fill in contingency tables with supplementary margins. It is also used to fit log-linear models when data are missing, fit latent class models (without or with missing data on observed variables), fit models with fused cells, and fill in tables or fit models to data when variables are more finely categorized for some cases than others.

200. Rubin, D.B. Inference and missing data. Biometrika 63:581-592, 1976.

This is the classical paper written by Donald Rubin on the drawing of inferences about the parameter of the data when missing values are present. It reviews the previous statistical literature on missing data, and it provides a thorough theoretical discussion of the problem. The paper outlines the different kinds of missing data generating mechanisms and discusses the implication of each for making inference about the true value of parameters. The paper also discusses separately the use of direct-likelihood inferences and Bayesian inferences in the context of missing data.

201. Schafer, J.L. Analysis of Incomplete Multivariate Data. London: Chapman and Hall, 1997.

The book presents a unified, Bayesian approach to the analysis of incomplete multivariate data, covering data sets in which the variables are continuous, categorical, or both. It is written for applied statisticians, biostatisticians, practitioners of sample surveys, graduate students, and other methodologically oriented researchers in search of practical tools to handle missing data. The focus is applied rather than theoretical, but technical details are also included.

202. Clogg, C.C. Rubin, D.B., Schenker, D., Schultz, B., & Weidman, L. Multiple imputation of industry and occupation codes from Census public-use samples using Bayesian logistic regression. J Am Stat Assoc 86:68-78.

The authors describe methods used to create a new Census database that can be used to study comparability of industry and occupation classification systems. The project consists of extensive application of multiple imputation to data and the fitting of hundreds of logistic regression models. The paper summarizes the strategies used in the project, and it shows how modifications of maximum likelihood methods were made for the modeling and imputation phases of the project. These methods include Bayesian methods that can deal with sparse data, which usually present problems for traditional ML methods.

203. Graham, J.W., & Schafer, J.L. On the performance of multiple imputation for multivariate data with small sample size. In: Hoyle, R., ed. Statistical Strategies for Small Sample Research. Thousand Oaks, CA: Sage Publications, 1999.

The purpose of the chapter is to investigate the performance of some of the latest missing-data technologies–in particular, multiple imputation using NORM–in realistic analyses where the sample size, N, is relatively small. The authors review from a user’s perspective the key ideas of multiple imputation and the use of NORM software developed by Schafer (1997; NORM is available at http://methcenter.psu.edu/). They then describe a simulation study designed to test the limits of NORM in a realistic setting where N is small and the data do not conform to assumptions of normality.

204. Schafer, J.L., & Olsen, M.K. Multiple imputation for multivariate missing-data problems: A data analyst's perspective. Multivariate Behav Res 33:545-571, 1998.

This article reviews the key ideas of multiple imputation, discusses the software programs currently available, and demonstrates their use on data from the Adolescent Alcohol Prevention Trial.

Back to Top

C. Selection of Covariates in Prevention/Epidemiological Studies

205. Baumrind, D. Familial antecedents of adolescent drug use: A developmental perspective. In: Jones, C.L., & Battjes, R.J., eds. Etiology of Drug Abuse: Implications for Prevention. National Institute on Drug Abuse Research Monograph 56. DHHS Pub. No. (ADM)87-1335. Washington, DC: Supt. of Docs., U.S. Govt. Print. Off, 1985, pp. 13-44.

The author considers the impact of early childhood and preadolescent socialization experiences on adolescent drug use from a developmental perspective. She first review the processes defining normal adolescent development and then present findings on the preadolescent phase of the Family Socialization and Developmental Competence Project. The findings do not support the presupposition that adolescent drug use arises from pathological personal characteristics or pathogenic socialization practices. The study also found that the use of illegal drugs, such as marijuana, is not a deviant behavior for adolescents.

206. Donovan, J.E., & Jessor, R. Structure of problem behavior in adolescence and young adulthood. J Consult Clin Psychol 52(6):890-904, 1985.

The authors carried out a multivariate test of the existence of a single behavioral syndrome that comprises problem drinking, illicit drug use, delinquent-type behavior, and precocious sexual intercourse among normal adolescents. Analyses were conducted using maximum likelihood factor analyses based on self-report data from several samples of adolescents and youth who participated in the 1978 National Study of Adolescent Drinking (Rachal et al., 1980). Results of the study consistently showed that one common factor accounts for the correlations among the different problem behaviors. The authors argue that the findings lend support to the notion of a syndrome of problem behavior in both adolescence and young adulthood.

207. Greenspan, S.I. Research strategies to identify developmental vulnerabilities for drug abuse. In: Jones, C.L., & Battjes, R.J., eds. Etiology of Drug Abuse: Implications for Prevention. National Institute on Drug Abuse Research Monograph 56. DHHS Pub. No. (ADM) 85-1335. Washington, DC: Supt. of Docs., U.S. Govt. Print. Off., 1985, pp. 136-154.

In this paper, Greenspan presents the developmental structuralist approach to etiologic and intervention research. Such an approach allows postulation of relationships between infancy and subsequent behaviors associated with drug use. The author later uses vignettes that illustrate how early intersensory integration, self-regulatory mechanisms, and affective development relate to subsequent impairments in affective relationships among individuals who develop antisocial behavior and substance use.

208. Hawkins, D.J., Catalano, R.F., & Miller, J.Y. Risk and protective factors for alcohol and other drug problems in adolescence and early adulthood. Psychol Bull 112(l):64-105, 1992.

The authors propose that a risk-focused approach to prevention of adolescent alcohol and other drug problems is the most promising route to effective intervention strategies. Such an approach requires the identification of risk factors for drug abuse, identification of methods by which risk factors have been effectively addressed, and application of these methods to appropriate high-risk and general population samples in controlled studies. The authors review risk and protective factors to drug abuse, assess a number of approaches for drug prevention potential with high-risk groups, and make recommendations for research and practice.

209. Hawkins, J.D., Lishner, D.M., & Catalano, R.F., Jr. Childhood predictors and the prevention of adolescent substance abuse. In: Jones, C.L., and Battjes, R.J., eds. Etiology of Drug Abuse: Implications for Prevention. National Institute on Drug Abuse Research Monograph 56. DHHS Pub. No. (ADM)871335. Washington, DC: Supt. of Docs., U.S. Govt. Print. Off., 1985. pp. 75-126.

The paper describes approaches to drug abuse prevention with preadolescent children. The authors review these approaches in light of the existing knowledge on the etiology of drug abuse among children and adolescents. They also identify gaps in the prevention intervention research and suggest etiological research that can aid in the development and refinement of preventive intervention focused on preadolescents.

210. Jessor, R. Bridging etiology and prevention in drug abuse research. In: Jones, C.L., & Battjes, R.J., eds. Etiology of Drug Abuse: Implications for Prevention. National Institute on Drug Abuse Research Monograph 56. DHHS Pub. No. (ADM) 85-1335. Washington, DC: Supt. of Docs., U.S. Govt. Print. Off., 1985.

Jessor comments on four papers in the monograph that deal with the etiology of drug use during the transition from adolescents to young adulthood. He concludes by highlighting two major issues for drug abuse prevention research. The first is the appropriate goal for prevention. Rather than targeting at complete abstinence, he presents a more differentiated specification. The author also discusses the implication of the findings on stages and sequences of drug initiation and the role early onset plays in drug use initiation for drug abuse prevention.

211. Johnston, L.D. Contributions of drug epidemiology to the field of drug abuse prevention. In: Leukefeld, C.G., & Brakoslu, W.J., eds. Drug Abuse Prevention Research: Methodological Issues. National Institute on Drug Abuse Research Monograph 107. DHHS Pub. No. (ADM) 91-1761. Washington, DC: Supt. of Docs., Govt. Print Off., 1991, pp. 57-80.

Johnston points out there are at least eight ways in which epidemiological studies could inform the development of drug prevention programs and the evaluations of such programs. He discusses each of them in detail in this book chapter. The eight areas are (1) drug use or drug-related problems that need to be prevented; (2) ages at which such use is initiated or problems are occurring; (3) subgroups in the population most "at risk" in terms of their demographic and lifestyle characteristics; (4) changing backdrop against which the effects of specific prevention efforts should be assessed; (5) importance of certain key intervening variables such as attitudes and beliefs; (6) behavioral and moral norms with regard to drug use among young people and other groups having influence on them; (7) extent to which major classes of prevention programming are reaching targeted segments of the population and the subjective opinions of those populations as to the helpfulness and effects of the interventions; and (8) combined effectiveness of all forces in society that tend to reduce drug use or abuse, including those that are planned programs, more spontaneous efforts of groups or individuals, and other historical events.

212. Jones, C.L., & Battjes, R.J., eds. Etiology of Drug Abuse: Implications for Prevention. National Institute on Drug Abuse Research Monograph 56. DHHS Pub. No. (ADM)87-1335. Washington, DC: Supt. of Docs., U.S. Govt. Print. Off., 1985.

The monograph consists of a number of comprehensive review papers written by distinguished scientists in the field of etiological research on substance use. It focuses on results of etiologic research on adolescent drug use could be utilized to improve efforts to prevent drug abuse. The papers in the monograph are grouped into two section, the first one deal with antecedents to drug use during childhood and the impact of the transition to adolescence. The second part deals with risk factors that become prominent during the transition from adolescent to young adulthood.

213. Kandel, D.B., & Yamaguchi, K. Developmental patterns of the use of legal, illegal, and medically prescribed psychotropic drugs from adolescence to young adulthood. In: Jones, C.L., & Battjes, R.J., eds. Etiology of Drug Abuse: Implications for Prevention. National Institute on Drug Abuse Research Monograph 56. DHHS Pub. No. (ADM) 85-1335. Washington, DC: Supt. of Docs., U.S. Govt. Print. Off., 1985, pp. 193- 235.

Kandel and Yamaguchi present their findings on the developmental patterns of drug use among a cohort of young adults in a followup study during 1980-1981. Using the methodology of Guttman Scaling, they test specific sequential models of progression of drug use. Furthermore, using event history analysis, they try to answer the question of whether the use of certain drugs lower in the sequence influences the initiation of higher drugs. The authors then discuss their findings on the initiation of different types of illicit drugs.

214. Robins, L.N., & Przybeck, T.R. Age of onset of drug use as a factor in drug and other disorders. In: Jones, C.L., & Battjes, R.J., eds. Etiology of Drug Abuse: Implications for Prevention. National Institute on Drug Abuse Research Monograph 56. DHHS Pub. No. (ADM)87-1335. Washington, DC: Supt. of Docs., U.S. Govt. Print. Off., 1985, pp. 178-192.

The paper presents findings from a large-scale epidemiological study in three sites on drug disorder among adolescents. The high rate of drug disorder is related to rise in drug use in this population but not to increase in vulnerability to addiction or abuse among users. Drug use disorders are associated with other psychopathology. Early onset of use predict increased risk of drug disorders and higher severity of the disorder. The findings suggest that a useful preventive strategy is to try to postpone first drug use to age 18 or later.

215. Shore, M.F. Correlates and concepts: Are we chasing our tails? In: Jones, C.L., & Battjes, R.J., eds. Etiology of Drug Abuse: Implications for Prevention. National Institute on Drug Abuse Research Monograph 56. DHHS Pub. No. (ADM)87-1335. Washington, DC: Supt. of Docs., U.S. Govt. Print. Off., 1985, pp. 127-135.

The paper is a response to three other papers in the monograph. The author first lists seven major advances in drug abuse research and then discusses the papers by Baumrind, Bush and Iannotti, and Hawkins, all of which focus on the impact of early childhood factors on later drug use. The author concludes by pointing out the difficulty in developing theories in prevention research, the challenges in developing useful research methodology for the field, and the need for longitudinal research paradigm.

Back to Top

D. Reducing Selection Bias and Adjusting for Initial Group Differences in Non-Randomized Treatment Studies

216. Berk, R.A. An introduction to sample selection bias, American Sociological Review 48:386-398, 1983.

This paper is a brief review of some of the methods in the diagnosis and correction for "sample selection bias." It describes, with the aid of diagram, the nature and consequence of sample selection bias. Drawing on Heckman’s (1979) exposition of the problem, the author further explains the phenomenon in statistical terms. Later parts of the paper introduce the readers to several remedies to the problem.

217. Cochran, W.G. The effectiveness of adjustment by subclassification in removing bias in observational studies. Biometrics 24:295-313, 1968.

In the adjustment by subclassification method, the distribution of the covariate x is broken up into two, three, or more subclasses. For each group of subjects, the mean value of the response variable y, for the two groups under comparison, is calculated separately within each subclass. Then a weighted mean of these subclass means is calculated for each group, using the same weights for every group. The effectiveness of such a method in removing bias in observational studies was examined in this paper. The extent to which adjustment reduces the sampling error of the estimated difference between the y is also examined.

218. Cochran, W.G., & Rubin, D.B. Controlling bias in observational studies: A review. Snkhya, Series A 35(4):417-446, 1973.

This paper reviews work on the effectiveness of different methods of matched sampling and statistical adjustment, alone and in combination, in reducing bias due to confounding x-variables when comparing two populations. The adjustment methods were linear regression adjustment for x continuous and direct standardization for x categorical. With x continuous, the range of situations examined included linear relations between y and x, parallel and nonparallel, monotonic nonlinear parallel relations, equal and unequal variances of x, and the presence of errors of measurement in x. The percent of initial bias that was removed was used as the criterion. Overall, linear regression adjustment on matched samples appeared superior to the matching methods, with linear regression adjustment on matched samples the most robust method. Several different approaches are suggested for the case of multivariate x.

219. McKinlay, S.M. The effect of bias on estimators of relative risk for pair-matched and stratified samples. J Am Stat Assoc 70(352):859-864, 1977.

The author used Monte Carlo methods to compare the effectiveness of pair-matched and independent stratified samples for estimating relative risk in the presence of bias. Three approximations to the maximum likelihood estimator for stratified samples suggested by Woolf, Mantel and Haenszel and Birch, respectively, are also compared. The results showed that the mean square error is always largest for the matched-pairs estimator, while of the stratified estimators, Woolf’s consistently produces the smallest MSE, equaled by Birch’s when the samples are equal.

220. Rosenbaum P.R., & Rubin, D.B. Reducing bias in observational studies using subclassification on the propensity score. J Am Stat Assoc 79(387):516-524, 1984.

The propensity score is the conditional probability of assignment to a particular treatment given a vector of observed covariates. It has been shown that subclassification on the propensity score will balance all observed covariates. The paper shows how the approach works using observational data on treatments for coronary artery disease. Subclasses are first defined by the estimated propensity score. These subclasses are applied within subpopulations, and model-based adjustments are then used to provide estimates of treatment effects within these subpopulations.

221. Rosenbaum, P.R., & Rubin, D.B. The central role of the propensity score in observational studies for causal effects. Biometrika 70(1):41-55, 1983.

Both large and small sample theory show that adjustment for the scalar propensity score is sufficient to remove bias due to all observed covariates in observational studies. The authors show that the approach can have the following applications: (1) matched sampling on the univariate propensity score, which is a generalization of discriminant matching, (2) multivariate adjustment by subclassification on the propensity score where the same subclasses are used to estimate treatment effects for all outcome variables and in all subpopulations, and (3) visual representation of multivariate covariance adjustment by a two-dimensional plot.

222. Rubin, D.B. Using multivariate matched sampling and regression adjustment to control bias in observational studies. J Am Stat Assoc 74(366):318-328, 1979.

Rubin studied the efficacy of multivariate matched sampling and regression adjustment for controlling bias due to specific matching variables X when dependent variables are moderately nonlinear in X, using Monte Carlo methods. The general conclusion he had is that nearest available Mahalanobis metric matching in combination with regression adjustment on matched pair differences is a highly effective plan for controlling bias due to X.

223. Rubin, D.B. Matching to remove bias in observational studies. Biometrics 29:159-183, 1973.

Several matching methods that match all of one sample from another larger sample on a continuous matching variable are compared with respect to their ability to remove the bias of the matching variable. One method is a simple mean-matching method, and three are nearest available pair-matching methods. The methods’ abilities to remove bias are also compared with the theoretical maximum given fixed distributions and fixed sample sizes. A summary of advice to an investigator is included.

224. Rubin, D.B. The use of matched sampling and regression adjustment to remove bias in observational studies. Biometrics 29:185-203, 1973.

The ability of matched sampling and linear regression adjustment to reduce the bias of an estimate of the treatment effect in two sample observational studies is investigated for a simple matching method and five simple estimates. Monte Carlo results are given for moderately linear exponential response surfaces, and analytic results are presented for quadratic response surfaces. One of the conclusions of the study is that the combination of regression adjustment in matched samples generally produces the least biased estimate.

225. Rubin, D.B. Multivariate matching methods that are equal percent bias reducing, I: some examples. Biometrics 32:109-120, 1976.

Multivariate matching methods are commonly used in the behavioral and medical sciences in an attempt to control bias when randomization is not feasible. Some examples of multivariate matching methods are discussed in Althauser and Rubin (1970) and Cochran and Rubin (1973) but otherwise have received little attention in the literature. The author presents examples of multivariate matching methods that will yield the same percent reduction in bias for each matching variable for a variety of underlying distribution. Eleven distributional cases are considered and for each one, matching methods are described that are equal percent bias reducing.

226. Winship, C., & Mare, R.D. Models for sample selection bias. Annu Rev Sociol 18:327-350, 1992.

This chapter reviews models that attempt to take account of sample selection and their applications in research on labor markets, schooling, legal processes, social mobility, and social networks. Variants of these models apply to outcome variables that are censored or truncated—whether explicitly or incidentally—and include the tobit model, the standard selection model, models for treatment effects in quasi-experimental designs, and endogenous switching models. It also discusses Heckman’s two-stage estimator.

Back to Top

E. Meta-Analysis

227. Bangert-Drowns, R.L. Review of developments in meta-analytic methods. Psychol Bull 99:388-399, 1986.

The author provides a brief history of the development of meta-analysis. The author also distinguishes five different approaches to meta-analytic method and makes suggestions for the use.

228. Bukoski, W.J., ed. Meta-Analysis of Drug Abuse Prevention Programs. National Institute on Drug Abuse Research Monograph 170. NTIS Pub. No. (ADM) 97-181598. Washington, DC: Supt. of Docs., U.S. Govt. Print. Off., 1997.

The monograph provides firsthand guidance in the application of research findings from meta-analysis and appropriate discussion of key technical procedures that should be considered in conducting future meta-analyses of drug abuse prevention research. It also helps to delineate what prevention programs and policies appear to be the most effective in combating drug abuse by adolescents and young adults who may be entering the workplace. In the first section of the monograph, Tobler presents a meta-analysis of adolescent drug abuse prevention research findings; Schmidt and colleagues provide a meta-analysis of integrity tests for predicting drug and alcohol abuse; and Becker provides an approach for meta-analysis of drug-related risk and protective factors research. In the second section of the monograph, several chapters explore the appropriateness and special methodological considerations that must be addressed when conducting a meta-analysis of the drug abuse prevention research literature. Perry's chapter focuses upon methods to calculate effect sizes; Devine's chapter discusses issues in coding prevention intervention studies; Shadish and Heinsman assess the differences in outcomes produced by experimental versus quasi-experimental studies; Matt explores issues concerning generalized causal inferences related to program effects; Hansen reviews approaches to classifying independent variables and types of correlational relationships between dependent and independent variables; in separate chapters, Lipsey and Hedges discuss potential applications of meta-analysis for policy development; and Bangert-Drowns presents general advantages and potential limitations of conducting and utilizing meta-analysis in drug abuse prevention research. Collectively these chapters provide a current overview of the efficacy of drug abuse prevention programs and related measurement systems and help define the techniques employed in meta-analysis of drug abuse prevention programs.

229. Hedges, L.V., & Olkin, L. Statistical Methods for Meta-Analysis. Orlando, FL: Academic Press, 1985.

The main purpose of this book is to address the statistical issues for integrating independent studies through the method of meta-analysis. Chapter 3 of the book provides a review of omnibus procedures for testing the statistical significance of combined results. Later chapters discuss various methods to estimate and combine effect sizes. Chapters 7, 8, and 9 describe the different kinds of analysis that can be used to analyze and compare effect sizes. Chapters 10 and 11 explore the properties of correlated effect size estimates and their analysis. Chapter 12 deals with outliers, and chapter 13 discusses clustering procedures. Chapter 14 demonstrates the effects of censoring of effect size estimates corresponding to nonsignificant mean differences.

230. Hunter, J., & Schmidt, F. Methods of Meta-Analysis: Correcting Error and Bias in Research Findings. Newbury Park, CA: Sage Publications, 1990.

The authors present clear and detailed descriptions of meta-analysis methods, with extensive examples that clarify the applications of these methods. New methods for correcting statistical artifacts not previously addressed are presented for both correlational and experimental studies. Other features of the book include advances in artifact distribution-based meta-analysis methods and applications to examples, examination of the use of regression slopes and intercepts in meta-analysis, detailed treatment of repeated measures of experimental designs and meta-analysis methods for these designs, detailed treatment of second-order sampling error and its associated problems, exploration of the current and future role of meta-analysis in the social sciences, and extensive analysis of criticisms of meta-analysis.

231. Schmidt, F. What do data really mean? Research findings, meta-analysis, and cumulative knowledge in psychology. Am Psychol 47(10):1173-1181, 1992.

In this article, the author presents an introduction on the technology of meta-analysis. He first contrasts the approach with traditional methods and then describes the application of meta-analysis in industrial organizational psychology. Further, Schmidt outlines the role of meta-analysis in theory development and the broader impact it has on scientific research. The author argues that the broader dissemination of meta-analysis will lead to major changes in the way psychologists view the general research process.

232. Rosenthal, R., & Rubin, D.B. Comparing effect sizes of independent studies. Psychol Bull 92(2):500-504.

This article presents a general set of procedures for comparing the effect sizes of two or more independent studies. The procedures include a method for calculating the approximate significance level for the heterogeneity of effect sizes of studies and a method for calculating the approximate significance level of a contrast among the effect sizes. Although the focus is on effect size as measured by the standardized difference between the means (d) defined as (M₁ -M₂)/S, the procedures can be applied to any measure of effect size having an estimated variance. This extension is illustrated with effect size measured by the difference between proportions.

233. Strube, M.J., & Miller, R.H. Comparison of power rates for combined probability procedures: A simulation study. Psychol Bull 99(3):407-415, 1986.

Rosenthal (1978) presented a thorough description of six procedures for combining the significance levels from independent tests of the same conceptual hypothesis. This simulation study compares the power of these six methods. The results indicated that for large numbers of studies to be combined, all procedures provide comparable power. The techniques vary in their ease of computation, however, making some procedures more preferable under certain conditions.

Back to Top

F. Sample Size Calculation

234. Cohen, J. Statistical Power Analysis for the Behavioral Sciences, 2nd ed. Hillsdale, NJ: Erlbaum, 1990.

The book provides a detailed description of the concept of power analysis, and it is written as a handbook on the topics for behavioral researchers. It covers the power calculation for various statistical tests, t-test for means, significance of a product moment r, differences between correlation coefficients, the test that a proportion is .t and the sign test, differences between proportions, chi-square test, ANOVA and ANCOVA, multiple regression and correlation analysis, set correlation, and multivariate methods. The book concludes with chapters on issues in power analysis and some computational procedures.

235. Donner, A. Approaches to sample size estimation in the design of clinical trials: A review. Stat Med 3:199-214, 1984.

The paper reviews various methods of sample size estimation in the design of clinical trials. It is restricted to the discussion of designs with the primary purpose of comparing two groups of patients with respect to the occurrence of some specified event, such as death or the recurrence of disease. It takes into account the special nature of clinical trials that might preclude the direct applications of formal sample size planning and simple and straightforward formulae.

236. Kraemer, H.C., & Theimann, S. How Many Subjects? Beverly Hills, CA: Sage Publications, 1988.

The book treats power in a unified fashion across hypothesis testing techniques by calculating what the authors called critical effect size. It offers a simple introduction for nonstatisticians to power analysis and sample size determination as well as discussion on other topics like the conditions under which a repeated measures design will be more or less efficient than a cross-sectional design; the considerations involved in deciding whether to match or stratify subjects; the selection of variables for multiple regression analysis; the value of equal (or near equal) N in analysis of variance designs; how to insure, in a correlational study, that the study will be valid; and the N required to make a reasonably rigorous test of one hypothesis using the chi-square technique.

237. Muller, K.E., LaVange, L.M., Ramey, S.L., & Ramey, C.T. Power calculations for general linear multivariate models including repeated measures applications. J Am Stat Assoc 87:1209-1226, 1992.

The paper reviews recently developed methods for power analysis, in particular those applicable to general linear multivariate models (GLMM). The paper first discusses the motivation for using detailed power calculations, focusing on multivariate methods in particular. Second, the authors survey available methods for the general linear multivariate model (GLMM) with Gaussian errors and recommend those based on F approximations. The paper covers the multivariate and univariate approaches to repeated measures, MANOVA, ANOVA, multivariate regression, and univariate regression. Third, the authors describe the design of the power analysis for an example of a study that examines the impact of mothers' verbal IQ on children’s intellectual development. Then the authors describe the result of their power analysis and evaluate the tradeoffs in using reduced designs and tests to simplify power calculations. Lastly, the authors discuss the benefits and costs of power analysis in the practice of statistics.

Back to Top

G. Human Subject Issues

238. Hoagwood K., Jensen, P.S., & Fisher, C.B. Ethical Issues in Mental Health Research with Children and Adolescents. Mahwah, NJ: LEA, 1996.

The purpose of the book is to surface the key ethical dilemmas that investigators who study child and adolescent emotional, behavioral, developmental, and mental disorders are encountering, and to offer practical suggestions for integrating ethical thinking into such research. The book is organized into four parts. Part I introduces the major scientific, regulatory, and community principles that guide ethical practices in research on children's mental health. Part II reviews major ethical issues across diverse research contexts. Part II focuses attention on illustrative cases. Part IV discusses the role of bioethicists.

239. Porter, J.P. Regulatory considerations in research involving children and adolescents with mental disorders. In: Hoagwood, K., Jensen, P.S., & Fisher, C.B., eds. Ethical Issues in Mental Health Research with Children and Adolescents. Mahwah, NJ: LEA, 1996, pp. 15-28.

The chapter highlights some of the regulatory considerations in the Department of Health and Human Services (DHHS) regulations for the protection of human subjects at Title 45 Code of Federal Regulations Part 46. These regulations provide the minimum standards for investigators. The author focuses on the regulatory aspects of research involving particularly vulnerable children.

240. Attkisson, C.C., Rosenblatt, A., & Hoagwood, K. Research ethics and human subjects protection in child mental health services research and community studies. In: Hoagwood, K., Jensen, P.S., & Fisher, C.B., eds. Ethical Issues in Mental Health Research with Children and Adolescents. Mahwah, NJ: LEA, 1996, pp. 43-58.

The authors identify major ethical and human subject challenges encountered in conducting research on services for children and conducting research in communities more broadly, including epidemiological surveys. They also identify strategies used by researchers to overcome these obstacles.

241. Hibbs, E.D., & Krener, P. Ethical issues in psychosocial treatment research with children and adolescents. Hoagwood K., Jensen, P.S., & Fisher, C.B., eds. Ethical Issues in Mental Health Research with Children and Adolescents. Mahwah, NJ: LEA, 1996, pp. 59-72.

In the chapter, the authors discuss five specific ethical issues that confront investigators who study the efficacy or effectiveness of psychosocial treatments for children. These issues are competence, informed consent/assent, confidentiality, use of incentives, and selection and involvement of control subjects.

242. Putnam, F.W., Liss, M.B., & Landsverk, J. Ethical issues in maltreatment research with children and adolescents. In: Hoagwood K., Jensen, P.S., & Fisher, C.B., eds. Ethical Issues in Mental Health Research with Children and Adolescents. Mahwah, NJ: LEA, 1996, pp. 113-134.

This chapter addresses some of the complex ethical and legal dilemmas raised in research with maltreated and victimized children. In particular, the authors discuss the general legal and ethical principles for informed consent/assent with minors and the role of parents in consenting for their children to participate in research studies. They also enumerate common concerns raised by researchers and institutional review boards (IRBs) considering research protocols with maltreated children. The authors also mention the mechanism available for clarification of these problems.

Back to Top