ISSUES AND STRATEGIES FOR ASSESSING BENEFITS, COSTS, AND COST EFFECTIVENESS OF PREVENTIVE INTERVENTIONS IN CHILD SERVICE SETTINGS
SUMMARY
This session was chaired by Jeffrey Evans (NICHD). Although there are programs for young children (e.g., the Nurse-Family Partnership [home visiting] and Incredible Years Series) that have demonstrated economic benefits, there is still a need for further economic analysis of early interventions. This panel addressed some of the issues involved in developing economic data for interventions in early childhood. Greg Duncan (Northwestern University) discussed the use of intervention effect sizes for guiding policies for early intervention (i.e., large apparent effect sizes may cost too much to be worth the investment, while inexpensive programs with modest effects may be cost-effective when viewed from a population-base perspective) among other economic considerations; Michael Foster (University of North Carolina) recommended a culture of open inquiry where data are accessible to researchers to assess the findings from effectiveness studies using state-of-the-art methodology; and Steve Aos (Washington State Institute for Public Policy) encouraged researchers to collaborate with policy experts to help translate findings into usable material for consideration by policy decisionmakers at the State level. Highlights of the discussion, moderated by Alka Indurkhya (Harvard University), included the need to have convincing evaluation designs and appropriate perspectives of economic analysis, as well as a focus on monetary and non-monetary benefits, not just effect sizes. There was also a discussion about incentives for sustainability and the need for support for replication research.
Penny Wise and Effect Size Foolish
Greg Duncan, Ph.D.
The pros and cons of relying on effect sizes of early interventions to guide policy are described, and an alternative approach to presenting policy-relevant results based on the costs and benefits of early intervention is described. Methods for estimating costs and benefits are suggested, and specific examples from early intervention findings are used to illustrate these points. Estimates of costs and benefits should capture a wide array of issues that may be salient to policymakers, such as costs to participants in addition to taxpayer costs, or benefits of impacting multiple related outcomes along with the target outcome. In sum, policy should be guided by good evaluation designs, consideration of benefits relative to costs (not just effect sizes), and a wide-ranging look at policy options.
A Culture of Open Inquiry: Producing Credible Evidence on Program Effects
E. Michael Foster, Ph.D.
The translation of research-based interventions into real-world settings will require higher standards of intervention evidence. The need to establish a culture of open inquiry for intervention findings is addressed, including better data sharing that will allow for outside analysis of robustness and representativeness of findings. In addition, improved methodology, analytic plans, and reporting are important, as is research that is conducted by a mix of investigators to match the expertise required to test interventions.
Statistics for the State House: Outcomes of Prevention Programs that Matter for Policy-Level Decisions
Steve Aos, M.S.
What research-based information motivates State-level policymakers to take action? Perspectives on the types of information from program evaluations that have proven useful (as well as those that have not) to state legislators and executive staff are considered. The outcomes identified as most salient are those that use some measure of costs and benefits, and those that focus on "big picture" outcomes of interest to legislators. The production of some of these big-picture outcomes places significant additional research demands on program evaluation. Approaches that have been tried in Washington State are used as examples.
|
