The National Institute on Alcohol Abuse and Alcoholism (NIAAA) is committed to helping colleges
and universities reduce alcohol-related problems on their campuses, protect students from harm,
and improve quality of life for the entire campus community. To guide future efforts, the Advisory
Council on Alcohol Abuse and Alcoholism established a Task Force on College Drinking to review and
report on the existing research on college student drinking, including evaluations of campus and
community policies, prevention programs, and early intervention strategies. A summary of the Task
Force’s report, A Call to Action: Changing the Culture of Drinking at U.S. Colleges,
provides college administrators and program specialists with a useful overview of these
evaluations, which can be used to inform future program and policy development (see http://www.collegedrinkingprevention.gov
for complete text of the report).
This brief guide is intended to provide some direction as to how this research can be
incorporated most effectively into an explicit planning process to not only maximize the impact of
any prevention strategy, but also to actively monitor any intervention’s implementation and local
impact. By tightly integrating evaluation into prevention planning and management, college
administrators and program staff can assure themselves that objectives are clear to all, and that
precious resources are being spent effectively. More broadly, our hope is that, when greater
numbers of college and university administrators commit their institutions to sound planning and
evaluation, all of us will benefit from their work. The following paragraphs show how the guide is
organized.
Steps for Effective Prevention Planning and Evaluation
Thinking about the evaluation as part of the planning process will sharpen everyone’s thinking
about the program: its mission, its goals, its objectives, and the activities designed to meet
those objectives. The process for developing and evaluating prevention programs and policies can
be divided into five basic steps:
Identifying specific goals and objectives
Reviewing the evaluation research
Outlining how the intervention will work
Creating and executing a data collection plan
Providing feedback to the intervention program.
1. Identify Specific Goals and Objectives
The problem—student drinking—is obvious, but exactly which goals and objectives should be
specified to guide campus prevention efforts is not. Is the goal to eliminate college student
drinking? Limit excessive consumption of alcohol? Eliminate alcohol-related behavior problems?
Protect student drinkers from harm? Should the prevention effort focus on student drinking on
campus, or should it also cover off-campus behavior? How college officials answer these questions
will depend on several factors: the philosophy and academic mission of the institution, the nature
of the student alcohol problem, the level of prevention resources available, the views and
opinions of key constituencies, the characteristics of the surrounding community, and the cultural
and political context in which the school operates.
Having an evaluator be part of the planning process from the beginning will help guarantee that
staff have listed out specific goals and objectives. In turn, these goals and objectives can be
translated into specific outcomes that are assessed through the evaluation process. The evaluator
can help a college’s officials reach consensus on their specific goals and objectives. This is a
good example of how planning an intervention can help shape the intervention as well.
2. Review Research on College Drinking Interventions
The next step is to review program and policy options that might be applied to achieve the
outlined goals and objectives. We present a typology of prevention interventions that comprises
programs and policies classified into one of the following levels: 1) individual, 2) group, 3)
institution, 4) community, and 5) State and Federal public policy. Many areas of strategic
intervention can be pursued at one or several levels of the social ecological framework.
Implementing multiple strategies from these various levels would greatly increase the likelihood
of the objective being achieved.
In this section we summarize some of the major findings from a review of the literature on
college-focused prevention, organized according to the levels of intervention (1-5 above).
3. Outline How the Intervention Will Work
A review of available research, plus consultations with other college and university prevention
specialists, will suggest a set of program and policy options that can be adopted. The next
planning step is to outline the chain of events that will lead from implementation of each
component program or policy to its specific (and measurable) objective. This is often called
building the “logic model” for the intervention. We provide a simple example of the kind of flow
chart that is often the clearest and most economical way of presenting this information (see
Figure 1).
There are several reasons why this step is important:
First, developing the logic model will pinpoint areas of uncertainty, confusion, or
disagreement among members of the planning team.
Second, work on the logic model can make transparent any false assumptions that need to be
addressed.
Third, development of the logic model will help guarantee that all program activities and
policies can be logically linked to the achievement of specific objectives.
Fourth, a logic model can later serve as an educational and communications tool when a new
program or policy is being implemented.
Fifth, a logic model can be a tool for tracking changes in the intervention or its
implementation.
Finally, the logic model helps inform the evaluation so that it can answer the fundamental
question of whether the program effects were smaller (say) because the fundamental concept behind
the intervention was wrong, the implementation was flawed, or one piece of the intervention
sequence fell apart.
4. Create and Execute a Data Collection Plan
Self-report surveys are a primary data source for program and policy evaluations, especially if
the goal is to reduce consumption or alcohol-related problem behaviors. If a student survey is to
be part of an evaluation, we briefly describe some basic requirements of a valid and useful
survey. It should be remembered, however, that a student survey is not the only source of useful
data, and in some cases may not even be the best source. Ideally, colleges and universities will
put in place a system for recording a wide range of alcohol-related incidents involving students.
These might include data from urgent or emergency care facilities, campus police student
counseling services, residence halls, athletic departments, and offices of student discipline.
On many campuses, the problem is that data are recorded but are not easily accessed, but this
situation is improving as offices move toward using computerized databases and automated data
entry. As these systems are put in place, administrators should be sure that records of campus
problems make note of alcohol involvement.
5. Communicating Evaluation Results: Feedback
The full value of any evaluation is only realized when it can provide ongoing feedback to the
program and the affected community at large. Often, this feedback is necessary just to create
support for the program or intervention to be continued. Important information on individual
program components may also prove valuable for continuously improving the intervention or for
guarding against degradation in the program’s impact.
Program Evaluation: The Big Picture
Newcomers to the topic of college student drinking are often puzzled to learn that the field’s
knowledge of “what works” is relatively slim. Apart from some recent and promising interventions
aimed at individual students, the conscientious program planner will find little empirical
evidence to guide choices of program and policy interventions aimed at the broader college
population. The broader field of prevention research, which has examined the impact of programs
and policies aimed at youth in the general population, provides useful guidance. Even so, it is
clear that evaluations of environmentally focused prevention strategies that focus specifically on
college populations are sorely needed.
We are urging higher education administrators to incorporate evaluation as an integral part of
program planning, which we view to be essential to developing more effective prevention programs
and policies. We hope that administrators will realize that the evaluations they undertake will
also contribute significantly to our knowledge of “what works.” Conducting and then sharing the
results of evaluations of alcohol prevention efforts is necessary to meet that larger goal.
Resources
The guide concludes with a number of references, both publications and Web sites, that directly
relate to the topic of alcohol-related problems among college students.
