Evaluation

Evaluation is the cornerstone to strong prevention planning, execution, and improvement. Senior campus and community officials will want to see evaluation data showing that the prevention efforts are a worthwhile endeavor deserving of continued support.

There are three basic types of evaluation. A process evaluation documents how the prevention work is being implemented and whether it is working as originally planned. An outcome evaluation looks at whether each program and policy is accomplishing its short-term and intermediate objectives. An impact or summative evaluation examines whether the overall prevention effort is reducing student alcohol and other drug (AOD) abuse and its consequences and other violent behavior.

With an evaluation plan in place, program directors can assess whether a particular program or policy was working as intended and then decide whether it should continue as is or be expanded, revised, or terminated. Evaluation is an essential part of strategic planning, because evaluation findings are used to guide plans for midcourse improvements. Effective strategic planning, therefore, is a cyclical process.

Evaluations are most useful when they are planned at the same time as the prevention effort itself, rather than after the fact. This approach ensures that the evaluation design is crafted to fit the program’s goals, objectives, and activities; that the short-term, intermediate, and long-term outcomes are clearly specified; that the data necessary to assess those outcomes are collected; and that the resources needed to conduct a proper evaluation are in place in advance. Planning the prevention effort and its evaluation simultaneously helps the planning group see evaluation as a management tool, rather than as an “add-on” that can be a drain on program resources or an external threat to the program.

A logic model is a valuable tool in the strategic planning process; it is also necessary for evaluation planning. The logic model diagram lists key components of the program and maps out the chain of expected events that show how the specific programs and policies will lead to and accomplish the intermediate and long-term outcomes. The evaluators and program planners’ first step should be to work with the staff to create a logic model that describes the program.

A well-designed evaluation will document how a prevention initiative was implemented and assess its intermediate and long-term outcomes. With these findings in hand, the planning group can develop plans for strengthening or improving that effort. In some cases it will be discovered that a program or policy was not implemented as planned, or that too few resources were invested in it. Or it may be that too few students knew about the initiative for it to be effective. In other cases, the evaluation will show that an entirely new approach is needed.

Embracing evaluation as a management tool is especially important in an academic environment, where the collection, assessment, and interpretation of evidence are highly valued activities. Evaluation will also address the challenge of competing demands on college budgets, allowing senior administrators to know that an initiative works or that evaluation is being used to improve it, before they commit the institution to a long-term investment of resources.