Guide 6: Define an Evaluation Plan Up Front

Think ahead--evaluation is integral to the progam plan

Up to this point in developing a community health information program, you have analyzed community needs and developed related goals and activities to address them. In addition, the underlying assumptions about intended accomplishments (outputs) and results (outcomes) are well defined and clarified in your program logic model.

The next step-- developing a realistic and meaningful evaluation plan-- is important to think through while in the planning stage, and not as an afterthought when your program is near completion.

Why is evaluation important?

If done well, an evaluation of selected outputs and outcomes helps to sharpen an outreach focus; provide accountability to our funders, managers, or administrators; improve quality and effectiveness; and better understand what has been achieved and the impact made.

Programs can usually track a program's outputs, in terms of the amount, quality, or volume of use of products or services. Output measurements help to track a program's implementation and whether the program is achieving what it set out to do.

But, even if the products, services, or activities are put in place as planned, there is always the nagging question about what differences (outcomes) the program makes for the participants and community involved.

At the 2003 Quintessential meeting in Philadelphia, the Institute of Museum and Library Services gave a thorough presentation about evaluation planning focusing on outcomes, titled A Case for Libraries: Why Outcomes Matter. Featured is a health information program called "Library RX" for public library staff in West Dakota. The intended outcomes are that public library staff will have basic computer skills, be information literate, and find high-quality health information quickly.

What does an evaluation plan include?

The challenge of planning an evaluation is to determine realistic priorities about what will be assessed and how best to gather and analyze the measurements you obtain.

Refer back to the program logic model, where you filled in the outputs and outcomes of your program.

Outpus and outcomes can be measured once you have identified the type of data you will collect. An evaluation plan puts it all together and sorts out:

Some negative perceptions and fears about evaluation stem from a feeling that it involves a lot of effort and that results are not always used. It's true that an evaluation plan should be realistic. As a general rule of thumb, allocate 5% of the total program budget to evaluation efforts. This means making priorities about what you want to find out and why, including what you will do with the results. Think about your stakeholders and what they most want to know. Imagine the range of possible results from your evaluation, and how you will use that information.

The efforts to think ahead about what you want to find out and why will not only help with data collection planning, it will help you frame the right questions and avoid collecting data just to say you have done an evaluation.

Resources for evaluation planning

To help you think through components of the evaluation plan, the Institute of Museum and Library Services tutorial for National Leadership Grants includes an excellent section on the Evaluation Approach. This tutorial includes downloadable templates for both output and outcome evaluation approaches.

For a completed example based on the IMLS templates, see an outputs and outcomes evaluation plan for Your Library Community Partnership for Breast Cancer Prevention.

For a guide to planning and evaluating health information programs, see Measuring the Difference: Guide to Planning and Evaluating Health Information Outreach. This 130-page guide is a primer (including tools and resources) developed by the National Network of Libraries of Medicine, Pacific Northwest Region and the National Library of Medicine. To receive a single free copy, send an email request for "evaluation guide" to nnlm@u.washington.edu. Include your name and mailing address.