View PDF

June 2000

Note that CDC’s “Evaluation Guidance” covers minimum expectations for the evaluation of HIV/AIDS community planning and prevention programming under Program Announcement 99004. You may add evaluation activities beyond those discussed in the “Guidance” based on recommendations from your community planning group (CPG) or an evaluation advisory committee you’ve set up to guide the development of your evaluation plan and oversee evaluation activities. Committee members could include CPG members, CBO staff, and health department contracts and computer staff.

In addition to the recommendations in CDC’s “Supplemental Handbook” on developing an evaluation plan, you may find it helpful to consider some basic questions as the framework for your plan. These questions are who, what, where, when, why, and how.

“Why” refers to the reasons you’re developing and carrying out the evaluation plan. Other than fulfilling CDC requirements, there are things you want to accomplish or questions you want to answer. For example, what you want to accomplish could be stated as goals:

• assess the quality of interventions;
• improve the community planning core objective to foster openness and participation;
• identify how resources are allocated among target population;
• use monitoring and evaluation findings to improve program operations.

You may want to answer certain questions, such as:

• how do CPG members rate the priority setting process?
• who are the clients that receive prevention case management?
• how effective are group education programs?
• in what ways should evaluation results be used to improve community planning and service delivery?

Center your evaluation plan on goals or questions.

ACTIONS

To operationalize the “whys” of your evaluation plan, you need to propose actions. Actions consist of the objectives or activities needed to address goals or answer questions. There will probably be a number of actions under each goal or question; for example:

• collect baseline data;
• review risk assessment form;
• analyze data;
• disseminate results.

It is up to you to decide how detailed your evaluation plan should be. A more detailed plan could include steps under each action.

“Who” refers to the resources you’ll use to carry out actions (i.e., objectives or activities). For example, resources could include:

• health department staff;
• grantee staff;
• CPG committee;
• consultant;
• intern;
• university researcher.

“What” includes the items or materials you need in addition to resources to carry out your objectives or activities; for example:

• quarterly reports from grantees;
• current computer run;
• specific data collection tool, such as CDC- adapted process monitoring form or
• questionnaire for CPG members.

Your evaluation plan may want to specify where activities will take place (or from whom data will be collected); for example:

• all grantees funded under program announcement 99004;
• a sample of CBOs that target African-American MSM (identify the agencies);
• final CPG meeting of the year.

It is important that you identify timeframes for your activities. You want to start with CDC’s deadlines and “back into” timeframes. For example,

• outcome evaluation is due September 2003;
• develop plan (or evaluation protocol) for outcome evaluation by September 2001;
  include a logic model for the intervention being evaluated;
• identify data collection tools (measures, instruments) by January 2002;
• pilot test data collection tools by July 2002;
• carry out data collection between September 2002 and March 2003;
• analyze data by August 2003;
• prepare report by September 2003.

“How” refers to ways in which actions will be carried out and should include activities for data analysis. Examples of “how” include:

• sort data by age, race, ethnicity and gender using relational data base;
• use “t” test to assess statistical significance of differences between pre- and posttest scores;
• reach consensus at evaluation advisory committee meeting.

TECHNICAL ASSISTANCE NEEDS

Remember that CDC’s “Evaluation Guidance” asks you to indicate your evaluation technical assistance needs.

EXAMPLE

It is crucial that each health department have an evaluation plan that satisfies CDC’s requirements and also meets the evaluation needs of CPG members and providers and consumers of prevention services. Therefore, each plan should be custom designed for the jurisdiction. However, in terms of format, you may want to include a table with explanatory narrative.

The following example may prove useful as you develop or revise your evaluation plan.

EVALUATION PLAN EXAMPLE

Evaluation Question:
Who are the clients that participate in health education and risk reduction programs?