Note that CDC’s “Evaluation Guidance” covers minimum
expectations for the evaluation of HIV/AIDS community
planning and prevention programming under Program
Announcement 99004. You may add evaluation activities beyond
those discussed in the “Guidance” based on recommendations
from your community planning group (CPG) or an evaluation
advisory committee you’ve set up to guide the development of
your evaluation plan and oversee evaluation activities.
Committee members could include CPG members, CBO staff, and
health department contracts and computer staff.
In addition to the recommendations in CDC’s “Supplemental
Handbook” on developing an evaluation plan, you may find it
helpful to consider some basic questions as the framework
for your plan. These questions are who,
what, where,
when,
why, and how.
WHY
“Why” refers to the reasons you’re developing and
carrying out the evaluation plan. Other than fulfilling CDC
requirements, there are things you want to accomplish or
questions you want to answer. For example, what you want to
accomplish could be stated as goals:
- assess the quality of interventions;
- improve the community planning core objective to
foster openness and participation;
- identify how resources are allocated among target
population;
- use monitoring and evaluation findings to improve
program operations.
You may want to answer certain questions, such as:
- how do CPG members rate the priority setting
process?
- who are the clients that receive prevention case
management?
- how effective are group education programs?
- in what ways should evaluation results be used to
improve community planning and service delivery?
Center your evaluation plan on goals or questions.
ACTIONS
To operationalize the “whys” of your evaluation plan, you
need to propose actions. Actions consist of the objectives
or activities needed to address goals or answer questions.
There will probably be a number of actions under each goal
or question; for example:
- collect baseline data;
- review risk assessment form;
- analyze data;
- disseminate results.
It is up to you to decide how detailed your evaluation
plan should be. A more detailed plan could include steps
under each action.
WHO
“Who” refers to the resources you’ll use to carry out
actions (i.e., objectives or activities). For example,
resources could include:
- health department staff;
- grantee staff;
- CPG committee;
- consultant;
- intern;
- university researcher.
WHAT
“What” includes the items or materials you need in
addition to resources to carry out your objectives or
activities; for example:
- quarterly reports from grantees;
- current computer run;
- specific data collection tool, such as CDC-
adapted process monitoring form or
- questionnaire for CPG members.
WHERE
Your evaluation plan may want to specify where activities
will take place (or from whom data will be collected); for
example:
- all grantees funded under program announcement
99004;
- a sample of CBOs that target African-American MSM
(identify the agencies);
- final CPG meeting of the year.
WHEN
It is important that you identify timeframes for your
activities. You want to start with CDC’s deadlines and “back
into” timeframes. For example,
- outcome evaluation is due September 2003;
- develop plan (or evaluation protocol) for outcome
evaluation by September 2001;
include a logic model for the intervention being
evaluated;
- identify data collection tools (measures,
instruments) by January 2002;
- pilot test data collection tools by July 2002;
- carry out data collection between September 2002
and March 2003;
- analyze data by August 2003;
- prepare report by September 2003.
HOW
“How” refers to ways in which actions will be carried out
and should include activities for data analysis. Examples of
“how” include:
- sort data by age, race, ethnicity and gender using
relational data base;
- use “t” test to assess statistical significance of
differences between pre- and posttest scores;
- reach consensus at evaluation advisory committee
meeting.
TECHNICAL ASSISTANCE NEEDS
Remember that CDC’s “Evaluation Guidance” asks you to
indicate your evaluation technical assistance needs.
EXAMPLE
It is crucial that each health department have an
evaluation plan that satisfies CDC’s requirements and also
meets the evaluation needs of CPG members and providers and
consumers of prevention services. Therefore, each plan
should be custom designed for the jurisdiction. However, in
terms of format, you may want to include a table with
explanatory narrative.
The following example may prove useful as you develop or
revise your evaluation plan.
EVALUATION PLAN EXAMPLE
Evaluation Question:
Who are the clients that participate in health
education and risk reduction programs?
ACTION |
WHO |
WHAT |
WHERE |
WHEN |
HOW |
Determine which data to collect |
Evaluation Coordinator and eval. advisory committee |
n/a |
HD |
July 1 eval. advisory committee meeting |
Review quarterly report forms; review CDC guidance on
monitoring implementation |
Collect data |
Evaluation Coordinator; contracts staff; HD computer
staff |
Revised quarterly report form that incorporates CDC
requirements |
All grantees will submit hard copies of the report to HD
contracts staff |
Reports are due April 15, July 15, Oct. 15, Jan. 15 |
HD computer staff will enter data and generate quarterly
aggregated report |
Analyze data |
Evaluation Coordinator; HD computer staff |
Aggregated quarterly reports |
HD |
One month after each quarterly report is due |
Using relational data base, sort data by risk
population, race, ethnicity, age, and gender; sort by agency, setting, #
of sessions |
Interpret data |
Evaluation Coordinator, CPG members |
Quantitative data analysis (see action above) |
HD |
CPG meeting each quarter |
Qualitative interpretation of data, with implications
for community planning |
|