Programs & Tools:

Immunization Program Evaluation (IPE)

General IPE Questions

1. How is CDC going to use the findings of our evaluation activities?

   The primary reasons for requiring grantees to engage in evaluation are to:

   1. ensure that the activities grantees are engaging in are yielding desired short- and long-term results;
   2. identify ways in which activities can be improved; and
   3. use findings to maximize grantee efforts and resources.

2. Will grantee funding change (increase or decrease) based upon the findings of our evaluation activities?

   No, changes in funding will not be linked to evaluation activities or their findings.

3. Once we have developed an evaluation plan, how long do we have to conduct our evaluation activities?

   Grantees are required to submit an evaluation plan on August 31, 2008; however, given that the scope and magnitude of evaluation activities will vary by grantee and component, there is no set time limit for completion of evaluation activities.

4. Do we need to choose a new component to evaluate when we finish with the current one?

   We expect that grantees will be engaged in program evaluation in an ongoing manner, i.e. once one evaluation is completed, another evaluation plan will be developed and implemented. The latter could be based on a different component or could be another aspect of the first component.

5. Do we have to submit a new evaluation plan with each grant application?

   No; however, because we expect that grantees will be engaged in program evaluation on an ongoing basis, once one evaluation is completed, another evaluation plan should be developed and implemented.

6. How can I find out what other grantees are evaluating?

   We plan to make this information available at the Immunization Program Evaluation website at http://www.cdc.gov/vaccines/programs/progeval/ once evaluation plans are submitted in August 2008. The Immunization Services Division is currently making arrangements to enable grantees to submit their plans and subsequent progress reports through the Program Annual Progress Assessments (“PAPA”) application (available at https://sdn.cdc.gov). This is the same application through which grantees currently submit other annual reports and surveys. In addition to the IPE website, grantees will be able to view other grantees’ evaluation activities through PAPA.

7. The IPOM does not have indicators or performance measures for the component or activity (within a component) we are planning on evaluating. Where can I find them?

   The Immunization Services Division is working to develop indicators for all components. In the meantime, grantees are encouraged to create their own indicators if the chosen component or activity (within a component) does not have existing indicators. Information about indicators, including examples, is provided in “Step 4,” of Chapter 2 of the Guide to Immunization Program Evaluation which can be found at the Immunization Program Evaluation website at http://www.cdc.gov/vaccines/programs/progeval/

8. What activity/aspect within our selected component should we evaluate?

   Grantees should make this decision based on answers to questions such as:

   1. Where do we need to improve?
   2. Where do we use substantial resources?
   3. How useful will this information be in enabling us to change our program?
   4. What evaluation is most feasible?
   5. Do we have the ability/authority to make changes based on the findings of this evaluation?
      
9. Are we expected to hire a contractor?

   No, there is no such expectation; however, the Immunization Services Division realizes that some grantees may wish to take this route. For this process to yield useful results, grantees must ensure that there is a close collaboration between the program, the involved stakeholders, and the contractor throughout the evaluation.

10. How much does it typically cost to hire a contractor to do an evaluation?

    The cost likely depends on their scope of work which will be determined by the component/activity grantees choose to evaluate.

11. What should we know in advance prior to hiring a contractor?

    Grantees should work closely with the contractor and their stakeholders to define evaluation questions that are useful for the program, or determine before contacting the contractor what activities the contractor should conduct. For example, contractors could be hired to conduct discrete tasks such as data gathering and analysis. Regardless of the scope and role of the contractor, grantees are more likely to attain the desired results (e.g. useful evaluation report, building evaluation capacity within the program) if they work hand-in-hand with the contractor.

IPE Evaluation Plans Questions

1. How does the 2008 Evaluation Plan relate to the 2008 grant submission requirements in eGrATIS?
   
   To help grantees simplify the submission in eGrATIS related to Program Evaluation (IPOM, Chapter 1, Requirement 1.2) we have developed an example (below) that grantees can customize based on the component/activity they are evaluating. This example is loosely based on the Sample AFIX Evaluation Plan that was distributed to grantees via email and available at the Immunization Program Evaluation website http://www.cdc.gov/vaccines/programs/progeval/default.htm
   
   Recommended eGrATIS EXAMPLE:
   I: Objective
   Start time: January 1, 2009
   End time: December 31, 2009
   Who will do the work: Evaluation team members
   Primary Action: Evaluate
   
   II. Evaluation Measure
   Progress towards implementation of the AFIX evaluation plan.
   Note: AFIX is an example, replace with the component/activity you are evaluating.
   
   III. Activities
   1. If needed, make changes to the evaluation plan based on feedback from CDC 2. Implement the evaluation plan
   3. Submit progress reports as required
   4. As appropriate, utilize evaluation findings in a timely manner
   5. Follow evaluation guidelines provided CDC
   
   IV. Associate objective with IPOM requirement
   Chapter 1, Program Requirement 1.2
   Note: If you choose to, you may associate the objective (above) to one or more additional requirements (e.g. this objective could also be associated with Chapter 4, Program Requirement 4.1); however, this is NOT required.
   
   
2. When is the evaluation plan due?

   
   Each grantee must submit their plan by August 31, 2008. The Immunization Services Division is currently making arrangements to enable grantees to submit their plans and subsequent progress reports through the Program Annual Progress Assessments (“PAPA”) application (available at https://sdn.cdc.gov). This is the same application through which grantees currently submit other annual reports and surveys.

3. What is CDC planning to do with the evaluation plans?

   Immunization Services Division (ISD) staff will review grantee plans using the 2008 Evaluation Plan Review and Feedback form (this form will be distributed to grantees and posted on the IPE website shortly) by October 31, 2008.

4. How long or short does the evaluation plan have to be?

   There is no set length; however, all aspects listed in the Evaluation Plan Template must be addressed. The template can be found at the Immunization Program Evaluation website at http://www.cdc.gov/vaccines/programs/progeval/



5. When I start submitting my 2008 Evaluation Plan via PAPA, can I copy and paste from a word document?
   
   Yes. Many grantees will likely have typed out their plan in a word document (using the 2008 Evaluation Plan Template). If that is the case, to save time, grantees can cut or copy text from the word document and paste into the fields in PAPA.
   
   
6. We’re planning on evaluating two components. Do we need to submit 2 separate plans?

   Most immunization programs have not previously conducted evaluations based on the CDC framework; therefore, we encourage grantees to carefully consider the size and scope of their first evaluation activity and suggest that grantees narrow their scope by concentrating on no more than one or two aspects/activities within one component. However, for programs who wish to undertake the evaluation of two components, programs should develop and write up the plans together if they are linked and separately, if they are not.

7. What will we be required to submit after the plan?

   Grantees will be required to submit annual progress reports; the due dates are not yet determined. The Immunization Services Division (ISD) of the National Center for Immunization and Respiratory Diseases (NCIRD) is currently making arrangements to enable grantees to submit their plans and subsequent progress reports through the Program Annual Progress Assessments (“PAPA”) application (available at https://sdn.cdc.gov). This is the same application through which grantees currently submit other annual reports and surveys.

8. Does CDC have a preference for the methodology for data collection (e.g. sample size; use of key informant interviews versus focus groups versus questionnaires)?

   There is no standard required methodology; the methodology grantees choose should be adequate to answer their chosen questions. Grantees should consider their resource availability and the feasibility of the methodology. We welcome discussions with grantees who would like to discuss methodology issues. Additional information is provided in “Step 3,” and “Step 4” of Chapter 2 of the Guide to Immunization Program Evaluation which can be found at the Immunization Program Evaluation website at http://www.cdc.gov/vaccines/programs/progeval/

9. What should I do if I have a question, comment or suggestion?

   Please contact your project officer or email the Immunization Program Evaluation workgroup directly at IPE@cdc.gov. We strongly encourage all grantees to contact us with any questions or concerns throughout their evaluation planning and implementation! 

• Email this page
• Printer-friendly version
• Get email updates

• Help
• Glossary / Acronyms
• Site Map