|
|
|
|||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||
TB Notes NewsletterReturn to TB Notes 2, 2006 Main Menu This is an archived document. The links and content are no longer being updated. No. 2, 2006HIGHLIGHTS FROM STATE AND LOCAL PROGRAMSLessons Learned in the Process of EvaluationLast year, the Metropolitan Chicago TB Consortium, the State of
Lesson 1: Allay fears about the evaluation process.We found that it was critical for all divisions or parties that will be affected by the evaluation to have a sense of security about what was being done. To allay fears and clear up misunderstandings, a process of education was put in place. Education and training sessions were arranged, and an introduction to the evaluation process was begun. A fair amount of time was directed toward reassuring staff that a program evaluation such as the one we were undertaking was not the same as a personnel review. As the staff members were educated on the concept of evaluation, they became a little more relaxed about it. We went through a step-by-step process of describing what was involved, and how it would require some time to discuss what the evaluation would accomplish. As we discussed the whole issue of stakeholders, it became clearer that the TB staff themselves were the major stakeholders, especially when it came to who had the most impact and who could effect change. Lesson 2: Develop a description that sets the stage for the evaluation.A significant amount of time was devoted to developing a comprehensive description of the TB program. The description included a discussion of how the TB program operates, number of clinics, staff, models of operation, relations within the department, relations with private providers, who oversees DOT, who manages the case, and how reporting works. We also included a description of the system from the patient’s perspective. We described the process that a patient goes through prior to being diagnosed with an active case. This description helped us narrow our ideas and select “completion of therapy” as the objective for evaluation. Lesson 3: Take time to determine the program’s stakeholders.Determining stakeholders is a vital part of the entire evaluation and can be improperly used when determined in haste. Discussions about who would be considered a stakeholder needed to be held at several different levels, both for input and buy-in and to ensure that we were not inadvertently missing a critical participant. Determining the stakeholders required extensive discussions, since both community members and health department employees have a stake in patients’ completion of treatment. Owing to the nature of the disease, it seems at first glance that stakeholders are almost everyone who may be impacted as well as an assortment of constituencies. Narrowing down the group of people to those who really have a stake in what is being evaluated is a major element in addressing the problem to be evaluated. It is necessary to determine not only who will be affected but, more importantly, who can do something about an identified situation and who can remedy any identified problems. It is crucial to understand the layers, if they exist, of the persons involved; a supervisor may understand that a problem exists or a physician may know what the specific symptom means, but it may require additional personnel to fix the problem or to set up therapy and manage the care of the patient. The stakeholders were all a bit apprehensive at being part of the process and were concerned about how it would reflect on them. However, they were guaranteed anonymity. Lesson 4: Program context and resources available for program evaluation are important.The TB program is a division of the City Department of Public Health (CDPH), with major funding coming from both CDC and CDPH corporate funding. The TB control program is currently short-staffed owing to an early retirement in April 2004. At the time the evaluation began, there was no one available for or dedicated to program evaluation and performance standards. Lesson 5: Program development can be affected by the evaluation process.We found that the process for implementing the evaluation—engaging stakeholders and developing a description that encompasses multiple perspectives—provided us with some valuable information about our program. For us, the evaluation questions became part of the evidence. The evaluation questions led to very specific thoughts on where and how to proceed. It told us some of the specific items that were amenable to change and that would make an immediate difference. This information has already led to some changes that are creating improvements. For example,
Lesson 6: Program evaluations have the capacity to detect both small and large problems.Although we are still in the process of analyzing data, our early analysis and interpretation of findings concluded that more education and training in the area of evaluation needed to be in place. Additionally, the findings are still being explored, but areas that needed immediate attention were found. —Reported by Phyllis J. Handelman, Evaluator, Handelman
Consulting Ltd.
Last Reviewed: 05/18/2008 |
|
|
If you would like to order any of the DTBE publications please visit the online order form. You will need Adobe Acrobat™ Reader v5.0 or higher to read pages that are in PDF format. Download the Adobe Acrobat™ Reader. If you have difficulty accessing any material on the DTBE Web site because of a disability, please contact us in writing or via telephone and we will work with you to make the information available. Division of Tuberculosis Elimination
Centers for Disease Control & Prevention |