Evaluation of AHRQ's Partnerships for Quality Program

You Are Here: AHRQ Home > About AHRQ > Mission & Budget > Evaluation of AHRQ's Partnerships for Quality Program > Appendix B (continued)

Appendix B: Summaries of PFQ Grantee Activities (continued)

PFQ Grant Summary: Striving Together, Improving Healthcare

Lead Organization: Texas A&M University System (TAMUS), Health Science Center
Partner Team: Texas A&M Rural and Community Health Institute, Texas A&M Health Science Center Office of Homeland Security, Altarum Research Institute, Inc., Air Force Texas Center for Medical Strategy Training and Readiness (first year only)
Title: Striving Together, Improving Healthcare
Topic Area: Bioterrorism/Emergency Preparedness
Principal Investigators: Josie R. Williams, director of Rural and Community Health Institute, Texas A&M University System Health Science Center; co-principal investigator Janine C. Edwards, research professor, TAMUS
AHRQ Project Officer: Sally Phillips
Total Cumulative Award: $399,816
Funding Period: 9/2002–9/2006
Project Status: Completed 9/29/06

1. Project Description

Goals. The project had two original aims: (1) to improve type 2 diabetes care in partner hospitals, clinics, and other organizations by implementing a care management intervention and (2) to conduct a case study of the management of bioterrorism (BT) funding on the readiness of public health and acute care systems in selected Texas Department of Health regions to respond effectively to BT threats. When the first component on diabetes care was not funded, the grantee changed its project to focus solely on the bioterrorism component. It revised its goal as "the formation of partnerships that will facilitate the study of important factors related to preparedness for bioterrorism and natural disaster."

Activities and Progress. During the first year, the project formed an Advisory Council to guide the study of selected regions' use of U.S. Centers for Disease Control bioterrorism preparedness funding and conducted and completed case studies of Public Health Region 8 (the San Antonio metropolitan area and 21 surrounding counties) and Region 2/3 (Dallas/Fort Worth metropolitan area). It found that (1) a regional strategy for resource allocation can be more effective in providing essential epidemiology services to small rural counties than a strict per capita allocation to each county; (2) regular disease surveillance systems can be used for bioterrorism incidents; (3) clear lines of authority and cooperation across those lines of authority are needed; (4) personal relationships and trust are critical to building relationships for preparedness, with such relationships developed through regular communication and the fulfillment of promises in allocating funds; and (5) continual and clear communication is necessary to achieve bioterrorism preparedness among an established network of people. The study found that Region 8 had one of the best emergency preparedness plans in the country, as confirmed by its subsequent response to Hurricanes Katrina and Rita.

The case study also found that public health officials experienced difficulty in obtaining the cooperation of physicians in all public health matters, even in state-required reporting of infectious disease cases. Therefore, the research team developed a learning exercise about Avian flu for medical students, which it taught to second-year students at the Texas A&M College of Medicine. The exercise emphasized the importance of reporting requirements and cooperation among all sectors for both emergency preparedness and day-to-day use.

Given that disease surveillance is such an important component of an effective disaster preparedness system, the project decided in its second year to study how disease surveillance methods in Texas and Mexico could affect the delivery of health care services in the event of bioterrorism or natural disaster along the U.S.-Mexico border. The project team conducted interviews with public health officers, emergency managers, the director of the U.S. Air Force surveillance agency, two health officers for the Mexican border town of Acuna, and the Texas state epidemiologist. The study found that information flows rely on a mix of statutory and informal networks; that public health officers working in the field often have no formal training in public health; that many doctors and hospitals do not routinely report on reportable diseases; and that obstacles prevent information sharing about disease surveillance on the Texas-Mexico border. It recommended improved information infrastructure at the local public health level and between U.S. and Mexican public health officials.

In the third year, the project team used the findings from the study of U.S.-Mexico border disease surveillance issues to help the Altarum Research Institute, another grantee and partner in the program, develop a causality prediction model to estimate the effects of early detection strategies for smallpox and influenza. It found, for example, that the effect of restricting casual contacts by infected individuals was greatest for the first couple of contacts, suggesting that absolute quarantines would not be necessary or cost-effective. This finding prompted the project team to expand its study of disease surveillance at international borders to the U.S.-Canada border.

Through Altarum's contacts, the study team formed an informal partnership with Michigan public health officials to undertake research on areas of similar and dissimilar concern about infectious disease surveillance at both the northern and southern U.S. borders. The research identified four issues that should receive priority: (1) robust bi-national health organizations that overcome jurisdictional obstacles to public health; (2) funding for border health security; (3) local-regional public health agencies able to function relatively independently during disaster; and (4) mechanisms to identify and properly manage emerging health disparities at both borders. At the state and federal levels in the United States, Canada, and Mexico, the findings recommended efforts to develop formal communication channels at the federal level among all three governments and to resolve differences in diagnostic standards and reporting requirements for communicable diseases. It also recommended creating and funding a bi-national border organization between the United States and Canada and providing adequate funding for existing U.S.Mexico bi-national organizations. Finally, the research recommended planning and exercising effective preparedness for all types of disasters across the international borders.

In the final year of the project, the team had two goals. It planned to complete its analysis of disease surveillance communication patterns and problems on both U.S. borders and to conduct disaster-training exercises in small rural hospitals that belong to a network of Texas A&M's Rural and Community Health Institute. The training exercises or drills focus on Avian flu to enable small, rural hospitals to approximate the preparedness achieved by urban hospitals with more extensive resources and training opportunities. The exercise used an AHRQ-developed tool called Evaluation of Hospital Disaster Drills: A Module-Based Approach.

2. Partnership Structure/Function

The project investigators created an Advisory Council that met on a quarterly basis to provide input into and feedback on the project and its findings. In addition to staff at Texas A&M Health Sciences Center, the Advisory Council included the director of Texas Public Health Region 8, the School of Rural Public Health, and the head of the Texas Department of Health's State Epidemiology Office. The Texas Department of Public Health's Region 8 was more the subject of the project's first case study than a partner in carrying out the research. The lead organization, TAMUS, also developed a partnership with the Altarum Research Institute during the first six months of the project after learning that both it and Altarum had a mutual interest in disaster preparedness.

Table 1. Major Partner Organizations and Roles in the Project

Partner Organizations

Organization

Role in Project

Lead Organization (grant recipient)

Texas A&M University Systems (TAMUS) Health Science Center, Rural & Community Health Institute (RCHI)

Co-principal investigator responsible for communicating with partners; deciding on research design, regions to be studied, staff Advisory Council; leads and directs all data collection and analyses and reports.

Directed by principal investigator, provides platform for disseminating lessons learned to hospitals in RCHI network

Key Collaborators

Altarum Research Institute, Inc.

Collaborator in conducting studies of disease surveillance using its electronic model for healthcare

Target Organizations

Medical students to test training program involving an Avian flu exercise

Conducted Avian flu disaster drills in 15 rural hospitals

3. Project Evaluation and Outcomes/Results

The project engaged an independent qualitative evaluator who reviewed the case study and wrote a report of the first year's work. Project outcomes consisted of (1) reports (see below) and publications whose findings have lessons and potential applicability elsewhere and (2) disaster preparedness training exercises for medical students and rural hospitals. Medical students provided feedback on the Avian flu training exercise, and independent public health officials observed and wrote reports for each participating hospital on the rural hospital training exercise.

The case studies produced several important recommendations for policy and practice. One recommendation is for state and national public health officials to develop policies that target funds to disease surveillance methods that produce the greatest impact in mitigating disease burden in BT and natural disasters, particularly in U.S. border areas, which are widely acknowledged to pose risks to homeland security. However, the existence of 50 state systems impedes rapid communication with Canadian and Mexican authorities, which operate centralized disease surveillance reporting systems. Additional policy recommendations include the need for robust bi-national health organizations to overcome jurisdictional obstacles to public health; the need for local-regional public health agencies that function relatively independently during disasters; and the need to understand and properly manage emerging health disparities at both borders.

4. Major Products

Akins, R. et al. "The Role of Public Health Nurses in Bioterrorism Preparedness." Disaster Management and Response Journal. Disaster Management & Response: DMR Vol. 3, No. 4, pp. 98-105.
Edwards, J. et al. "Lessons Learned from a Regional Strategy for Resource Allocation." Biosecurity and Bioterrorism: Biodefense Strategy, Practice, and Science 2005. Vol. 3, No. 2, pp. 113-118.
Silenas, R. et al. "Influenza Pandemic: A Disaster Preparedness Exercise for Medical Students." Submitted to Teaching and Learning in Medicine, March 2006.
Silenas, R. et al. Presentation at Academy Health Conference in Boston, June 2005, on "Closing the Gap between Biological Agent Detection and Response."
Silenas, R. et al. Presentation at TRIP Conference in Washington, DC, July 2005 on "Improving Disparities in Healthcare through Disease Surveillance at the Field Level."
Silenas, R. et al. "Syndromic Surveillance: Potential Meets Reality." Proceedings of the National BTR 2005 Conference. University of New Mexico.
Williams, J. et al. "A Case Study of Surveillance in Texas Department of State Health Services, Region 8." Technical Report. Rural and Community Health Institute, Health Science Center, Texas A&M University System, October 2004.
Williams, J. et al. "Study of Disease Surveillance Policy Issues across the International Borders of the United States." Technical Report. Rural and Community Health Institute, Health Science Center, The Texas A&M University System, April 2006.

5. Potential for Sustainability/Expansion after PFQ Grant Ends

The hospital exercises conducted in March 2006 merged the Rural and Community Health Institute (RCHI) network with the work of this project, which holds potential for sustainability of disaster preparedness work in small, rural Texas hospitals. For example, three hospitals that did not participate in the March training program have asked the team to conduct the exercise again. The RCHI network offers the potential for sustaining disaster preparedness activities. The team also plans to pursue funding for continued work with Altarum, the delivery of training exercises for rural hospitals, and additional studies of U.S. border disease surveillance systems.

Return to Appendix B Contents

PFQ Grant Summary: Partnership for Achieving Quality Homecare

Lead Organization:Visiting Nurse Service of New York (VNSNY)
Partner Team: VNSNY with 8 home health agencies, and starting in year 3, Delmarva and other QIOs
Title: Partnership for Achieving Quality Homecare (PAQH)
Topic Area: Better use of evidence-based quality improvement approaches by home care agencies serving the elderly
Principal Investigators: Penny Hollander Feldman, Director, Center for Home Care Policy and Research, VNSNY
AHRQ Project Officer: Judy Sangl
Total Cumulative Award: $913,667
Funding Period: 10/2002–9/2006
Project Status: Received a no cost extension through September 2007

1. Project Description

Goals. This project sought to improve home care for elderly individuals by creating a learning collaborative—the Partnership for Achieving Quality Homecare (PAQH)—through which selected home care agencies throughout the nation could (1) identify and prioritize improvement goals and (2) gain access to methods, tools, and materials that would enable them to conduct more sophisticated, evidence-based quality improvement activities than they could individually. The project originally planned to focus on one clinical condition prevalent in the home care population. Over the four-year project period, however, it considered the possibility of expanding either by adding partners and/or target conditions. The project also planned to develop a "toolkit" of materials and techniques that could be disseminated to home care agencies for use in translating research findings into daily practice.

Activities and Progress. The first year was devoted primarily to planning and setting the foundation for the project. The lead agency, VNSNY, established a partnership steering committee, which selected diabetes as the clinical focus for the project. The project invited home health agencies to join the improvement initiative if they had a reputation for innovation and the capacity to participate, i.e., interested staff, information systems, ability to pay for participants' trips, etc.

The eight agencies selected were dispersed geographically, were a mixture of nonprofit and for-profit entities, and varied in size. The agencies formed three-person QI teams, collected baseline performance data according to the instruments developed by VNSNY, and participated in a collaborative learning model, which was based on the Institute for Healthcare Improvement (IHI) Breakthrough Series. Agencies participated in three face-to-face meetings, with the first meeting highlighting the Model for Improvement. The collaborative adopted the rapid cycle "Plan-Do-Study-Act" (PDSA) approach to quality improvement in order to test and implement clinical practice guidelines developed by the American Diabetes Association.

During the second year, collaborative agencies worked on three common targets for diabetes quality improvement—glycemic control, foot care, and medication management—and on two other areas of their choosing (e.g. hypertension, lipid control, lifestyle changes). Each agency assessed the gap between current and desired performance targets and worked to achieve the targets with support via phone (coaching) calls with the VNSNY staff and consultants, and from each other at two subsequent meetings. Using chart review data submitted by each agency on diabetes patients, VNSNY prepared monthly feedback reports containing data on outcomes and processes of care, including data from the supplemental Outcome and Assessment Information Set (OASIS) collected at two points in time. VNSNY also established a listserv for informal communication among collaborative members.

In the third year, VNSNY evaluated the results and lessons from the diabetes learning collaborative and created a strategic expansion plan, which involved not only adding new partners to extend the reach of QI activities, but also a new clinical focus—reduction of acute care hospitalization among home health recipients. Seven of the eight PAQH home health agency members agreed to participate in the second collaborative. With help from the project's AHRQ program officer, VNSNY secured a commitment from the Delmarva Foundation, the QIO for Maryland and DC and the QIO Support Center for home health improvement for all QIOs at the time, to help recruit several QIOs from around the country, and a few additional home health agencies, to participate in the new collaborative. VNSNY planned to use a different learning collaborative model, relying on web-based technology to hold training and on seminars to hold down costs while sustaining the core elements of the learning collaborative. VNSNY developed pilot training materials and outcome measures for this acute care hospitalization collaborative.

In the fourth year, to extend the reach of home health QI initiatives, VNSNY began working with 10 QIO representatives from around the country on a strategy to develop a "wholesale" model for disseminating evidence-based strategies for home care practice tailored to the needs and issues unique to home health care agencies working with decentralized staff and led by nurses. The focus is on Reducing Acute Care Hospitalization, hence the name "ReACH." The lead QIO changed to Quality Insights of PA, which helps recruit and support communication with participating QIOs. VNSNY also developed a system for collecting measures on acute care hospitalization, which is in the OASIS data set submitted to CMS. The ReACH Collaborative was implemented in two overlapping waves over two years. The 1^st wave ends in December 2006, while the second wave began in September 2006 and will end in August 2007. Participating home health care agency teams attended three Learning Sessions hosted by their respective QIOs to hear and share best practices for improvements in the multiple content areas. At each session, teams reported on the activities, methods, and results surrounding their improvement efforts. With the expansion of the partnership, VNSNY utilized distance-learning technology (WebEX, teleconference) to allow simultaneous learning and sharing while minimizing project costs to expand access to a wide audience of participating home health agencies.

2. Partnership Structure/Function

The Diabetes Collaborative had a partnership steering committee made up of CEOs and other management-level representatives from the participating organizations who were a critical part of the planning process. They provided the human and financial resources needed to implement the project and supported the cross-agency learning process and evaluation.

The ReACH Collaborative also has an advisory group, which was more involved than the first collaborative's steering committee in project design. Those on the advisory group include QIO representatives, the QIOSC, Quality Insights of PA, and ReACH Collaborative faculty. In the early part of this initiative, VNSNY had weekly or biweekly calls with the QIOs to support project design and initiation. Currently, the advisory group conducts monthly conference calls with QIOs. In addition, the ReACH Collaborative has engaged a partners group that includes key stakeholders such as CMS, Visiting Nurse Associations of America, and other leaders from the home care industry and professional organizations. This group is convened quarterly to assess the project design, implementation, and opportunities for expansion and additional support.

Table 1. Major Partner Organizations and Roles in the Project

	Organization	Role in Project
Lead Organization (grant recipient)	VNSNY PI: Penny Hollander Feldman, PhD	Provide overall leadership and direction to the Collaboratives; create and staff expert panels and steering committees to guide project development and content; develop and implement evaluation plans and activities on project impact; provide training and technical assistance to participating home health agencies and QIOs; assess opportunities for expansion and sustainability of project outcomes
Key Collaborators	Delmarva Foundation (the QIO for MD & DC). In year 3, switched to Quality Insights of PA—the QIO support center for HH quality improvement 10 QIOs, beginning in year 3	To recruit QIOs and home health agencies from the acute hospitalization pilot test as participants for the second ReACH Collaborative QIOs recruit and work with participating agencies to actively support the implementation and spread of the initiative throughout the project period; QIOs host participating agencies for each learning session and provide direct coaching and technical assistance to the teams to support their improvement efforts during the action periods
Target Organizations	8 home health agencies located throughout the country	Commitment to achieving explicit goals in selected common areas of collaborative; involvement of three team members in both collaborative learning sessions and bi-monthly conference calls; willingness to share outcomes and assessment information set and other data on achievement of process and outcomes goals; commitment to providing their change results in a timely manner; willingness to have a site visit
Target Organizations	69 home health agencies participating in REACH National Demonstration Collaborative	Home health agencies designate a senior leader, or "spread sponsor," for the initiative to support the necessary systems redesign, staff training, and practice improvements across the agency to reduce avoidable hospitalizations; agency participants designate a 3- to 5-member team to participate in the full implementation of the collaborative; agency teams test and implement key changes to meet the Collaborative aims, report monthly data on process measures, and share key lessons learned within and across Collaborative teams; agencies are expected to participate in each wave of the Collaborative to support spread of successful changes throughout the agency

3. Project Evaluation and Outcomes/Results

The evaluation of the first learning collaborative found that all eight teams integrated change into systems or standard operative procedures. Many accomplished this by redesigning agency-wide forms and documentation, while some worked more closely with their diabetes nurse specialists or revamped the orientation for new staff. All of the teams also codified change into their training manuals and other systems by, for example, adding new competencies around the core topics for their nursing staff or creating standards of care for diabetes patients to be used throughout the agency. Five of the eight teams had used or were planning to use the PDSA model for other quality improvement initiatives, and six teams had integrated or intended to integrate the improvement process into their other improvement initiatives.

The main domains and measures/research questions used for the evaluation of the first diabetes learning collaborative, which were very comprehensive, included (1) collaborative reach in numbers of patients affected; (2) leadership experience, engagement, and satisfaction, including perceived value of participation in the Collaborative and its impact on each organization's strategic objectives, (3) team/staff experience, expectations, engagement, and satisfaction, (4) success in implementing the improvement model, and in collecting and submitting data; team use of data to make changes in clinical care practices, (5) spread beyond pilot group and use for other quality initiatives, and sustainability of change via integration into existing systems and processes, training manuals, and other systems or through commitment from leadership for continuation and integration of the QI process with other initiatives; (6) clinical improvement (discussed below); and (7) cost of the Collaborative's direct costs.

A complete review of the outcomes is beyond the scope of this summary, but some examples suggest that the outcomes were very positive. In terms of leadership's perceived value of the project, a majority of home health agency CEOs and clinical managers surveyed after the diabetes collaborative ended agreed or strongly agreed that their agency's participation led them to revise their approach QI initiatives and helped to identify changes that they intended to spread to the entire organization. Over 70 percent of the CEO/managers strongly agreed that their agency's participation in the Collaborative was likely to lead to lasting improvement in care provided to patients with diabetes.

Agencies were required to submit monthly data on the following clinical measures:

Glycemic Control

Patients with an individualized glycemic control plan ("target" blood sugar range).
Patients testing their blood glucose according to their plan most or all of the time (among patients with a control plan).
Patients whose blood glucose is in their target range most or all of the time.

Foot Care

Patients who received a comprehensive foot exam (visual inspection, vascular assessment and testing for sensation ) within 10 days of home care admission.
Patients (and/or their caregivers) who received education about foot care.
Patients who did not develop a new foot ulcer during home care.

Medication Management

Patients (or their caregiver) who can return-demonstrate administration of their insulin (among patients who are taking insulin).
Patients taking their diabetes medications as prescribed most or all of the time (among patients taking one or more diabetes medications).
Patients whose prescribed medications have been reviewed for possible drug interactions or contraindicated medications.

In terms of clinical outcomes, chart review data from monthly reports submitted by participating agencies showed that the greatest improvement, Collaborative-wide, was in the proportion of persons with diabetes who received a comprehensive foot exam within 10 days of their admission to home care, with an increase of over 50 percentage points during the course of the Collaborative. Increases of over 30 percentage points, Collaborative-wide, were also demonstrate for 1) percent of patients with an individualized glycemic control plan, 2) percent of patients testing their blood glucose according to plan most or all of the time, 3) percent receiving education about foot care, and 4) percent whose medications were reviewed for contraindications. These results should be interpreted with caution because there was no control group, but the clinical change data suggest that performance on eight of the nine clinical measures increased over the course of the collaborative and for three months after it ended. The one exception was in "no new foot ulcer," which did not change substantially, as it was already quite good at the start.

VNSNY developed an evaluation plan to assess the implementation and impact of the ReACH National Demonstration Collaborative. The primary objective is to evaluate the effectiveness of the Collaborative in reducing acute care hospitalization rates among participating home care agencies. The four key components of the evaluation plan include: 1) assess the improvement work of participating home care agencies (monthly performance data); 2) document the strategies employed to reduce acute care hospitalizations at participating home care agencies; 3) assess QIO supports to facilitate the improvement work of participating home care agencies; and 4) determine the effectiveness of the virtual Collaborative Learning Model approach to reduce avoidable hospitalizations. Data will be collected in interviews with key home health agency staff from a random sample of participating home care agencies, surveys of participating QIO staff, online evaluations of learning sessions, and monthly performance data of key clinical indicators. Project staff will assess the change in performance on each of 5 clinical indicators, comparing results from a baseline study period with results from a post-implementation study period. These data will be assessed for each Wave of the Collaborative (Jan-Dec 2006; Nov-Aug 2007).

4. Major Products

Acute Care Hospitalization Toolkit.
Diabetes Toolkit and Dissemination Document (for each collaborative).
ReACH Project Website (paqh.org/ReACH). PAQH engaged IANet technology partners to support development of a project Web site to serve as the core infrastructure for the national virtual Learning Collaborative. The ReACH project Web site is a resource for participating agencies to submit data, view agency-specific and national performance, and download or link to valuable tools and resources to support improvement efforts aimed at reducing acute care hospitalizations. All registered users are automatically enrolled on the agency listserv to support communication and sharing of information with peers across the country.
Presentations: (1) October 2002, Deans from the Rutgers, Yale, U Penn, NYU, Columbia, Hunter, and Pace nursing schools; (2) January 2003, New England Health Care Summit in Boston; (3) September 2003, "A National Quality Agenda and Experiences from the Field" at the National Association for Healthcare Quality's Annual Education Conference; and (4) July 2006, Translating Research Into Practice Meeting in Washington, DC. Collaborative participants also presented about the project to state departments of health and agency boards.
Organized a national meeting in July 2003, "Charting the Course for Home Health Quality: Action Steps for Achieving Sustainable Improvement," New York City, June 30-July 1, 2003. The proceedings were published in Home Healthcare Nurse December 2004. An interview with the PI (and the commissioned papers from this meeting) was published in the May/June 2004 edition of the Journal for Healthcare Quality (JHQ).
Organized the national meeting, "Advancing the Agenda for Home Healthcare Quality," held on March 31-April 1, 2005. Proceedings were published in Home Healthcare Nurse May 2006, and the commissioned papers were published in JHQ, Jan/Feb 2006.
"The Importance of Screening for Depression in Home Care Patients," Caring, November 2003.
"Improving the Delivery of Care for Diabetes Patients with a Collaborative Model," Home Healthcare Nurse 23(3): 177-182, March 2005.

5. Potential for Sustainability/Expansion After PFQ Grant Ends

As noted, the Diabetes Collaborative appeared to have long-lasting effects on quality improvement initiatives within the eight participating home health agencies. Seven of the eight that decided to continue with the ReACH collaborative have demonstrated their interest in and commitment to continuing QI activities, at least in an advisory capacity.

The Reducing Acute Care Hospitalization Collaborative will continue until August 2007 with additional funding obtained from the Robert Wood Johnson Foundation. Additionally, the project received a no cost extension until September 2007. VNSNY hired a business consultant to help them develop a strategic sustainability plan. The plan included research and interviews with current and prospective partners, clients and key stakeholders. Initial findings of the plan have revealed opportunities to extend the Partnership and serve a key role with a variety of local and national stakeholders to support translation of evidence-based strategies to frontline home care practice. The plan will be finalized by the end of Project Year 5.

Return to Appendix B Contents
Return to Contents

AHRQ Publication No. 08-M010-EF
Current as of December 20, 2006

Internet Citation:

Evaluation of AHRQ's Partnerships for Quality Program. Program Evaluation. AHRQ Publication No. 08-M010-EF, December 20, 2006. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/about/evaluations/partnerships/