Advances in Patient Safety: From Research to Implementation Advances in Patient Safety: From Research to Implementation Volume 3. Implementation Issues

Using Reported Primary Care Errors to Develop and Implement Patient Safety Interventions: A Report from the ASIPS Collaborative

David R. West, John M. Westfall, Rodrigo Araya-Guerra, Laura Hansen, Javán Quintela, Rebecca VanVorst, Elizabeth W. Staton, Bethany Matthews, Wilson D. Pace

Abstract

Background and objectives: Approaches to translating medical error information into effective interventions have not been well described. The Applied Strategies for Improving Patient Safety (ASIPS) Collaborative developed a mixed-methods approach to analyze medical errors to (1) develop an initial conceptual framework for depicting specific clinical processes at risk for error, (2) validate the framework through critical inquiry with clinicians and staff from participating practices and through concurrent analysis of malpractice insurance data, and (3) implement practice-specific quality improvement interventions to reduce medical errors. Methods: We identified two areas for possible practice-level intervention: laboratory errors and prescription errors. Expert panels of local stakeholders provided grounded input into the refinement of the frameworks and causal flows, resulting in the development of realistic "principles for process improvement" (PPIs) for developing flexible and locally relevant interventions. Results: The intervention for laboratory tests involved the use of a portable bar coding utility to support an electronic laboratory test tracking system. The prescription/medication interventions were based upon an electronic mechanism designed to ensure timely and accurate transmittal of prescription data from practices to pharmacies; incorporate important data elements such as the purpose/indication on each prescription; and develop timely, accurate, and accessible medication lists for the medical record. Conclusion: Using multiple data sources, locally developed and relevant quality improvement interventions to improve patient safety can be successfully implemented in primary care. However, a clear understanding of the processes that require change is essential to successfully address implementation challenges and put interventions into routine use.top link

Introduction

The ultimate aim of collecting and analyzing medical error data is to implement change that results in safer care. 1 Safer care, in this sense, is the expected outcome of a series of linked efforts that allow health care professionals to practice in a manner that minimizes the harm that stems from the processes of health care delivery. 2

To date, most efforts that translate medical error information into effective patient safety interventions have emerged from inpatient hospital systems, which, compared with ambulatory clinics, have extensive financial and personnel resources for quality improvement. This inpatient focus persists despite recent indications that errors in primary care may also have serious consequences. 1, 3 Only a few studies have used medical error data to develop specific interventions to prevent patient injury in primary care. 4 -- 7 To address the knowledge deficit within primary care patient safety, a conference sponsored by the Agency for Healthcare Research and Quality and others was held in late 2000 to establish a research agenda for ambulatory patient safety. From this conference came a set of recommendations for improving ambulatory patient safety that included the need to strengthen processes and systems (including the use of health information technology) and improve practice culture by fostering teamwork and shared responsibility. 8

To plan, implement, evaluate, and refine interventions designed to improve the quality, safety, efficiency, and effectiveness of health care, one can draw on numerous process improvement models available from the manufacturing industry. Some contemporary examples include Total Quality Management (TQM) and its health care analogue, Continuous Quality Improvement (CQI); Six Sigma; and Healthcare Failure Modes and Effects Analysis (HFMEATM). 9 Whether used singly or in combination, these models can be used to assess risks inherent to structured processes, to plan and implement internal solutions, and to refine and improve these solutions over time. 10

One example of a project designed to translate information about ambulatory primary care medical errors into quality and safety improvement interventions is the Applied Strategies for Improving Patient Safety (ASIPS) project. This paper describes the interventions implemented by the ASIPS project and our process for developing those interventions.top link

Methods

ASIPS was a 3-year demonstration project funded by the Agency for Healthcare Research and Quality. The core of the project was a reporting system that collected voluntarily reported medical errors from two primary care, practice-based research networks in Colorado, the Colorado Research Network (CaReNet) and the High Plains Research Network (HPRN). 11 We used the secondary analysis of events reported to two malpractice insurance carriers to corroborate our findings from the reporting system.

The common features of quality improvement (QI) models used in contemporary health care, as summarized by the Institute of Medicine, 9 were central to the development of our interventions. These include:

Reports to the ASIPS system were coded using a detailed, process-oriented taxonomy that allowed for thorough documentation of the processes leading to errors. 12 -- 14 We analyzed encoded error data and the associated narratives, using mixed methods for the purpose of developing targeted quality and safety improvement interventions at the practice level. 15 Based on these analyses, we focused on the development of interventions to improve specific practice functions that depended on (1) successful coordination and management mechanisms to communicate internally, (2) accurate transmission of information to outside organizations, and (3) integrity of systems to track and receive information from outside organizations. Once the general focus areas were established, we followed the processes outlined below to design useful, effective interventions.

Assuring clinical practice relevance

Two specific groups were convened to inform and advise ASIPS regarding intervention development, providing what Miles and Huberman 16 call valid, evaluative judgments from subject matter experts.

Clinical Steering Committee. ASIPS was guided by a Clinical Steering Committee of 12 individuals, including 2 representatives of malpractice insurers, a health care regulator, a patient safety advisor, a pharmacist, and physicians, nurses, and administrative staff from the practices participating in ASIPS. The group met regularly to review data, direct additional study, and provide policy oversight.

Learning Groups. Practice-level stakeholders in patient safety/quality improvement (including providers and staff from ASIPS participating clinics) were invited to join Learning Groups that guided the development of a framework for each intervention. Participants had particular interest or expertise in the identified topic. Each Learning Group tackled a different patient safety issue identified from the analysis of ASIPS error reporting system data and worked collaboratively with ASIPS project staff. Project staff supported the work of the Learning Groups by establishing the roles and responsibilities of participants, facilitating a series of meetings, keeping and distributing meeting minutes, providing background material, assisting with between-meeting assignments, and drafting reports of the group's work and recommendations. In general, Learning Groups served to:

Mapping processes

The second step in the process to design useful, effective interventions was to map processes in practices. Process flow mapping has long been established as a useful method for making the implicit steps of complex activities both visible and clear. This technique has been used extensively for analyzing recurring decisions and processes involving multiple people and complex situations and is recognized as a critical component of quality improvement activities. 9, 16

We found this technique to be extremely useful for summarizing our onsite observations of how practice-level processes occur. Process flow maps provided a tool for documenting various formal and informal process steps, coordination and communication activities, and interfaces with external organizations. Observation teams collected data through both passively observing and directly questioning clinicians, nursing staff, and administrative staff, using semistructured interview guides. When developing an intervention, we mapped process flows at multiple practice sites to improve our understanding of basic processes common to all sites. The process maps included steps that were unique to a specific practice that may have been causally linked to medical errors, or steps that represented a "best practice" that appeared to address identified problems. Learning Groups used the process flow maps in discussions concerning the causal flows of phenomena that result in medical errors; thus, the maps provided insight into specific strategies to mitigate the propagation of error.top link

Assuring a culture of safety at the individual practice level

Our process of designing interventions encouraged a culture of safety within participating practices. The ASIPS interventions were based upon the analysis of reported medical errors, multiple process flow maps, and the judgment of Learning Group members. However, specific interventions implemented at the practice level were designed around the work processes and needs of each individual practice. This attention to the needs of the individual practice assured that process improvement efforts were "owned" (through the investment of time, ideas, and internal debate) by the multidisciplinary teams that had to live with them, and fostered teamwork, explicit communication, and further development of a culture of patient safety within the practice.top link

Evaluating/refining intervention

When implementing a practice intervention, ASIPS project staff maintained an onsite presence to (1) facilitate work process changes using a series of quick "huddles" 18 involving both providers and office staff, (2) observe any design problems that needed immediate resolution, (3) observe work flows to determine if the interventions were having the desired effect without unintended consequences, and (4) to assist the practice in "staying the course" until new work flow processes became institutionalized. After the initial implementation ASIPS project staff returned regularly to further observe the changes effected by the interventions and gather data for review and discussion by the Learning Group.top link

Results

Using the analysis of the ASIPS medical error reports and malpractice data, the Clinical Steering Committee identified two error domains for the initial ASIPS interventions: prescription drug errors and diagnostic testing errors. These decisions were based upon the following data:

Two separate Learning Groups developed a framework for the interventions within these error domains. Each group convened to describe a complete picture of errors and error-related events from the event data and their own experiences, which informed discussions and development of PPIs for each error domain. The result was the implementation of interventions in a small number of practices involving the use of information technology to reduce the frequency of errors and identify errors immediately when they occur. Our experience in developing and implementing these interventions is outlined below.

Addressing prescription drug errors

The Prescription Drug Learning Group reviewed the process flow maps of the prescription and refilling process (Figure 1) and the results from the mixed-methods analysis of prescription drug error reports. When combined with their own clinical experiences, a resultant framework of high priority errors and error-related events emerged. From this framework, the Learning Group developed the following PPIs for prescribing drugs:

Figure 1. Prescription drug process flow map

top link

Based upon the range of issues addressed by these PPIs, both the Clinical Steering Committee and the Learning Group determined that the most viable strategy to improve patient safety for prescriptions was to adopt the use of a commercially available, computerized physician order entry (CPOE), electronic prescribing system. Further, it was determined that any vendor would be required to provide features competitive within the industry (e.g., incorporate a complete product and drug-interaction database), as well as have the flexibility to modify the system to directly address the PPIs and any additional requirements specified by participating practices. After a search, the Center for Drug Safety was chosen as the vendor. 20

Working with this vendor, several practices have participated in ASIPS-sponsored interventions for electronic prescribing. The interventions included customized features to (1) check prescriptions for possible contraindications ---when desired, (2) require that prescriptions include dosage instructions and the purpose in plain English, (3) facilitate the creation and maintenance of accurate medication lists for inclusion in the medical record (in either a paper or electronic medical record), and (4) support electronic transmission of the prescriptions directly to the pharmacy ---even in rural communities where the only available pharmacy is in another town. As of this writing, practices continue to use the system and to specify additional CPOE system modifications to assure prescribing accuracy and efficiency.top link

Addressing laboratory test errors

After reviewing the composite process flow map for the ordering of lab tests (Figure 2), the results of the diagnostic test error report analysis, and their own clinical experiences, the Learning Group developed a framework of high priority errors and error-related events. The Learning Group then developed the following PPIs for diagnostic lab test processes:

Figure 2. Diagnostic lab test process flow map

top link

Assure that the lab tracking process is accessible by multiple users. Process flow mapping illustrated that in numerous practices, lab tracking was done diligently and efficiently, but only by a single "key" individual within the practice. When that individual was away from the practice, tracking of ordered and received test results did not occur and searches for missing lab test information became grossly inefficient. By ensuring that multiple users have access to lab tracking data, and by further ensuring that they have a working knowledge of the tracking system (either electronic or manual), this "key person dependency" source of error can be addressed.

The ASIPS Clinical Steering Committee and the Learning Group believed that the problems of tracking labs would be solved by a practicewide comprehensive health information technology solution, such as an electronic health record with CPOE capability. However, making such a system available to all ASIPS participating practices was not economically feasible. Furthermore, few electronic health records perform all of these lab-tracking functions. 21, 22 As an alternative, we developed and implemented a stand-alone, Web-based "lab tracker" system to address the laboratory test-specific issues identified in the PPIs. The system tracks all lab tests ordered and results returned using bar code technology. The system captured the identity of the laboratory to which each test was sent and calculated a due date for results to be returned to the practice. The lab tracker also provided automated daily reports to inform practice personnel of overdue tests, as well as the status of "stat" tests and "alarm" values.top link

Discussion

Using an integrated approach to quality improvement, we were able to design and implement interventions that addressed patient safety issues relevant to primary care practice. Learning Groups were able to use the analysis of medical error reports, process maps, and their own experiences to produce PPIs that are potentially "best practices" for primary care in the areas of prescription writing and diagnostic testing ---two domains we know pose significant risks to patient safety.

As the project progressed, we identified challenges to successful implementation of interventions; we also identified important factors that facilitated our progress. The major challenges we faced included time requirements, resistance to change, lack of resources within practices, and practice variation (e.g., differing management and communication styles). We addressed these challenges in a number of ways, as summarized in Table 1, and briefly discuss them below.

Table 1. Addressing the challenges to implementation

Issue Challenge How we addressed the challenge
Time The time required to participate in the development and implementation of locally relevant interventions by providers and staff was considerable. The use of automated systems to support the interventions provided the opportunity, once new processes were learned, to create efficiencies for practitioners and staff.
Resistance to change While most individuals at the practice level were enthusiastic about changes to improve patient safety, a small number of providers and staff were not interested in adhering to the redesigned processes of the intervention. Additionally, because many of the processes we sought to revise were originally dependent on one key person, that person at times felt threatened by the changes and resisted. Within each practice, a "champion" emerged to assure the successful implementation of the intervention. This champion successfully enlisted buy-in from key people to reduce resistance. The involvement of providers and staff on the Clinical Steering Committee and the Learning Groups, and use of "huddles" at the practice level, served to create a team atmosphere that served to galvanize individuals and to create and maintain momentum.
Resources None of the practices had the resources to independently develop the interventions, nor to procure the required hardware and software required to implement them. Grant funding enabled ASIPS to provide new computers and bar code scanners, develop tracking databases, and fund the use of an electronic prescription writing vendor.
Practice variation Process flow maps developed at a number of practices reinforced our awareness of the wide variation in practice process. Other characteristics of practices, from management styles to patient population, also varied widely. Each intervention was customized to the particular practice in which it was implemented. ASIPS staff maintained contact with practice staff to continually modify the intervention as needed.

The interventions were generally valued, supported, and continued by the practices that initially adopted them. Formal and informal leaders within the practice embraced the opportunity to redesign systems to improve patient safety. Notably, however, a very small number of providers within these practices chose not to adopt the changes. This may have been due to a resistance to any change in processes, a concern over the level of effort required to learn a new system, or both. Given that participation in the ASIPS project was voluntary and that no "line authority" existed to ensure full participation by all providers, we encouraged but could not require participation of individuals ---a problem that led to unintended consequences. For example, if one provider in a practice did not use the electronic prescribing system, the prescribing history for the patients seen by that provider would not be in the system. If all other providers learned to rely on the system and always expected to find medication history there, they would encounter trouble when they cared for another provider's patient and could not find the patient's medication history where expected. Clearly, all providers and staff within a practice must be committed or required to commit to the adoption and maintenance of any intervention for it to fully succeed.

Throughout the process of developing the interventions, a number of patient safety "champions" emerged from the participating practices, and many of these champions ultimately became the leaders for the intervention that was implemented within their practices. These individuals came from the ranks of physicians or nursing staff, as well as from administrative staff. Their individual efforts were remarkable, as was their dedication to assuring that a culture of patient safety became institutionalized within their practices.

We initially expected to implement, on a rapid cycle basis, 1 numerous interventions using the processes described above. Instead, we implemented two large-scale, technology-driven interventions in a handful of practices. We underestimated both the time and effort required to complete the cycle of analyzing the report data, developing process flow maps, convening Learning Groups, and implementing practice-specific interventions. We believe that to move more quickly we will need to identify interventions built upon the PPIs that are somewhat less ambitious, more focused, but not less important than those currently implemented. These new interventions should be capable of being fully implemented, evaluated, and refined within 4 months or less. For example, we were unable to test the use of a single, office-specific laboratory order form. With many insurance carriers contracting directly with laboratories, it is common for a primary care office to send tests to multiple laboratories. Each laboratory typically requires a unique form with different names and locations for identical tests. Standardizing the forms could potentially prevent transcription errors, which were commonly reported to the ASIPS project. Further work in this area appears to be warranted.

The Learning Groups developed the PPIs, which are essentially testable hypotheses for improvement in ambulatory primary care safety. While we have implemented a series of linked efforts or processes to improve patient safety, we continue to evaluate the outcomes of these changes. Our evaluation of the interventions will be completed by the end of our funding period (September 2005).

In the future, we wish to be able to determine the economic costs and benefits of our interventions, to estimate the return on what has proven to be a substantial investment on the part of all project participants.

Finally, we gained insight into the paucity of resources available within ambulatory primary care practices to develop and implement interventions for improving patient safety. While hospitals have the available workforce and information technology resources to undertake interventions, the primary care practices we worked with are unable to support or undertake these efforts without the resources provided by ASIPS.top link

Conclusion

The ASIPS project collected information about real medical errors and used this information to create a culture and process of safer care in multiple ambulatory primary care practices. We implemented a process by which practicing clinicians and staff identified and prioritized problem areas; developed relevant, practice-specific interventions to improve patient safety; and implemented the interventions in practices. Future efforts should focus more narrowly on processes amenable to rapid-cycle testing, and should also examine the economic costs and benefits of such processes in primary care.top link

Acknowledgments

We would like to acknowledge all the clinicians and staff of the High Plains Research Network and the Colorado Research Network for opening their doors to us and allowing us to learn the reality of clinical processes, and for embracing the implementation of the interventions. Thanks to Daniel Harris, Ph.D., Debbi Main, Ph.D., and Daniel Burke, M.D., for their work with us and the Learning Groups, and Susan Karp, R.N., for her dedication to excellence. Finally, thanks to Walker Freeman, information technology specialist, for his many hours of work on the design of the lab order/results tracking program.

Funding for this study was provided by the Agency for Healthcare Research and Quality, grant #U18-HS011878, Wilson D. Pace, M.D., principal investigator.top link

Author affiliations

All of the authors are affiliated with the University of Colorado Health Sciences Center, Department of Family Medicine, Aurora, CO. One author is also affiliated with the University of Colorado Health Sciences Center School of Pharmacy, Denver, CO (LH); another author is also affiliated with the Denver Health and Hospital Authority, Denver, CO (BM).

Address correspondence to: David R. West, Ph.D., University of Colorado Department of Family Medicine, P.O. Box 6508, Mail Stop F-495, Aurora, CO 80045-0508; phone: 303-724-9762; e-mail: david.west@uchsc.edu.top link

References

1. Aspden P, Corrigan JM, Wolcott J, et al. Comprehensive patient safety programs in health care settings. Patient safety: achieving a new standard for care. Washington, DC: National Academies Press, 2004; pp. 127 -- 41.

2. Jha AK, Kuperman GJ, Teich JM, et al. Identifying adverse drug events: Development of a computer-based monitor and comparison with chart review and stimulated voluntary report. J Am Med Inform Assoc 1998. 5(3):305-14. May/Jun. (PubMed) (Full Text in PMC)

3. Leape LL. The godfather of patient safety sees progress. Interview by Mark Crane. Med Econ 2003 Aug;80(15):29, 30, 33 -- 4.

4. Wilf-Miron R, Lewenhoff I, Benyamini Z, et al. From aviation to medicine: applying concepts of aviation safety to risk management in ambulatory care. Qual Saf Health Care 2003. 12(1):35-9. Feb. (PubMed) (Full Text in PMC)

5. Gandhi TK, Weingart SN, Borus J, et al. Adverse drug events in ambulatory care. N Engl J Med 2003. 348(16):1556-64. Apr. (PubMed)

6. Gurwitz JH, Field TS, Harrold LR, et al. Incidence and preventability of adverse drug events among older persons in the ambulatory setting. JAMA 2003. 289(9):1107-16. Mar. (PubMed)

7. Elder NC, Dovey SM. Classification of medical errors and preventable adverse events in primary care: a synthesis of the literature.[erratum appears in J Fam Pract. 2002 Dec;51(12):1079.]. J Fam Pract 2002 Nov;51(11):927 -- 32.

8. Hammons T, Piland NF, Small SD, et al. Ambulatory patient safety. What we know and need to know. J Ambulatory Care Manage 2003. 26(1):63-82. Jan/Mar. (PubMed)

9. McDonough JE, Solomon R, Petosa L. Quality improvement and proactive hazard analysis models: deciphering a new Tower of Babel. In Aspden P, Corrigan JM, Wolcott J, et al. (eds.). Patient safety: achieving a new standard for care. Washington, DC: National Academies Press; 2004; pp. 471 -- 508.

10. McDonough JE. Proactive hazard analysis and health care policy. Milbank Memorial Fund/ECRI; 2002.

11. Pace WD, Staton EW, Higgins GS, et al. Database design to ensure anonymous study of medical errors: a report from the ASIPS Collaborative. J Amer Informatics Assoc 2003. 10(6):531-40. Nov/Dec.

12. Pace WD, Fernald DH, Harris D, et al. Developing a taxonomy for coding ambulatory medical errors: a report from the ASIPS Collaborative. In: Henriksen K, Battles JB, Marks ES, Lewin, DI, editors. Advances in patient safety: from research to implementation. Vol. 2, Concepts and methodology. AHRQ Publication No. 05-0021-2. Rockville, MD: Agency for Healthcare Research and Quality; Feb 2005.

13. The ASIPS Collaborative. Dimensions of medical outcome. The ASIPS -- Victoroff taxonomy. [Web Page]. 2003. Available at: http://fammed.uchsc.edu/carenet/asips/taxonomy/ . (Accessed 2004 Apr 23.)

14. Fernald DH, Pace WD, Harris DM, et al. Event reporting to a primary care patient safety reporting system: A report from the ASIPS Collaborative. Ann Fam Med 2004. 2(4):327-332. (PubMed) (Full Text in PMC)

15. Harris DM, Westfall JM, Fernald DH, et al. Mixed methods analysis of medical error event reports: a report from the ASIPS Collaborative. In: Henriksen K, Battles JB, Marks ES, Lewin, DI, editors. Advances in patient safety: from research to implementation. Vol. 2, Concepts and methodology. AHRQ Publication No. 05-0021-2. Rockville, MD: Agency for Healthcare Research and Quality; Feb 2005.

16. Miles MB, Huberman AM. Qualitative data analysis: a sourcebook of new methods. 2nd ed. Thousand Oaks, CA: Sage Publications; 1994.

17. West D. Applied strategies for improving patient safety. Presented at the 20th Annual AcademyHealth Research Meeting; 2003; Nashville, TN.

18. Institute for Healthcare Improvement. Huddles [Web Page]. 2004. Available at: www.ihi.org/IHI/TOPICS/ ImprovementMethods/tools/Huddles.htm . (Accessed 2004 Jul 8.)

19. Institute for Safe Medication Practices. ISMP medication safety alert: treat medication samples with respect. [Web Page]. 2004. Available at: www.ISMP.org . (Accessed 2004 Jul 8.)

20. Center for Drug Safety. E-Prescribing, personalized drug interaction and secure email [Web Page]. 2000. Available at: http://www.centerfordrugsafety.org/ . (Accessed 2004 Apr 23.)

21. Poon EG, Wang SJ, Gandhi TK, et al. Design and implementation of a comprehensive outpatient Results Manager. J Biomed Informatics 2003. 36(1 --2):80-91. Feb/Apr.

22. Rehm S, Kraft S. Electronic medical records: the FPM vendor survey. Fam Pract Manag 2001. 8(1):45-54. Jan.