Use of reporting system information as data. Participants in the conference generally agreed that a large scale epidemiological study of risks and adverse events in ambulatory care would be valuable if feasible, but difficult to do, and not essential to the research effort. A robust and complete reporting system, which is in essence an observational study, may provide an alternative to large scale epidemiological studies, and support improvement in the bargain. The two sources of data can be complementary, as trends and case studies from the reporting system can serve as data to stimulate hypothesis generation and controlled studies of systems interventions.
The following table provides a framework for learning from incidents (Gelissen 1997). Select for Text Version.
| Modeling | Monitoring | Alertness | |
| Detection | Everything | Known problems only | Recognizing and reporting |
| Selection | New reports only | (not relevant) | Convincing, detailed examples of new and old hazards |
| Description | Detailed | (not relevant) | same |
| Classification | Flexible—looking for new root causes | Routine—standard set of root causes | (not relevant) |
| Computation | (not relevant) | Periodic analysis of updated large database | (not relevant) |
| Interpretation and implementation | Finding new ways of improving, preventing, recovering | (not relevant) | Near misses as precursors; focus on recovery mechanisms |
| Evaluation | (not relevant) | Comparing actual and predicted effects | (not relevant) |
Organizational change and leadership. Tradition and culture are powerful inertial forces, and consistent sustained pressure and resources need to be brought to bear to alter attitudes, behaviors, and values that cut across disciplines and hierarchies in healthcare organizations. Change itself can be difficult to initiate and sustain, and a number of programs and tools have become widely available to address this issue (for example, see the Breakthrough Series of the Institute for Healthcare Improvement at http://www.ihi.org). These methods are being applied with increasing frequency to improving reporting systems in organizations.
The Kennedy School Executive Session on Patient Safety, the Leadership Forum at the National Press Club in July 2000, and derivative leadership colloquia have created an ongoing focus on the critical need for top leadership to create and sustain movement for safety improvement. Consonant with the empiric observations theorists have published in regard to high-reliability organizations, leadership sets the tone for a culture of safety by first centralizing the core values that enshrine safety and then facilitating a flexible culture that decentralizes how these core values are actualized at the operational end (Sagan 1993, Chapter 8 of Institute of Medicine 1999). As Paul O'Neill has emphasized in the dramatic turnaround he engineered at Alcoa with a total focus on safety, safety is not a priority—people know that priorities can be rearranged. In that model, safety is a precondition for all other priorities. Ultimately, top leadership views safety as both the right thing to do and a means of coping with and better understanding complex processes.
Education and training. It will be necessary to provide "training" about what to report and how, and to structure the reporting vehicle to ask questions that elicit the information that is needed in the report. For instance, many clinicians may be much more aware of errors by individuals than of latent errors that are properties of processes and systems. They may not notice the latter, nor be able to report on them accurately. Education and ongoing training linked to feedback from reporting also provides a mechanism for standardizing reports and reporting.
Sustained feedback and demonstrable improvement. Reporting must be linked to action to improving patient safety. If there is no link, clinicians (and patients) may feel little incentive and willingness to report—nothing constructive would be done with the information. If this logic holds, it may be best to organize reporting at a level where it can be acted upon to improve care and safety.
Nested models may have advantages over an all-inclusive national reporting system. Local reporting with local feedback is most effective for leading to action, but aggregate data across many physicians, organizations, and locations is needed to identify larger patterns and to see infrequent occurrences. Local reporting could be organized according to particular hospitals, group practices, or physicians covered by a particular liability insurer, with those entities reporting up to the State or national level. Different approaches or systems for reporting may be appropriate for different kinds of occurrences. For example, mandatory reporting may be appropriate for adverse events with substantial injury to patients, or for never-events (ref NQF), and voluntary reporting for near misses and incidents resulting in only minor injuries. Different approaches or separate systems could be used for different settings (hospitals, physician offices, ambulatory surgical centers, dialysis units) and different persons (patients and families, physicians, nurses, pharmacists). Again, aggregation would be required to gain a more complete picture.
The issue of improving disclosure of errors and untoward outcomes to patients has taken on a new primacy with the arrival of JCAHO standards that require disclosure. Despite natural experiments such as the experience of the Veterans Administration Hospital in Kentucky (Kraman and Hamm 1999). The Veterans Affairs (VA) Medical Center in Lexington, Kentucky, recently reported encouraging data on a courageous and ethical institutional policy to disclose errors to patients and to equitably compensate patients/families for their actual loss. When compared to other VA medical centers, the Lexington facility is in the top quartile for number of claims filed, but at the bottom quartile for payments. Full disclosure of errors and just compensation for injuries seems to be a solution that is both ethical and cost effective. It restores organizations and practitioners to a patient advocacy role, putting the patient's interests first, while potentially minimizing the adverse financial impact on the organization.24
While full and immediate disclosure policy is believed to have led to lower malpractice costs, organizations are struggling with how to do what most believe to be the right thing to do. Complicating the issue is the lack of universally agreed upon definitions of terms, difficulty of assigning causation and separating in some cases the effects of medical management from the effects of disease processes.
It is likely that many of the physician practices and other small organizations in ambulatory care will not have the expertise or capability to analyze their own data and understand its implications. These capabilities might be provided by an entity to which the practice or other organization belongs and to which it reports, such as a large affiliated care system, or an accrediting program like those for ambulatory surgery centers. Such an arrangement would enable the larger entity to protect confidentiality, to put the data and experience of each small organization in the context of similar organizations, and to provide support for improving safety in both the organization reporting the incident and others with the risk of similar incidents. An alternative may be "turn key" programs for small organizations that provide, in standardized form, the knowledge, tools, training, and support needed to succeed.
A reporting system, like other approaches to gathering information about patient in ambulatory care, will need to accommodate the fact that a patient will often move among a number of sites during a single episode of care (e.g., physician office, laboratory or imaging facility, consulting physician practice). The actions and factors that contribute to an occurrence or incident may occur in a different time and place that where the consequences are found. It is useful to recall the advice to "follow the patient."
There are costs to reporting and the time and effort to report errors and incidents will have to compete with all the other claims on the time of clinicians, administrators, and staff in ambulatory care. And if reporting relies on written medical records (e.g., to examine a sample of patient records each month), the "thinness" of those records may limit the entity's ability to find incidents and information about each incident to complete a report. One tactic to improve both reporting and the documentation upon which it depends, is to limit data collection to selected, manageable elements of care, such as to particular care processes that lend themselves to establishing good documentation and for which there are proven methods to improve safety (e.g., medication). In order to build capability in ambulatory care, and to achieve significant progress early on, it may be useful to begin with a focus on specific areas of ambulatory care that are relatively well understood, such as medication, to develop models for reporting, analysis, and feedback, and for action to improve safety. Pilot projects could focus on particular aspects of ambulatory care that are believed to be high risk: office based surgery and anesthesia, or transitions of patients from care of one physician to another or to an inpatient facility.
A successful system of reporting in ambulatory care will require the same steps required for inpatient care (e.g., standard ways of defining and capturing events and data), and particular attention to establishing an infrastructure to support data gathering, analysis, feedback, and effective action to improve.
There are a number of existing reporting systems in ambulatory care from which we can learn (Kaplan et al. 1998, Battles et al. 1998). The American Association for Accreditation of Ambulatory Surgery Facilities (AAAASF) has had a paper trail event reporting system for ambulatory surgical facilities in place for several years, and recently moved to a confidential Web-based reporting system for outcomes of all cases. Accreditation reviews every three years take reporting compliance into account. Despite confidential, cross State lines transfer of sensitive case data, only a few legal challenges have occurred to this relatively new data base and they have not progressed beyond early stages. The Medical Event Reporting System for Transfusion Medicine, another Web-based system, has been used to capture data from blood banks and other non-hospital transfusion facilities including ambulatory donor sites and is now being phased in for nearly twenty hospitals nationwide as an approach to comply with new and broader FDA reporting requirements for near misses.
Reporting to Accrediting Organizations. Accrediting organizations active in ambulatory care (the Joint Commission on Accreditation of Healthcare Organizations (JCAHO), the National Committee for Quality Assurance (NCQA), the Institute for Medical Quality or IMQ in California, and the aforementioned AAAHC and AAAASF) have or are planning to offer programs for reporting and for learning from and acting on reports. Accrediting agencies can establish standards for personnel, equipment, procedures, and processes (including documentation), for particular ambulatory care sites (e.g., ambulatory surgery center) or particular activities and procedures (e.g., conscious sedation and anesthesia, office based surgery, tumescent liposuction).
The Accreditation Association for Ambulatory Health Care (AAAHC) Institute for Quality Improvement has recently published the results of a large survey of attitudes and reporting practices among the population of organizations that AAAHC accredits. 491 (43 percent) of the 1240 organizations solicited responded, and respondents included physician offices performing surgery, group practices, and other organizations. Approximately one-third of respondents reported that they participate in a medical event reporting system, and this was also true for the main subgroups—i.e., about one-third of respondents from each subgroup (e.g., group practices, offices performing surgery) reported that they participate. Organizations that had a larger clinical staff or were part of a network were more likely to participate in medical event reporting systems, but many independent ambulatory surgical centers (i.e., not part of a network) were involved in medical event reporting through their professional organization. Other findings included:
Interestingly, 11 percent (55/491) of respondents reported that nothing would motivate them to become involved in a medical event reporting system. These organizations on average had smaller size clinical staffs, and the report of the study suggested that lack of staff resources and greater risk of being identified may be responsible for these organizations' lack of willingness to participate in medical event reporting programs.
The Medical Event Reporting System (MERS) is a best practice reporting system, having evolved from a Delphi study and modified using the nominal group technique with a cadre of international, cross disciplinary safety science experts. Unique aspects of MERS include two sets of codes: a clinical set, in the case of transfusion medicine, 80 specific codes delineating what, where, and how, and, a second set of codes that are universal for why—based on organizational, technical, and human factors probes. MERS is linked to institutional quality improvement/assurance bodies, is confidential while allowing anonymous reporting, and provides an administrator supported by risk assessment and decision tools who enters all reports into the database and decides which ones require further investigation. MERS is incorporating case-based reasoning tools and fuzzy matching algorithms as approaches to data mining—essentially, to deal with large numbers of reports, identify case clusters of interest, and assist systems administrators and decision makers by providing data with which to allocate resources. MERS is set up to favor users by allowing one to benchmark one's own qualitative (case based) and quantitative (trending) data confidentially using Web-based de-identified data submission and searching with confidential passwords. Currently, MERS is being expanded in a trial in ambulatory care settings (funded by AHRQ). Patients will be trailed as reporters after a preliminary assessment and training period given the increasing evidence that they wish to take a more active role in their care and safety, and, have a considerable amount of information about system processes and gaps in those processes (Hanlon et al. 1998).
It appears that the period of sweeping away old taboos about discussing limitations in current reporting systems and barriers to collecting data is ending. We are now entering a period where large sums have been allocated to move forward a range of mandatory and voluntary reporting projects linked to future improvements in patient safety across the spectrum of health care delivery, including ambulatory care. The key issues in the next few years will include:
A new regulatory model may emerge that helps create a positive cycle in which accountability and learning can both increase, with organizations demonstrating how well they have learned from analysis of reported data, and how they have improved patient safety.
It may well be that the greatest potential gains in patient safety—and more generally, in quality and efficiency—will prove to be from improving the infrastructure supporting management of information, communication, and coordination of care.25 Improving the more clinical elements of care such as choosing the most appropriate antibiotic or ensuring a high level of skill in the performance of an ambulatory surgery procedure will continue to be necessary and important, but many improvements in clinical processes and clinical decisions can be more easily achieved with a stronger infrastructure (Kilo and Berwick 1999).
While conference participants recognized the importance of improved information management and decision support systems, most suggested that these and other tools should be viewed within a larger context. Improving support systems for physicians and patients may require improved information systems and devices, but regardless of the type of infrastructure support, it is important to first consider the tasks and functions needed, and how best to accomplish them.
A conceptual model of ambulatory care should help identify where complete information, good decisions, reliable communication, and effective coordination are essential to good care and outcomes, and where better support systems are most needed.26 Where are the high-risk points, where having the right information for clinicians and patients to make a good decision—for a patient to seek care for a symptom like difficulty in breathing, for a physician to prescribe an appropriate antibiotic? Are there widespread shortcomings in providing the needed information, or just a handful of key points? For example, many practices report difficulty in keeping track of lab tests, imaging procedures, and specialty consultations, with consequent risk of lost, missed, and delayed diagnoses (Garr and Marsh 1986, Kravitz et al. 1997, Puopolo and Brennan 2000). What is the effect of a "tickler" system—manual or electronic—to improve follow-up and prevent delays and other errors? There is abundant evidence of shortcomings and errors in prescribing antibiotics and other pharmaceutical agents.
Physicians are the principal managers of most patients' care and of the clinical processes of care, and physicians make or guide many of the most important decisions about care. The processes and systems of ambulatory care must support physicians by providing the information when and where it is needed, decision support where it adds value, and mechanisms for tracking and managing steps in the patient's diagnostic and therapeutic process. Therefore, a physician practice or other ambulatory care entity must manage a number of clinical processes (e.g., evaluation and treatment of acute asthma episode), clinical support processes (e.g., ordering, performing, reporting, and interpreting clinical laboratory tests), administrative processes (e.g., patient scheduling) and business functions (e.g., billing).
Dr. Brent James and colleagues at Intermountain Health Care have identified the patient conditions that account for most inpatient care, and the key processes through which care is provided, including the support functions and systems upon which clinicians rely.27 James and colleagues have also begun to identify the key patient groups and the clinical and support processes for ambulatory care. The results of these studies should provide a substantial amount of the content of an ambulatory care model, and their methods may useful in further exploration and characterization. Excellent conceptual models of entities that provide patient care have been developed by Nelson, Batalden, and colleagues, and by the Idealized Design of Clinical Office Practice project of the Institute for Healthcare Improvement (Batalden et al. 1997, Nelson et al. 1998, http://www.IHI.org/idealized/idcop). These sources and perhaps others can be used to describe the functions, processes, and systems that must be managed for a physician practice to provide good care and to survive.
It is also useful to focus on other "key players" in ambulatory care, but to combine this point of view with a focus on patients' paths through care. It is generally assumed that the physician is most important in determining patients' care and outcomes, so most attempts to improve care have focused on the activities of the physician. But in some situations, the patient and family are equally or more important. And in others, the most useful focus is the care team, which may include several physicians and other clinicians, such as nurse practitioners, physician assistants, pharmacists, and social workers. There is need to focus particular attention on key decisions and how they can best be supported.
The processes and systems of ambulatory care must support physicians. What are the physicians' decisions and actions that have the greatest impact on the patient's outcomes? Where are there risks of harm to the patient from a suboptimal decision or action? Where does the physician need support—for example, to have ready access to the patient's past medical records, or information on antimicrobial sensitivities of a particular bacterium—in order to make the right decision and take the right action? And how can this support best be provided using available technology, processes, and systems? Finally, how can changes in technology and processes be implemented for the most gain? To what extent should the technology used and the systems built be adapted to the work of the physician (and the patient) rather than force the physician to adapt to the technology?
To improve management of information and communication, high reliability may require consistent content and methods, though full standardization would conflict with the necessity to customize care for individual patients. Consistency enables people to know what to expect, where to find information, and how to interpret it.
The use of information technology such as the Internet, electronic mail, electronic medical records, provider order entry, and decision support has the potential to transform the provision of care to patients. Considerable attention has recently been devoted to the potential of information systems and other devices to improve patient safety. This has been driven in significant part by growing evidence that supports the effectiveness of computerized systems as a means to improve quality and safety (Evans et al. 1994, 1999, Classen 1998, Raschke et al. 1998, Bates et al. 1998, 1999, Delaney et al. 1999, Bates 2000, Cullen et al. 2000, Nightingale et al. 2000, Gandhi et al. 2001b, Balas et al. 2000, Balas 2001, Institute of Medicine 1999, 2001, Dexter et al. 2001, James 2001, Kaushal et al. 2001). Decision support systems are increasingly being used in inpatient and ambulatory settings to improve patient safety. These decision support systems can provide computerized reminders about preventive services to prevent errors of omission, as well as highlight drug-drug interactions to prevent errors of commission. Decision support may be embedded into the electronic medical record. Using decision support at the point of care, it is possible for providers to access and utilize evidence-based information to provide better care for their patients. For example, computerized systems can help remind providers about needed preventive services and alert providers to possible drug-drug interactions.
From the safety perspective, decision support systems also can help to ensure that clinical practice is based on the best possible evidence available. There are a number of studies that have found that decision support systems help to improve compliance with guidelines for appropriate care (Shea et al. 1996, Evans et al. 1998, Schiffman et al. 1999, 2000, Goldberg et al. 2000, Dexter et al. 2001). In a review by Hunt et al. (1998), 66 percent of studies demonstrated benefits from decision support systems related to increased use of preventive services and accurate drug dosing.
There are several important examples where computerized decision support systems, particularly computerized physician order entry (CPOE) systems, have been shown to improve patient safety (Evans et al. 1994, 1998, 1999, Classen 1998, Bates1998, 1999, 2001, Bates 2000, Schiff and Rucker 1998, Raschke et al. 1998, Delaney et al. 1999, Gawande and Bates 2000, Nightingale et al. 2000, Teich et al. 2000, Gandhi et al. 2001b). On the inpatient side, Bates and colleagues demonstrated that physician order entry systems decreased serious medication errors by 55 percent and potential adverse drug events by 84 percent. From the cost perspective, there is some evidence that these systems can be both effective and cost-effective in hospital settings. For example, Bates estimated $5-10 million in savings per year, compared to the $2 million in estimated costs (Bates et al. 1998). However, these systems have been slow to disseminate to smaller practices and hospitals, and relatively few hospitals in the United States have CPOE systems in place. The evidence for improving inpatient safety has led the Leapfrog Group, a coalition of large purchasers, to advocate widespread adoption of CPOE by hospitals (Leapfrog Group 2000, Birkmeyer 2000). Evidence of the effectiveness of CPOE to reduce medication errors and adverse drug events is beginning to emerge for outpatient care (Gandhi et al. 2001b, Rothschild et al. 2001). While CPOE is very promising, there are questions that should be posed and answered through rigorous evaluation as CPOE is implemented in a variety of settings.
Conference participants raised a number of specific questions that should be addressed in evaluating CPOE: