Triggers and Targeted Injury Detection Systems (TIDS) Expert Panel Meeting: Conference Summary Report

You Are Here: AHRQ Home > Quality & Patient Safety > Medical Errors & Patient Safety > Triggers and Targeted Injury Detection Systems (TIDS) > Diagnostic Event Triggers

Diagnostic Event Triggers: Current State of Science and Future Directions

Hardeep Singh, M.D., M.P.H.^a
Eric J.Thomas M.D., M.P.H.^b

Background

The spectrum of patient safety events in ambulatory care is quite different from the inpatient setting. For instance, in addition to medication events, diagnostic and other types of care management events are more likely to be common and harmful in ambulatory care.^E1 Errors in diagnosis are expensive^E2,E3 and are the leading basis for ambulatory malpractice claims.^E2,E4,E5 Despite their importance, diagnostic errors are, in general, an underemphasized and understudied area of patient safety research.^E6 Considered as errors of omission, they bring about complex questions of causation and appropriateness and are at times difficult to identify.^E4 Tracking a patient's diagnostic process over time is also not easy in a fragmented outpatient environment, especially when clear standards defining "delays" are lacking.

In our preliminary work, carried out in the Nation's largest electronic health record (EHR) system (the Veterans Affairs [VA] health care system), we developed and tested two computerized triggers to identify patient records that may contain evidence of diagnostic errors.^E7 Triggers are signals that can alert providers to review the medical record to determine if an actual or potential patient safety event occurred.^E8 Our triggers were based on primary care visit patterns in an internal medicine trainee clinic of a tertiary care VA facility. Although their performance was comparable with that of electronic trigger tools used to identify ambulatory medication errors, the positive predictive value (PPV) was only modest: 16.1 percent for one trigger and 9.7 percent for the other.

In work funded by the Agency for Healthcare Research and Quality (AHRQ), we are now refining these trigger tools by integrating them with additional clinical variables (predictive variables) and by reducing false positive triggers. Our efforts focus on increasing the signal-to-noise ratio of positive triggers and could lead to a higher PPV. We have expanded our research beyond the VA to a large primary care network in Texas that has an EHR comparable in many aspects to that of the VA. Hence, our settings will now include internal medicine and family medicine; academic and nonacademic practices; urban and rural patients; and significant racial, gender, ethnic, age, and socioeconomic diversity. Since diagnostic errors due to a lack of followup of abnormal test results are also a significant concern in ambulatory care,^E9 we are now testing a computerized method that potentially can be used as a new trigger tool to detect these problems. Such triggers may be useful to detect and learn about diagnostic errors in ambulatory health care systems that use an advanced EHR.

Development of Methods To Trigger Ambulatory Diagnostic Events

Based on our preliminary research and experience, we believe that the trigger methodology may be useful to advance the study of diagnostic events in ambulatory care. Many opportunities as well as challenges exist. For instance, many diagnostic events, including loss of followup of patients and test results, occur in the outpatient setting,^E10 and triggers to address them have not been well developed.

We propose a conceptual model (Figure 1) to illustrate how the use of two types of triggers (henceforth called Type A and B) may be useful to advance the detection of diagnostic events in ambulatory care. Type A triggers target patterns of visits (such as a primary care visit followed by a hospitalization in the next 14 days) that may be able to identify patients whose diagnosis was missed at the initial visit and who returned to seek care. Electronic medical record review of available progress notes, laboratory and imaging tests, consultations, and other subsequent appointments could confirm or refute the presence of a diagnostic error at the primary care visit. Our current work focuses on developing the next generation of Type A triggers by enriching these trigger tools with additional clinical data from the primary care visit, such as information about abnormal vital sign data, laboratory values, and imaging studies. It may result in higher PPV and the subsequent detection of more diagnostic errors. Due to the nature of this methodology, it also holds promise in identifying other care management problems that occur in ambulatory care in addition to diagnosis. For instance, patients may return to seek care not just because of diagnosis problems but also due to some treatment or monitoring errors.

Type B triggers address events related to loss of followup, either of patients or their abnormal diagnostic test results. These triggers are still in the developmental stages. Currently, we are in the process of testing actionable, concurrent triggers to prevent loss of followup of certain abnormal diagnostic test results in the outpatient setting. If validated, this type of trigger can be used in advanced EHR systems that use a computerized test result notification system to "alert" providers about abnormal results.

Key Considerations in Applying Proposed Diagnostic Event Triggers

Our proposed Type A triggers are global and retrospective. Even though they are considered "nonactionable," they provide useful information for system-level interventions. For instance, once practices detect errors using our triggers, a review of these cases could be conducted by multidisciplinary teams to ensure that all contributing factors are identified. Multidisciplinary interventions can be designed in the future to prevent these errors. This is similar to the goal of voluntary incident reporting systems, except it does not depend upon providers identifying and then taking the time to report the events.

Conversely, Type B triggers are more specific, actionable, and concurrent, and they offer potential for putting into place novel monitoring and surveillance tools that can significantly reduce diagnostic errors in ambulatory care.^E11 For instance, once abnormal diagnostic results that have not received any diagnostic followup within a certain time interval are triggered positive, several actions could be put in place to ensure that they receive prompt attention. Similarly, a missed consultation with a subspecialist could be an indication of a delayed diagnostic evaluation. We would caution, though, that much of this work is untested and is still undergoing development.

The key considerations in defining relative advantage over other methods to detect similar adverse events are PPV, feasibility of use, and limitations imposed by the trigger itself. Methodological constraints do not allow calculation of the true sensitivity and specificity of our triggers; however, PPV provides a reliable indication of trigger effectiveness. PPV must be higher than for some other comparable methods to identify these types of events. (PPVs of our two triggers, although modest, were much higher than those for random chart reviews.) The types of diagnostic triggers we propose may not be feasible in clinical settings where the information management system does not integrate the EHR with the inpatient setting and with other ancillary systems (such as with consultants and with radiology and laboratory information systems). They also will underestimate the error rate for Type A triggers if any patients sought medical care outside the study setting after the initial visit. Other limitations that would affect usability and implementation of such triggers are issues such as hindsight bias and disagreements among reviewers about the presence or absence of a diagnostic error. Hence, rigorous reviewer training is critical.^E12 Lastly, these triggers will inevitably miss some errors (as seen by the presence of errors even in controls in our previous work) and should not be used to determine rates of diagnostic error or compare performance across practices.

Conclusions

We believe it is possible to identify diagnostic events and advance the science of their prevention through the application of trigger methods. Current methodology has encouraging prospects but is relatively underdeveloped compared with triggers for other types of medical errors. The available preliminary triggers are most apt to be used in systems that have an integrated, advanced EHR. A significant investment in further development and refinement of current methods is needed prior to large-scale implementation.

References

E1. Hammons T, Piland NF, Small SD, et al. Ambulatory patient safety. What we know and need to know. J Ambul Care Manage 2003;26:63-82.

E2. Chandra A, Nundy S, Seabury SA. The growth of physician medical malpractice payments: evidence from the National Practitioner Data Bank. Health Aff (Millwood) 2005;Suppl Web Exclusives:W5-240-W5-249.

E3. Thomas EJ, Studdert DM, Newhouse JP, et al. Costs of medical injuries in Utah and Colorado. Inquiry 1999;36:255-264.

E4. Graber M. Diagnostic errors in medicine: a case of neglect. Jt Comm J Qual Patient Saf 2005;31:106-113.

E5. Phillips RL Jr., Bartholomew LA, Dovey SM, et al. Learning from malpractice claims about negligent, adverse events in primary care in the United States. Qual Saf Health Care 2004;13:121-126.

E6. Schiff GD, Kim S, Abrams R, et al. Diagnosing diagnosis errors: Lessons from a multi-institutional collaborative project. In: Henriksen K, Battles JB, Marks ES, et al., eds. Advances in Patient Safety: From Research to Implementation. Agency for Healthcare Research and Quality. Rockville, MD. AHRQ Pub No. 05-0021-2. 2005. pp. 255-278.

E7. Singh H, Thomas EJ, Khan M, et al. Identifying diagnostic errors in primary care using an electronic screening algorithm. Arch Intern Med 2007;167:302-308.

E8. Agency for Healthcare Research and Quality. Patient Safety Network. 2008. http://psnet.ahrq.gov/glossary.aspx#T.

E9. Bates DW, Leape LL. Doing better with critical test results. Jt Comm J Qual Patient Saf 2005;31:66-67.

E10. Singh H, Arora HS, Vij MS, et al. Communication outcomes of critical imaging results in a computerized notification system. J Am Med Inform Assoc 2007;14:459–466.

E11. Singh H, Naik A, Rao R, et al. Reducing diagnostic errors through effective communication: harnessing the power of information technology. J Gen Intern Med 2008;23:489-494.

E12. Thomas EJ, Lipsitz SR, Studdert DM, et al. The reliability of medical record review for estimating adverse event rates. Ann Intern Med 2002;136:812-816.

^a Health Policy and Quality Program, Houston VA Health Service Research and Development (HSR&D) Center of Excellence, and The Center of Inquiry to Improve Outpatient Safety Through Effective Electronic Communication, both at the Michael E. DeBakey Veterans Affairs Medical Center and the Section of Health Services Research, Department of Medicine, Baylor College of Medicine.
^b University of Texas—Houston-Memorial Hermann Center for Healthcare Quality and Safety, Division of General Medicine, Department of Medicine, University of Texas Medical School at Houston.

Note: The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.

Previous Section Contents Next Section