Skip Navigation
< Back
Innovation Attempt

Electronic Alerts Related to Black Box Warnings Do Not Affect Overall Physician Prescribing Habits in Outpatient Clinics


Tab for The Profile
Comments
(0)
   

Snapshot

Summary

Partners HealthCare System developed and integrated electronic alerts based on black box warnings (developed by the U.S. Food and Drug Administration) into an existing, internally developed computerized decision support system. The nature of the alerts varied based on the potential consequences of a violation, with intrusive popup boxes used if death or serious injury could occur and nondisruptive informational alerts used in other situations. In a test conducted in 51 affiliated outpatient clinics, the alert system did not affect physicians’ overall prescribing habits, with the proportion of orders violating black box warnings rising by a statistically insignificant amount after implementation. It did, however, have a positive impact on prescribing habits related to certain types of warnings, particularly alerts about the most serious potential drug–drug and drug–pregnancy interactions. Based on these findings, program leaders scaled back the program by turning off those alerts that did not change physician behavior.
begin doxml

Developing Organizations

Partners HealthCare System
Boston, MAend do

Date First Implemented

2006

What They Did

Back to Top

Problem Addressed

A common, largely preventable problem, adverse drugs events (ADEs) can bring great harm to patients. Although black box warnings (BBWs) are intended to help avoid serious ADEs, physicians sometimes still prescribe in violation of these warnings. Electronic ordering systems can be set up to include alerts to prevent medication-related injuries in the hospital, but this approach remains largely unused and untested in outpatient settings.
  • A common, preventable problem: Each year at least 1.5 million preventable ADEs occur in the United States,1 mostly in outpatient settings (where the prevalence of such errors has been found to be as high as 27 percent.2)
  • Causing great harm: Each year, more than 770,000 injuries or deaths occur in hospitals due to ADEs,3 with many additional injuries and deaths in outpatient settings.
  • Failure to heed BBWs: Physicians continue to prescribe medications in violation of BBWs, due in part to lack of awareness that the proposed prescription conflicts with guidance contained in the warnings. A study of 51 ambulatory practices in the Boston area found that roughly 7 percent of prescriptions for drugs with a BBW violated the warning.4
  • Unrealized potential of alerts electronic ordering systems: Multiple studies suggest that revamped electronic ordering systems that make use of alerts can reduce prescribing errors in hospitals.5,6,7 Yet relatively little experience exists with such systems in outpatient settings.8,9

Description of the Innovative Activity

Partners HealthCare System developed and integrated electronic alerts based on BBWs into an existing, internally developed computerized decision support (CDS) system. The nature of the alerts varied based on the potential consequences of a violation, with intrusive popup boxes used if death or serious injury could occur and nondisruptive informational alerts used in less potentially harmful situations. Key elements of the system include the following:
  • Broad universe of alerts: Based on a knowledge base created by expert panels of physicians and pharmacists, the alerts covered situations judged to be clinically relevant to the ambulatory care setting, including all BBWs related to drug–pregnancy interactions, most related to drug–drug interactions and the need for laboratory test monitoring, and some related to drug–disease interactions.
  • Tiered alerts based on potential consequences: The system reviewed each patient’s active medication list, problem list, and laboratory results, using applied logic rules to identify potential contraindications. Different types of alerts appeared based on the potential for harm, with more important warnings listed first. The BBW warnings appeared alongside other warnings already embedded within the CDS system, with no specific notation distinguishing BBW-based alerts from others. More details on the various levels of alerts appear below:
    • Intrusive popups for life-threatening, other serious situations: If the prescribed drug could lead to a fatal interaction or serious injury (deemed “level 1” and “level 2” alerts, respectively), the system used popup boxes that displayed the alert clearly and required the doctor to take a specific action before proceeding.
      • Level 1 alerts: For potentially fatal situations, the system did not let the doctor continue without taking specific action. For example, a physician ordering isotretinoin received a popup warning about the potential for death or serious injury for patients who are pregnant or of childbearing potential. The physician could not proceed without entering the date of the last pregnancy test and attesting to the fact that the patient was not of childbearing potential, or was not pregnant and had been made aware of the risk of using this drug. The alert also included a reminder to document a negative pregnancy test within 24 hours of initiation of therapy and every month during therapy. In essence, these level 1 alerts represented a “hard stop” that prevented the doctor from violating the BBW.
      • Level 2 alerts: These alerts covered situations with the potential for serious injury. Physicians could continue with the order only if they documented a legitimate reason for doing so. For example, a physician ordering the drug ritonavir to a patient already taking a certain dosage of cisapride received an alert warning of potentially fatal cardiotoxicity, along with instructions to discontinue one of the drugs. The doctor could proceed with the order only if he/she documented a reason for the override, such as adjusting the dosage of one of the drugs to a safe level, agreeing to monitor as recommended, noting that the patient already tolerated this combination of drugs or that no reasonable alternatives existed, or providing another reason.
    • Informational, unobtrusive warnings for less serious situations: For situations with little or no potential for immediate harm, the system embedded an informational alert requiring no specific action within the patient record, as outlined below:
      • Level 3 alerts: These alerts reminded the physician to collect certain information in conjunction with prescribing the drug, usually laboratory test data. For example, a doctor initiating clozapine received a reminder to collect baseline information on white blood cell count.
      • Level 4 alerts: The alerts reminded physicians to monitor certain laboratory test results on an ongoing basis after initiating the drug. For example, the system reminded the doctor to run a serum creatinine test for patients on metformin for more than a year.

References/Related Articles

Yu DT, Seger Dl, Lasser KE, et al. Impact of implementing alerts about medication black-box warnings in electronic health records. Pharmacoepidemiol Drug Saf. 2011;20:192-202. [PubMed]

Contact the Innovator

Diane L. Seger, RPh
Pharmaco-informatics Specialist
Clinical & Quality Analysis, Information Systems
Partners HealthCare System
93 Worcester Street, 2nd Floor
PO Box 81905
Wellesley, MA 02481
Tel: (781) 416-8546
Fax: (781) 416-8771
E-mail: dseger@partners.org

Innovator Disclosures

Ms. Seger has not indicated whether she has financial interests or business/professional affiliations relevant to the work described in this profile.

Did It Work?

Back to Top

Results

In a test conducted in 51 outpatient clinics, the BBW alerts did not affect physicians’ overall prescribing habits, with the proportion of orders violating BBWs rising by a statistically insignificant amount after implementation. The program had a positive impact in some specific areas, particularly for the alerts about the most serious potential drug–drug and drug–pregnancy interactions.
  • No impact on overall likelihood of BBW violation: Post-implementation, 5.1 percent of the 56,869 orders violated a BBW, slightly higher than the 4.8-percent rate in 28,359 orders before implementation.4
  • Positive impact in specific areas: The alerts reduced BBW violations related to potential drug–drug and drug–pregnancy interactions (particularly high-priority alerts in these areas) and tended to be more effective with primary care clinicians and in clinics outside the hospital.4
    • Fewer potential drug–drug and drug–pregnancy interactions: The proportion of orders violating a BBW related to drug–drug interactions fell from 6.1 to 1.8 percent after implementation. Similarly, the proportion violating a BBW related to drug–pregnancy interactions fell from 5.1 to 3.6 percent. Both declines met the test of statistical significance. In general, reductions tended to be concentrated in level 1 and 2 alerts. (As noted, the system prevented physicians from violating level 1 alerts.)
    • Positive impact in primary care and nonhospital settings: Primary care physicians prescribed fewer drugs in violation of BBWs after implementation, while specialists prescribed more. Similarly, physicians working in clinics outside the hospital had fewer violations after implementation, while those in hospital-based clinics had more.

What They Learned

Back to Top
  • Be selective in choosing alerts: Implementing too many alerts can be disruptive to physicians and, as Partners’ experience shows, yield little overall benefit. The better approach is to alert physicians only to the most important potential violations, particularly level 1 and level 2 alerts related to drug–drug and drug–pregnancy interactions. Based on their experience, program leaders at Partners have scaled back this program by turning off specific alerts that had no impact on physician behavior.
  • Start with level 1 alerts: Organizations without an existing CDS-based alert system should consider starting with level 1 alerts. By emphasizing these important, widely accepted alerts, would-be adopters can gain buy-in and support among physicians. This approach will also help physician practices in meeting "meaningful use" electronic health record requirements developed by the Office of the National Coordinator (ONC) for Health Information Technology, which mandate implementation of alerts about important drug–drug interactions. To assist with this effort, Partners has developed a set of level 1 drug–drug interactions for ONC. This set is not yet available to the general public; those interested can contact the program developer for a status update on its availability.
  • Screen vendor products carefully: Most organizations purchase health information technology and CDS systems rather than building their own (as Partners did). Those buying such products should evaluate them carefully to make sure they meet organizational needs. Some systems include a very large universe of built-in alerts and may not offer the ability to “turn off” those that prove ineffective or otherwise undesirable.


 
1 Institute of Medicine. Preventing Medication Errors: Quality Chasm Series. Washington, DC: National Academies Press; 2006.
2 Gandhi TK, Weingart SN, Borus J, et al. Adverse drug events in ambulatory care. N Engl J Med. 2003;348(16):1556–1564. [PubMed]
3 Classen DC, Pestotnik SL, Evans RS, et al. Adverse drug events in hospitalized patients: Excess length of stay, extra costs, and attributable mortality. JAMA. 1997;277(4): 301–306. [PubMed]
4 Yu DT, Seger Dl, Lasser KE, et al. Impact of implementing alerts about medication black-box warnings in electronic health records. Pharmacoepidemiol Drug Saf. 2011;20:192-202. [PubMed]
5 Leape LL, Bates DW, Cullen DJ, et al. Systems analysis of adverse drug events. ADE Prevention Study Group. JAMA. 1995;274(1):35–43. [PubMed]
6 Shamliyan TA, Duval S, Du J, et al. Just what the doctor ordered. Review of the evidence of the impact of computerized physician order entry system on medication errors. Health Serv Res. 2008;43:32–53. [PubMed]
7 Wolinsky JI, Gurwitz JH, Field TS, et al. The effect of computerized physician order entry with clinical decision support on the rates of adverse drug events: A systematic review. J Gen Intern Med. 2008;23:451–458. [PubMed]
8 Gurwitz JH, Field TS, Harrold LR, et al. Incidence and preventability of adverse drug events among older persons in the ambulatory setting. JAMA. 2003;289(9):1107–1116. [PubMed]
9 Lazarou JM, Pomeranz BHM, Corey PN. Incidence of adverse drug reactions in hospitalized patients: A meta-analysis of prospective studies. JAMA. 1998;279(15):1200–1205. [PubMed]
Comment on this Innovation

Disclaimer: The inclusion of an innovation in the Innovations Exchange does not constitute or imply an endorsement by the U.S. Department of Health and Human Services, the Agency for Healthcare Research and Quality, or Westat of the innovation or of the submitter or developer of the innovation. Read more.
Innovation Attempt Classification

IOM Domains of Quality:
Organizational Processes:
Funding Sources:

Original publication: July 06, 2011.
Original publication indicates the date the profile was first posted to the Innovations Exchange.

Last updated: July 03, 2012.
Last updated indicates the date the most recent changes to the profile were posted to the Innovations Exchange.

Date verified by innovator: June 15, 2012.
Date verified by innovator indicates the most recent date the innovator provided feedback during the annual review process. The innovator is invited to review, update, and verify the profile annually.