Advances in Patient Safety: From Research to Implementation Volume 1. Research FindingsDetection of Inpatient Health Care Associated Injuries: Comparing Two ICD-9-CM Code Classifications
Carol J. Masheter, Paul Hougland, Wu Xu
AbstractThis paper compares two complementary International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) code classifications for detection of adverse events (AEs) at the hospital, State, and national levels. The classifications are the Agency for Healthcare Research and Quality's Patient Safety Indicators 2003, June release, Version 2.1 (PS Indicators) and the Utah/Missouri Adverse Event Classification 2002, Version 1 (AE Classes). First, the paper describes similarities and differences between the two classifications, such as intended purpose, process of development, grouping of ICD-9-CM codes, specificity, and sensitivity. Second, it compares the ways each classification categorizes ICD-9-CM codes into indicators or classes of potential AEs. Third, the paper presents the number and percentage of Utah inpatient discharges (UTIDs) with any PS Indicator over 3 years (2000, 2001, and 2002) and compares the percentage of UTIDs by PS Indicator to published values derived from a national database, the Healthcare Cost and Utilization Project (HCUP) Nationwide Inpatient Sample 2000 (NIS). UTIDs have significantly higher values for five PS Indicators and significantly lower values for seven PS Indicators than NIS. Fourth, the paper presents the annual percentage of UTIDs with any AE Class code for 2000, 2001, and 2002, which shows a small but significant increase over this 3-year period. The AE Classes are more sensitive; they detect far more potential injuries due to medical care, or AEs, but may include more false positives than the PS Indicators. The PS Indicators are more specific; they detect fewer potential AEs but may include fewer false positives than the AE Classes.
We expect miracles from modern medicine. In fact, advances in medicine have increased life expectancy and enhanced quality of life. People who would have died or been bedridden by cancer, diabetes, heart disease, or other diseases even a decade ago now survive, and even thrive, thanks to new medical devices, procedures, and medications. However, the very care that is intended to heal can also cause harm. Highly publicized cases, such as a teenage girl who died after lung transplant surgery due to a blood type error, 1 serve as tragic reminders. Unfortunately, these cases are only the tip of the iceberg. Frequently cited studies have estimated between 44,000 and 98,000 deaths per year due to medical management nationwide. 2 The Institute of Medicine has reported findings from approximately 30 works published in the 1990s substantiating "serious and widespread errors in health care delivery that resulted in frequent avoidable injuries to patients." 3 Such incidents and publications have increased awareness of errors in, and injuries due to, medical management (AEs) rather than the patient's underlying disease or condition. 2
AE underreportingAEs are underdetected and underreported, both within health care organizations and externally.
2
Interviews in 19 States, including Utah, indicated numerous reasons for underreporting, such as health care facilities lacking internal systems to identify events, uncertainty about reporting requirements, a culture of nonreporting, a lack of enforcement at the State level, bureaucratic burden, competition and market share, fear of publicity, fear of liability,
4
and lack of a common AE taxonomy.
5
These findings suggest that AE reporting systems may be most effective if they are easy to use, but not so simple that the information reported is of limited value.
5
In its call for improved understanding of patient safety epidemiology, the Agency for Healthcare Research and Quality (AHRQ) has identified hospital discharge data as one of six useful sources of information on AEs.
5
Hospital discharge data are among the few forms of data used nationwide. Because the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) coding system is widely used in hospital discharge data, it can provide an accessible, low cost, and efficient means of detecting AEs in hospitals, State and nationwide.
6
8
Because it is already in use, this system adds few new burdens on hospital and State resources.
6
As retrospective surveillance and case-finding tools, ICD-9-CM codes in discharge data can complement other detection and reporting methods. Though hospital discharge data and ICD-9-CM codes have substantial limitations, the codes provide an attractive starting place to improve AE detection.
6,
7
Hospitals already use ICD-9-CM codes in discharge data for their own patient safety surveillance systems. For example, at a large teaching hospital in Utah, ICD-9-CM codes related to medical device codes detected more medical device-related hazards and AEs than five other detection methods, including computer flag surveillance, online incident reporting, telemetry checklists, clinical engineering database, and a postdischarge patient survey. Also, ICD-9-CM codes related to medical devices detected different kinds of device-related hazards and AEs than the other detection methods. Review of a random-selection sample of patient records in a large teaching hospital in Utah with ICD-9-CM device codes (N = 141) revealed that 72 percent had a confirmed medical device AE. 8 Other Utah teaching hospital researchers have investigated enhanced case findings based on ICD-9-CM codes, followed by retrospective chart review using explicit criteria to detect adverse drug events (ADEs), or adverse events related to medications. Initial results suggested that such methods can roughly double the total number of ADEs detected relative to those found by computer flag surveillance. 9 These studies give compelling evidence of the effectiveness of ICD-9-CM codes for AE detection. However, they are limited to specific teaching hospitals and specific kinds of AEs.
Several States also have developed methods based on ICD-9-CM codes for statewide surveillance of AEs and to assist hospitals with their own internal quality improvement of patient care. These classifications differ in purpose of AE detection as well as scope and kind of AEs they detect. For example, the Florida Agency for Health Care Administration released a one-time statewide study in 2000 on drug-caused illnesses, such as allergies and side effects, that was based on codes associated with these adverse drug effects. 10 To date, the Utah Department of Health has sent five quarterly reports that contain statewide numbers and rates of inpatient discharges with at least one ADE or misadventure and related information to all acute care hospitals in Utah. The Department also has posted the reports on its Web site. 11 Additionally, all 40 Utah acute care hospitals received their own hospital level numbers and rates for ADEs and misadventures. The Department uses the term "misadventures" to include obvious errors of or injuries due to medical care, such as accidental punctures or lacerations and foreign bodies accidentally left in patient's body during procedures, wrong surgery, etc.
Adverse event classes of ICD-9-CM codesAHRQ-funded patient safety reporting demonstration projects in Missouri, New York, Utah, and Wisconsin have developed similar, but not identical, classifications for several categories of AEs.
5
Wisconsin uses four categories of ICD-9-CM codes for AEs (drug, device, procedure, and radiation), including principal and secondary diagnosis codes among all Wisconsin general acute care hospitals.
6
One of New York's AHRQ-funded projects focuses on three specific kinds of AEs: new acute pulmonary embolism, acute myocardial infarction, and postoperative wound infection.
5
Based on published research, research in progress, and input from a national expert panel, the Utah and Missouri studies expanded Rolfs and associates' previous AE classification.
12
The expanded version, Utah/Missouri Adverse Event Classification, 2002 Version 1 (AE Classes),
13
is being validated by medical chart reviews in Missouri and Utah (N = approximately 7,200 charts per State). All 40 Utah acute care hospitals are participating in the chart review. In Missouri, 36 study hospitals (a convenience sample of all 123 Missouri acute care hospitals) are participating in the chart review. (See Van Tuinen, Elder, Link, Li, Song, Pritchett, "Surveillance of Surgery-Related Adverse Events in Missouri: Using ICD-9-CM Codes," in this volume for more details about the Missouri sample.) The Utah study emphasizes misadventures, a subset of two of the surgery-related AE Classes, (e.g., patient injuries likely to be due to medical care, such as foreign object accidentally left in patient's body), and ADEs in the entire patient population; the Missouri study emphasizes a larger number of surgery-related AEs among surgery patients. The States have provided their statewide and hospital information on AEs to their hospitals. Based on this information, some Utah hospitals have requested additional patient-level information for in-depth examination of AEs and changed or added programs to reduce risk to patients.
AHRQ and Stanford University have developed 20 Patient Safety (PS) Indicators based on ICD-9-CM codes.
14
These indicators are based on published research, input from a national expert panel, medical coding experts, and empirical results from analysis of Healthcare Cost and Utilization Project (HCUP) data. The intended purpose of the PS Indicators is to detect potential AEs, prioritize areas of action, or to provide a starting point for further analysis to reduce preventable errors through system or process changes. Users have reported that the PS Indicators have demonstrated high specificity and low sensitivity.
15
In other words, the PS Indicators capture relatively few false positives (potential AEs that further inquiry showed were not caused by medical care,) but they probably miss a considerable portion of true positives, or actual AEs. In this paper, "potential AEs" are errors or injuries detected by selected ICD-9-CM codes that are more likely than not to be due to medical care, such as accidental puncture or laceration during a procedure (998.2).
In summary, several teaching hospitals, States, and a federal agency have developed ICD-9-CM code classifications as accessible, low cost, efficient methods for detection of potential AEs for State public health surveillance and to assist their hospitals with internal patient safety improvement. However, AHRQ and most of these States have developed their own ICD-9-CM code classifications. Their classifications differ in number of codes, ways the codes are categorized, ranges of AEs they detect, and the degree of verification for their ability to detect actual AEs. To date, no national findings are available for any of the state-developed classifications, and no State findings have been published on the PS Indicators. This paper addresses this gap. It compares one of the State classifications, the Utah/Missouri Adverse Event Classification, 2002 Version 1 (AE Classes),
13
and the AHRQ Quality Indicators, Patient Safety Indicators 2003, Version 2.1 (PS Indicators).
14
Data Sources
The national dataset is the HCUP National Inpatient Sample (NIS), which consists of approximately 36 million discharges from 986 nonfederal United States acute care hospitals in the year 2000. For this paper, the authors have used published values for number and percentage of NIS discharges by PS Indicator.
16
The Utah dataset (Utah inpatient discharges or UTIDs) consists of approximately 710,000 inpatient discharges from all 40 nonfederal Utah acute care hospitals in 2000, 2001, and 2002.
17
First, this paper compares the features of the two classifications, the PS Indicators and the AE Classes. Second, it compares the ways each classification categorizes ICD-9-CM codes into indicators or classes of potential AEs. Third, the paper presents and compares the number and percentage of inpatient discharges by PS Indicator for UTIDs and NIS. Fourth, the paper presents the number and percentage of inpatient discharges by AE Class for UTIDs.
Similarities between PS Indicators and AE ClassesThe two classifications have followed similar development processes and share several similarities. Both use ICD-9-CM codes in hospital inpatient discharge data to identify potential AEs. Input from national expert panels guided development of both classifications. Both exclude principal diagnosis codes from the numerator for each indicator or class to focus on in-hospital AEs, rather than AEs that originated before the patient's hospital stay. Both classifications restrict the denominator of some indicators or classes to at-risk patient populations (e.g., only surgery patients' discharges were included in the denominator for potential AEs related to surgical procedures).
Closer inspection reveals several differences (Table 1). First, each classification has a different emphasis. The PS Indicators, which target events with a high likelihood of representing errors in medical care, are intended primarily for institutional case-finding and patient safety improvement initiatives. 14 The AE Classes are intended primarily for statewide public health surveillance, though hospitals have been encouraged to use State and hospital information based on the AE Classes for internal improvement of patient care. Second, the PS Indicator expert panel consists of physicians who reviewed PS Indicator codes prescreened by medical records coders, 14 whereas the panel for the AE Classes includes pharmacists, nurses, medical records experts, as well as physicians, all of whom reviewed each AE Class code. Third, PS Indicators were developed based on previous findings. 14 The development of the AE Classes included chart reviews, currently in progress, in Utah and Missouri (N = approximately 7,200 charts per State.)
Fourth, the classifications differ in scope. The PS Indicators consist of 143 ICD-9-CM diagnosis codes and codes for external causes of injury and poisoning (E-codes) in three categories: three obstetric or birth trauma indicators, eight medical indicators, and nine surgery-related indicators, 14 but none specifically for ADEs. The AE Classes consist of 1,003 codes in 3 categories of 64 classes: 26 ADE classes, 22 medical classes, and 16 surgery-related classes (including two misadventure classes). 13 The Utah study has emphasized the ADE classes, because the Harvard Study reported that they are the most frequent cause of iatrogenic injury. 18
Table 1. Comparison of AHRQ Patient Safety Indicators and Utah/Missouri Adverse Event Classes| Comparison | AHRQ Patient Safety Indicators | Utah/Missouri Adverse Event Classes | |
| 1 | Purpose | Hospital internal quality improvement | Statewide public health surveillance |
| 2 | Expert panel | Physicians reviewed codes prescreened by medical records coders. | Physicians, pharmacists, nurses, medical records experts reviewed each code |
| 3 | Medical chart review | Literature based | 14,400 charts reviewed, analysis in progress |
| 4 | Number of ICD-9-CM codes | 143 | 1,003 |
| Number of indicators/classes | 20 | 64 | |
| Categories of indicators/classes | 3 obstetric or birth adverse event indicators | 26 adverse drug event classes | |
| 8 medical or surgery-related adverse event indicators | 22 medical adverse event classes | ||
| 9 surgery-related adverse event indicators | 16 surgery-related adverse event classes (including two misadventure classes) | ||
| 5 | Mapping onto ICD-9-CM codes | Medical injury topic, e.g., complications of anesthesia | Adverse event type, e.g., poisoning by antibiotics and other infectives |
| 6 | Emphasis | Exclusive, e.g., may capture fewer false positives but may miss more true positives | Inclusive, e.g., may capture more true positives but may include more false positives |
| Numerators | ICD-9-CM codes and subgroups of patients | ICD-9-CM codes | |
| Denominators | At-risk patient subpopulations | Entire patient population, except surgery-related adverse events in surgery patients only | |
The classifications also differ in how they group the ICD-9-CM codes. For example, PS Indicator complications of anesthesia (18 codes) is an injury "topic" (anesthesia,) whereas AE Class adverse effects of anesthetics and other central nervous system (CNS) agents (17 codes) names a kind of injury (by certain kinds of drugs.) Differences in the grouping of codes vary by Indicator/Class. For example, the PS Indicator complications of anesthesia includes some (but not all) of the codes in several AE Classes (Table 2). In contrast, the PS Indicator and the AE Class for decubitus ulcer use the same single code, 790.7, an example of 100 percent overlap in code use for both classifications. However, the PS Indicator has more inclusion and exclusion criteria than the AE Class for decubitus ulcer. The PS Indicator includes only medical and surgery-related discharges and only patients with a length of stay longer than 4 days. It excludes patients with any diagnosis of hemiplegia, paraplegia, or quadriplegia. The AE Class includes all patients with no exclusions except principal diagnosis of decubitus ulcer. The PS Indicator postoperative sepsis and AE Class septicemia/bacteremia are an example of partial overlap in code use that is less complex than the PS Indicator complications of anesthesia. The PS Indicator and the AE Class use the same 13 sepsis codes, but the former also includes two additional codes (995.91, 995.92 systemic inflammatory response syndrome due to infectious process without and with organ dysfunction, respectively), and the latter also includes a single bacteremia code, 790.7. The PS Indicator postoperative sepsis includes only elective surgery cases or a patient with length of stay longer than 3 days, and excludes obstetric patients and principal diagnoses of infection or any code for immunocompromised state and cancer. AE Class sepsis excludes only principal diagnosis of septicemia or bacteremia.
Table 2. Comparison of AHRQ Patient Safety Indicators and Utah/Missouri Adverse Event Classes for complications of anesthesia, infection, sepsis, and decubitus ulcer| Patient Safety Indicator | AHRQ Patient Safety Indicator Codes | Utah/Missouri Adverse Event Class Codes |
| Anesthesia | ||
| Indicator 1. Complications of Anesthesia | Class 49. Other misadventures of medical care | |
| E876.3 (one code) Endotracheal tube wrongly placed during anesthetic procedure | E876.3 is one of 51 codes in Class 49. | |
| 968 (9 codes) Poisoning by specified anesthetics, unspecified local anesthetics and muscle-tone depressants | Class 40. Poisoning by other CNS depressants, stimulants, anesthetics, nervous system agents | |
| E855.1 (one code) Poisoning by muscle-tone depressants | 968 and E855.1 includes 10 of 17 codes in Class 40. | |
| E939 (8 codes) Adverse effects by specified anesthetics, unspecified general anesthetics and muscle-tone depressants | Class 60. Adverse effects of other CNS depressants, stimulants, anesthetics, nervous system agents | |
| E939 includes 8 of 18 codes in Class 60. | ||
| Total: 19 codes | Total: 86 codes | |
| Infection | ||
| Indicator 7. Infections due to medical care | Class 44. Complications peculiar to specified procedures | |
| 996.62 (1 code) Infection & inflammatory reaction to other (not cardiac) device, implant, and graft | 996.62 is 1 of 57 codes in Class 44. | |
| Class 47. Other medical complications not specified. | ||
| 999.3 (1 code) Other infection due to medical care not elsewhere specified | 999.3 is 1 of 10 codes in Class 47. | |
| Total: 2 codes | Total: 67 codes | |
| Sepsis | ||
| Indicator 14. Postoperative sepsis | Class 3. Sepsis/bacteremia | |
| 038.00038.9 (13 codes) |
Septicemia due to specified organisms, unspecified gram-negative organisms, and bacteremia | 038.00038.9 include 14 of 15 codes in Class 3. |
| 995.91, 995.92 (2 codes) | 790.7 (one code for bacteremia) in Class 3. | |
| Total: 15 codes | Total: 14 codes | |
| Decubitus ulcer | ||
| Indicator 3. Decubitus ulcer | Class 23. Decubitus ulcer | |
| 707.0 | 707.0 | |
| Total: 1 code | Total: 1 code | |
Fifth, definitions of numerators and denominators for the PS Indicators and the AE Classes influence the specificity and sensitivity of these two classifications. The PS Indicators use fewer codes in the numerator and more restricted denominators (patients at risk) than most of the AE Classes. For example, for PS Indicator infection due to medical care, only two infection codes are used for the numerator, and patients with any diagnosis code for immunocompromised state or cancer were excluded from the denominator. The comparable AE Class uses more codes and the entire inpatient population for the denominator, except principal diagnosis for infection (see Table 2). In fact, users have reported that the PS Indicators have demonstrated excellent specificity, although their sensitivity is quite low, 15 whereas the authors of this paper have found the AE Classes to be more sensitive and less specific. In other words, the PS Indicators may miss some true positives (actual AEs) yet include fewer false positives (a PS Indicator detects a potential AE but further investigation shows an actual AE has not occurred,) while the AE Classes probably capture more true positives and more false positives.
Since AHRQ released the Patient Safety Indicators (PSI) software,
14
researchers have reported national rates of selected PSI.
16
Funded by AHRQ, several large health systems and hospital chains are exploring methods to use PSI for internal quality improvements.
The number and percentage of UTIDs with any potential AEs detected by the PS Indicators were determined using modified versions of the AHRQ SAS programs. For each PS Indicator, the annual number of UTIDs was small, and differences among the years 2000, 2001, and 2002 were not statistically significant. These 3 years of UTIDs were combined to calculate the percentage of discharges by PS Indicator. The authors also developed SAS programs to obtain the annual number and percentage of discharges with any potential AE Class code. The AE Classes are not mutually exclusive, nor are the PS Indicators. Throughout this paper, 95 percent confidence intervals (95% CI) were used as the criterion for statistical significance (alpha = 0.05.)
PS Indicators for UTIDs and the NIS
The number and percentage of UTIDs with any PS Indicator are 22,298 of 710,077 total discharges (3.141 percent). The published number and percentage of NIS discharges with any PSI Indicator are 1.07 million of 36.32 million discharges 16 (2.946 percent). This difference is statistically significant but not substantially different.
At the individual indicator level, the differences in percentage of discharges for UTIDs and NIS are statistically significant for 12 of the 20 PS Indicators (Table 3.) Five are significantly higher for UTIDs than for NIS. Of particular interest is obstetric (OB) vaginal trauma with instrument (26.517 percent for UTIDs versus 24.408 percent for NIS), which has the highest values for both samples. Accidental puncture/laceration also is noteworthy, because the UTIDs percentage is over twice the NIS percentage (0.718 percent for UTIDs versus 0.324 percent for NIS). Seven differences in percentage are significantly lower for UTIDs than NIS. Of particular interest is failure to rescue, which has the second highest values for both samples (11.132 percent for UTIDs versus 17.424 percent for NIS), and OB vaginal trauma without instrument (7.689 percent for UTIDs versus 8.659 percent for NIS), which has the third highest values for both samples. Differences for the remaining eight PS Indicators are not significant.
Table 3. Number and percentage of inpatient discharges by AHRQ Patient Safety Indicator, national sample 2000 (NIS), and Utah sample 2000, 2001, and 2002 (UTIDs), nonfederal acute care hospitals| Patient Safety Indicator | US Numerator 2000 | US % 2000 | US 95% CI | US 95% CI Range | UT Numerator 2000-2002 | UT % 2000-2002 | UT 2000-2002 95% CI Range | |||
| 1 | Complications of Anesthesia* | 5,305 | 0.056 | 0.005 | 0.051 | 0.061 | 156 | 0.077 | 0.065 | 0.089 |
| 2 | Death in Low Mortality DRGs | 5,912 | 0.043 | 0.003 | 0.040 | 0.046 | 97 | 0.040 | 0.032 | 0.048 |
| 3 | Decubitus Ulcer* | 201,459 | 2.130 | 0.107 | 2.023 | 2.237 | 1,546 | 1.443 | 1.371 | 1.514 |
| 4 | Failure to Rescue* | 267,541 | 17.424 | 0.329 | 17.095 | 17.753 | 1,633 | 11.132 | 10.623 | 11.641 |
| 5 | Foreign Body Left in During Procedure* | 2,710 | 0.008 | 0.001 | 0.007 | 0.009 | 74 | 0.013 | 0.010 | 0.016 |
| 6 | Iatrogenic Pneumothorax | 19,397 | 0.067 | 0.004 | 0.063 | 0.071 | 288 | 0.080 | 0.071 | 0.089 |
| 7 | Infection due to Medical Care* | 54,490 | 0.193 | 0.009 | 0.184 | 0.202 | 843 | 0.166 | 0.155 | 0.177 |
| 8 | Postop Hemorrhage or Hematoma | 17,014 | 0.206 | 0.012 | 0.194 | 0.218 | 318 | 0.184 | 0.164 | 0.204 |
| 9 | Postop Hip Fracture* | 5,207 | 0.080 | 0.005 | 0.075 | 0.085 | 48 | 0.047 | 0.034 | 0.061 |
| 10 | Postop Physiological and Metabolic Derangements | 4,003 | 0.089 | 0.009 | 0.080 | 0.098 | 60 | 0.078 | 0.059 | 0.098 |
| 11 | Postop Pulmonary Embolism or Deep Vein Thrombosis | 75,811 | 0.919 | 0.050 | 0.869 | 0.969 | 1,458 | 0.845 | 0.802 | 0.889 |
| 12 | Postop Respiratory Failure* | 12,842 | 0.359 | 0.026 | 0.333 | 0.385 | 154 | 0.229 | 0.193 | 0.265 |
| 13 | Postoperative Sepsis* | 14,055 | 1.091 | 0.082 | 1.009 | 1.173 | 175 | 0.831 | 0.709 | 0.954 |
| 14 | Postop Wound Dehiscence | 3,858 | 0.193 | 0.014 | 0.179 | 0.207 | 68 | 0.145 | 0.111 | 0.180 |
| 15 | Accidental Puncture/Laceration* | 89,348 | 0.324 | 0.021 | 0.303 | 0.345 | 2,942 | 0.718 | 0.692 | 0.744 |
| 16 | Transfusion Reaction | 138 | 0.0004 | 0.0001 | 0.0003 | 0.0005 | 8 | 0.0014 | 0.0004 | 0.0024 |
| 17 | Birth Trauma* | 27,035 | 0.667 | 0.140 | 0.527 | 0.807 | 1,307 | 0.931 | 0.880 | 0.981 |
| 18 | OB Trauma Vaginal With Instrument* |
60,622 | 24.408 | 1.250 | 23.158 | 25.658 | 3,495 | 26.517 | 25.764 | 27.271 |
| 19 | OB TraumaVaginal Without Instrument* |
249,243 | 8.659 | 0.437 | 8.222 | 9.096 | 7,989 | 7.689 | 7.527 | 7.852 |
| 20 | OB TraumaC-Section |
5,523 | 0.593 | 0.064 | 0.529 | 0.657 | 132 | 0.528 | 0.438 | 0.617 |
| Any PS Indicator* | 1,070,000 | 2.946 | 0.006 | 2.940 | 2.952 | 22,298 | 3.141 | 3.100 | 3.181 | |
| Denominator for Overall Percentage | 36,318,000 | 710,077 | ||||||||
|
* Percentage is significantly different (alpha = 0.05), according to 95% CIs. Indicators are not mutually exclusive. Data Sources:Healthcare Cost and Utilization Project (HCUP) National Inpatient Sample 2000 (NIS)Utah Hospital Database, 2000, 2001, and 2002 (UTIDs), Utah Department of Health | ||||||||||
The annual number and percentage of UTIDs with any AE (Figure 1) increased from 2000 through 2002 (53,351 and 23.07 percent to 58,706 and 24.21 percent). For the overall percentage for any kind of AE, and for the three categories of AE classessurgery-related AE (10,378, 16.91 percent to 10,715, 17.95 percent), medical AE (44,374, 19.19 percent to 49,121, 20.26 percent), and ADE (6,916, 2.99 percent to 8,110, 3.34 percent)the increases from 2001 to 2002 and from 2000 to 2002 were significant, but no increases from 2000 to 2001 were significant. For subcategories within "any ADE," increases for clinical side effects of drugs (1,659, 0.72 percent to 1,983, 0.82 percent) and adverse effects of (a specified group of) drugs (4,679, 2.02 percent to 5,206, 2.15 percent) mirrored significant differences for "any ADE." For poisoning by (a specified group of) drugs (1,155, 0.50 percent to 1,539, 0.63 percent, the increase from 2000 to 2002 is significant. All other increases are not significant.
| Adverse Event Type | N, 2000 | D, 2000 | %, 2000 | 95% CI, 2000 | 2000 lower | 2000 upper | N, 2001 | D, 2001 | 2001 | 95% CI, 2001 | 2001 lower | 2001 upper | N, 2002 | D, 2002 | 2002 | 95% CI, 2002 | 2002 lower | 2002 upper |
| Any AE | 53351 | 231208 | 23.07 | 0.172 | 22.90 | 23.25 | 55263 | 236365 | 23.38 | 0.171 | 23.21 | 23.55 | 58706 | 242504 | 24.21 | 0.170 | 24.04 | 24.38 |
| Any Surgery-related AE | 10378 | 61372 | 16.91 | 0.297 | 16.61 | 17.21 | 10807 | 62805 | 17.21 | 0.295 | 16.91 | 17.50 | 10715 | 59682 | 17.95 | 0.308 | 17.65 | 18.26 |
| Any Medical AE | 44374 | 231208 | 19.19 | 0.161 | 19.03 | 19.35 | 45907 | 236365 | 19.42 | 0.159 | 19.26 | 19.58 | 49121 | 242504 | 20.26 | 0.160 | 20.10 | 20.42 |
| Any ADE | 6916 | 231208 | 2.99 | 0.069 | 2.92 | 3.06 | 7310 | 236365 | 3.09 | 0.070 | 3.02 | 3.16 | 8110 | 242504 | 3.34 | 0.072 | 3.27 | 3.42 |
| Clinical Side Effects of Drugs | 1659 | 231208 | 0.72 | 0.034 | 0.68 | 0.75 | 1733 | 236365 | 0.73 | 0.034 | 0.70 | 0.77 | 1983 | 242504 | 0.82 | 0.036 | 0.78 | 0.85 |
| Poisoning by Drugs | 1155 | 231208 | 0.50 | 0.029 | 0.47 | 0.53 | 1377 | 236365 | 0.58 | 0.031 | 0.55 | 0.61 | 1539 | 242504 | 0.63 | 0.032 | 0.60 | 0.67 |
| Adverse Effects of Drugs | 4679 | 231208 | 2.02 | 0.057 | 1.97 | 2.08 | 4731 | 236365 | 2.00 | 0.056 | 1.95 | 2.06 | 5206 | 242504 | 2.15 | 0.058 | 2.09 | 2.20 |
| 2000, 2001 | 2000, 2002 | 2001, 2002 | ||||||||||||||||
| Adverse Event Type | ||||||||||||||||||
| Any AE | N | Y, H | Y, H | |||||||||||||||
| Any Surgery-related AE | N | Y, H | Y, H | |||||||||||||||
| Any Medical AE | N | Y, H | Y, H | |||||||||||||||
| Any ADE | N | Y, H | Y, H | |||||||||||||||
| Clinical Side Effects of Drugs | N | Y, H | Y, H | |||||||||||||||
| Poisoning by Drugs | N | Y, H | N | |||||||||||||||
| Adverse Effects of Drugs | N | Y, H | Y, H | |||||||||||||||
The AE Classes detect considerably more potential adverse effects in UTIDs than the PS Indicators (about 53,400 to 58,700 versus about 7,400 to 7,600 annually). The annual percentage of UTIDs with any AE Class is more than seven times that for the PS Indicators (23.07 percent to 24.21 percent versus 3.14 percent.)
Overview: differences in design, purpose, and use
The higher number of potential AEs detected by the AE Classes was expected, since the AE Classes use more ICD-9-CM codes (1,003) than the PS Indicators (143). The developers of the AE Classes wanted this classification to be inclusive in the early stages of use and may later refine the classes based on chart review findings. For State public health surveillance, the AE Classes detect more potential AEs for further investigation and trend analysis over time. For internal patient care quality improvement at the individual hospital level, PS Indicators can help hospital personnel identify and examine "high yield" cases, or those that are most likely to involve actual AEs.
There could be several explanations for statistically significant differences in percentage of discharges with any PS Indicator or AE Class. First, the statistical differences could be due to variations in coding between UTIDs and NIS. On the one hand, implementation in 2001of Utah patient safety administrative rules, which require health care facilities to report ADEs using their ICD-9-CM codes and to report sentinel events, may account for improved reporting of some codes. 19, 20 For example, Utah reports more E-codes than the national average.
Specifically, an AHRQ internal analysis reveals that on average, in 2001, the AHRQ HCUP database contained 33.7 percent of medical misadventure discharges with a corresponding misadventure E-code; but UTIDs reported 39.7 percent of the cases (AHRQ HCUP internal memo describing an analysis of State E-code reporting and accuracy, dated March 1, 2004). On the other hand, significantly lower UTIDs values for some of the PS Indicators may mean that Utah hospitals have miscoded or underreported the conditions these PS Indicators detect compared to the NIS.
Second, significant differences could reflect differences in actual AEs, which might be due to moderating variables in Utah compared to the rest of the Nation, such as variations in patient characteristics, medical practice, or prenatal care. Increases for AE Classes from 2000 to 2002 could mean that that rate of actual AEs has increased. Improved detection would be "good news," because more detected AEs could increase understanding, leading to their reduction and prevention. More actual AEs are "bad news," suggesting more harm to patients is occurring.
Though statistically significant differences alone do not provide definitive answers, they do provide a useful starting point for both health care quality/patient safety improvement and statewide surveillance. Users, such as health care facilities (e.g., hospitals) and public health institutions (e.g., State departments of health), can start with these differences and explore possible explanations, such as those suggested above.
The expert panel of the Utah/Missouri Patient Safety Project recognizes several limitations of use of ICD-9-CM codes in hospital discharge data for detection of potential AEs. ICD-9-CM codes sometimes are incomplete. Some health care facilities and public health institutions do not capture all of the codes that may be in a patient's discharge records. For example, currently the UTIDs database has nine fields for ICD-9-CM diagnosis codes plus a single additional field for an E-code. Some patients' discharge records may contain additional codes that these 10 fields do not capture. Because some ICD-9-CM E-codes, which document conditions related to an external cause of injury, are not directly related to reimbursement, there is little incentive to report these codes at this time. Also, only conditions documented by physicians in patients' medical records are coded; conditions in pharmacist, nurse, or other health care professional notes are not coded.
In addition to possible incomplete recording of codes, other limitations include the inability to categorize the degree of harm to patients, capture all priority AEs, capture near misses, and provide reliable inter-institutional comparisons due to coding variations among facilities. Additionally, ICD-9-CM codes in discharge data do not distinguish reliably between AEs that occurred during hospitalization (in-hospital AEs) from AEs that occurred before hospitalization (arrived-with AEs).
6,
7
Currently codes in discharge data are analyzed after the patients' hospitalizations and can provide retrospective, but not real-time, information.
The findings of this paper and previous works suggest that no "gold standards" for AE detection and ICD-9-CM AE code classification currently exist. While some detection methods appear to be more promising than others, optimal tools depend on the user's purpose. The PS Indicators were designed with the primary purpose of helping health care facilities with internal patient care quality improvement. Because the PS Indicators are more specific and probably include fewer false positives, they provide a useful starting place, a subset of "high-yield" discharges for in-depth examination of AEs. The AE Classes were designed with the primary purpose of statewide surveillance of potential AEs and the secondary purpose of providing facility-level information that could help facilities with their own internal patient care quality improvement practices. Because the AE Classes are more sensitive, they detect a larger number of discharges with potential AEs, capturing more information at the State surveillance level, providing health care facilities more discharges to examine, and addressing the patient safety issues of the entire inpatient population.
Possible revisions of the AE ClassesDevelopment of improved AE detection tools based on ICD-9-CM codes still is in its early stages. Further improvements and changes are needed. How indicators or classes group codes is one design consideration. The PS Indicators group codes by AE topic, such as complications of anesthesia, whereas the AE Classes group codes by type of health care associated injury (or AE), such as adverse effects of anesthetics and other CNS agents. Each of these groupings has advantages and disadvantages. For example, the PS Indicator for complications of anesthetics tells the user to start with the patients, departments, personnel, procedures, policies, medications, and equipment involved with anesthesia; but this indicator does not tell the user whether the AE involved wrong position of the endotracheal tube, malfunction of anesthesia equipment, or the anesthesia medication. The AE Class for adverse effects of anesthesia and other CNS agents tells the user the kind of injury (or AE), but this particular AE Class includes drugs other than anesthetics. To examine adverse effects of anesthesia, apart from these other drugs, the user would need to access anesthesia codes only.
Possible revisions of the AE Classes could include changes in the numerator or denominator. The next development stage for the AE Classes includes refinement based on forthcoming medical chart review and other findings. These refinements may include dropping existing codes that identified few or no AEs compared to other similar codes in their AE Class, and adding new ICD-9-CM and ICD-10 codes that identified AEs in the chart reviews that are not already in the AE Classes. Currently, both the PS Indicators and the AE Classes exclude principal diagnosis in order to focus on in-hospital AEs in contrast to arrived-with AEs. However, inclusion of principal diagnosis could be useful if the intention is to detect AEs in a larger health care context than acute care hospitals only. For example, a patient may experience an AE at one facility, but the AE may be detected later during a subsequent hospitalization or doctor's appointment. Examination of principal diagnosis codes would capture at least some of these additional cases and improve the detection of AEs in general. Also, as mentioned earlier, the AE Classes denominators have fewer restrictions than the PS Indicators; the developers are considering additional restrictions based on forthcoming chart review findings.
Unit of analysis is another consideration. The PS Indicators detect the number of discharges by individual indicator. The AE Classes can detect the number and rate of discharges for top-level groupings of AEse.g., the most general category (any AE) or its subcategories (any ADE, any surgery-related AE)as well as at the individual AE class level (e.g., adverse effects of antibiotics and other anti-infectives). Other possible units of analysis are individual AE codes, specific AEs, and patient conditions. For example, the number and percentage for the code E876.5 (performance of inappropriate surgery) would be useful for learning more about wrong-site surgeries. The number and percentage of adverse effects of psychotropic medication for a particular patient or group of patients would provide useful information about whether a particular program of treatment for mental illness is effective, as would "patient condition," which could include readmission.
The findings in this paper are based on retrospective AE detection. As mentioned earlier, retrospective and real-time detection methods can detect different AEs; using them in combination could improve AE detection. Improvements in current methods, such as computer flag surveillance, and development of entirely new methods are possibilities. Advances in medical and health knowledge in current "gray areas," such as pulmonary embolism, may clarify the distinction between disease processes and AEs. New medical treatments, medications, and devices may expose patients to new health care-related risks, such as the recently approved drug eluting vascular stents.
21
Existing and new AE detection systems will need to adapt to both advances in medical knowledge about current gray areas as well as new potential risks to patients, and provide timely information for better understanding of AEs and how to prevent them.
This project is supported by grant number U18 HS 11885 from the Agency for Healthcare Research and Quality, the Utah and Missouri Patient Safety Consortium, and the Missouri Department of Health and Human Services Patient Safety Research Team.
All of the authors are affiliated with the Office of Health Care Statistics, Utah Department of Health.
Address correspondence to: Carol Masheter, Ph.D.; Office of Health Care Statistics, Utah Department of Health, P.O. Box 144004, Salt Lake City, UT 84114-4004. Phone: 801-538-6355; fax: 801-538-9916; e-mail: cmasheter@utah.gov.
1. Medical Malpractice Referral Network. Transplant victim dies
teens family treated poorly. February 22, 2003. Available from: http://www.medical-malpractice-lawyers-attorneys.com/ transplant_victim.html . 2. Kohn LT, Corrigan JM, Donaldson, MS, editors. To err is human: building a safer health system. A report of the Committee on Quality of Health Care in America, Institute of Medicine. Washington, DC: National Academy Press; 2000.
3. Committee on Quality of Health Care in America, Institute of Medicine. Crossing the quality chasm: a new health system for the 21 st century. Washington, DC: National Academy Press; 2001.
4. Marchev M, Rosenthal J, Booth M. How States report medical errors to the public: issues and barriers. Portland, ME: The National Academy for State Health Policy (NASHP); 2003.
5. Agency for Healthcare Research and Quality (AHRQ). AHRQ's patient safety initiative: building foundations, reducing risk. Interim report to the Senate Committee on Appropriations. AHRQ Publication No. 04-RG005. Rockville, MD: AHRQ; December 2003. Available from: http://www.ahrq.gov/qual/ pscongrpt .
6. Layde P. WMIRS Report Demonstration Project Report Summary, Patient Safety Reporting Research Demonstration Project, AHRQ Grant # 5U18HS77893. 2003.
7. Williams, SD. Comments on using ICD-9-CM and ICD-10-CM codes to identify medical errors or adverse events. Letter to the Institute of Medicine Committee on Data Standards for Patient Safety. January 8, 2003. Unpublished.
8. Samore M, Evans RS, Lassen A, et al. Surveillance of medical device-related hazards and adverse events in hospitalized patients. JAMA 2004. 291(3):325-70. (PubMed)
9. Xu W, Hougland P, Pickard S, et al. Detecting adverse drug events using ICD-9-CM codes. Poster presented at the AHRQ Second Annual Patient Safety Research Conference. 2003 March 2
4; Arlington, VA. 10. Florida Agency for Health Care Administration, State Center for Health Statistics. Health outcome series: adverse drug effects. AHCA; May 2000.
11. Utah Department of Health. Utah Patient Safety Updates. Vol. I, No. 1
5. Salt Lake City, UT: Utah Department of Health; 2002, 2003. Available from: http://health .utah.gov/psi/pubs/psup_v1n1.pdf . 12. Shah HG, Rolfs RT, Xu W, et al. Adverse events related to medical care, Utah: 1995-1999. 2001. Salt Lake City, UT: Utah Department of Health, Utah Health Data Committee, Center for Health Data, 2001.
13. Utah/Missouri Patient Safety Project. Utah/Missouri adverse event ICD-9-CM classification. 2002 Version 1. Salt Lake City, UT: Utah Department of Health; 2002. Available from: http://health.utah.gov/psi .
14. Agency for Healthcare Research and Quality. AHRQ Quality indicators
guide to patient safety indicators. Version 2.1, Revision 1. AHRQ Publication No. 03-R203. Rockville, MD: Agency for Healthcare Research and Quality; 2003. 15. Zhan C, Miller RM. Excess length of stay, charges, and mortality attributable to medical injuries during hospitalization. JAMA 2003. 290(14):1868-74. (PubMed)
16. Romano PS, Geppert JJ, Davies S, et al. A national profile of patient safety in U.S. hospitals. Health Aff 2003. 22(2):154-66.
17. Utah Hospital Discharge Database. Salt Lake City, UT; Utah Health Data Committee, Utah Department of Health 2000, 2001, 2002.
18. Leape LL, Brennan TA, Laird N, et al. The nature of adverse events in hospitalized patients: results from the Harvard Medical Practice Study II. N Engl J Med 1991. 324:377-84. (PubMed)
19. Division of Administrative Rules. Rule R380-210. Health care facility patient safety program. State of Utah. Available from: http://www.rules.utah.gov/publicat/code/ r380/r380-210.htm .
20. Division of Administrative Rules. Rule R380-200. Patient safety sentinel event reporting. State of Utah. Available from: http://www.rules.utah.gov/publicat/code/ r380/r380-200.htm .
21. U.S. Food and Drug Administration, Center for Devices and Radiological Health. FDA Public health notification: updated information for physicians on sub-acute thromboses (SAT) and hypersensitivity reactions with use of the Cordis CYPHER TM Sirolimus-eluting coronary stent. FDA; 2003 Nov 25. Available from: http://www.fda.gov/cdrh/safety/cypher.html .