Advances in Patient Safety: From Research to Implementation Volume 3. Implementation IssuesThe Impact of a Web-based Reporting System on the Collection of Medication Error Occurrence Data
William J. Rudman, Jessica H. Bailey, Carol Hope, Paula Garrett, C. Andrew Brown
AbstractThis paper examines the impact of an Internet-based method on the collection of medication error occurrence reports. Data for this study was collected for two time periods. The first data collection effort used a paper-based method and produced our baseline data (January 1994 to December 2000). A second dataset was created with information collected from January to December 2003, using a Web-based software application. Four priority areas were developed from the baseline data: (1) increase the overall number of reported medication errors; (2) increase the number of intercepted medication errors; (3) increase the documented number of physician-attributed medication errors; and (4) improve data quality and specificity. Findings from the Web-based method of collecting data show significant improvements across all four areas. The number of documented medication errors increased from an average of 414 between 1994 and 2000, to 959 in 2003. In this same time period, intercepted errors increased from an average of 17.3 percent to 58.2 percent, and physician-attributed errors increased from an average of 4.8 percent to 27.2 percent. Finally, the missing or unspecified data from the cause-of-error variable decreased from 18.6 percent to 2.1 percent. The electronic system's deliberate anonymity and ease of use, coupled with its ability to expedite reported medication error investigations and the educational efforts directed at the creation of an open reporting environment, are the most likely explanations for the positive impact of the Web-based reporting system.
Medication error (ME) is a significant problem within our health care system, in terms of patient harm and cost. In July 2002, the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) cited the need to reduce medication errors as a top priority. Several studies suggest that medical error is the third-leading cause of death in the United States, closely trailing heart disease and cancer.
1,
2
In fact, at least 7,000 inpatient deaths occur annually as a direct result of medication errors in hospitals and 106,000 deaths occur each year, due to adverse effects of medications.
1,
2
Kohn reported that 44,000 to 98,000 Americans die annually as a result of medication errors.
2
It has been estimated that medication-related problems cost the U.S. health care system $132
$182 billion annually, at a price of approximately $1.5 million per hospital.
3,
4
It has been further estimated that the cost per adverse drug event (ADE) ranges between $2,300 and $4,685.
5,
6
As greater attention is given to medication errors and the need to reduce their incidence, technology often is seen as an important tool for addressing the problem. In the Institute of Medicine's 2001 report, Crossing the Quality Chasm, emphasis is placed on the important role that information technology may play in the creation of a reporting system that will effectively reduce medication errors. 7 Technologies that can detect adverse drug events and medication errors in a swift and cost effective manner are becoming increasingly available to health care institutions. 8 Technological innovations that may be used for the detection and reduction of medication errors continue to improve patient safety in health care. 9 Information technology, such as computerized medication records, bar coding, robots, and computerized prescriber order entry, are likely to further reduce medication errors. 10, 11 In a survey conducted by the Healthcare Information Systems Society, health care industry leaders ranked the implementation of patient safety information technology as their number one priority for the year 2003. 12 Health care institutions traditionally restricted the use of technology to quality assurance efforts, as the potential for using information technology as a cost-efficient means of addressing medication error problems is only now being realized. 13, 14
Web-based reporting systems are one example of how information technology may be used to improve patient safety and quality of care initiatives, while at the same time improving operational efficiency. 15, 16 Automated databases and their ease of use have brought about drug safety studies that might otherwise be impossible to carry out. 17 Web-based technologies have made anonymous, automated, and integrated reporting systems a reality. Moreover, the use of Web-based technologies ensures reporting consistency and data quality, as well as the ability to survey and detect adverse events. 18, 19 This, in turn, provides data that may be measured quantitatively and, analyzed for trends, then used to generate feedback for the quality assurance process. 18 Thus, technology has the potential to improve workflow and the fluid transmission of vital information on errors that surface from a system perspective, without placing blame on fallible individuals and institutions. 20 Safety may be improved in an ongoing manner with the use of technology, particularly as the effects of systemic changes are being assessed. 21 The use of information technology can reduce errors in three ways: (1) adverse events and medication errors may be prevented before they can occur; (2) the response time needed to resolve the root cause of adverse events can be decreased, thus preventing reoccurrence; and (3) trends can be tracked and pertinent feedback about medication errors and adverse drug events then can be disseminated. 22
Technology for reporting medication errors has evolved over the years from a card with the name of the suspected drugs handwritten on it and dropped into a box,
23
to sophisticated electronic systems with computerized order entry.
24
These advances have improved the quality of patient care and safety.
25
Web-based reporting allows data to be collected from any Internet-accessible computer terminal, anywhere in a medical facility. Practitioners, pharmacists, nurses, and other staff can enter data into the system from multiple locations, while an analyst tracks trends and errors from a central location. The reports generated from a Web-based system enable hospitals to better track and categorize medication errors that can be further analyzed to bring about changes in the health care system necessary to eliminating critical errors and improving patient care. Web-based databases have the ability to store lab test results and medication orders in one place; they can be used to organize information on possible drug reactions and their prevention; and they can capture patient care data in a timely manner, making them a valuable tool in the effort to reduce medication errors of all types.
26
28
In addition, Web-based systems allow individuals to anonymously report errors and the circumstances contributing to them.
20
This frees the health care provider making the report from the fear of embarrassment or liability. Stump reported a five-fold increase in the number of captured error events, once the concept of anonymity and an online system were accepted in his facility.
29
Academic medical centers across the Nation have used Web-based reporting systems in a variety of ways as an integral component of their patient safety initiatives. In 2002, Kivlahan and colleagues 30 developed an electronic reporting system for the University of Missouri Medical Center. Initially, all staff, physicians, patients, visitors, and families were allowed to enter data. It became apparent, however, there was a need to limit family data entry, so steps were taken to have nurses assist with the entry of family generated data. The staff of different departments collaborated, which eliminated redundancies and led to greater knowledge sharing. Since the Web-based system was more secure than the paper-based system, the staff were more likely to report mistakes.
The New York Patient Occurrence Reporting and Tracking System (NYPORTS) was created to comply with mandatory State error reporting legislation. NYPORTS identified the root causes of errors, examined possible risk reduction strategies, and provided the ability for improved measurement of intervention effectiveness. Across the various regions of New York, the number of reported medication errors (MEs) increased steadily from 1999 through 2001. By 2002, reports indicated an average of 29 medication errors per 100 discharges. 31 Although the number of reports increased dramatically, the volume of harm associated with the reported events was small in comparison with the total volume of errors. 32
Vanderbilt Medical Center has built a Web-based system for reporting pediatric chemotherapy incidents. 33 The system includes wireless, handheld reporting tools. Of the 10,000 prescriptions per day, 300 to 500 had advisories warning of common prescribing errors. This system resulted in a $5 million annual decrease in pharmacy costs.
When the Baylor University Medical Center transitioned from paper-based to Web-based reporting, the leadership wanted to capture a broad range of events that included anything "unusual, unexpected, or unanticipated."
34
The system was a Web-based arrangement that permitted online patient report submissions. It was unique and of particular interest in that it was linked to, and provided event reporting services for, multiple sites.
35
After the Web-based system was fully implemented, the number of ADEs decreased more than 250 percent and the cost of data collection decreased by approximately $30,000 annually. Finally, the time required to investigate medication errors decreased 2550 percent.
36
Data collection
The data for this paper was obtained from inpatient medication error occurrence reports during two specific time periods. One dataset was created from a paper-based system, using reports generated between January 1994 and December 2000 (n = 2,965). The second dataset was created with the aid of an anonymous, Web-based medication error reporting system, using occurrence reports generated between January 2003 and December 2003 (n = 959).
Data from the initial collection period (i.e., that generated by the paper-based system) were used to establish baseline criteria for measuring success and for identifying system needs and future action. Data from the subsequent collection period (i.e., that from the Web-based system) were used to assess whether needs identified in the initial data collection period were met. We did not include data in our analysis from medication error occurrence reports collected in 2001 and 2002. Although the total number of medication errors identified in the 2001 and 2002 occurrence reports were consistent with the volume of reports generated by the paper-based system (337 and 394 per year, respectively), we had experimented with several methods for collecting medication error occurrence reports during this two-year period. The method of data collection used in conjunction with the medication error reports generated by the Web-based system was first tested in the Health Information Management (HIM) lab at the University of Mississippi Medical Center, before its adoption in December 2002. Forty medication errors were reported during the initial month of data collection, using the Web-based form.
Prior to December 2002, occurrence reports were collected using a variety of traditional paper-based systems. At that time, we introduced a Web-based system for collecting occurrence reports associated with the University of Mississippi Medical Center (UMMC) inpatient population. A medication error occurrence report icon was placed on all personal computers (PCs) within the UMMC system of care. Health care providers were given the option of filing the Web-based occurrence reports either from work stations on the various floors of the medical center, or from private offices, to better facilitate anonymous medication error reporting. An electronic form would appear on the screen of the PC when the clinician clicked the mouse on the appropriate icon, with drop-down menus to ensure response accuracy and consistency, reducing the time needed to report a medication error.
Unlike most Web-based reporting systems, which are self-contained and do not interface with other information systems within the health care system, the occurrence report Web page for the medical center was interfaced with the Softmed Clin TracTM data system that contains clinical, financial, and demographic patient data. As a result, staff were better able to eliminate duplicate and non-patient reporting from the system. To ensure accuracy, the patient name, medical record number, and billing number entered by the individual making the medication error report had to match the system data on file, or the report could not be filed.
From a functional perspective, data entered from the electronic form into the occurrence report system is sent in real time to the risk management department. The risk manager begins the medication error report investigation by sending details of the event to the appropriate floor and health care providers involved, using the medical center's e-mail system. A nurse manager investigates the report, then returns it
along with any changes and/or clarificationsto risk management via e-mail for final analysis. The risk manager enters the report details into the Clin Trac system, making the data available for inhouse reports (e.g., the Patient Safety Council report) or for further analysis related to Quality Improvement (QI) reports and initiatives. Automated time logs are included at each stage of the process to improve accountability and establish work standards for the processing of medication error occurrence reports.
Paper-based format: identified areas of improvement
Findings from the initial data collection period, derived from the paper-based system, provided a baseline against which the impact of the Web-based reporting system could be measured. We identified from our initial analysis of the paper-based forms four priority areas. First, the data suggested the overall number of reported medication errors was low and needed to be increased to accurately represent the actual number of incidents. A second problem area centered on the low number of intercepted medication errors that were captured using the paper-based occurrence reporting system. The third target goal involved the documentation procedures for physician-attributed medication errors. The fourth and final priority was improving the quality of reporting data and medication error reporting specificity.
Priority area 1: increase the number of reported medication errorsThe average monthly number of medication error occurrence reports was 34.5, based on data collected from the paper-based system (Figure 1); this resulted in a medication error rate of 1.62 errors per 100 admissions (Table 1). Reported errors increased to an average of 79.9 per month, using the Web-based electronic form. This represents an average annual increase of 545 occurrence reports, using the Web-based form. Moreover, the number of reported errors was higher for every reporting month of the Web-based data collection period (Figure 1). Finally, the rate of errors per 100 admissions increased from 1.62 under the paper-based format to 3.62 using the Web-based electronic form.
| Year | Annual admission | Total medication errors | Medication error rate / 100 admissions | ADE /100 admissions | Intercepted errors /100 admissions | % Error intercepted |
| Paper | ||||||
| Average of 1994-2000 | 25,713 | 416 | 1.62 | 0.46 | 0.30 | 17.3 |
| Web | ||||||
| 2003 | 26,490 | 959 | 3.62 | 0.39 | 2.01 | 58.2 |
In addition to increasing the overall number of intercepted medication errors, our goal was to increase the number of intercepted error threats, categorized by documented health care provider (e.g., nurse or doctor), and by the type of error. For the purposes of this study, intercepted error threats were those that did not reach the patient, while harm is defined as a range from increased monitoring to death. Prior to the implementation of the Web-based electronic form, 17.3 percent of the medication errors were intercepted. This resulted in an intercept rate of 0.30 per 100 admissions (Table 1). After implementing the Web-based form, the number of intercepted medication errors increased to 58.2 percent with an interception rate of 2.01 per 100 admissions. In addition to an overall increase in the number of intercepted medication errors, there also was an increase in intercepted nursing medication errors (15.9 to 22.3 percent), intercepted physician medication errors (28.6 to 88.9 percent), intercepted pharmacist medication errors (30.0 to 65.9 percent), and intercepted staff medication errors (19.0 to 52.0 percent) (Table 2). The percentage of intercepted errors increased across all categories related to medication error type, with the largest increases for prescribing errors (2.1 to 14.8 percent) and improper dose errors (13.4 to 30.3 percent) (Table 3).
Table 2. Person documented in the occurrence report as responsible for the medication error by severity of outcome| Severity of Outcome | ||||||||
| No.(%) Intercepted | No.(%) No Harm | No.(%) Harm | No.(%) Total | |||||
| Paper | Web | Paper | Web | Paper | Web | Paper | Web | |
| (1994-2000) | (2003) | (1994-2000) | (2003) | (1994-2000) | (2003) | (1994-2000) | (2003) | |
| Nurse | 15.9% (367) | 22.3% (67) | 55.6% (1280) | 55.3% (166) | 28.5% (656) | 22.3% (67) | 78.3% (2303) | 31.3% (300) |
| Pharmacist | 30.0% (125) | 65.9% (245) | 45.6% (190) | 28.8% (107) | 24.5% (102) | 5.4% (20) | 14.2% (417) | 38.8% (372) |
| Physician | 28.6% (40) | 88.9% (232) | 44.3% (62) | 6.1% (16) | 27.1% (38) | 5.0% (13) | 4.8% (140) | 27.2% (261) |
| Staff/Other | 19.0% (15) | 52.0% (12) | 43.0% (34) | 26.1% (6) | 38.0% (30) | 19.2% (5) | 2.7% (79) | 2.4% (23) |
| Missing | 0.1% (2) | 0.3% (3) | ||||||
| Severity of Outcome | ||
| No.(%) Total | ||
| Paper | Web | |
| Extra dose | 4.8%(140) | 4.6%(44) |
| Improper dose | 13.4%(392) | 30.3%(290) |
| Omission | 24.0%(702) | 16.6%(159) |
| Prescribing | 2.1%(62) | 14.8%(141) |
| Wrong administration technique | 5.2%(153) | 1.2%(11) |
| Wrong dose form | 0.0%(1) | 2.0%(19) |
| Wrong drug preparation | 1.9%(54) | 3.4%(33) |
| Wrong patient | 6.6%(192) | 5.2%(50) |
| Wrong route | 1.7%(49) | 2.0%(19) |
| Wrong time | 9.0%(262) | 6.3%(60) |
| Unauthorized drug | 12.7%(371) | 11.5%(110) |
| Other | 7.0%(204) | 0.0%(0) |
| Missing data | 11.6%(338) | 2.1%(20) |
Table 2 shows a significant shift in medication error culpability, or "blame." Data from the paper-based collection period indicates 92.5 percent of all reported medication errors were attributed to nurses or pharmacists; and, of the total number of medication errors reported, only 4.8 percent were attributed to physicians (Table 2). By contrast, medication errors attributed to physicians increased to 27.2 percent of the total volume, following implementation of the Web-based electronic form, while the combined total of errors attributed to nurses and pharmacists dropped to 70.1 percent of the documented totals. The number of medication errors attributed to the staff remained consistent at 2.7 to 2.4 percent overall. Interestingly, the average number of medication errors attributed to nurses decreased from 329 to 300 per year, following the implementation of the Web-based system, while the average number of pharmacist-attributed errors increased from 59.5 per year to 372 per year. The documented number of physician-attributable medication errors for the 1-year period of Web-based data collection (261) exceeded the combined total of the previous 7 years (140), gathered using the paper-based data collection method.
A primary concern arising from our baseline analysis was the number of medication errors that were reported either as missing or "other." Seven percent of the reported errors from the paper-based system had been placed in the "other" category, and 11.6 percent of the data was missing. No errors in the dataset generated from the Web-based system had been placed in the "other" category, and just 2.1 percent of the medication errors were categorized as missing.
Findings from our study supported prior research concerning the positive impact of a Web-based medication error occurrence data collection implementation. As noted previously, our study found an overall increase in reported medication errors, as well as increases in intercepted medication errors and physician-attributed medication errors. Moreover, the reporting specificity improved and the volume of missing data decreased. After changing from a paper-based to a Web-based system of collecting occurrence reports, the medication error reporting rate increased by 130 percent. This compares with other research, such as the Baylor study, where the reporting rate increased from 79.5 to 83.5 percent, and the Missouri study, where the ME occurrence rate increased from 1.3 to 1.5 per 100 patient days.
30,
34,
37
In addition, the intercepted medication errors described in our study increased more than three-fold following the Web-based system implementationand was consistent across all three hospital staff groups (nurses, physicians, and pharmacists). In addition to the increased reporting and intercepted errors patterns, these findings generated insight into the myriad ways Web-based systems may affect reporting patterns among various health care providers, and physicians in particular. The data we collected from the paper-based system were consistent with prior research that documented the nurse as the care provider most responsible for medication errors. Similar to findings from other studies in which the volume of nurse-attributed errors ranged from 60.3 to 72 percent and physician-attributed errors ranged from 2.6 to 9 percent when a paper-based form was used, the present study showed 78.3 percent of the errors were attributed to nurses and only 4.5 percent to the physician. After implementation of a Web-based system, however, 31.3 percent of the errors were found to be linked to nurses, while 27.2 percent were attributed to physicians. Finally, the data from our study showed the clinical care staff was willing not only to report medication errors, but to complete the occurrence report form as well. The volume of missing cause-of-error data decreased from more than 11 percent to approximately 2 percent, following the Web-based system implementation.
Whenever a system change of this magnitude occurs, it is often difficult to pinpoint the exact source of change. We introduced a Web-based method of collecting medication error occurrence reports, but external to our efforts, the subject of medication errors has become an important topic of discussion in the popular and academic press, as well as in the accreditation field. From our informal work on patient safety committees within the hospital and education efforts organized by the patient safety center and hospital leadership, there are three plausible explanations for the success of the Web-based system initiative. First, there is an anonymity and ease of use associated with the Web-based system that cannot be denied. Those reporting errors could do so from PCs located throughout the hospital, or from the privacy of their office. Second, the Web-based system reduces the time involved in the investigation of adverse drug events, while at the same time improving individual accountability. Medication error reporting on the Web-based system is done in real time, so the time needed to initiate an ADE investigation (i.e., the interval from the occurrence of the error to the time the event report reached Risk Management) was reduced from 34 weeks to less than 1 day. This was similar to a significant decrease in reporting time described in the Baylor study.
34,
36
Furthermore, the Web-based system provided an audit log for systematically tracking the time devoted to an ADE medication error investigation. Finally, since the Web-based error reporting system was integrated with the clinical and financial data processing systems, the risk manager did not have to collect paper records for use in an ADE investigation. As a result, those individuals who reported errors could see in the data record that immediate action was being taken on their behalf. Educational efforts on the part of the patient safety center and hospital leadership stressed the importance of error reporting, for the insight it provides into the processes and various factors that contribute to errors. The medical center and hospital administration, in conjunction with grant efforts, used articles in the medical center newspaper to build interest and trust in the Web-based reporting system, and the hospital's CEO became chair of the Patient Safety Council. This further reinforced the importance of medication error detection within the medical center community and advanced the hospital's movement toward patient safety-driven culture focused on the systemic, rather than individual, correction of medication errors.
We would like to thank Bill Pierce for helping in the research process. This paper was supported by AHRQ grant number 1 U18 HS22923-01.
Department of HIM, University of Mississippi Medical Center, Jackson, MS (WJR, JHB), Department of General Medicine, University of Mississippi Medical Center, Jackson, MS (CAB, CH).
Address correspondence to: William J. Rudman, Ph.D., University of Mississippi Medical Center, 2500 N. State Street, Dept. HIM, Jackson MS 39216; phone: 601-984-6346; e-mail: Brudman@shrp.umsmed.edu.
1. Starfield B. Is U.S. health really the best in the world. JAMA 2000. 284(7):483-5. (PubMed)
2. Kohn LT, Corrigan JM, Donaldson MS, editors. To err is human: building a safer health system. A report of the Committee on Quality of Health Care in America, Institute of Medicine. Washington, DC: National Academy Press; 2000.
3. Johnson JA, Bootman JL. Drug related morbidity and mortality: a cost of illness model. Arch Intern Med 1995. 155:1949-56. (PubMed)
4. Schneider PJ, Gift MG, Lee YP, et al. Cost of medication related problems at a university hospital. Am J Health Syst Pharm 1995. 52:2415-18. (PubMed)
5. Classen DC, Pestotnik SL, Evans RS, et al. Adverse drug events in hospitalized patients: excess length of stay, extra cost, and attributable mortality. JAMA 1997. 227(4):301-6.
6. Bates DW, Spell N, Cullen DJ, et al. The cost of adverse drug events in hospitalized patients. JAMA 1997. 227(4):307-12.
7. Crossing the quality chasm: A new health system for the 21st Century. Institute of Medicine. Washington, DC: National Academy Press; 2001.
8. Bates DW, Evans S, Murff H, et al. Detecting adverse events using information technology. JAMIA 2003c. 10(2):115-28. (PubMed) (Full Text in PMC)
9. Shea B, Belmont C, Williams R. Communication, standards, and technology. Health Forum J 2002;May-June:34
5. 10. Kaushal R, Barker KN, Bates DW. How can information technology improve patient safety and reduce medication errors in children's health care? Arch Pediatr Adolesc Med 2001. 155:1002-7. (PubMed)
11. Weitzman M. Information technology and the future of child health care. Arch Pediatr Adolesc Med 2001. 155:990-1. (PubMed)
12. Johnson VR, Hummel J, Kinninger T, et al. Immediate steps towards patient safety. Healthc Finance Manage 2004. 58(2):56-61. February.
13. Bates DW. Using information technology to screen for adverse drug events. Am J Health-Sys Pharm 2002. 59:2317-19.
14. Anderson JG, Jay SJ, Anderson M, et al. Evaluating the capability of information technology to prevent adverse drug events: a computer simulation approach. JAMIA 2002. 9(5):479-90. (PubMed) (Full Text in PMC)
15. Ball MJ, Douglas JV. IT, patient safety, and quality care. J Healthc Inf Manag 2002. 16(1):28-33.
16. Matthews P. Leveraging technology for success. J Healthc Inf Manag 2000. 14(2):5-12.
17. Jick H. A database worth saving. Lancet 1997. 350(9084):1045-6. (PubMed)
18. Memel DS, Scott JP, Mcmillan DR, et al. Development and implementation of an information management and information technology strategy for improving healthcare services: a case study. J Healthc Inf Manag 2001. 15(3):261-85. (PubMed)
19. Walley T, Mantgani A. The UK General Practice Research Database. Lancet 1997. 350(9084):1097-9. (PubMed)
20. Jackson K. Shedding light on medication errors. For the Record 2004. 16(4):27-9.
21. Bates DW, Evans S, Murff H, et al. Policy and the future of adverse event detection using information technology. JAMIA 2003b. 10:226-8. (PubMed) (Full Text in PMC)
22. Bates DW, Gawande AA. Patient safety with information technology. N Engl J Med 2003. 348(25):2526-34. (PubMed)
23. Finn B, Carlstead BC. Reporting adverse drug reactions in ambulatory care setting. Am J Health Syst Pharm 1995. 52(23):2704-6. (PubMed)
24. Jha AK, Kuperman GJ, Rittenberg E, et al. Identifying hospital admissions due to adverse drug events using a computer-based monitor. Pharmacoepidemiol Drug Saf 2001. 10(2):113-9. (PubMed)
25. Snoots WM. Information technology and the medical profession: a curse or an opportunity? BUMC Proceedings 2002. 15:138-40. (PubMed) (Full Text in PMC)
26. Kelly JJ, Sweigard KW, Shields K, et al. Safety, effectiveness, and efficiency: a Web-based virtual anticoagulation clinic. Jt Comm J Qual Saf 2003. 29(12):646-51. (PubMed)
27. Batsakes PJ, Hancock HE, RogersWA , et al. A medication screening tool for cognitive aging researchers. Psychol Aging 2002. 17(1):169-73. (PubMed)
28. Dixon JF, Wielgosz C, Pires ML. Description and outcomes of a custom Web-based patient occurrence reporting system developed for Baylor University Center and other system entities. BUMC Proceedings 2002. 15:199-202. (PubMed) (Full Text in PMC)
29. Stump LS. Re-engineering the medication error-reporting process: removing the blame and improving the system. Am J Health Syst Pharm 2000. 57(Suppl 4):S10-7.
30. Kivlahan C, Sangster W, Nelson K, et al. Developing a comprehensive electronic adverse event reporting system in an academic health center. Jt Comm J Qual Improv 2002. 28(11):583-94. (PubMed)
31. The New York Patients Reporting and Tracking System 2000/2001. Albany: New York State Department of Health; September 2003.
32. Tuttle D, Panzer RJ, Baird T. Using administrative data to improve compliance with mandatory state event reporting. Jt Comm J Qual Improv 2002. 28(6):349-58. (PubMed)
33. France DJ, Miles P, Cartwright J, et al. A chemotherapy incident reporting and improvement system. Jt Comm J on Qual and Safety 2003. 23(4):171-80.
34. Dixon JF. Going paperless with custom-built Web-based patient occurrence reporting. Jt Comm J Qual Improv 2002. 28(7):387-95. (PubMed)
35. Joshi MS, Anderson JF, Marwaha S. A systems approach to improving error reporting. J Healthc Inf Manag 2002. 16(1):40-5.
36. Raju TN, Kecskes S, Thornton JP, et al. Medication errors in neonatal and pediatric intensive-care units. Lancet 1989. 2(8659):374-6. (PubMed)
37. Atherton T. Description and outcomes of the DoctorQuality incident reporting system used at Baylor Medical Center at Grapevine. BUMC Proceedings 2002. 15:203-8. (PubMed) (Full Text in PMC)