Advances in Patient Safety: From Research to Implementation Volume 2. Concepts and MethodologyPreparing for Ambulatory Computerized Prescriber Order Entry by Evaluating Preimplementation Medication Errors
Emily Beth Devine, Jennifer L. Wilson-Norton, Nathan M. Lawless, Thomas K. Hazlet, Ryan Hansen, Kerry Kelly, Stephanie Te, Carolyn Wong
AbstractThe objective of this report is to describe the methods used in and lessons learned from a research study that evaluated prescriptions for medication errors prior to the implementation of an ambulatory computerized prescriber order entry (ACPOE) system in a community-based, integrated health system. Several aspects of the study are described: practice setting, research team, project scope and study design, data elements, database creation, data sources, study methods, evaluation methods, refinements, and current project status. Lessons learned are summarized. Focused attention to these aspects, a priori, resulted in the collection of data that have been used to characterize the epidemiology of medication errors at baseline. By repeating these same methods post-ACPOE implementation, the impact of the ACPOE system on medication errors will be determined. The methods used to measure the baseline rate of medication errors provided useful results and proved practical in a practitioner-based organization. This design can be replicated in similar organizations desiring to evaluate the impact of an ACPOE system on medication errors.
During the millennium period, the issue of patient safety was elevated to national focus, based largely on the publications of the Institute of Medicine.
1
3
To proactively address these patient safety concerns, in 2002, the leadership of the Everett Clinic, Snohomish County, Washington, began adding an ambulatory computerized prescriber order entry (ACPOE) system to their existing computerized medical record (CMR). As a priority, the clinic leadership initiated this study to evaluate the impact of the ACPOE system on patient safety. The purpose of this study was twofold: (1) to measure the baseline incidence of medication errors prior to implementation of an ACPOE system, and (2) to characterize the epidemiology of these errors. This report describes the methods used to conduct the preimplementation evaluation in one internal medicine clinic selected as the pilot site. Addressed are the developments of research methods; decisions made to streamline the methods; and lessons learned in addressing the challenges inherent in measuring the baseline rate of medication errors in a practitioner-based, ambulatory care setting.
The Everett Clinic is an integrated, multispecialty, physician group practice that provides comprehensive ambulatory care services for 275,000 residents in the northern Puget Sound region of Washington State. The clinic espouses three core values: (1) We do what is right for each patient; (2) we provide an enriching and supportive work environment; and (3) our team focuses on value, service, quality, and cost. An integral part of this value system is the promotion of a culture of safety.
The clinic is both owned and managed by local physicians in the community. The clinic health care team includes 225 physicians working collectively with 600 other health care professionals. Care is delivered in 13 locations, in 60 clinics, throughout Snohomish County, Washington. The care delivery system includes primary care and specialty clinics, walk-in clinics, two outpatient surgery centers, a cancer center, comprehensive laboratory, and imaging services. A hospitalist team functions 24 hours daily, 7 days weekly, and admits to the local community hospital. The clinic physicians write more than 2.1 million prescriptions each year, amounting to approximately $136 million dollars. Ten percent of these prescriptions are filled at three onsite pharmacies owned and operated by the clinic. The clinic has estimated the ACPOE system will save $1.8 million dollars annually in prescription processing costs (2002 data).
The clinic has a strong medication management team that manages the formularies of the 12 health plans the clinic contracts with. Two clinical pharmacists (JWN and NML) optimize the quality of medication use and provide cost management by collaboratively working with the physicians. 4 Together, the physicians and pharmacists lead the Pharmacy and Therapeutics Committee and are members of the Quality Improvement Committee. These two committees oversee the patient safety aspects of the ACPOE implementation.
The information technology (IT) infrastructure at the clinic is provided by employees of a wholly-owned subsidiary, Clinitech© Information Services. Clinitech employees have worked collaboratively with the clinical and administrative staff since 1995 to develop the clinic's CMR. 5 At present, the CMR incorporates physician dictation (chart notes summarizing each ambulatory care visit), laboratory values, and radiology reports. The ACPOE system is the next module being implemented. The CMR also serves as the backbone for future development of clinical decision support (CDS) tools for clinicians to use throughout the health care system. These CDS tools will largely be presented as programmed enhancements of the ACPOE system. The Clinitech team has also developed and maintained an extensive intranet for physician and staff use, which helps the latter easily access clinical and administrative information geared to helping them provide patient care.
The ACPOE system is Web-based and involves point and click functionality. It integrates patient scheduling, chart notes, laboratory values, radiology, and prescribing into one system. The ACPOE software module was built by the Clinitech team de novo, not purchased from an outside vendor. A discussion of software development and the selection of portable, wireless order entry devices will be provided in a separate publication. When a prescription is written electronically, it is saved on the clinic mainframe computer and can be printed and handed to the patient, or it can be automatically faxed to a retail pharmacy. The clinic currently auto faxes to more than 100 pharmacies. The prescribing software does not currently allow for full electronic transfer to retail pharmacy order entry software. Therefore, the printed prescription is not eliminated, and the retail pharmacy maintains this printed order as part of the patient record.
Establishing the research team
The clinic leadership partnered with investigators from the University of Washington Pharmaceutical Outcomes Research and Policy Program to evaluate the impact of the ACPOE system on medication errors. From mid-2002 through early 2003, this research team developed the plan and conducted the research that evaluated medication errors in one internal medicine clinic, prior to implementing the ACPOE system in July 2003. The team consisted of four core investigators (JWN, NML, EBD, TKH), three clinical pharmacists who served as research associates in evaluating the prescriptions for errors (KK, ST, CW), and a database programmer who was also a clinically trained pharmacist (RH). One of the research associates was a specialist in geriatrics (KK) and one in primary care (CW). Physician leaders and administrators of the clinic provided oversight. For 7 months the team met weekly, in person. All decisions related to study design were captured in written meeting minutes and in an electronic e-mail file. When the study was launched, communication took place almost daily. At that juncture, efficiencies were gained by managing communications via teleconferences; e-mail was also extensively used.
The first step in defining the scope of the study was to diagram the prescription-flow process in the ambulatory care setting at the clinic (Figure 1). This exercise validated the notion that the process of prescribing in the ambulatory care setting is complex, and that there are numerous points in that process where errors can occur.
The second step was to decide whether to adopt a concurrent or retrospective method for data collection. The team considered using a concurrent, observational study design, following prescriptions as they made their way through the clinic, from the point of prescriber writing to the point of dispensing at the pharmacy. Barker and Flynn have recommended this approach for conducting drug dispensing and administration studies. 6, 7 However, the goals of the present study were to identify errors occurring solely during the prescribing process, which could be measured more efficiently by using a retrospective design. This provided a method for identification and characterization of errors, albeit without observing the specific point in the prescription process-flow where each error occurred.
The third step was to decide which pharmacies to include in the study. Using pharmacies onsite at the clinic would facilitate capture of errors inclusive of the point at which they were entered into the pharmacy computer system. Using pharmacies external to the clinic would preclude information obtained at this point. As the potential exists for electronically written prescriptions to be electronically transmitted directly into the pharmacy computer system (eliminating the faxing step), it seemed prudent to include this point in the measurement process. Thus, study methods included prescriptions transmitted to and filled solely at a pharmacy owned and operated by the clinic. The identification and characterization of medication errors occurring in the dispensing process was outside the scope of this study, as the ACPOE system would not impact this activity.
The fourth step was to decide which clinic should be first brought online with the ACPOE system. An internal medicine clinic, where prescribers were eager to prescribe electronically, was selected as the pilot site. The research team reasoned that selecting this clinic would maximize the chances of successful implementation not only within the pilot clinic, but that the spread of this positive news would facilitate implementation at the additional 59 clinics that comprise the Everett Clinic integrated health system.
The research team conducted a comprehensive review of the literature
8
16
and used this information to draft the data elements. Many of the data elements used by previous investigators were selected for their use in the inpatient setting;
8
10,
12,
14
16
not all of them were applicable to the ambulatory setting. The team selected those that were applicable, including error characteristics that specified type of error and underlying cause of error, as well as patient-, prescriber-, and prescription-level characteristics. The availability of the CMR chart notes and laboratory values made it possible to evaluate clinical errors, as well (Table 1). Data were recorded on every prescription evaluated, regardless of whether an error was identified.
Error characteristics type of error
|
| Illegible prescription |
| Information missing |
| Inappropriate abbreviations (drug name, apothecary system, leading/trailing zeros, use of "u" for units) |
| Wrong information (wrong patient, wrong drug, wrong route, wrong dosage form, wrong dose, wrong strength, wrong directions) |
| Clinical criteria - patient allergy |
| Clinical criteria - drug-drug interaction |
| Clinical criteria - drug-disease interaction |
| Clinical criteria - therapeutic duplication |
| Clinical criteria - Contraindication - pregnancy, geriatric criteria, Laboratory Harm Table |
| Clinical criteria - multiple prescribers prescribing the same drug or drug class |
|
Error characteristicsunderlying cause of error
|
| Drug name confusion |
| Lack of standardization of prescription processing rules |
| Lack of knowledge of drug or lack of drug information by/for prescriber |
| Lack of information about the specific patient (missing lab values, disease states, interactions) |
| Staffing problems |
| Unable to determine |
| Patient level |
| Patient year of birth |
| Patient gender |
| Prescriber level |
| Prescriber specialty |
| Prescriber year of birth |
| Prescriber years in practice |
| Prescription level |
| Method of prescription transmittal |
| Error detected and corrected during the routine course of the prescribing process |
| Formulary-therapeutic interchange required |
| Formulary-prior authorization required |
| Prescription data elements |
| Name, dose and directions for prescription (both as written and as should have been written) Therapeutic drug class |
| NCC MERP Risk Assessment Index - Severity Levels A through I |
|
NCC MERP = National Coordinating Council on Medical Error Reporting and Prevention |
To conduct a risk assessment on each error identified, the definition of error and the Risk Assessment Index published by the National Coordinating Council on Medication Error Reporting and Prevention (NCC MERP) 17 was adopted. According to NCC MERP, "A medication error is any preventable event that may cause or lead to inappropriate medication use or patient harm while the medication is in the control of the health care professional, patient, or consumer. Such events may be related to professional practice, health care products, procedures, and systems, including prescribing; order communication; product labeling, packaging, and nomenclature; compounding; dispensing; distribution; administration; education; monitoring; and use." We purposefully limited the scope of our study to include errors related solely to prescribing. The NCC MERP severity levels range from "A" (circumstances that have the capacity to cause error), to "I" (an error occurred that resulted in patient death) (Table 2).
Table 2. NCC MERP Definition of a medication error and Risk Assessment Index 17| Category | Description of category |
| No error | |
| A | Circumstances or events that have the capacity to cause error |
| Error, no harm | |
| B | An error occurred, but the medication did not reach the patient |
| C | An error occurred that reached the patient but did not cause patient harm |
| D | An error occurred that resulted in the need for increased patient monitoring but no patient harm |
| Error, harm | |
| E | An error occurred that resulted in the need for treatment or intervention and caused temporary patient harm |
| F | An error occurred that resulted in initial or prolonged hospitalization and caused temporary patient harm |
| G | An error occurred that resulted in permanent patient harm |
| H | An error occurred that resulted in a near-death event (e.g., anaphylaxis, cardiac arrest) |
| Error, death | |
| I | An error occurred that resulted in patient death |
|
NCC MERP = National Coordinating Council on Medical Error Reporting and Prevention | |
Variation among research associates in identifying errors is inherent in the methodologic process chosen. To prevent this, a concerted effort was made to codify definitions and rules, to ensure a standardized approach to the identification of errors and interpretation of data reviewed by the research associates. For example, a standardized definition was needed for the data element "information missing from prescription." In the clinic setting, when a prescriber orders a medication to be taken orally, the abbreviation "PO" (per os, by mouth) is seldom written on the prescription. Thus, the team established a rule that prescriptions intended for oral consumption, from which a route of administration was absent, did not constitute an error. In contrast, prescriptions that were intended for other than oral administration, that did not specify the route of administration, did constitute an error. An example of the latter was a prescription written for "Rhinocort, 2 sprays each side, bid" (bis in d'ie, twice daily). The team considered this prescription to contain an error, as the route of administration, "intranasally," was not included on the prescription. This decision engendered lively debate within the team. It was argued that a health care professional would likely know the route of administration, whereas the patient would not. After much discussion the rule described above was codified. Separately, the team created a list of "inappropriate abbreviations." It included drug name abbreviations, the use of the terminology from the apothecary system, the use of "u" for the word "units," and the lack of leading or the presence of trailing zeros.
The research team next established rules for clinical errors that constituted "contraindications." Specifically, the team was very interested in identifying errors occurring in patients greater than 65 years of age. The use of Beers criteria
18
a list of medications that should be avoided in the elderlywas adopted for this purpose. The team also established rules for what constituted appropriate laboratory monitoring in patients of all ages. To achieve this end, a Laboratory Harm Table, specific to the clinic's population and prescribing patterns, was created. The Laboratory Harm Table specified laboratory-monitoring parameters that, if not ordered on a patient-specific basis, could result in patient harm, and consequently be considered a medication error. In creating this evaluation tool, the team was careful to differentiate laboratory values that would optimize medication effectiveness from those that are necessary for preventing harm; only the latter were included. For example, ordering a baseline set of liver function tests prior to instituting therapy with hydroxymethyl-glutaryl coenzyme A (HMG-CoA) reductase inhibitor (statin) was included in the Laboratory Harm Table, whereas the ongoing monitoring of a lipid panel to assess statin efficacy was not. A copy of the Laboratory Harm Table is available from the investigators. The clinic Pharmacy and Therapeutics Committee approved the use of Beers criteria and the Laboratory Harm Table specifically for this research project. The interpretation of clinical errors due to drug-disease interactions and drug-drug interactions was left to each individual research associate. All had recently completed their clinical training and were employed in clinical practices. The team reasoned that drug-disease interactions are addressed, in part, by the Laboratory Harm Table. In hindsight, creating a list of the 50 most clinically significant drug-drug interactions was certainly a manageable task and could easily have been codified.
Each rule was codified in the Operations Manual. The investigators also created a Reference Packet for use by the research associates. The Reference Packet included an example of the signature of each prescriber whose prescriptions were undergoing review, a copy of the Laboratory Harm Table, the NCC MERP Risk Assessment Index, and contact information for the investigators.
A member of the team who had both clinical and IT expertise created the database (RH). The database was created in a popular relational database program, with five screens that facilitated entry of data elements, utilizing a design schema that was clinically logical and relevant. Important features of the database included the automatic downloading of eligible prescriptions from the pharmacy computer system and numerous background tables, including therapeutic drug class tables and contraindication tables. Automatic downloading of information from the pharmacy computer system made useful information immediately accessible, without the research associates having to "retrieve" it from the CMR. The background tables included those that linked drugs to dosage forms and doses, provided directions for use ("sigs"), provided diagnosis codes based on the International Classification of Diseases 9th Revision Clinical Modification system (ICD-9-CM),
19
and provided documented allergies.
Three data sources were utilized for the evaluation of medication errors: the prescription itself, the chart notes and laboratory values of the CMR for clinical data, and the prescription as entered into the pharmacy computer system onsite.
Approval for the study was obtained from the Human Subjects Review Committee of the University of Washington, using a waiver of consent. The sample size was calculated based on information from the current literature, using a baseline error rate of 5 percent. 9 The study was powered to provide an 80 percent chance of finding a 40 percent reduction in the rate of medication errors when comparing pre- to postimplementation (two-sided test of proportions with an alpha level of 0.05). Thus 1,500 prescriptions were evaluated in the preimplementation phase. The same number will be evaluated for the postimplementation phase.
The method by which prescriptions were selected for evaluation was as follows. A 5-month time frame was identified; in an effort to minimize bias, one that occurred immediately prior to the time when implementing an ACPOE system was first discussed. Using the beginning of this 5-month time frame as the index date, every prescription written for a patient in the internal medicine clinic and filled at the onsite pharmacy, from that date forward, was evaluated for inclusion in the study. A total of 2,250 prescriptions were reviewed to find the evaluable 1,500 that met the inclusion criteria. The evaluation was limited to newly written prescriptions, inclusive of those written for patients between the ages of 18 and 89 years. Prescriptions transferred in from, or out to, outside pharmacies were excluded. Other exclusions were prescriptions for devices or laboratory monitoring equipment (e.g., glucose test strips), prescriptions filled via medication vouchers or coupons, and voided prescriptions. The initial plan was that each prescription would be evaluated by two research associates with discrepancies adjudicated by the third; problem prescriptions would be evaluated by the entire research team.
The team agreed at the outset that chart notes in the CMR would be evaluated for the 15 months prior to the writing of the prescription undergoing evaluation. The team reasoned that each patient should be seen a minimum of once yearly; 15 months provided an additional three months allowing for delays in the appointment-making process. Laboratory data were followed, retrospectively, for 1 year prior to prescription writing. The medication regimen of each patient was reviewed in a comprehensive fashion, evaluating drug-drug and drug-disease interactions for all current medications, not limited to those impacting the prescription undergoing evaluation. The team also outlined a substudy that would compare information found in the CMR with that found in the pharmacy computer system. The intent was to compare, from these two sources, the active drug list, the list of current disease states, and allergies.
The research associates evaluated each prescription for medication errors using a structured, computer-based system. Using five screens, they progressed in sequence, entering all data, one prescription at a time, into the database (except that which was downloaded automatically). The first screen prompted entry of data directly from the handwritten prescription. The second screen prompted the evaluation of clinical information from the chart notes and laboratory values contained in the patient's CMR. The third screen prompted entry of data that facilitated a comparison between the prescription as it had been written by the prescriber, and as it had been entered into the pharmacy computer system by the dispensing pharmacist. Close attention was paid to accurate entry to minimize the introduction of errors into the study process. Once the data were entered, the fourth and fifth screens prompted the research associate to record the detail of any medication errors found. If more than one error was found per prescription, each error was evaluated separately.
Refinements in the process took place during the pretest phase of data entrydata entry of the first 35 prescriptions. Several issues arose and each was addressed in turn. At the outset, each prescription required approximately 45 minutes for data entry and evaluation. Lack of proficiency with the process, at least initially, contributed to the unexpectedly long time frame for review, although other reasons also became apparent. To achieve the study goals, several modifications were necessary. Through discussions with the research associates, the investigators refined the process, decreasing the time required for data entry, while increasing time for meaningful analysis.
The first set of refinements streamlined the overall methods. Because discrepancies between the CMR and the pharmacy computer system were ubiquitous, the substudy was eliminated. Because comprehensive review of each medication regimen was burdensome, review was narrowed to focus on data relevant solely to the prescription undergoing evaluation. The number of months included in the retrospective review of the CMR was decreased from 15 to 6, unless the specific instance required additional evaluation, at the discretion of the research associate. Double entry of data was eliminated, resulting in each prescription being reviewed once by one research associate. This refinement will be accounted for in the statistical analysis, by conducting a test of interrater reliability.
The next set of refinements involved revision of several aspects of the data entry process. Although the numerous background tables were intended to create efficiencies during the data entry process, only some tables did, and others did not. The background tables presented too may choices from which to choose. The allergy table was comprised of more than 400 possibilities; the "directions for use" ("sig," signe'tour) table was comprised of more than 11,000 "sigs," and the table of ICD-9-CM codes was comprised of more than 15,000 records. Further, the background tables that linked drugs to dosage forms and doses were too restrictive. The research associates were frequently unable to enter the letters of the alphabet (or numbers) that called up the data that would match exactly, the prescription they were evaluating. Consequently, the database programmer created two shortcuts: an "express sig" list of 20 common "sigs," and a consolidated ICD-9-CM list comprised of 58 composite disease states (sorted alphabetically). Hard copies of each of these lists were added to the Reference Packets at each computer terminal. The programmer also unlinked drugs from dosage forms and doses, allowing the research associates to enter each of these elements separately. As a final feature, free-text fields were added in strategic places to facilitate the entry of data without restrictions. Once added, these free-text entries were saved by the system and could be recalled for future use. A free-text field was also added for comments.
Two of the investigators conducted quality audits of 10 percent of the prescriptions entered. These reviews pointed up the need for additional decision rules about what constituted a medication error. These were addressed in a systematic fashion, codified in the Operations Manual, and added to the Reference Packet. The team created a decision rule that required a distinction be made between tablets and capsules (for purposes of identifying an error) only when the dosage form impacted the pharmacokinetics or pharmacodynamics of the drug (e.g., an immediate- versus sustained-release formulation). Separately, the team addressed the issue of whether the dispensing of a drug with a particular strength and directions different from those prescribed constituted an error, for example, dispensing "atorvastatin 20 mg, 1/2 tablet daily" for a prescription written for "atorvastatin 10 mg once daily." As this is considered standard practice in the managed care setting, this was codified as not an error.
With close attention to details and revisions such as these, the time for evaluation of each prescription decreased to approximately 56 minutes for a minimally complex patient, and 710 minutes for a moderately complex patient. Even so, the evaluation of 1,500 prescriptions required more than 400 hours of labor. The evaluation was completed in May 2003.
This pilot, pre-ACPOE data, describing the incidence and characterizing the epidemiology of medication errors using the handwritten prescribing process, are now being analyzed. These results will be published in a subsequent paper. Implementation of the ACPOE system in the pilot clinic took place during July 2003. The prescribers have had 12 months to become proficient with the use of the new system. The collection of the postimplementation phase of the study is imminent. In the meantime, the ACPOE system is being rolled out to additional clinics within the Everett Clinic integrated health system.
This report describes our experience in conducting a small pilot study that evaluated medication errors prior to implementation of an ACPOE system. Although it is standard to include a "methods" section in the publication of the results of any similar study, we have not found a paper in this field that has focused solely on providing the details of study design and data collection methods. Our goal in so doing is to convey to others the complexity of even such a seemingly simple evaluation process.
With implementation of ACPOE systems becoming more widespread in the coming years, this experience may be useful to others as they undertake similar evaluations. Although aspects of our evaluation are unique to the Everett Clinic setting, others are generalizable. This study was conducted in the primary care setting (internal medicine) of a community-based integrated health system. The types of errors that are characterized in the ambulatory setting are often different from those occurring in the inpatient environment, but perhaps similar among different ambulatory settings. Our taxonomy for characterizing errors may be useful in creating a standard taxonomy for the ambulatory setting. Our approach to evaluating medication errors associated with laboratory monitoring may inform decision support programming in the future.
The study design enabled the capture of information on severity levels for patients who were harmed by an error and subsequently seen in the clinic setting. It did not include those for whom a hospital admission or emergency room visit was necessary. Because the design did not include an evaluation of medications dispensed, the results will be limited in distinguishing between severity levels "B" and "C" on the NCC MERP Risk Assessment Index (Table 2). The investigators will further explore these limitations during data analysis.
Several commonly held beliefs about prescribing were noted in this study. Prescribers seldom write with the same degree of detail and the same level of accuracy, as required in the drug dispensing process. Definitions of medication errors differ, depending on one's perspective. A prescription considered error-free by a prescriber may be considered by a pharmacist to contain an error; a prescription considered error-free by a pharmacist may still cause a patient to be confused, thus qualifying to be an error. Patient information contained in the medical record is seldom consistent with that contained in the pharmacy computer systemeven in an integrated health system with a systemwide CMR.
By paying close attention to detail during the design of the study, addressing each challenge as it arose and streamlining methods, the research team implemented a process to evaluate prescriptions for medication errors. This method will next be used to identify and characterize medication errors in the postimplementation phase of ACPOE implementation.
The potential for improvements in our nation's health care system offered by IT, and specifically by ACPOE, is substantial. Even using IT to evaluate medication errors requires careful planning. Realizing the benefits of ACPOE systems will require the same level of careful planning and attention to detail. A carefully crafted methodology, implementation plan, and response to challenges as they arise are critical to the success of any IT-related patient safety endeavor.
We gratefully acknowledge grant support from the Merck Foundation, the American Medical Group Association Foundation, and the American Society of Health-System Pharmacists Research and Education Foundation. We wish to thank our colleagues for their guidance and support: Sean Sullivan, Ph.D., University of Washington; Al Fisk, M.D., M.M.M., and Richard Rafoth, M.D., the Everett Clinic.
University of Washington Pharmaceutical Outcomes Research and Policy Program, Seattle, WA (EBD, TKH). The Everett Clinic, Everett, WA (JLW, NML, RH). Overlake Hospital Medical Center, Bellevue, WA (KK). University of Washington Medical Center, Seattle, WA (ST, CW).
Address correspondence to: Emily Beth Devine, Pharm.D., M.B.A.; Research Assistant Professor, Pharmaceutical Outcomes Research and Policy Program, Department of Pharmacy, University of Washington, Box 357630, Seattle, WA 98195-7630. Phone: 206-221-5760; Fax: 206-543-3835; e-mail: bdevine@u.washington.edu.
1. Kohn LT, Corrigan JM, Donaldson MS, editors. To err is human: building a safer health system. A report of the Committee on Quality of Health Care in America, Institute of Medicine. Washington, DC: National Academy Press; 2000.
2. Institute of Medicine. Crossing the quality chasm: a new health system for the twenty-first century. Washington, DC: National Academy Press; 2001.
3. Corrigan JM, Greiner A, Erickson SM. Fostering rapid advances in health care: learning from system cemonstrations. Washington, DC: Institute of Medicine; 2002.
4. Wilson-Norton JL, Fisk AW, Lawless NM. Managing prescribing cost and quality: one group's experience. Group Practice Journal 2002 Oct;11
18. 5. Hood B, Clarke J, Crouch R, et al. An incremental approach to a web-based computerized medical record. J Healthc Inf Manag 2001. 15(3):199-205. Fall. (PubMed)
6. Barker KN, Flynn EA, Pepper GA, et al. Medication errors observed in 36 health care facilities. Arch Intern Med 2002. 162:1897-1903. (PubMed)
7. Flynn EA, Barker KN, Carnahan BJ. National observational study of prescription dispensing accuracy and safety in 50 pharmacies. J Am Pharm Assoc 2003. 43:191-200.
8. Leape LL. Errors in medicine. JAMA 1994. 272:1851-7. (PubMed)
9. Bates DW, Boyle DL, Vander Vliet MB, et al. Relationship between medication errors and adverse drug events. J Gen Intern Med 1995. 10:199-205. (PubMed)
10. Leape LL, Bates DW, Cullen DJ, et al. Systems analysis of adverse drug events. JAMA 1995. 274:35-43. (PubMed)
11. Abood RR. Errors in pharmacy practice. US Pharm 1996. 21:122-132.
12. Bates DW, Teich JM, Lee J, et al. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc 1999. 6:313-21. (PubMed) (Full Text in PMC)
13. Meyer TA. Improving the quality of the order-writing process for inpatients orders and outpatient prescriptions. Am J Health-Syst Pharm 2000. 57(Suppl4):S18-22.
14. Bates DW, Cohen M, Leape LL, et al. Reducing the frequency of errors in medicine using information technology. J Am Med Inform Assoc 2001. 8:299-308. (PubMed) (Full Text in PMC)
15. Lesar TS. Prescribing errors involving medication dosage forms. J Gen Intern Med 2002. 17:579-87. Aug. (PubMed) (Full Text in PMC)
16. McNutt RA, Abrams R, Aron DC. Patient safety efforts should focus on medical errors. JAMA 2002. 287(15):1997-2001. Apr. (PubMed)
17. National Coordinating Council for Medication Error and Reporting and Prevention. Available at: http://www.nccmerp.org/aboutMedErrors.html . Accessed 4/12/04.
18. Beers MH. Explicit criteria for determining potentially inappropriate medication use by the elderly. An update. Arch Intern Med 1997. 157:1531-6. (PubMed)
19. INGENIX ICD-9-CM and AMA CPT with RVU Coding Resources. STAT!Ref. Available at: http://online.statref.com/toc.aspx?StartDoc=1&EndDoc=28227&FxID=24&offset=0&level=0&State=False&GRPALIAS=UofWash . Accessed 9/15/02.