Advances in Patient Safety: From Research to Implementation Advances in Patient Safety: From Research to Implementation Volume 2. Concepts and Methodology

Patient Safety Research in Medical Group Practices: Measurement and Data Challenges

Amy R. Wilson, Bryan E. Dowd, John E. Kralewski

Abstract

This paper attempts to identify and discuss some of the major challenges to conducting patient safety research in medical group practices. First, we identify the most important attributes of medical group practices to be considered in this type of research. The measurement and specification of these variables are discussed, and the problems associated with past research are noted. Alternate ways of specifying key variables are discussed, and examples of methods that have been shown to be effective are provided. We then propose a definition of patient safety and discuss issues around its measurement. Finally, we discuss how the data requirements for this type of research are very complex and outline benefits and challenges related to using certain types of data.top link

Introduction

There is wide agreement that many, if not most, patient care adverse events originate in the ambulatory care sector. 1 -- 3 The magnitude of these events is just now becoming evident. A recent study of prescription drug errors found that 13.5 percent of the prescriptions written for a managed care population resulted in an error. 4 Ernst and Grizzle recently estimated the additional cost of health care resulting from medication-induced morbidity and mortality to be $177.4 billion, or about 7 percent of total national health expenditures. 5 These large values result in part from the volume of care provided by physicians in ambulatory care settings and the increased complexity of that care. As noted by Hammons et al., nearly 80 percent of all medical procedures are now performed in ambulatory care settings. 6 Moreover, the services provided during these visits are becoming more complex as enhanced technologies and treatment modalities are transferred from hospital settings to ambulatory care. 7

While medical group practices are rapidly becoming the practice form of choice in the ambulatory setting, little is known about what is, can, or should be done to assure patient safety and quality of care. As noted by Hammons, a physician's knowledge, skills, and judgments influence patient care outcomes, but the systems within which they practice are factors as well. 6 In fact, the practice organization may be as important as the clinician's skills in assuring quality of care. Much of the patient safety research has focused on inpatient care and the influence that internal hospital systems and procedures have on medical errors and the quality of care. This has resulted, in part, because hospital data is often more readily available for study, and hospitals are often more willing to participate in studies than are physicians in office-based practices. 8

Research in medical group practices presents several methodological challenges related to defining a group practice, measuring patient safety, and synthesizing the data necessary for the estimation. The remainder of this paper describes those methodological challenges and potential resolutions.top link

Specifying and measuring group practice variables

The standard definition of medical group practices used by the American Medical Association and Medical Group Management Association (MGMA) is three or more physicians practicing together with a common medical record and billing system. 9 This definition may not be specific enough to support rigorous research or even to monitor changes in the field. Several additional attributes are important defining characteristics of medical groups, but most are difficult to assess from existing datasets. For example, one of the most important attributes of group practices is the degree to which the physicians consider themselves part of an organized way of providing health care, rather than independent clinicians practicing in the same location and sharing support services. This has been described as a "me vs. us" perspective about all aspects of the group practice organization. 10 Unfortunately, despite its importance, assessing it would constitute a research project in and of itself.

Then, how should group practices be classified for studies of patient safety, and how can the important organizational variables be specified? A useful approach is to classify the defining variables according to those that can be expected to be reasonably stable and those that are amenable to short-term changes by the practice leadership. Variables that have been shown to be stable over 5- to 10-year time periods and influence practice performance include location (urban vs. rural), ownership, employment of nurse practitioners or physician assistants, tax status (for profit or not), decisionmaking structure, and practice culture. 11 Attributes that are more amenable to short-term change include practice size, specialty mix, degree of financial risk sharing for patient care, physician compensation method, and clinical information system.

Thus, we propose the basic unit of analysis to be established using the more stable variables. Theoretical models will drive the inclusion of other attributes. For example, a study of prescription drug errors in medical group practices using claims data from a managed care plan can specify the unit of analysis as multispecialty practices with three or more physicians practicing together, using a common billing system, and including at least one primary care physician. All of these attributes are available in the claims data. The theoretical model might then include variables such as ownership, urban location, physician compensation method, and clinical information system. Some of these variables might be available from claims data, but most would need to be obtained from the practices.

Defining the unit of analysis with respect to group practices is not straightforward, and neither is the specification of several significant practice attributes. As mentioned earlier, ownership has been shown to be an important factor influencing practice performance, and especially, access to capital. The following is the conventional approach to defining different types of ownership:

While this definition captures the main influence of ownership on practice performance, three cautions should be noted. First, there is a trend for physician-owned practices to be located in clinic facilities owned by some other organization, usually a hospital or health plan. When the unit of analysis is the group practice, the ownership of the practice, rather than the clinic, should be the central variable. Clinic owners, however, may influence access to capital improvements and support services, and for this reason, a separate variable should address that issue.

A second ownership issue relates to the trend toward forming consortia of medical group practices to share information and negotiate contracts with managed care plans. These umbrella organizations, sometimes called individual practice associations, may own some of the participating practices and may even claim all of them as their clinics. For research purposes, however, it is important to trace each practice to its legal corporate ownership in order to untangle the influence of important explanatory variables.

A third ownership issue relates to satellite clinics. Often a large group practice will establish satellite clinics to improve access to services and to capture additional referrals. In some cases, the parent group purchases existing practices. While the legal ownership in this case is clear, these satellites often operate as separate entities. We have found that responses to questions about practice attributes (e.g., the use of clinical guidelines) from satellite administrators and physicians differ from those provided by the parent. 12 -- 14 Moreover, the culture of the satellite may differ significantly from that of the parent, especially if the parent purchased, as opposed to established, the satellite. Therefore it is clear that in some patient safety research, these satellites should be treated as separate units of analysis. The decision whether to specify the satellites as separate units of analysis depends on the variables included in the model and the sources of data. A model that includes physician compensation methods, clinical information systems, or use of drug formularies can usually treat the parent and the satellites as one practice by assessing the degree to which the responses pertain to all of the clinic sites. However, a model that includes a practice culture variable or degree of financial risk sharing in the revenue streams may need to treat each site as a unit of analysis. A useful signal from claims data that helps resolve this issue is the source of the bills. A satellite clinic that bills separately from the parent indicates that the clinic operates as a relatively autonomous unit and should be treated as such.

Financial incentives in group practices have been shown to have an important influence on the costs and quality of care. 15 -- 17 Group practice performance is influenced by both how the practice is paid and how the physicians within the practice are paid. While early studies specified the financial incentive variable at the practice level as capitation or fee-for-service, this dichotomy is no longer sufficiently discerning. 18 -- 20 A useful alternative is to assess the degree of financial risk sharing for patient care in the payment plan. Using this approach, capitation payment for all services defines one end of the continuum, and fee-for-service based on billed charges defines the other. In between exist multiple arrangements such as capitation for physician services only, fee-for-service with withhold provisions, and negotiated fee schedules. A useful way to assess the mix of these payment plans is to ask the practice administration to indicate the proportion of revenue derived from each of the different payment plans for the study period. The following categories have been successfully used in several past studies: 15, 21

A useful method of assessing the influence of physician-level financial incentives on group practice performance is to measure the degree to which the physician's income is at risk. A predetermined fixed salary carries the least risk, while some share of the practice net revenue has the most risk of variation in annual income. Moreover, the risk compensation plans may emphasize different types of patient care incentives. For example, a plan that bases a large portion of the compensation on some share of net practice revenue in a largely capitated practice provides incentives to reduce service use. A structure that pays on a discounted fee-for-service basis provides the opposite incentive. An increasingly popular compensation plan in medical groups is to base about 60 percent of the physicians' income on some measure of productivity and the remainder on either a fixed salary or a share of net revenue. This provides the incentives to work hard, yet aligns the incentives with practice goals imposed by the revenue streams. Physician-level payment categories can be specified as the portion of compensation based on

All of these measures focus on the influence of financial incentives, rather than level of compensation, on patient care practices. Level of compensation clearly relates to practice styles, but is not a driver of performance.

Two additional variables warrant discussion: clinical information systems and practice culture. Both have been shown to influence practice performance and present complex measurement problems. A growing number of medical groups are adopting electronic clinical information systems, and patient safety research must take this important variable into account. There is little agreement, however, on the terminology describing this technology. Consequently, while some research focuses on the presence or absence of an electronic medical record (EMR), the findings are often suspect because of a lack of agreement on the definition of an EMR. 22, 23 The definition problem is twofold. First, some practices consider an EMR to be a computer-based supplement to a paper record rather than a replacement. Second, an EMR has several components, and practices may implement only one or two components. For example, it is not uncommon for practices to have electronic patient information at the care site, but to write lab orders and drug prescriptions on paper. This distinction would be very relevant to studies of prescription drug errors, and a research model should capture such variations.

Including practice culture variables in analytic models presents a different problem. While several studies have found that the culture of organizations influences performance, the translation of that research to the health care field has been slow to develop. 24 Moreover, there is concern that the instruments used to measure cultures in industrial firms may not capture the nuances of organizations such as medical group practices. 25 Researchers focusing on patient safety in medical groups should be attentive to this issue when selecting a culture instrument and look carefully at the types of organizations used to develop and test the instrument. A second caution is that researchers often describe organizational culture using single-dimension variables constructed as the mean response from the physicians and nurses to each item on the culture instrument. While this is a valid first step in assessing practice cultures, the next important step is to measure the degree of clinician agreement on the culture. Expressed together as interaction terms, these variables provide a much more powerful analysis.

This discussion of group practice attributes is not meant to be exhaustive. Many other factors may be hypothesized to have an effect on patient safety. The variables we discuss here, however, have been shown to influence group practice performance and are known to present special specification and measurement issues.top link

Measuring patient safety

Defining and measuring patient safety poses a significant challenge. Researchers have put forth many definitions of patient safety, errors, and adverse events. We support a definition of patient safety that encompasses care that is not only free of unwanted consequences, but meets a given standard of care. We would like to capture the idea that both commission of inappropriate procedures and omission of appropriate procedures can compromise patient safety; so for the purposes of this discussion, let us define patient safety as the patient receiving the care that was intended and merited. Patient care in a group practice is comprised of processes, even if they are not formally defined as such. Steps in these processes can go awry when an individual takes an inappropriate action or fails to take an appropriate action. We refer to these instances as errors. An error may or may not result in an adverse event, which is an instance of a patient suffering the consequences of an error. In other words, errors may be detected by monitoring the process, while adverse events may be detected by monitoring the patient. It is important to note that not all errors will result in adverse events. Some health care processes can tolerate a certain number of errors. Similarly, whether or not an error results in an adverse event may be related to patient characteristics. For these reasons, an analyst studying patient safety cannot choose between studying errors or adverse events ---but must study both. The importance of studying adverse events is obvious. Patients suffer when they occur. The importance of studying the errors that precede the adverse events is also obvious. The errors show how the process failed the patient. What may be less obvious is that it is also important to study errors that did not result in adverse events. Not only are they more plentiful, but they offer the opportunity to learn about why the error did not result in an adverse event.

An analysis of errors and adverse events thus requires an understanding of both the process of patient care and patient outcomes. Determining whether a patient received the care that was intended and merited is not a straightforward task. Best practices must be defined to determine if safe care was provided. Previous research has demonstrated that the standards of care vary geographically and are often different from best practices. 26 Of course, for many conditions, there still exists uncertainty about treatment options, especially when patient preferences are taken into account. Fisher and Wennberg 26 address this issue by making the distinction between "effective care" and "price-sensitive care." The former includes "...only interventions that virtually all well-informed patients would want, and that such patients would expect physicians to recommend or prescribe." The latter refers to "...those interventions in which there is a choice between at least two treatments that have different risks and benefits," from which different patients may draw different conclusions.

A researcher commencing a patient safety study must define the events and errors that might arise in any given setting. Several studies have attempted to identify and classify the types of errors encountered in family practice. While these studies may serve as useful starting points, researchers should be aware that the definitions of errors and adverse events often vary between studies. The implementation of an error reporting system in general practices in the United Kingdom (U.K.) led to a classification system for the errors that included the categories prescriptions, communication, appointments, equipment, clinical care, and other. 27 A survey of general practitioners from six countries resulted in a taxonomy of errors they encountered in their work. 28 The study identified 171 types of errors and concluded that they could be classified as either process errors or knowledge/skills errors. Family practitioners in the United States reported errors in the following processes: ordering medication, implementing laboratory investigations, filing, implementing medication orders, and responding to abnormal test results. 29 They reported consequences of those errors such as health outcomes (e.g., pain, worsening of condition), care consequences (e.g., delay), and financial and time costs. 30 A review of the few studies related to errors in primary care also led to the development of a classification scheme. Elder and Dovey distinguished between "preventable adverse events" and "process errors." 31 In their taxonomy, adverse events included outcomes related to diagnosis (e.g., missed diagnosis), treatment (e.g., incorrect drug), and preventive care (e.g., inappropriate). Note that this definition of adverse events differs significantly from ours in that it does not address health outcomes. Process errors in that taxonomy included factors related to clinicians (e.g., judgment), communication, administration, and "blunt end" (e.g., government regulations).

This discussion of errors, adverse events and standards of care emphasizes that it is very difficult to divorce the discussion of patient safety from the discussion of health care quality. In fact, there are disadvantages to focusing on errors and adverse events to the exclusion of other quality indicators:

A common problem arises in the attempt to define the performance of a system by a single measure or a few measures concentrated in one dimension. Given one measure of success, almost any group can be successful in the short term by optimizing that measure at the expense of other important measures. 25

For example, the Institute of Medicine (IOM) has called for health care to improve in six dimensions: safety, effectiveness, timeliness, patient-centeredness, efficiency, and equity. 32 The definition of patient safety we have put forth encompasses the IOM's criteria of safety and effectiveness. There are certainly aspects of timeliness that affect the safety of a patient, but there are also aspects that play a role in patient satisfaction. Finally, the remaining goals are not likely to be addressed by an emphasis on patient safety as we have defined it.top link

Data sources

Because of the nature of both ambulatory care and patient safety, research in this area will likely involve data from multiple sources. As discussed earlier, errors and adverse events should be measured by looking at both processes and patient outcomes. In the absence of a specially designed error reporting system, these two types of information are unlikely to be found in a single data source. There is a significant body of research studying errors in hospitals. This setting has the advantage of being a (nearly) closed system. If an error occurs in a process in the hospital, the adverse event is likely to occur in the hospital. This is not the case in ambulatory care, where an error that occurs in the practice setting may result in an adverse event at, for example, the patient's home, which then results in the patient going to the emergency department. These points are illustrated by a recent study of adverse drug events among elderly patients in an ambulatory setting. Gurwitz et al. 33 found it necessary to identify adverse events using several data sources: "Reports from health care providers; review of hospital discharge summaries; review of emergency department notes; computer-generated signals; automated free-text review of electronic clinic notes; and review of administrative incident reports concerning medication errors." In addition to the challenge of synthesizing multiple sources of data, each type of dataset offers potential advantages and challenges. We discuss those trade-offs in the rest of this section.

Claims data

Claims submitted by medical group practices for services provided to enrollees of a health insurance plan offer a promising source of large datasets at low cost. These data usually include physician, patient, and practice identifiers along with both International Classification of Diseases (ICD)-9 and Current Procedural Terminology (CPT)-4 codes for services. These data can provide important process and, in some cases, health outcome measures to assist in the identification of errors and adverse events.

Claims data may, however, present practice identification problems related to those discussed earlier. While many, if not most, health insurance plans can identify the medical groups serving enrollees, information about the practice rarely extends beyond the number and types of physicians and the location. If the plan requires enrollees to select a practice to provide and manage their care, costs and quality measures can be attributed to a specific group practice. If enrollees are allowed to select any physician in the plan for care on an episodic basis, the responsible physician must be determined from the claims. A conventional method to addressing this issue is to assign responsibility for prevention services (e.g., immunizations, PAP smears) to the primary care physician accounting for most visits during the year, or to the physician to whom the last primary care visit was made. Other patient safety incidents can be traced to the physician responsible for the episode of care when the incident occurred or, in the case of drug errors, to the physician who wrote the prescription. If the health plan database does not include a practice identifier, the physicians can be placed in their medical group by the tax codes used for claims billing. When verified against health plan and phone online directories, this approach has been found to place about 90 percent of the physicians in their practice organizations. 34 top link

Practice surveys

We have discussed the importance of practice structure and culture to patient safety research. While some of this data is available from sources such as the MGMA, 35 in most cases the data must be collected from the group practices themselves. Several mechanisms have been used to acquire this data. Casalino et al. successfully used phone interviews with group practice administrators and physicians to obtain practice data. 36 Others have combined phone interviews or mailed surveys with site visits. 15 Mailed or electronic questionnaires are by far the least expensive methods to obtain group practice data, but the response rates are often disappointing. This results, in part, from failure to gain prior cooperation of the practice administrators in the project and from including too many items in the questionnaires. Researchers who have successfully attended to these issues have consistently obtained response rates in the 70 to 80 percent range. While most, if not all, practice structure data can be obtained from the administrator, the practice culture data must be obtained from the clinicians. This presents different methodological problems. Many researchers obtain only 40 percent response rates to physician surveys, and there is a great deal of concern about potential biases in the responses.

Organizational culture studies risk serious flaws if the response rate falls below 80 percent because different professions view the culture differently, and response bias is highly likely to have lower return rates. Various methods have been used to improve these rates, including payment for completing the survey instrument and holding raffles for trips to professional meetings. 31 Neither has been very successful. A method successfully used in Minnesota appealed to the professional commitments of group practices to improve their patient care process. The group practice medical directors were asked to distribute the culture instrument to the physicians at one of their routine meetings, with instruments to be returned anonymously to the researchers. This approach obtained high levels of participation (84 percent) while protecting the identity of the respondents. 11 top link

Patient surveys

An alternate method of acquiring a sufficient amount of data to support patient safety research is to interview patients. Most research using this method focuses on patient satisfaction and access to care. However, some studies have acquired patient safety data using phone interviews. Hanlon et al. 37 found that patients were able to identify a broad range of prescription drug errors, including the degree of adverse effects. Moreover, MGMA has developed a method of conducting group practice patient interviews by phone while protecting the confidentiality of the patient, an important consideration when using this method of data collection.top link

Self-reporting systems

There is growing recognition of the potential value to health care of incident reporting systems (both voluntary and mandatory). 38, 39 For example, the Veterans Health Administration has announced its intent to establish such a system. 40 The American Academy of Family Physicians is conducting an evaluation of doctor- staff-, and patient-reported medical errors, which is underway in five primary care clinics and five family physician offices. 41 As self-reported error and adverse event data become increasingly available, researchers should be aware of the incentives and disincentives that an individual may have for reporting an error or adverse event, and how that may affect the quality of the data. As Barach and Small 39 report in their review of nonmedical "near miss" reporting systems, disincentives to use of a reporting system include perceptions of additional work, fear of reprisal, and lack of trust in the effectiveness of the system itself.top link

EMRs

Some authors have suggested that EMR is the next big development that must occur to ensure safety and quality in all health care settings. 42 Research suggests that EMRs offer advantages over paper records with respect to legibility and completeness of information related to diagnoses, prescriptions, advice given, and referral. 43 The implementation of EMRs in group practices could greatly expand the kinds of patient safety research that is practical and may increase the quality of that research. An EMR could obviate the need for costly chart reviews, provide a useful counterpoint to claims data, and provide background information to supplement self-reported errors. We have noted that the implementations of EMRs in group practices are heterogeneous and that patient safety entails looking at both the process and the outcomes of patient care. In the ambulatory care setting, the outcomes can occur in various locations and at various times, which may mean that they occur outside the purview of the EMR. For this reason, the EMR may be most informative with respect to the process, rather than the outcome of patient care.

Ultimately, rigorous patient safety research requires multiple sources of data and approaches. The costs of acquiring this data depend on the degree to which the practice and the external organizations have electronic data systems in place and are willing to collaborate in the research.top link

Conclusions

Clearly, patient safety is and should be an important part of the national health care agenda. To date, very little research has occurred in the ambulatory care setting, including group practices. Patients are, however, receiving increasingly greater proportions of their care in these settings. For these reasons, we believe that patient safety in group practices should be a priority.

In this paper, we have outlined some of the challenges of working in this area. Group practices are heterogeneous in nature, and we have shown how that presents specification and measurement challenges and how those challenges may be addressed. The definition and measurement of patient safety is also a complex task. We have called for a definition that includes errors and adverse events, representing both the health care process and patient outcomes. The complex nature of group practices introduces many estimation problems. When the data represent multiple levels of structure, attributing effects to the proper level is difficult. We have discussed several approaches to dealing with these challenges. Finally, we described how a complex definition of patient safety could result in a complex set of data requirements. Many potential data sources exist, and each presents its own challenges. No particular data source, however, is likely to be sufficient for a rigorous study of errors and adverse events.top link

Author affiliations

All of the authors are affiliated with the Division of Health Services Research and Policy, School of Public Health, University of Minnesota, Minneapolis, MN.

Address correspondence to: Amy R. Wilson; Division of Health Services Research and Policy, School of Public Health, University of Minnesota, 420 Delaware Street SE, MMC 729, Minneapolis, MN 55455. Phone: 612-626-3732; fax: 612-624-2196; e-mail:arwilson@umn.edu.top link

References

1. Kohn LT, Corrigan JM, Donaldson MS, editors. To err is human: building a safer health system. A report of the Committee on Quality of Health Care in America, Institute of Medicine. Washington, DC: National Academy Press; 2000.

2. Gandhi TK, Burstin HR, Cook EF, et al. Drug complications in outpatients. [See Comment.] J Gen Intern Med 2000. 15(3):149-54. Mar. (PubMed) (Full Text in PMC)

3. Weingart SN, Wilson RM, Gibberd RW, et al. Epidemiology of medical error. [See Comment.] BMJ 2000. 320(7237):774-7. Mar 18. (PubMed) (Full Text in PMC)

4. Kralewski J, Dowd BE, Heaton A. The influence of the structure and culture of medical group practices on prescription drug errors. Academy Health Annual Research Meeting; 2004 Jun 6 -- 8; San Diego, CA.

5. Ernst FR, Grizzle AJ. Drug-related morbidity and mortality: updating the cost-of-illness model. [See Comment.] J Am Pharm Assoc 2001. 41(2):192-9. Mar --Apr.

6. Hammons T, Piland NF, Small SD. Ambulatory patient safety: what we know and need to know. Englewood, CO: The Medical Group Management Association; 2002.

7. National Center for Health Statistics. Advance data, No. 305NCHS. Washington, DC: United States Department of Health and Human Services; 1999.

8. Gandhi TK, Puopolo AL, Dasse P, et al. Obstacles to collaborative quality improvement: the case of ambulatory general medical care. nt J Qual Health Care 2000. 12(2):115-23. Apr.

9. Medical Group Management Association. Practice management FAQs * answers: governance & organizational dynamics. 2004. Available at: http://www.mgma.com/members/infocenter/goanswerfaqs.cfm . Accessed April 28, 2004.

10. Kralewski J. The management of managed care in medical group practices. Minneapolis: Division of Health Services Research and Policy, University of Minnesota; 2002.

11. Kralewski J, Dowd B, Silversmith J, et al. Managing patient care cost in Minnesota medical group practices. Minn Med 2004. 87:48-54.

12. Rich EC, Kralewski J, Feldman R, et al. Variations in the management of primary care: effect on cost in an HMO network. Archives of Internal Medicine 1998. 158(21):2363-71. Nov 23. (PubMed)

13. Gans D. Medical Group Management Association; Englewood, CO.; 2004. (personal communication)

14. The adoption of clinical guidelines by medical group practices. Division of Health Services Research and Policy, University of Minnesota; 1999.

15. Kralewski JE, Rich EC, Feldman R, et al. The effects of medical group practice and physician payment methods on costs of care. Health Serv Res 2000. 35(3):591-613. (PubMed) (Full Text in PMC)

16. Conrad DA, Sales A, Liang SY, et al. The impact of financial incentives on physician productivity in medical groups. Health Serv Res 2002. 37(4):885-906. Aug. (PubMed) (Full Text in PMC)

17. Barrow J, Beaulieu N. Selection and improvement: physician responses to financial incentives. National Bureau of Economic Research, Inc., Working Paper 10017; 2003.

18. Luft H. Health maintenance organizations: dimensions of performance. New York: Wiley-Interscience; 1981.

19. Kralewski JE, Rich EC, Bernhardt T, et al. The organizational structure of medical group practices in a managed care environment. Health Care Management Review 1998. 23(2):76-96.

20. Pedersen CA, Rich EC, Kralewski J, et al. Primary care physician incentives in medical group practices. Archives of Family Medicine 2000. 9(5):458-62. May. (PubMed)

21. Kralewski JE, Wallace W, Wingert TD, et al. The effects of medical group practice organizational factors on physicians' use of resources. Journal of Healthcare Management 1999. 44(3):167-82. discussion 182 --3 (PubMed)

22. Waegemann CP. Status report 2002: electronic health records. Boston: Medical Records Institute; 2002.

23. Institute of Medicine, Committee on Data Standards for Patient Safety. Key capabilities of an electronic health record system: letter report. Washington, DC: National Academy of Sciences; 2003.

24. Kralewski J, Dowd BE, Kaissi A, et al. Measuring the culture of medical group practices. Minneapolis: Division of Health Services Research and Policy, University of Minnesota; 2004.

25. Shortell SM, Jones RH, Rademaker AW, et al. Assessing the impact of total quality management and organizational culture on multiple outcomes of care for coronary artery bypass graft surgery patients. Med Care 2000. 38(2):207-17. Feb. (PubMed)

26. Fisher ES, Wennberg JE, et al. Health care quality, geographic variations, and the challenge of supply-sensitive care. Perspect Biol Med 2003. 46(1):69-79. Winter. (PubMed)

27. Rubin G, George A, Chinn DJ, et al. Errors in general practice: development of an error classification and pilot study of a method for detecting errors. [See Comment.] Qual Saf Health Care 2003. 12(6):443-7. Dec. (PubMed) (Full Text in PMC)

28. Makeham MA, Dovey SM, County M, et al. An international taxonomy for errors in general practice: a pilot study. [See Comment.] Med J Aust 2002. 177(2):68-72. Jul 15. (PubMed)

29. Dovey SM, Phillips RL, Green LA, et al. Types of medical errors commonly reported by family physicians. Am Fam Physician 2003. 67(4):697. Feb 15. (PubMed)

30. Dovey SM, Phillips RL, Green LA, et al. Consequences of medical errors observed by family physicians. Am Fam Physician 2003. 67(5):915. Mar 1. (PubMed)

31. Elder NC, Dovey SM. Classification of medical errors and preventable adverse events in primary care: a synthesis of the literature. J Fam Pract 2002. 51(11):927-32. Nov. Erratum appears in J Fam Pract 2002 Dec;51(12):1079. (PubMed)

32. Institute of Medicine, Committee on Quality of Health Care in America. Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academy Press; 2001.

33. Gurwitz JH, Field TS, Harrold LR, et al. Incidence and preventability of adverse drug events among older persons in the ambulatory setting. [See Comment.] JAMA 2003. 289(9):1107-16. Mar 5. (PubMed)

34. Town RJ, Finch M, Kralewski J, et al. The impact of payments on the screening behavior of physicians. Minneapolis: Division of Health Services Research and Policy, University of Minnesota; 2004.

35. Medical Group Management Association. Physician compensation and production Survey. 2004 report based on 2003 data. Englewood, CO: Medical Group Management Association; 2004.

36. Casalino L, Gillies RR, Shortell SM, et al. External incentives, information technology, and organized processes to improve health care quality for patients with chronic diseases. JAMA Jan 22-29. 289(4):434-41. 2003.

37. Hanlon JT, Schmader KE, Koronkowski MJ, et al. Adverse drug events in high risk older outpatients. [See Comment]. Journal of the American Geriatrics Society 1997. 45(8):945-8. (PubMed)

38. Thomas EJ, Helmreich RL. Will airline safety models work in medicine? In: Rosenthal MM and Sutcliffe KM, editors. Medical error: what do we know? what do we do? San Francisco: Jossey-Bass; 2002. p. 325.

39. Barach P, Small SD. Reporting and preventing medical mishaps: lessons from nonmedical near miss reporting systems. BMJ 2000. 320(7237):759-63. Mar 18. (PubMed) (Full Text in PMC)

40. Veterans Administration National Center for Patient Safety. Creating a Culture of Safety; 2004. Available at: http://www.patientsafety.gov/vision.html . Accessed April 21, 2004.

41. American Academy of Family Physicians. Pilot study #1: physician, staff, and patient-reported errors in primary care. 2004. Available at: http://www.aafp.org/x16063.xml . Accessed Dec 2004.

42. Halvorson GC, Isham GJ. Epidemic of care: a call for safer, better, and more acountable health care. San Francisco: Jossey-Bass; 2003.

43. Hippisley-Cox J, Pringle M, Cater R, et al. The electronic patient record in primary care ---regression or progression? A cross sectional study. [See Comment.] BMJ 2003. 326(7404):1439-43. Jun 28. [Erratum appears in BMJ 2003 Aug 30;327(7413):483.] (PubMed) (Full Text in PMC)