Medical Errors and Adverse Drug Events Reporting System (MEADERS): Networks' Experiences

Slide Presentation from the AHRQ 2008 Annual Conference

---

On September 10, 2008, John Lynch, made this presentation at the 2008 Annual Conference. Select to access the PowerPoint® presentation (2 MB).

---

Slide 1

Networks' Experiences

• Connecticut Center for Primary Care: ProHealth Physicians Practice Based Research Network (Principal Investigator [PI]: John Lynch, MPH).
• The Oregon Rural Practice-based Research Network (ORPRN) (PI: LJ Fagnan, MD).
• CenTexNet (PI: Samuel N. Forjuoh, M.D., M.P.H., Dr.P.H).
• LA Net (Lyndee Knox, PHD).

Slide 2

Connecticut

• The Connecticut Center for Primary Care (CCPC) is an independent 501c3 non-profit research and education foundation.
• On incorporation in 2002, CCPC assumed management of the ProHealth Physicians Practice Based Research Network (PBRN), a PBRN affiliated with the American Academy of Family Physicians (AAFP) Federation of Practice Based Research Networks.

Slide 3

PBRN Setting: ProHealth Physicians

• CT's largest primary care group practice.
• 75 practice sites throughout Hartford, Middlesex, Litchfield, Tolland, and New Haven Counties.
  • 9 practice sites participated in the Medical Errors and Adverse Drug Events Reporting System (MEADERS).
• 221 primary care providers:
  • 166 MD/DO, 55 APRN/PA (72 Family Practice, 73 Pediatric, 75 Internal Medicine, and 3 specialists).
• Patient population: 349,000 (approx 10% of CT).
• Patient Contacts: over 750,000 encounters/yr.

Slide 4

CCPC MEADERS 9 Study Sites

1. Pediatric—9 clinicians.
2. Pediatric—6 clinicians.
3. Family Practice—7 clinicians.
4. Family Practice—2 clinicians.
5. Internal Medicine—4 clinicians.
6. Internal Medicine—3 clinicians.
7. Internal Medicine—3 clinicians.
8. Internal Medicine—2 clinician.
9. Internal Medicine—1 clinician.

Slide 5

CCPC Results continued

• Responses by type of event.
  • 61.6% (n=77) Adverse drug events.
  • 20.0% (n=25) Med error-provider.
  • 8.8% (n=11) Med error-patient.
  • 4.0% (n=5) Pharmacy.
  • 5.6% (n=7) other (Managed care organization [MCO], manufacturer, multiple, ...)

Slide 6

CCPC Results continued

• When happened:
  • 9.6% Today.
  • 44.8% Past week.
  • 26.4% Past month.
  • 18.4% Over month ago.
  • 0.8% Don't know.

Slide 7

Discussion: Experience

• Recruiting—easy.
  • 20 practices for 10 spots.
• Extended wait—Office of Management and Budget (OMB).
  • Lost one site—left ProHealth.
  • Sites didn't remember they had signed up.
• System—Surprisingly easy/straightforward.
• Retention—consistency by week.

Slide 8

Discussion: Lessons

• Pilot test quirks.
  • Participants say they completed many more forms.
• Form utility.
  • Comfort reporting adverse drug events.
  • Patient/pharmacist/MCO failed to...
    • Blame culture—continuous quality improvement (CQI) opportunity.
    • Need for better education/support.
• Legal concerns.
  • Need for Patient Safety Organization.

Slide 9

The Oregon Rural Practice-based Research Network (ORPRN)

Slide 10

Participating ORPRN Clinics

• Map of Oregon indicates clinics at Scappoose, Enterprise, The Dalles, Baker City, John Day.

Slide 11

Participant Job Categories

• Bar graph shows percentages of job categories in OPRN and Nationally.
  • Primary Care Clinicians.
    • OPRN 38.4%
    • Nationally 70.9%
  • Medical Assistants or Nurse.
    • OPRN 46.5%
    • Nationally 19.1%
  • Other.
    • OPRN 15.1%
    • Nationally 10.0%

Slide 12

Reporting Facilitators

• Adequate Training.
• Reminders & Feedback.
  • Weekly "Chalk Talk" reports.
  • Access to online error report page.
  • Occasional "fun" reminders.
• MEADERS Desktop Icon.
• User-friendly Program.
• Active engagement of lead clinician & staff.

Slide 13

Qualitative Findings in ORPRN

1. Rural Family Practice Clinicians and office staff will report to MEADERS as evidenced during a 10 week reporting trial.
   • "It was painless."
   • "It went well. All participated and seemed to enjoy it. It did not take an excessive amount of time."
   • "People in the practice picked up on the type of events to report quite quickly and required little direction. Melinda helped a lot too."

Slide 14

Qualitative Findings in ORPRN

2. Barriers to reporting included: making the time to report, breadth of reporting options, and confusion regarding what to report.
   • "It was more difficult for the staff it seemed because it was one more thing that they had to do."
   • "I am assuming that some errors did not get reported because the criteria were too broad and clinicians didn't think that they were important."

Slide 15

Qualitative Findings in ORPRN

3. Some practices used MEADERS data for internal Quality Improvement.
   • "Overall, I think that the MEADERS reporting system was helpful to staff and providers to remind us that errors occur and we should be mindful of them and work to prevent them in the future...my MA and I worked especially hard with one patient who kept taking the wrong dose of her Coumadin... Her care improved because of MEADERS."
   • "The biggest thing it changed was making us more attentive to documentation and keeping med lists up to date - especially between institutions like Long Term Care (LTC) and hospitals and hospices."
   • Qualitative Findings in ORPRN.

Slide 16

Qualitative Findings in ORPRN

4. Some clinics report a desire to continue using MEADERS while others do not.
   • "It was easy to use, we would like to continue using it in our practice."
   • "I doubt the practice would be excited about reporting events on a regular basis...This will be a significant obstacle to over come if this or a similar system becomes required in clinical practice."

Slide 17

ORPRN MEADERS Conclusion

• "The overall thing is that the direct access made the process much easier—most of our staff had never made any reports regarding adverse drug reactions so in my eyes—if the goal is to improve the process by which reports are made—this is a huge improvement."
  —Family Physician, Clinic 2

Slide 18

CenTexNet—Central Texas Primary Care Research Network

Slide 19

Scott & White Health Care System

• Map of central Texas shows clinics at Killeen, Gatesville, Waco, Temple, and Taylor.
• CenTexNet Practices.
  • 28 Scott & White Regional Clinics in 22 Locations.
    + Brazos Family Medicine Residency Clinic, Bryan.
    + Family Practice Residency Clinic, Waco.

Slide 20

Follow-up Phone Call about Status of Survey Response

• "Clinic has been really busy and just hasn't had time to complete survey. Will try to get to it ASAP."
• "Clinic is short of nurses this week. Will try to get to it TODAY."
• "Contact has been out sick and just got back to work today. Will try to get to it."
• No responses from two clinics.

Slide 21

Facilitators & Barriers

• Facilitators:
  • Quality & Safety Officers.
  • Clinic staff & MD champions.
  • Adequacy of training.
  • Part of network mission.
  • Staff familiarity with Information Technology (IT).
• Barriers:
  • Time constraints.
  • Flu season.
  • Staff turnover.
  • Workflow conflict.

Slide 22

LA Net

• www.lanetpbrn.net
• Reducing Health Care Disparities in Southern California
  Through Participatory Research Partnerships With the Health Care Community

Slide 23

LA Net—5 Federally Qualified Health Centers (FQHCs)

• Clinica Msr. Oscar A. Romero Alvarado St.
• Clinica Msr. Oscar A. Romero Marengo St.
• QueensCare, Echo Park.
• Cleaver Family Wellness Center.
• East Valley Community Health Center West Covina.
• East Valley Community Health Center Pomona.

Slide 24

LA Net—Sites

• Site—MDs—NPs/PAs—Pharmacy.
• Clinica Msr. Oscar A. Romero Alvarado St.—5—4—1.8
• Clinica Msr. Oscar A. Romero Marengo St.—4—1
• Cleaver Family Wellness Center—1—2
• East Valley Community Health Center West Covina—8—7—3
• East Valley Community Health Center Pomona—6—4—2

Slide 25

Weekly Data Submissions by Clinic

• Line graph depicts number of submissions from each clinic over a 10-week period; submissions dropped off toward end of reporting period.
• Note: QueensCare dropped out week 5 and Cleaver joined at week 6.

Slide 26

Reporting Facilitators

• Agreed w/ concept reporting/tracking is important.
• Liked MEADERS system
  • Easy to use.
  • Fast - 20 seconds to enter.
• Said project increased their awareness.

Slide 27

Reporting Barriers: Not about time

• Culture: Administrative significance not enough—not clinically significant, why report?
• Ergonomics/Work flow.
  • Interrupted paper-based work flow for clinicians.
  • Dispensary uses computer but also interrupted process.
• Translation to QA processes not easy.
  • Feedback function came on-line toward end.
  • When did, not used in QI.
  • Duplicates existing paper based ME systems for County and State (logs).
  • Likely will be a key motivator in future.

Slide 28

Learning & recommendations (LA Net)

• Need training & implementation supports for program that aid:
  • Identifying administrative value of detecting & reporting non-harm ADEs and MEs.
  • Translation of data into existing QI processes.
• On-demand reports that meet existing reporting requirements.
• Multiple options for entering data:
  • Electronic (Electronic ICON integrated into Electronic Health Record [EHR]—No site had EHR or eRxing capabilities at time of study but all are either considering or in process of transition at present).
  • Paper (Paper to electronic).

Slide 29

Total Submissions by Week

• Bar graph depicts submissions from four sites over the 10-week period.
  • Note with graph says "From a total of 48 submissions in the first week, there was a dramatic drop to the teens in the 4 subsequent weeks. This was obviously a sign of the initial excitement about any project and possibly the weekly raffle draw. Since the majority of these submissions were from one clinic, we believe that it represented a single 'excited' individual. "Of course, the excitement waned in the following weeks. Thus we believe the first week was an aberration, while the subsequent weeks really represent the true results from our system. "We also think that the low trend of submissions in our network may have resulted from the coincidence of the project with the high flu season since the study ran from October to December."

Slide 30

Total Submissions by Network/Site

• Bar graph depicts submissions from four sites over the 10-week period.
  • Note with graph says "From a total of 48 submissions in the first week, there was a dramatic drop to the teens in the 4 subsequent weeks. This was obviously a sign of the initial excitement about any project and possibly the weekly raffle draw. Since the majority of these submissions were from one clinic, we believe that it represented a single 'excited' individual. "Of course, the excitement waned in the following weeks. Thus we believe the first week was an aberration, while the subsequent weeks really represent the true results from our system. "We also think that the low trend of submissions in our network may have resulted from the coincidence of the project with the high flu season since the study ran from October to December."

Slide 31

Post-reporting survey of the experience of the participants

• A survey was sent to the 220 clinicians and staff members who agreed to participate in the project.
  • Purpose: to learn about participants experience with MEADERS.
  • 164 completed the survey, a 75% response rate.

Slide 32

Practice role of the 164 respondents

• Role in the practice—Number (%)
• MD, DO—51 (31%)
• NP, PA—31 (19%)
• RN, LPN, MA, Lab Tech—55 (33%)
• Pharmacist—2 (1%)
• Office Manager—16 (10%)
• Front Office Staff—8 (15%)
• Billing/Administrative Staff—1 (1%)

Slide 33

Demographics of 164 respondents

• Demographics of 164 respondents.
• Male: 40 (25%)
• Female: 120 (75%)
• Age 18 37: 50 (33%)
• Age 38 47: 35 (23%)
• Age 48 57: 52 (34%)
• Age 58 87: 15 (10%)

Slide 34

Demographics of 164 respondents

• Race—Number (%)
• American Indian/Alaska Native—1 (1%)
• African American—1 (1%)
• Asian—2 (1%)
• Hispanic/Latino—11 (6%)
• Pacific Islander.Native Hawaiian—1 (1%)
• White, no Hispanic—129 (79%)
• Multi-racial—3 (2%)
• No response—16 (9%)

Slide 35

Training for MEADERS

• Percent Trained on Use of MEADERS—Number (%)
  • Trained—156 (96%)
  • Untrained—7 (4%)
• Effectiveness of Training in Preparation
  • Effective: 76 (48%)
  • Somewhat effective: 23 (15%)
  • Not effective: 59 (37%)
• Extent of understanding on what events to report
  • Understand: 133 (82%)
  • Somewhat Understand: 26 (16%)
  • Do Not Understand: 3 (2%)

Slide 36

Difficulty using MEADERS

• Have you reported any events in the past 3 months using MEADERS?
  • Yes: 93 (57%)
  • No: 71 (43%)
• Degree of difficulty accessing the electronic reporting form and submitting a report.
  • No or little difficulty: 101 (62%)
  • Moderate to a great deal of difficulty: 3 (2%)
  • Unable to access system: 1 (1%)
  • Did not make a report in MEADERS: 58 (35%0)

Slide 37

Experience using MEADERS-1

• Experience—Agree—Neither Agree or Disagree—Disagree.
• It is easy to use—90%—9%—1%
• It allows me to be candid when reporting errors—79%—18%—3%
• It protected my anonymity when filing reports—70%—28%—2%
• It encourages me to learn from my mistakes and the mistakes of others—64%—29%—7%
• It is viewed positively by this practice—60%—38%—2%
• It has increased my own awareness of how errors affect patient care—61%—29%—10%

Slide 38

Experience using MEADERS-2

• Experience—Agree—Neither Agree or Disagree—Disagree.
• It has helped my to improve patient care at my practice—41%—44%—16%
• It has increased the fear of repercussion in the practice—36%—47%—17%
• It takes too much time to submit a report—26%—20%—54%
• It has led to changes in how we practice medicine—25%—54%—21%
• It made me feel like I was informing on my co-workers—10%—20%—70%
• It has not worked in this practice—9%—26%—65

Slide 39

Did you have any concerns about...

• Concerns—Number (%)
• The time it took to complete a report—29 (33%)
• Knowing what should be reported—26 (29%)
• The privacy of your reports—12 (13%)
• Knowing how to correctly make a report—10 (11%)
• The layout or format of the reporting system—9 (10%)
• Others in your practice finding out you had made a report—6 (7%)
• The type of question asked—5 (6%)
• How your practice might react to your making a report—5 (6%)

Slide 40

Forwarding event data to MedWatch

• Were there adverse drug events or medication errors that you reported in MEADERS that you did not forward to the Food and Drug Administration (FDA) MedWatch database?
  • Yes: 38 (40%)
• What were the reasons you elected not to forward an adverse drug event report to the MedWatch?
  • Did not see any benefit to myself or my practice by reporting an error or adverse event to MedWatch—25 (42%)
  • Unclear on the procedure for reporting to MedWatch—13 (22%)
  • Uncomfortable about the possibility that the FDA could contact me about the report I filed—6 (10%)
  • Concerned about repercussions of reporting to MedWatch—4 (7%)
  • Worried about provider confidentiality (mine and/or others)—3 (5%)
  • Worried about patient confidentiality—3 (5%)
  • Other—5 (9%)

Slide 41

Under what circumstance might you have reported more events?

• Instances—Likely—No difference—Unlikely.
• If more errors occurred—78%—19%—3%
• If there were a greater awareness of the system's benefits—50%—36%—14%
• If I had more time or opportunity to access the system—50%—30%—20%
• If it took less time to file a report—45%—37%—18%
• If I had a better understanding of what should be reported—25%—57%—18%
• If I had more assurance of the system's confidentiality—13%—63%—24%
• If there was a change in the attitudes of my practice towards reporting—11%—59%—30%

Slide 42

Did the study affect you personally?

• Has your participation in this study affected you personally?
  • Yes: 24 (26%)
• How has your use of this system affected you?
  • More cognizant of potential errors and adverse events. More vigilant about not repeating them.
  • More aware of medication errors and the importance of careful med review/prescribing practices.
  • More aware of all the difference types of medication errors that go on in day to day practice.
  • Made me more aware of the number of errors and made me more careful when doing my job.
  • Made me less complacent about refilling prescriptions.
  • It has taken up a fair amount of my time that I would typically use on different tasks.
  • I'm more cautious in giving shots and med samples.

Slide 43

Conclusions

• The MEADERS tool was used by practices and felt to be of value.
• FDA MedWatch is rarely used by clinicians.
• Reporting on medication errors is was not considered "safe" by many participants. They expressed concerns about repercussions of reporting and a few people were uncomfortable reporting on the mistakes of others.
• Comments of participants note that MEADERS improved the quality of their care; they learned from mistakes and changed how they did their job.
• Time is money: the sustainability of using a reporting tool such as MEADERS is unclear. Health systems might consider implementing MEADERS across their organization.

Slide 44

Discussion

Current as of January 2009

---

Internet Citation:

Networks' Experiences. Slide Presentation from the AHRQ 2008 Annual Conference (Text Version). January 2009. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/about/annualmtg08/091008slides/Lynch.htm

---