Chapter 45. Simulator-Based Training and Patient Safety
Ashish K. Jha, M.D.
University of California, San Francisco School of
Medicine
Bradford W. Duncan, M.D.
Stanford University School of Medicine
David W. Bates, M.D., M.Sc.
Harvard Medical School
Background
For a number of years, simulators have been used in aviation,
nuclear power, military flight operations and other industries as a training
tool and method to assess performance. Their use is nearly universal in high
reliability organizations.1 Recently the use of simulation in
medicine has increased markedly, in part due to greater awareness of the
importance of patient safety.
Defined broadly, a simulator replicates a task environment with
sufficient realism to serve a desired purpose.1 In medical training,
simulators can substitute for actual patients and can be as simple as utilizing
pigs' feet to practice suturing, or as complex as virtual reality machines and
re-creations of actual clinical environments for surgeons, radiologists and
anesthesiologists. In a general sense, they improve patient safety by allowing
physicians to become better trained without putting patients at risk. For
example, in a randomized controlled trial, Peugnet and colleagues used a virtual
reality simulator to train physicians to perform retinal photocoagulation and
found that surgeons who trained with the simulator performed the procedure as
well as those who trained with patients.2 Gaba lists several other
advantages of simulation, among them3:
- Presentation of uncommon but critical scenarios in which a rapid
response is needed (e.g., malignant hyperthermia, which occurs once in every
40,000 anesthesia cases).4 To conduct systematic training about
managing such critical events there is little alternative but to use
simulation.
- Errors can be allowed to occur and reach their conclusion—in real life a
more capable clinician would have to intervene—so participants can see the
results of their decisions and actions.
- With mannequin-based simulators clinicians can use actual medical
equipment, exposing limitations in the human-machine interface.
- Complete interpersonal interactions with other clinical staff can be
explored and training in teamwork, leadership, and communication provided.
In a number of medical fields, simulation has been used in crew resource
management (CRM) training1 (see Chapter 44), where the focus is
on behavioral skills such as inter-team communication during critical
incidents.
Human error is a leading cause of adverse anesthesia events
that lead to poor patient outcomes5 and may play an important role in
surgical errors as well.6 However, it may be difficult to demonstrate
improved patient outcomes from simulation because adverse events are unusual and
there are an extreme number of potential confounders. Given these difficulties,
performance has been used as a surrogate outcome, despite concerns that it may
be a less than perfect measure. It is yet unclear which attributes of
performance matter most to patient outcomes. Additionally, the methods (e.g.,
reliability and consistency) and timing (e.g., duration of follow-up) by which
performance is measured also remain ill-defined.1 Studies of the
effectiveness of simulators often are limited in that they measure performance
using the same training simulator, which may favor those who have trained on the
simulator itself. In other words, seemingly improved performance may not
translate to actual patient care. Of the studies that have extended laboratory
simulation to patient care, few have evaluated the impact on medical error or
established a clear link between simulator training and patient outcomes.
This chapter reviews the evidence regarding the use of
simulators in the training and ongoing education of healthcare providers as a
patient safety measure. Not included in this review is the use of simulators in
the planning or preparation of diagnostic or therapeutic interventions in
specific patients (e.g., calculation of radiotherapy doses using computer
simulations, planning a surgical procedure for a specific patient using 3-D
simulations based on the anatomical data from the actual patient). While these
kinds of simulation clearly decrease the morbidity and mortality associated with
their particular procedures, they are omitted here because they focus on unique
characteristics of individual patients that may not be generalizable to other
patients. Other uses of simulators beyond the scope of this discussion include
simulators to measure worker proficiency,7-10 to identify areas for
educational intervention, and to target improvements in
training.11,12
Simulators in Anesthesia
Patient simulators have been most widely studied in anesthesia
where human error may account for over 80% of critical incidents.5
Simulators range from simple mannequins to high-fidelity simulators that
recreate the operating room experience. According to one study, 71% of medical
schools in Canada, the United Kingdom and other western nations used mannequins
or some other form of simulator to teach anesthesia to medical
students.13 There is a growing body of literature on the different
roles that simulators can play in anesthesia training.14
Studies have found that simulators can effectively identify
errors and appropriateness of decision making in anesthesia. For example, during
19 comprehensive anesthesia simulations, DeAnda et al documented 132 unplanned
incidents, of which 87 (66%) were due to human error and 32 (27%) were
considered critical incidents.15 Schwid and O'Donnell have also used
simulators to document the type of errors that anesthesiologists make in
critical incidents, finding errors in monitor usage (37%), airway management
(17%), ventilator management (13%), and drug administration (10%).16
Gaba and colleagues studied both the appropriateness of decisions and response
time of anesthesia trainees to simulated critical incidents.8 They
found great individual variability as well as variability by incident in the
accuracy and timeliness of response. Some simulated incidents, such as cardiac
arrest, had major errors in management a majority of the time. Based on these
studies, patient simulators can be used to identify areas for further education
or training of anesthesia providers.
We identified 2 studies of the effect of simulators on
anesthetists' performance. Schwid and colleagues studied the impact of a
computer screen-based anesthesia simulator in a randomized, controlled trial of
31 first-year anesthesia residents.17 Residents that had trained on
the simulator with individualized debriefing responded better to critical events
on a mannequin-based simulator than those who received standard training without
the simulator. Using a randomized, controlled design, Chopra and colleagues
studied management of simulated critical situations by 28 resident or staff
anesthesiologists.18 The performance of subjects who trained on the
simulator was superior to that of subjects who did not receive that
training.
Another setting where simulators may play an important role is
in crew resource management (CRM). Though establishing the effectiveness of
simulation in CRM training may be difficult,19 initial work has been
done on reliably and consistently rating performance.10 CRM is
discussed further in Chapter 44.
Proficiency on a simulator does not ensure proficiency in
clinical settings. Simulator fidelity (i.e., how accurately the simulator
replicates reality) is imperfect. It is much more difficult to "re-create" a
human being than to do so for, say, an airplane. This limitation is illustrated
by a study conducted by Sayre and colleagues. They studied emergency medical
technicians (EMT) who learned intubation techniques on anesthesia
mannequins.20 After successfully intubating the mannequins 10 times,
they were permitted to intubate patients in the field, where their proficiency
was only 53%. Other factors can inhibit optimum learning using simulation or the
applicability of learning to real practice. Some participants may be more
vigilant than usual during simulator sessions. Others may be unable to "suspend
disbelief," may treat the simulation only as a game, or act in a cavalier
fashion, knowing that the simulator is not a real patient.21
Refinement of simulators to make them more sophisticated and life-like may help
to improve the quality of the training that simulators can provide. Appropriate
construction of curricula and debriefings can also minimize the potential
problems of simulation training.
Simulators in Radiology
The number of radiologic examinations that require sedation,
analgesia, or contrast media has increased rapidly in recent years.22
Despite their rarity, serious medication reactions do occur and require prompt,
appropriate management. Some evidence of suboptimal management23 has
prompted the creation of computer-based simulators to improve training in these
areas.24 Simulators have also been used to measure the effectiveness
of strategies to teach trainees about critical incidents, but studies have not
reported the effectiveness of simulators as adjuncts to training. Sica and
colleagues developed a computer-based simulator that was used to study the
effectiveness of a lecture and videotape-based intervention on critical
incidents for radiology housestaff.25 Those residents who underwent
the intervention scored better on the simulator than those that had only
received basic, standard training. The authors concluded that the simulator was
an effective way of assessing the utility of the educational course.
Simulators in Surgery
As surgical technique and expertise has changed drastically
over recent decades, some methods used to train surgeons have evolved as well.
Simulators in the surgical setting are aimed at improving surgeons' technical
skills and dexterity. Training on simulators and virtual reality machines,
though still in its nascent stages, is becoming increasingly
accepted.26,27 Surgical simulators have been developed for a variety
of procedures: endovascular repair of abdominal aortic aneurysms,28
sinus surgery29,30 gynecologic surgery,31 orthopedic
surgery,32 prostatic surgery,33 amniocentesis
procedures,34 and oral surgery.35 Nonetheless, many of
these have yet to be formally evaluated in terms of efficacy in improving
physician performance in patient care.
We identified several studies that evaluated physician
performance after training on a surgical simulator. Derossis evaluated surgical
residents and attendings in a randomized study and found that those trained with
a simulator had greater proficiency in suturing, transferring, and mesh
placement, when tested on the simulator, than did the control
group.36 They subsequently found that when tested in vivo in
pigs, surgeons (both attendings and residents) who had been randomized to the
simulator arm were more proficient at the same skills.37 Scott and
colleagues studied the impact of a video-trainer on laparoscopic cholecystectomy
skills.38 They randomized surgical residents to training on a
video-trainer versus no formal training over a 30-day period. At the end of the
period, when tested on pre-specified tasks on the video trainer, those who
trained on the video trainer did uniformly better.
Intuitively, improved technical skills should lead to fewer
complications during surgery. Wallwiener and colleagues developed a surgical
trainer that, when used in conjunction with other improvements in their training
program, led to lower rates of hysteroscopic complications.39,40
However, for most simulators the link between improvements in technical skills
and dexterity from simulator training and prevention of adverse events has yet
to be established and deserves formal investigation. Further, problem-based
surgical simulation (e.g., avoiding inadvertent ligation of the ureter during
hysterectomy) may improve patient safety not only by improving skills, but also
by training surgeons to better anticipate and avoid complications and to manage
them should they occur.
Simulators in Gastroenterology
Simulators have been developed to train physicians in the
technical skills required in endoscopy.41-43 We identified one study
that evaluated the effect of simulator training on physician performance. In a
small randomized controlled trial enrolling 10 residents, Tuggy and colleagues
found residents who trained for flexible sigmoidoscopy using a virtual reality
simulator were faster, visualized a greater portion of the colon, and made fewer
directional errors in actual patients.44
Simulators in Cardiology
Cardiology training has long used a variety of simulators from
audiocassettes of heart tones to full patient simulators. Two simulators have
been evaluated. Champagne and colleagues demonstrated that a heart sound
simulator could increase medical students' recognition of pathologic heart
sounds.45 Ewy and colleagues studied the efficacy of "Harvey," a
cardiology patient simulator.46 In a study enrolling 208 senior
medical students at 5 medical schools, participants were randomized to receive
training on Harvey versus a standard cardiology curriculum during their
cardiology elective. Students who had been trained with Harvey performed skills
better both when tested with the simulator and when tested with real patients.
Some physicians have expressed concern that training on simulators may decrease
professionalism. In this study, there was no difference in the way patients
perceived the professionalism of the students trained on Harvey compared with
students who received standard training.
Systems that simulate the cardiovascular anatomy and physiology
have also been developed. Swanson and colleagues created a cardiovascular
simulator to train physicians on the workings of mechanical valves and balloon
assist devices, and to recognized diseased vessels.47 As cardiac
procedures have become more invasive, simulators to train cardiologist in these
procedures have become more common,48, 49 but their effect on
resident performance has not been evaluated formally.
Comment
Although simulators have been used for many years in a variety
of settings, data on their efficacy are still emerging. While there is currently
no evidence that simulation-based training leads to improved patient outcome, it
may prove difficult to conduct such studies. These would require large cohorts
of patients to be followed during and after care by clinicians who were
randomized to have undergone different cumulative amounts of simulation
training. Because adverse events are uncommon, and there are a large number of
patient-based and system-based factors that contribute to negative outcomes, any
such study would have to be massive and prolonged. Instead, provider
performance, with its known limitations, has been and will continue to be used
as a surrogate outcome. Nonetheless, as Gaba has asserted "...no industry in which
human lives depend on skilled performance has waited for unequivocal proof of
the benefits of simulation before embracing it."50 Certain benefits
are clear. In training for procedures, simulators have high face validity
because they ease trainees' transition to actual patients, which seems
inherently beneficial as a means to avoid adverse events. Further, procedural
success is related to the experience of the operator, known as the
volume-outcome relationship51, 52 (see Chapter 19). As simulators
become more advanced, they may be reasonable substitutes to improve proficiency
of both trainees and low volume physician operators. This increase in
proficiency may have an important impact in patient outcomes. Future studies of
the link between simulator-based training and performance on actual patients
will improve our ability to better assess the appropriate role of simulators in
training and patient safety.
The costs of simulators vary widely and need to be considered.
"Home-made" or simple trainers are far less expensive than complex simulators or
full-scale simulation centers. The average cost of high-fidelity patient
simulators is on the order of $200,000. Medium fidelity simulators may be as
little as $25,000. Establishing a dedicated simulation center can cost up to
$1,000,000 (including the simulator) depending on the amount of space, the type
of clinical equipment to be used, the extent of renovations needed, and the
sophistication of the audio-visual equipment. However, such capital costs are
amortized over a long period of time, and such centers typically are used for a
wide variety of training curricula for diverse target populations. Further, for
most simulation training the dominant cost is that of instructor time. Another
indirect cost is that of removing clinical personnel from revenue producing work
to undergo training. The healthcare industry currently does not fully embed time
or costs of training into the system, but instead often leaves these costs for
the individual clinicians to bear.
There are potential risks to simulation-based training. Where
the simulator cannot properly replicate the tasks or task environment of caring
for patients, there is a risk that clinicians might acquire inappropriate
behaviors (negative training) or develop a false sense of security in their
skills that could theoretically lead to harm. Although there are no data to
suggest that this currently happens, such risks will have to be weighed and
evaluated as simulators become more commonly used.
In summary, although there is currently little evidence that
simulation training improves patient care, the experience with simulation in
other industries and the high face validity of their applications in healthcare
has led many institutions to adopt the technology. It is likely that simulators
will continue to be used and their role in training of medical personnel will
grow. Definitive experiments to improve our understanding of their effects on
training will allow them to be used more intelligently to improve provider
performance, reduce errors and ultimately, promote patient safety. Although such
experiments will be difficult and costly, they may be justified to determine how
this technology can best be applied.
References
1. Bushell E, Gaba, D.M. Anesthesia Simulation and Patient Safety.
Problems in Anesthesia 2001;In Press.
2. Peugnet F, Dubois P, Rouland JF. Virtual reality versus conventional
training in retinal photocoagulation: a first clinical assessment. Comput
Aided Surg 1998;3:20-26.
3. Gaba DM, DeAnda A. A comprehensive anesthesia simulation environment:
re-creating the operating room for research and training. Anesthesiology.
1988;69:387-394.
4. Dunn D. Malignant hyperthermia. AORN J 1997;65:728,731,passim.
5. Cooper JB, Newbower RS, Long CD, McPeek B. Preventable anesthesia mishaps:
a study of human factors. Anesthesiology 1978;49:399-406.
6. Spencer FC. Human error in hospitals and industrial accidents: current
concepts. J Am Coll Surg 2000;191:410-418.
7. Keyser EJ, Derossis AM, Antoniuk M, Sigman HH, Fried GM. A simplified
simulator for the training and evaluation of laparoscopic skills. Surg
Endosc 2000;14:149-153.
8. Gaba DM, DeAnda A. The response of anesthesia trainees to simulated
critical incidents. Anesth Analg 1989;68:444-451.
9. Gaba DM. Anaesthesia simulators. Can J Anaesth 1995;42(10):952-953.
10. Gaba DM, Howard SK, Flanagan B, Smith BE, Fish KJ, Botney R. Assessment
of clinical performance during simulated crises using both technical and
behavioral ratings. Anesthesiology 1998;89:8-18.
11. Ruppert M, Reith MW, Widmann JH, et al. Checking for breathing:
evaluation of the diagnostic capability of emergency medical services personnel,
physicians, medical students, and medical laypersons. Ann Emerg Med 1999;34:720-729.
12. Dorfsman ML, Menegazzi JJ, Wadas RJ, Auble TE. Two-thumb vs. two-finger
chest compression in an infant model of prolonged cardiopulmonary resuscitation.
Acad Emerg Med 2000;7(10):1077-1082.
13. Cheung V, Critchley LA, Hazlett C, Wong EL, Oh TE. A survey of
undergraduate teaching in anaesthesia. Anaesthesia 1999;54:4-12.
14. King PH, Blanks ST, Rummel DM, Patterson D. Simulator training in
anesthesiology: an answer? Biomed Instrum Technol 1996;30:341-345.
15. DeAnda A, Gaba DM. Unplanned incidents during comprehensive anesthesia
simulation. Anesth Analg 1990;71:77-82.
16. Schwid HA, O'Donnell D. Anesthesiologists' management of simulated
critical incidents. Anesthesiology 1992;76:495-501.
17. Schwid HA, Rooke GA, Michalowski P, Ross BK. Screen-based anesthesia
simulation with debriefing improves performance in a mannequin-based anesthesia
simulator. Teach Learn Med 2001;13(2):92-96.
18. Chopra V, Gesink BJ, de Jong J, Bovill JG, Spierdijk J, Brand R. Does
training on an anaesthesia simulator lead to improvement in performance? Br J
Anaesth 1994;73:293-297.
19. Gaba DM, Howard,S.K., Fish,K.J., Smith,B.E., Sowb,Y.A. Simulation-based
training in anesthesia crisis resource management (ACRM): a decade of
experience. Simulation and Gaming 2001;32(2):175-193.
20. Sayre MR, Sakles JC, Mistler AF, Evans JL, Kramer AT, Pancioli AM. Field
trial of endotracheal intubation by basic EMTs. Ann Emerg Med 1998;31:228-233.
21. New Technologies in anesthesia practice: anesthesia simulators
[Audiocassette]. Kansas City, MO: Audio-Digest Anesthesiology; 1998.
22. Chait P. Future directions in interventional pediatric radiology.
Pediatr Clin North Am 1997;44(3):763-782.
23. Sadler DJ, Parrish F, Coulthard A. Intravenous contrast media reactions:
how do radiologists react? Clin Radiol 1994;49:879-882.
24. Medina LS, Racadio JM, Schwid HA. Computers in radiology. The sedation,
analgesia, and contrast media computerized simulator: a new approach to train
and evaluate radiologists' responses to critical incidents. Pediatr Radiol.
2000;30:299-305.
25. Sica GT, Barron DM, Blum R, Frenna TH, Raemer DB. Computerized realistic
simulation: a teaching module for crisis management in radiology. AJR Am J
Roentgenol 1999;172(2):301-304.
26. Carrico CJ, Satava,R.M. Advanced simulation technologies for surgical
education. Bull Am Coll Surg 1996;81:77.
27. Ota D, Loftin B, Saito T, Lea R, Keller J. Virtual reality in surgical
education. Comput Biol Med 1995;25:127-137.
28. Chong CK, How TV, Black RA, Shortland AP, Harris PL. Development of a
simulator for endovascular repair of abdominal aortic aneurysms. Ann Biomed
Eng 1998;26:798-802.
29. Rudman DT, Stredney D, Sessanna D, et al. Functional endoscopic sinus
surgery training simulator. Laryngoscope 1998;108(11 Pt
1):1643-1647.
30. Ecke U, Klimek L, Muller W, Ziegler R, Mann W. Virtual reality:
preparation and execution of sinus surgery. Comput Aided Surg 1998;3:45-50.
31. Szekely G, Bajka M, Brechbuhler C, et al. Virtual reality based surgery
simulation for endoscopic gynaecology. Stud Health Technol Inform 1999;62:351-357.
32. Poss R, Mabrey JD, Gillogly SD, et al. Development of a virtual reality
arthroscopic knee simulator. J Bone Joint Surg Am 2000;82-A(10):1495-1499.
33. Ballaro A, Briggs T, Garcia-Montes F, MacDonald D, Emberton M, Mundy AR.
A computer generated interactive transurethral prostatic resection simulator.
J Urol 1999;162:1633-1635.
34. Maher JE, Kleinman GE, Lile W, Tolaymat L, Steele D, Bernard J. The
construction and utility of an amniocentesis trainer. Am J Obstet Gynecol 1998;179:1225-1227.
35. Bettega G, Payan Y, Mollard B, Boyer A, Raphael B, Lavallee S. A
simulator for maxillofacial surgery integrating 3D cephalometry and orthodontia.
Comput Aided Surg 2000;5:156-165.
36. Derossis AM, Bothwell J, Sigman HH, Fried GM. The effect of practice on
performance in a laparoscopic simulator. Surg Endosc 1998;12:1117-1120.
37. Fried GM, Derossis AM, Bothwell J, Sigman HH. Comparison of laparoscopic
performance in vivo with performance measured in a laparoscopic simulator.
Surg Endosc 1999;13:1077-1081; discussion 1082.
38. Scott DJ, Bergen PC, Rege RV, et al. Laparoscopic training on bench
models: better and more cost effective than operating room experience? J Am
Coll Surg 2000;191:272-283.
39. Wallwiener D, Rimbach S, Bastert G. The HysteroTrainer, a simulator for
diagnostic and operative hysteroscopy. J Am Assoc Gynecol Laparosc 1994;2:61-63.
40. Wallwiener D, Rimbach S, Aydeniz B, Pollmann D, Bastert G. Operative
Hysteroscopy: Results, Security Aspects, In Vitro Simulation Training
(Hysterotrainer). J Am Assoc Gynecol Laparosc 1994;1(4, Part 2):S39.
41. Williams CB, Saunders BP, Bladen JS. Development of colonoscopy teaching
simulation. Endoscopy 2000;32:901-905.
42. Aabakken L, Adamsen S, Kruse A. Performance of a colonoscopy simulator:
experience from a hands-on endoscopy course. Endoscopy 2000;32:911-913.
43. Neumann M, Mayer G, Ell C, et al. The Erlangen Endo-Trainer: life-like
simulation for diagnostic and interventional endoscopic retrograde
cholangiography. Endoscopy 2000;32(11):906-910.
44. Tuggy ML. Virtual reality flexible sigmoidoscopy simulator training:
impact on resident performance. J Am Board Fam Pract 1998;11:426-433.
45. Champagne MT, Harrell JS, Friedman BJ. Use of a heart sound simulator in
teaching cardiac auscultation. Focus Crit Care 1989;16:448-456.
46. Ewy GA, Felner JM, Juul D, Mayer JW, Sajid AW, Waugh RA. Test of a
cardiology patient simulator with students in fourth-year electives. J Med
Educ 1987;62:738-743.
47. Swanson WM, Clark RE. A simple cardiovascular system simulator: design
and performance. J Bioeng 1977;1:135-145.
48. Wang Y, Chui C, Lim H, Cai Y, Mak K. Real-time interactive simulator for
percutaneous coronary revascularization procedures. Comput Aided Surg 1998;3:211-227.
49. Dawson SL, Cotin S, Meglan D, Shaffer DW, Ferrell MA. Designing a
computer-based simulator for interventional cardiology training. Catheter
Cardiovasc Interv 2000;51:522-527.
50. Gaba DM. Improving anesthesiologists' performance by simulating reality.
Anesthesiology 1992;76:491-494.
51. Jollis JG, Romano PS. Volume-outcome relationship in acute myocardial
infarction: the balloon and the needle. JAMA 2000;284:3169-3171.
52. Jollis JG, Peterson ED, Nelson CL, et al. Relationship between physician
and hospital coronary angioplasty volume and outcome in elderly patients.
Circulation 1997;95:2485-2491.
Return to Contents
Proceed to Next Chapter