Summary
Evidence Report/Technology Assessment: Number 95
Under its Evidence-based Practice Program, the Agency for Healthcare Research and Quality (AHRQ) is developing scientific information for other agencies and organizations on which to base clinical guidelines, performance measures, and other quality improvement tools. Contractor institutions review all relevant scientific literature on assigned clinical care topics and produce evidence reports and technology assessments, conduct research on methodologies and the effectiveness of their implementation, and participate in technical assistance activities.
Select for PDF File (155 KB). PDF Help.
Introduction / Methods / Results / Discussion / Availability of Full Report / References
Introduction
Disaster scenarios that once seemed merely theoretical have become a disturbing reality. The emergence of state-sponsored terrorism, proliferation of chemical and biological agents, availability of materials and scientific weapons expertise, and recent increases in less discriminate attacks all point toward a growing threat of a mass casualty incident (MCI). Governmental agencies, healthcare professionals, and public health advocates have sought to determine the best ways to mitigate the potential impact of an MCI that results in multiple casualties that overwhelm local resources and that may include natural, biological, chemical, nuclear, or other agents.
Hospital disaster preparedness has therefore taken on increased importance
at local, State, and federal levels. Hospitals themselves are taking renewed
interest in disaster preparedness, reexamining their disaster plans, and conducting
disaster exercises. Preparing for MCIs is a daunting task, as unique issues
must be considered with each type of event. For example, the systemic stress
of a biothreat is entirely different from that of a chemical disaster or any
other acute onset disaster. These differences hold challenging implications
for preparedness training.
Hospitals must play a key role in developing disaster preparedness plans, and they need to coordinate efforts with public health systems and appropriate governmental agencies. The Joint Commission on Accreditation of Healthcare Organizations (JCAHO) actually requires hospitals to test their emergency plan twice a year, including at least one community-wide drill.1 However, it is not known whether this type of training is effective.
The current evidence report updates the evidence report Training of Clinicians for Public Health Events Relevant to Bioterrorism Preparedness2 and focuses specifically on the effectiveness of hospital disaster drills, computer simulations, and tabletop or other exercises in training hospital staff to respond to an MCI. The following key questions were addressed:
- What is the effectiveness of hospital disaster drills in training hospital staff to respond to an MCI?
- What is the effectiveness of computer simulations in training hospital staff to respond to an MCI?
- What is the effectiveness of tabletop or other exercises in training hospital staff to respond to an MCI?
- What methods or tools have been used to evaluate the effectiveness of hospital disaster drills, computer simulations, and tabletop exercises or other exercises in training hospital staff to respond to an MCI?
Return to Contents
Methods
Data Sources
The Evidence-based Practice Center (EPC) searched for articles published through January 2003 using six electronic databases, including PubMed®, the Cochrane CENTRAL Register of Controlled Trials, the Exerpta Medica database (EMBASE), the Educational Research Information Clearinghouse, the specialized register of the Effective Practice and Organization of Care Cochrane Review Group, and the Research and Development Resource Base in Continuing Medical Education. Search terms included mass casualty, disaster, disaster planning, and drill. The EPC also conducted a hand search of references and selected journals.
Study Selection
Paired investigators reviewed the abstracts of citations located by the search to identify pertinent articles. Exclusion criteria were: not written in English; no human data; no original data; meeting abstract (no full article for review); did not include hospital staff; did not include response to an MCI or a disaster; did not include training or education; no evaluation of the training; or did not apply to any of the key questions.
Data Extraction
Paired reviewers evaluated study quality in terms of the representativeness of the targeted hospital staff, potential bias and confounding, description of the intervention, assessment of outcomes, and analysis. The reviewers then extracted information on the studies (e.g., geographic location, MCI type, training intervention, hospital staff targeted, other entities involved, objectives, evaluation methods, and results).
Return to Contents
Results
The literature search process identified 243 unique, potentially relevant citations, of which 208 were excluded at abstract review. Twenty-one of the remaining 35 articles were deemed eligible for data abstraction. Sixteen of these studies addressed the effectiveness of hospital disaster drills in training hospital staff to respond to an MCI (key question 1);3-18 one study addressed the effectiveness of computer simulations in training hospital staff to respond to an MCI (key question 2);19 and four studies addressed the effectiveness of tabletop or other exercises in training hospital staff to respond to an MCI (key question 3).20-23
The reviewed studies represented a heterogeneous body of literature. They ranged from descriptions of local drills, including transportation incidents, fires, and radiological exposures, to sophisticated telecommunication exercises, such as a large regional drill involving multiple agencies.22
Studies also varied in terms of targeted staff, learning objectives, identified outcomes, and evaluation methods. Because of the wide range of foci for the studies, it was difficult to draw definitive conclusions about the most effective approaches for training hospital staff to respond to an MCI. However, some potentially valuable points could be identified in the literature:
- Internal and external communications were the key to effective disaster response.
- A well-defined incident command center reduced confusion.
- Conference calls were an inefficient way to manage disaster response.
- Accurate phone numbers for key players were vital and regular updating was necessary.
- Disaster drills appeared to be an effective way to improve clinicians' knowledge of hospital disaster procedures.
- Computer simulation may be an economical method to educate key hospital decisionmakers and improve hospital disaster preparedness before implementation of a full-scale drill.
- A tabletop exercise can help to motivate hospital staff to learn more about disaster preparedness and can help to teach staff about aspects of disaster-related patient care in a way that simulates the practice setting.
- A regional exercise involving top government officials can help to increase awareness of the need for better disaster response planning.
- Video demonstrations may be an inexpensive, convenient way to educate a large number of staff about disaster procedures and equipment use in a short time.
Nineteen studies included specific evaluation methods (key question 4), and 13 of these used more than one type of evaluation method.4,5,7,8,10,13,14,16-19,22,23 Group or individual debriefings were the most common,5,7-10,12,14,17-20,22 followed by "smart" observers (medical personnel).4,5,7,8,10,17 Other observers4-6,8,19,23 and trained "smart" casualties4,13,14 were also used in several studies. Four studies used a written exam.14,16,21,23 Other methods of evaluating the educational intervention included:
- Individual interviews.22
- Inspection and review by chemical spill specialists.4
- Self-assessment.23
- A computer-generated detailed picture of the situation.19
- Observer checklists.10
- Mock disaster patient charts.11
- Victim tracking cards.13
- Videotaping.16
Due to the heterogeneity of the evaluation methods and the lack of data on their validity and reproducibility, the evidence was insufficient to support any firm conclusions about the usefulness of reported evaluation methods.
Return to Contents
Discussion
Hospital disaster drills, computer simulations, and tabletop and other exercises are designed to test the hospital's disaster plan and to allow employees to become familiar with disaster procedures. Based on the review of the literature, discussion with experts, and analysis of disaster response plans,24 the EPC team identified several important aspects of hospital disaster response that may be useful to evaluate. Most of the lessons learned relate to one or more of the following aspects of disaster response:
- The incident command system.
- Communications (both internal and external).
- Clinical care, including triage, patient care, patient flow, and patient tracking.
- Security.
- Materials and resources.
- Decontamination.
Enough studies were available to suggest that hospital disaster drills can help to identify problems with incident command, communications, triage, patient flow, security, and other issues. Evidence also indicated that computer simulations and tabletop and other exercises may help to train key decisionmakers in disaster response. The studies demonstrated that different types of training exercises may have different roles to play in educating hospital staff in disaster response. However, the evidence was insufficient to support firm conclusions about the effectiveness of specific training methods because of the marked heterogeneity of studies, weaknesses in study design, and the limited number of exercises that have been reported in the literature. Future disaster preparedness efforts would benefit from increased reporting of hospitals' experiences in disaster response training.
Return to Contents
Availability of Full Report
The full Evidence Report from which this summary was taken was prepared for the Agency for Healthcare Research and Quality (AHRQ) by the Johns Hopkins University Evidence-based Practice Center, Baltimore, MD, under Contract No. 290-02-0018. Printed copies may be obtained free of charge from the AHRQ Publications Clearinghouse by calling 800-358-9295. Requesters should ask for Evidence Report/Technology Assessment No. 95, Training of Hospital Staff to Respond to a Mass Casualty Incident.
The Evidence Report is also online on the National Library of Medicine Bookshelf, or can be downloaded as a PDF File (505 KB). PDF Help.
Return to Contents
References
1. Joint Commission on Accreditation of Healthcare Organizations. Comprehensive Accreditation Manual for Hospitals: The Official Handbook. Oakbrook Terrace, IL: Joint Commission Resources; 2003. p. EC-21.
2. Catlett C, Perl T, Jenckes MW, et al. Training of Clinicians for Public Health Events Relevant to Bioterrorism Preparedness. Evidence Report/Technology Assessment No. 51. Rockville, MD: Agency for Healthcare Research and Quality. 2002 Jan. AHRQ Publication No. 02-E011.
3. Chobin N, Engle L. Are you ready for an ethylene oxide spill? J Healthc Mater Manage 1989; 7(3):56-7, 60.
4. Tur-Kaspa I, Lev EI, Hendler I, et al. Preparing hospitals for toxicological mass casualties events. Crit Care Med 1999; 27(5):1004-8.
5. Gretenkort P, Harke H, Blazejak J, et al. Interface between hospital and fire authorities—a concept for management of incidents in hospitals. Prehospital Disaster Med 2002; 17(1):42-7.
6. Baughman KS, Calvert DR. Internal disaster drill critique. J Emerg Nurs 1990; 16(4):240-1.
7. Halstead MA. Fire drill in the operating room. Role playing as a learning tool. AORN J 1993; 58(4):697-706.
8. Weston CJ. The operating theatre's on fire. Health Serv Manage 1988; 84(3):20-3.
9. Classic KL, Knutson AH, Smith GD. Radiation safety role in institutional disaster planning. Health Phys 2000; 78(5 Suppl):S35-9.
10. Cook L. Hospital disaster drill game: a strategy for teaching disaster protocols to hospital staff. J Emerg Nurs 1990; 16(4):269-73.
11. Eisner ME, Waxman K, Mason GR. Evaluation of possible patient survival in a mock airplane disaster. Am J Surg 1985; 150(3):321-3.
12. Lau PF, Lau CC. A disaster drill in Hong Kong. Accid Emerg Nurs 1997; 5(1):34-8.
13. Paris PM, Stewart RD, Pelton GH, et al. Triage success in disasters: dynamic victim-tracking cards. Am J Emerg Med 1985; 3(4):323-6.
14. Gofrit ON, Leibovici D, Shemer J, et al. The efficacy of integrating "smart simulated casualties" in hospital disaster drills. Prehospital Disaster Med 1997; 12(2):97-101.
15. Saxena K, Cicero JJ, Hass WF. Chemical disaster drill in the State of Minnesota. Minn Med 1986; 69(3):137-9.
16. Menczer LF. The Hartford disaster exercise. N Engl J Med 1968; 278(15):822-4.
17. Fishel ER. Exercise "Med-Ex" 73. Md State Med J 1974; 23(9):46-8.
18. Maxwell C, Cherneski M. Post-disaster drill analysis. Using victim-tracking cards. J Emerg Med Serv 1987; 12(2):43-6.
19. Levi L, Bregman D, Geva H, et al. Hospital disaster management simulation system. Prehospital Disaster Med 1998; 13(1):29-34.
20. Levy K, Aghababian RV, Hirsch EF, et al. An Internet-based exercise as a component of an overall training program addressing medical aspects of radiation emergency management. Prehospital Disaster Med 2000; 15(2):18-25.
21. Gray D. Disaster plan education: how we made and tested a video. J Accid Emerg Med 1996; 13(1):21-2.
22. Inglesby TV, Grossman R, O'Toole T. A plague on your city: observations from TOPOFF. Clin Infect Dis 2001; 32(3):436-45.
23. Burns KA. Experience in the use of gaming and simulation as an evaluation tool for nurses. J Contin Educ Nurs 1984; 15(6):213-7.
24. The HEICS plan. Available at: http://www.emsa.cahwnet.gov/dms2/download.htm. Accessed 2003 Feb 28.
Return to Contents
AHRQ Publication Number 04-E015-1
Current as of April 2004
Internet Citation:
Hsu EB, Jenckes MW, Catlett CL, et al. Training of Hospital Staff To Respond to a Mass Casualty Incident. Summary, Evidence Report/Technology Assessment: Number 95. AHRQ Publication Number 04-E015-1, April 2004. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/clinic/epcsums/hospmcisum.htm
540 Gaither Road Rockville, MD 20850