Chapter 1. Background

Hospitals must be prepared to respond to natural and manmade mass casualty incidents that may cause sudden demand on services. Disaster drills have been identified as a critical component of preparedness because they allow the institution to test response capabilities in real time. Evaluation of these activities is essential to understand the strengths and weaknesses of an institution's disaster response.

Evaluation is based on accurate observation. Accurate observation requires careful documentation of events before, during, and after a drill. Using a standardized observation and evaluation approach allows for a consistent record each time a drill occurs within an institution. A standardized approach helps both to capture the specific strengths and weaknesses of hospital responses during the drill and to draw comparisons across hospitals participating in similar drills. Using a standardized evaluation also allows comparison from one drill to the next to determine improvements in areas where weaknesses have been identified.

Drawing from the published literature on disaster drills as well as input from a variety of experts in the field, the Johns Hopkins University Evidence-based Practice Center (JHU-EPC) developed a set of evaluation modules and addendums for operationalized hospital disaster drills in 2004 entitled Evaluation of Hospital Disaster Drills: A Module-Based Approach.¹ This document includes substantial detail on evaluation of hospital disaster drills. In 2005, the Agency for Healthcare Research and Quality (AHRQ) requested that the JHU-EPC develop an abridged version of the original evaluation tool that focuses on the critical elements of drill evaluation that all hospitals should address as part of disaster preparedness.

This document describes the principles behind the approach used to develop the abbreviated evaluation modules and addendums and recommendations for their use.

Return to Contents

Purpose

The purpose of the abridged evaluation tool is to provide institutions with an instrument designed to capture the most critical aspects of disaster drill activities. In cases in which a comprehensive drill evaluation is desired, the 2004 document, Evaluation of Hospital Disaster Drills: A Module-Based Approach should be used.¹ The abridged disaster drill evaluation modules should be used to efficiently identify the most important strengths and weaknesses in hospital disaster drills. The results gained from evaluation then can be applied to further training and drill planning. Although the evaluation modules can be used to identify improvement in repeated drills, they are not intended to be used to make final or complete judgments about whether a hospital passes or fails in its planning and training endeavors. This approach is designed to identify specific weaknesses that can be targeted for improvement and to promote continued strengthening of hospital disaster preparedness. Systematized and standardized observations are essential to assess overall process improvement.

When an institution decides to use this abridged evaluation tool, it is strongly recommended that the original evaluation tool be reviewed prior to the drill, as the original tool provides a more complete discussion of this evaluation approach and more detailed evaluation questions.¹ The original tool can be found at http://www.ahrq.gov/research/hospdrills/hospdrill.htm.

Return to Contents

Guiding Principles in Developing the Evaluation Modules

In developing the 2004 document, Evaluation of Hospital Disaster Drills: A Module-Based Approach, the following principles were taken into account, and they continue to be relevant to the abridged evaluation tool.

Need for observing multiple hospital zones. Because evaluation of a disaster drill requires an understanding of drill activities in all areas of the hospital, the JHU-EPC developed an approach that allows evaluation of disaster response activity through documentation by trained observers in identifiable functional and geographic zones. To assist the hospitals in planning a drill, the JHU-EPC designed a Pre-drill Module to identify the active zones for the drill. The four zones identified in this approach are Incident Command, Decontamination, Triage, and Treatment. Each of these zones has a separate evaluation module. In addition, for exercises that include biological or radiation scenarios, a Biological Incident Addendum or a Radiological Incident Addendum is to be attached to each of the zone-specific forms. Table 1 illustrates the use of the modules and addendums for different scenarios.

Need for documentation of time points. Over-reliance on simple documentation of time points by a single observer detracts from the primary goal of the evaluation. Documenting the many simultaneous activities in the zone as a whole is necessary for an evaluation that identifies strengths and weaknesses of the hospital's response. The JHU-EPC addressed the issue of recording time points by designing the modules to document limited specific time points. These are listed in the first section of each zone module.

Documenting clinical care outcomes. The zone evaluation modules track the volume of victims in each zone and the adequacy of the provisions made for them, including space, staff, supplies, and other issues. The modules are not designed to collect individual victim level data.

Need for debriefing (after-action review). In addition to real-time observations, a comprehensive evaluation must include methods to obtain feedback from participants, including organizers, staff, and victims, at the end of the drill. This allows for discussion of issues that span more than one zone, for example, the effectiveness of communication between the incident command center and patient treatment areas. Thus, the JHU-EPC designed not only zone-specific modules that focus on issues that can be ascertained by an observer during the drill, but also a debriefing module that can be used to evaluate and integrate cross-zone issues at a post-drill debriefing session.

Return to Contents
Proceed to Next Section