This Web-Assisted Audioconference, broadcast on April 15, 2003, was the second in a series on bioterrorism conducted throughout 2003 via the World Wide Web and telephone. It was designed to address the issues and activities associated with the planning and conduct of disaster drills and on new tools being developed to assess health care facility and clinician readiness. The User Liaison Program (ULP) of the Agency for Healthcare Research and Quality (AHRQ) developed and sponsored the program.
Cindy DiBiasi: Good afternoon. Welcome to Disaster Planning Drills and Readiness Assessment. This is the second event in a series of Web-Assisted Audioconferences on bioterrorism and health system preparedness. These events are designed for State and local health policymakers and health systems decision-makers. This series is sponsored by the U.S. Department of Health and Human Services Agency for Healthcare Research and Quality, often referred to by the acronym "AHRQ" or AHRQ. My name is Cindy DiBiasi and I will be your moderator for today's session.
The context for these calls is clear and compelling. Bioterrorism represents a significant public health threat to the United States. Addressing this threat requires the rapid development of Federal, State and local capacity to respond to potential bioterrorism events. This means improving the abilities of both our public health and healthcare delivery systems to detect and respond to such threats. It also means ensuring that public health, health systems, facilities and providers communicate and coordinate effectively with one another as well as with other related systems such as emergency preparedness and law enforcement.
The intended audience for these calls are State and local health policymakers, program administrators and health system decision-makers, all play an essential role in these efforts. Within their own jurisdictions, regions or delivery systems they must develop capacity and coordinate efforts across public health, healthcare, law enforcement and related systems. It is therefore extremely important that they have information about emerging health services research, promising approaches and available tools that can assist them in the development of readiness plans.
In addition to today's event, three more calls will be conducted as part of this series. The next call, scheduled for June 17, will focus on surge capacity assessment and regionalization issues. The fourth call on October 21 will examine information and communication technology as well as surveillance and monitoring systems. The topic for the fifth call will be selected at a later date. I will tell you more about these calls later in the broadcast, but right now let's turn to today's call.
We will be discussing the development of new tools to assess how prepared healthcare facilities and clinicians are to respond to a bioterrorism event. We will also look at issues and activities associated with the planning and conduct of disaster drills.
Let me begin by introducing today's panelists. In the studio with me I have Sue Skidmore, a nurse and an associate at Booz Allen Hamilton. Dr. Howard Levitan, an emergency physician and consultant from Disaster Planning International. Dr. Gary Green, an associate professor of emergency medicine and pathology at the Johns Hopkins University School of Medicine. And Mary Massey, disaster coordinator at Anaheim Memorial Medical Center in Anaheim, California. Welcome everyone.
Before we begin our discussion I would like to tell the audience a bit about the format of this audio conference. First we will talk with our four panelists, then open the lines to take your questions. We will give instructions on how to send these questions to us later on in the program.
In the meantime, if you experience any Web-related technical difficulty during this event, please click the "help" button in your window to troubleshoot your Web connection. If it appears that the slides are not advancing, you may need to restart your browser and log on again. If you are on the phone, dial "*0" to be connected to technical assistance. Also if you have any difficulty with the audio stream or if there is an uncomfortable lag time between the streamed audio and slide presentation, we encourage you to access the audio via your phone. The number is 1-888-496-6261. Give the password "bioterrorism" to be connected to the call.
Now I think we are ready to discuss today's topic, Improving Health Systems Preparedness Through Readiness Assessment and Disaster Planning Drills. Let's begin with Sue Skidmore from Booz Allen Hamilton and Dr. Howard Levitan from Disaster Planning International.
During the first phase of their AHRQ-sponsored project, Sue and Howard developed a questionnaire to help hospitals and health systems assess their level of preparedness and their capacity to respond to bioterrorist attacks. Howard, can you tell us more about this initial questionnaire and why was it developed and what the most important findings from the assessment were?
Dr. Howard Levitan: Sure. The initial questionnaire was developed for several reasons. The first was to begin assessing hospitals' readiness for bioterrorism based on newly developed benchmarks for preparedness. Basically we wanted to see where hospitals were in their planning process based on some preconceived norms. Second was to gather information so we could begin gauging the cost of preparedness and finally it was to develop and collect data that could be used for computer modeling purposes.
During the project's first stage, we assessed four hospitals to determine their readiness for a bioterrorist incident. Two of the hospitals were in Georgia and two were in Maryland. Several findings really stood out. In general, it seemed that the hospitals were working to improve their readiness, but each one seemed to be focusing their efforts on different areas that they thought was applicable to bioterrorism. I think this occurred because there was really no nationally accepted benchmarks of adequate preparedness on which to gauge or direct their readiness efforts. In addition, there was no mechanism in place to share best practices or useful solutions between hospitals. Most of the bioterrorism plans that we reviewed to develop our assessment tool placed the majority of their emphasis on the management of specific biological agents rather than really addressing a number of regional and planning issues. Such issues as hospital staffing; how to increase inpatient and outpatient treatment capacity; the stockpiling of resources; internal and external communications; security, which is always essential in these events; the ability to provide mass prophylaxis and vaccinations; tracking of local and regional bed utilizations; the problem of emergency department overcrowding and the interaction with the EMS services.
Finally, we found out that really across the board administrative support was key to bioterrorism readiness as well as preparedness in general.
Cindy DiBiasi: Sue, how does this new questionnaire project differ from the original one?
Sue Skidmore: The addition will be developing the objectives measures that users need in order to interpret the results of the questionnaire and later in the project will develop some planning templates that will be useful to facilities and systems and also identify best practices along that preparedness continuum.
Cindy DiBiasi: It seems that there are a lot of different assessment tools and preparedness questionnaires available throughout the country. Why do you think there is so much variation?
Dr. Howard Levitan: That is a good question. You are correct, there are a number of assessment tools available. We actually reviewed over 12 prior to the development of our first assessment tool for AHRQ. I think it has occurred for several reasons.
First, all 50 States as well as several locales were tasked to assess their readiness for funding purposes, and as a result either developed their own assessment tool or simply modified an existing one to reflect their unique local and regional issues.
Secondly, a number of State and Federal grants were also awarded to develop an assessment tool. As a result, a number of assessment tools were developed to gather data on readiness. Yet there are still no established objective measures of bioterrorism preparedness for hospitals nor was there any generally accepted attack scenarios that hospitals could use as a basis for their preparedness. They have prepared for 20 patients, they prepared for 100 patients. We just don't know.
In addition, most of the assessment tools have not been evaluated for validity or in other words evaluated to determine if the questions that they ask or the information sought are true predictors of preparedness. In my opinion this aspect alone is the biggest challenge and one of the most important components of this project.
Finally, most of the assessment tools failed to obtain any buy-in from various government agencies, national organizations or regulatory bodies.
Cindy DiBiasi: Well since many of the tools out there probably have not been evaluated to see if they are collecting the information expected or if buy-in has been obtained from key groups, how are you approaching the development of your assessment tools to ensure that you are collecting the right information and getting the buy-in you need?
Sue Skidmore: What we did in this project is we took our original questionnaire and we then took questionnaires from across the country that were either very well known, very comprehensive, well done, or had some other reason why they were used in a specific area. We took all of those and pitted them against each other and assessed the questions, categorized the questions and rated them in terms of whether it was pertinent to hospital disaster planning in general or whether it was relevant specifically to bioterrorism. We weeded everything out and focused just on bioterrorism. Then we brought in a working group that was composed of those various government agencies and national organizations and regulatory bodies and we asked them to walk through all of the categories and questions with us and to really help us focus on that minimal level of preparedness.
Finally, we engaged a market and national survey research firm who has experience and is focused on just developing healthcare surveys and questionnaires in order to help us really focus the questionnaire, the question design on content, clarity, design and administration in order to make sure the questions we asked were tight and got the information we asked.
Cindy DiBiasi: Let's talk a bit about that. How are you specifically going to redesign your questionnaire to make it easy to use but yet still specific enough so that it addresses the issues that are most relevant to hospital bioterrorism preparedness?
Sue Skidmore: The biggest challenge in the questionnaire design is to craft questions that allow users to get the information they need without making it too long or too cumbersome.
In addition, the HRSA grantees need a questionnaire that closely follows the priority areas that they are being asked to benchmark themselves in. So we are keeping our categories consistent with the HRSA priority areas and in addition, we looked at the question design itself and we chose only questions that only focused on issues that are under the responsibility and control of the hospital leadership. Questions that pertained to the unique response requirement of a bioterrorism event and that did not duplicate the generic mass-casualty protocols of an existing facilities emergency management plan and we limited regional issues to those activities involving the hospital participation and role. Questions were redesigned for benchmarking purposes in that the answers would not be open-ended answers; there would be less data collection, they would be more scaled so we have "no we don't do this at all", "yes we do this a little" and "oh boy, we have a best practice over here" so this is scaled along the continuum.
Cindy DiBiasi: Howard, in your opinion, what makes one hospital better prepared for bioterrorism compared to another?
Dr. Howard Levitan: Well, that is the big unknown and the challenge that AHRQ has tasked us with. However, instead of really focusing on comparing one hospital to another, we sought to determine the essential planning issues that ever hospital must address.
During phase one we began the process by developing a list of preparedness criteria based on routine joint commission requirements, lessons learned from natural and man-made disasters that we gleaned from the literature and from the opinions of our working group who represented really most of the major healthcare organizations in the United States. Our objective now is to reach consensus on identifying the critical components of bioterrorism preparedness, then on establishing indicators of preparedness along a continuum rather than really delineating good, better or best preparedness.
Cindy DiBiasi: What areas does your questionnaire focus on, Sue?
Sue Skidmore: After reviewing all of the tools and questionnaires that we compared our questions against, a dozen or more, we came up with twelve main categories, regional linkages, bioterrorism planning and structure, training and exercise, triage diagnosis and treatment and so on. We found that they are actually closely aligned or will ultimately align with the HRSA priority areas.
The current timeline for the questionnaire is that the revised version will be ready to pilot near the beginning of June and the plan is that a small sampling of HRSA grantees will be asked to volunteer to participate in the pilot questionnaire. Data and feedback will be obtained and used to make further revisions and then the final questionnaire will be delivered to AHRQ.
Cindy DiBiasi: OK. Thank you Sue and Howard. We will get back to you with some questions.
But first I would like to turn now to Dr. Gary Green from the Johns Hopkins University Evidence-based Practice Center. Gary, I understand that you are part of a group that has been working on two AHRQ-sponsored projects and they are evaluating the existing evidence about healthcare worker and institutional preparedness for bioterrorist events.
Tell me, do you think there is adequate evidence out there concerning effective planning and preparedness efforts to guide those responsible for hospital disaster planning or does every hospital and health system have to come up with their own method for disaster drill planning and evaluation?
Dr. Gary Green: There is some evidence out there to guide us, but it is extrapolated information based on the evidence in the general disaster response literature. Our Evidence-based Practice Center's first evidence-based report on this topic examined the existing literature on training clinicians for a public health event relevant to bioterrorism preparedness. The evidence report was released last year and is now available on the AHRQ Website.
A second project, which is not yet complete, examines, evaluates and summarizes the current evidence specifically concerning hospital (unclear). Without going into detail, I can tell you that this is very much a field in infancy and there really are not yet any high quality scientifically-based, peer reviewed publications addressing either disaster planning or disaster drills specifically related to bioterrorist events.
The good news is we did find that there is a rapidly growing body of evidence concerning disaster preparedness in general and that much of this literature is also applicable to bioterrorism preparedness.
Cindy DiBiasi: Gary, specifically what does the literature tell us?
Dr. Gary Green: Well, general disaster preparedness literature is also in a relatively early stage. However, there is some limited conclusions that can be drawn from the existing literature that the Evidence-based Practice Center has used.
First, the basic components of an adequate disaster response system have been defined. The basic steps needed for disaster preparedness capacity building have been established. In addition, a variety of training and assessment techniques have been described and have already been successfully used. However, the terminology in the field has not yet been standardized nor have the best techniques and practices been established. For example, even the term "disaster drill" means different things to different people. Some use the term to describe all disaster simulation including tabletop and computer simulation of disaster events, while others use the term in a more specific way only referring to activities if the event is physically operationalized and mock patients are used.
Cindy DiBiasi: What steps should a hospital or health system take to improve their disaster response preparedness?
Dr. Gary Green: I will start by saying that there is no cookbook recipe that every institution can follow that will lead to the product of adequate preparedness. However, based on existing literature and based on my own experience, I can outline a basic framework that will help each institution determine what they need to do given their individual environments and resources.
One way to look at this process for disaster preparedness capacity building is to consider how similar it is to a continuous quality improvement initiative. One of the principles of CQI is that the project is never completed. In other words, no institution ever reaches a total State of preparedness for any disaster. Rather disaster preparedness efforts should be viewed as a continuous cycle.
The basic steps within the cycle are as follows. First, an interdisciplinary team has to be assembled. That includes all the key stakeholders. That team will then review the current resources as well as the strengths and weaknesses of their individual health system with regards to disasters. That will include both looking at the history of previous disasters that that institution has responded to and doing an objective assessment of that response as well as trying to do an objective assessment of the risk of future events based on the available evidence.
The next step would be to develop a detailed written disaster response plan. To this end, to disseminate and practice that plan, disseminate the plan throughout the institution and practice it with all the staff through both educational interventions and through disaster drills.
As your education and training progresses, you then have to evaluate the knowledge and skills of the staff using objective data gathering techniques and that will lead you to a determination of the specific strengths and weaknesses of the staff, of the disaster plan and the training techniques. Based on that, your next step is to reengineer the disaster plan based on that data and then go back and modify your educational interventions and training to better target those areas of weaknesses.
As seen in this slide that is then a continuously repeating cycle with training, evaluation and reengineering. This cycle can be applied at the institutional level or the departmental level or the regional level. Of course the length of the cycle can vary according to the needs of the individual institution.
Cindy DiBiasi: What are the differences between preparing for conventional disasters and preparing for bioterrorism events?
Dr. Gary Green: As illustrated in the next diagram, preparedness for conventional disasters is really the foundation for bioterrorism preparedness. You cannot be prepared for a bioterrorist event or another non-conventional attack or event without first being prepared for a conventional disaster. This is because most of the knowledge, the skills and the resources needed are the same. Also, healthcare institutions are likely to be called on to respond to conventional disasters far more often than non-conventional disasters. Therefore, putting all the pieces in place for general disaster preparedness is the first and most important step for preparedness for responding to a bioterrorist event.
So, in this schematic, we can see that the solid circles represent the response to a conventional attack, either at the hospital level or in pre-hospital level, whereas the dotted lines surrounding them represents the additional activities and preparedness efforts necessary to respond to an attack or an event based on a biological agent or radiation or chemical event. Also included is the relationship between the pre-hospital team and the hospital response and how an Incident Command System really is important and effective throughout all of these different components.
Cindy DiBiasi: You said earlier that the basic building blocks of a disaster response system have been defined. Can you elaborate a little bit on that?
Dr. Gary Green: Yes Cindy. The fact is that to many of us disaster response is by its very nature complex and confusing. One of the reasons that the task of preparing an institution for such an event seems to be so overwhelming is that it is very difficult to break down into manageable components. The many simultaneous activities necessary to respond to a disaster all have to happen simultaneously and have to be intricately coordinated. To deal with this, there have been several attempts made by a variety of organizations and groups to come up with a basic structure for a disaster preparedness effort and to break down disaster response activity into more clearly defined and understandable components.
As I mentioned earlier, there is still no agreement on the exact terminology and different groups stress somewhat different boundaries around the various response components. But the basic building blocks are more or less agreed upon. In a disaster, each one of those components must be actively managed by a designated commander according to a pre-determined action plan.
Perhaps the most important component is the Incident Command System. This is also referred to as an Incident Management System or some people still use the term "Command and Control." To a large extent, this component defines the boundaries and the functions of the other response systems.
Another important component is system integration, which is absolutely key and largely refers to communication functions. Both communication functions within institutions as well as communication between the institution and outside agencies.
Logistics refers to supply and management of materials, facilities as well as transportation of patients, providers and materials. Security, as mentioned earlier, is critical to all the other components in order to keep the other components working the way they are planned to. Clinical care, human resources, public relations of course each have very specific requirements that have to be addressed as well. Then individual institutions may also categorize other components according to their needs.
Cindy DiBiasi: Gary, what do we know about effective techniques for training healthcare workers to be prepared for disaster?
Dr. Gary Green: Well, based on our EPC's first evidence report on this topic we know that a variety of techniques have been used to train both clinicians and other members of the healthcare system in disaster preparedness. Unfortunately, very few of any of the reports about this technique applied scientifically rigorous evaluation methods so there is really only very limited data concerning which of these techniques has been successful.
There is also almost no objective data about the relative cost of the various techniques. The training techniques that have been prepared, that have been reported in the literature, include the traditional educational techniques that we are all familiar with, lectures, discussions and use of audio-visual aids and written materials.
Standardized or smart patients. Standardized patients or use of smart patients refers to training healthcare workers to be mock victims. That way during an interaction with providers, mock victims can evaluate and teach the providers as they are treated. The use of smart patients has been found to be very acceptable to physicians and it is an effective technique for one-on-one training. However, it is questionable whether it is useful for training large numbers of people. It may be very cost ineffective; it is unclear whether the cost would be prohibitive to train a large number.
Teleconferencing, the satellite broadcasting, has also been used. Reports of using this as an educational technique for disasters has been recorded in the literature. This seems to be very useful for reaching large numbers of trainees and seems to be as effective as traditional classroom learning based on what evidence there is out there.
Tabletop exercises is another technique that is in use. It seems to be used quite commonly. A tabletop exercise is a sort of theoretical drill without physically operationalizing patients or equipment or personnel movement. It is usually sort of focusing on the Incident Command System and system integration components. It has been successfully applied to teaching clinicians bioterrorism preparedness specifically. It seems to be a very useful technique although this technique might be best used when combined with other techniques as part of a comprehensive plan.
Computer simulation is something that there is a lot of buzz about, but there has really only been one or two evaluations of computer simulations have been recorded in the literature. There is very limited data out there. However, the authors of those papers have suggested that it may be able to replace expensive drills and also might allow identification of deficiencies in disaster planning and implementation.
Then of course there is what we all usually all refer to as disaster drills. Disaster drills are the most generally accepted training tool and they are really sort of at the cornerstone of disaster preparedness efforts. There is very significant collective experience in their use. Unfortunately as mentioned before, there is very little standardization and very limited data concerning objective evaluation of disaster drills. Drills have been shown to improve knowledge of the disaster plan and they have also been successfully used to identify problems and plan execution.
Cindy DiBiasi: Gary, we are going to get to come back to you for questions as well. But before we do that, we have been talking a lot about disaster planning drills and what the literature tells us about them. Let's first go to Mary Massey, the disaster coordinator at Anaheim Memorial Medical Center, who has been at the center of many drills. Mary, if you could share some of the insights that you have gained from being at the ground level. My understanding is that you and your colleagues at Anaheim Memorial Medical Center have been involved in numerous disaster drills.
Mary Massey: We have been holding disaster drills for over 20 years. Home of some high-venue sights like Disneyland, Arrowhead Pond, Edison Field, Anaheim Convention Center, as well as our share of earthquakes and fires, we found that it is important as a community that we have been practicing these drills for the different scenarios.
We have been practicing it at different levels too. Not only department-specific type drills, but hospital-wide ones. We move on to community-wide ones. We are an MMRS city, Metropolitan Medical Response System city. There are three of them within Orange County, Anaheim, Santa Ana and Huntington Beach, where the government came in and helped us prepare for bioterrorism. At a minimum we have to be prepared for 2,000 chemical victims and 10,000 biological ones. Going larger than that, we practice in a net, our county is divided into separate nets, that practice a large, community-wide drill once a year. Our net has 17 hospitals and several provider agencies, police and fire.
Then a step above that is the annual Statewide drills. This past year we did a radiation drill, which was a new one for us. We found the change shifting the last few years from more manmade-type disasters up into terrorism-type focuses. So we have found that we have had to learn how to deal with mass-casualty decon, how to handle fear and large levels of the worried well.
Cindy DiBiasi: Based on that 20 years of experience, what would you say the key lessons are in getting a community started in planning a drill?
Mary Massey: The first thing, it sounds kind of simple, but you need to figure out what you are trying to do. What is the objective you are trying to answer in this drill and then figure out a way to measure it. So the way we do it is we make a universal critique for everyone that measures what our objectives are we stated. Then we bring all the players together. A community-wide drill requires law enforcement, fire, EMS, public health, DWP, hospitals, other healthcare organizations and the vital industry. Sometimes I think we forget who is in our community. We have a high tourism industry in Anaheim so we need to involve them in it too. This helps you build the relationships that will lead to trust and cooperation in collaboration.
Cindy DiBiasi: That seems like a lot of relationships to manage. Is everybody getting along?
Mary Massey: We have been doing this for so long in Orange County that we have built really good relationships between the different agencies. Not that we are always perfect; there is always progress to be made, but we realize that this isn't always the case. Don't let long-standing differences or personalities block working together. It isn't a battle to see which agency is the best; it is a battle to prepare your community to handle your own citizens. It might by your own agency personnel or your family. The better prepared you and your competitors are, the better chance of survival.
Cindy DiBiasi: So once you bring everybody together and you build the relationships, then what?
Mary Massey: Then we need to develop a community-wide plan. We have all worked on having our own agency's emergency plans, but we need to have a plan that works well when we are working with the other agencies. (Unclear) requires disaster plans and testing twice a year for hospitals. What we need to do now is develop an integrated response.
Some of the major points are definitions of roles and responsibilities. We use HEICS or "HIKES", depending on where you are in the country for pronunciation, equipment and supplies and communication. This is a good time to test your redundant communications because you don't know what you will be able to use when an actual disaster strikes. Some examples are phones, cell phones, walkie-talkies, and don't forget the low-tech paper, pencil and runners.