Chapter  95:  Training of Hospital Staff to Respond to a Mass Casualty Incident

Introduction

Disaster scenarios that once seemed merely theoretical have become a disturbing reality. The emergence of state-sponsored terrorism, proliferation of chemical and biological agents, availability of materials and scientific weapons expertise, and recent increases in less discriminate attacks all point toward a growing threat of a mass casualty incident (MCI). Governmental agencies, healthcare professionals, and public health advocates have sought to determine the best ways to mitigate the potential impact of an MCI that results in multiple casualties that overwhelm local resources and that may include natural, biological, chemical, nuclear, or other agents.

Hospital disaster preparedness has therefore taken on increased importance at local, state, and federal levels. Hospitals themselves are taking renewed interest in disaster preparedness, reexamining their disaster plans, and conducting disaster exercises. Preparing for MCIs is a daunting task, as unique issues must be considered with each type of event. For example, the systemic stress of a biothreat is entirely different from that of a chemical disaster or any other acute onset disaster. These differences hold challenging implications for preparedness training.

Hospitals must play a key role in developing disaster preparedness plans, and they need to coordinate efforts with public health systems and appropriate governmental agencies. The Joint Commission on Accreditation of Healthcare Organizations (JCAHO) actually requires hospitals to test their emergency plan twice a year, including at least one community-wide drill. ¹ However, it is not known whether this type of training is effective. The current evidence report updates the evidence report Training of Clinicians for Public Health Events Relevant to Bioterrorism Preparedness ² and focuses specifically on the effectiveness of hospital disaster drills, computer simulations, and tabletop or other exercises in training hospital staff to respond to an MCI. The following key questions were addressed:

• What is the effectiveness of hospital disaster drills in training hospital staff to respond to an MCI?

• What is the effectiveness of computer simulations in training hospital staff to respond to an MCI?

• What is the effectiveness of tabletop or other exercises in training hospital staff to respond to an MCI?

• What methods or tools have been used to evaluate the effectiveness of hospital disaster drills, computer simulations, and tabletop exercises or other exercises in training hospital staff to respond to an MCI?

Methods

Results

The literature search process identified 243 unique, potentially relevant citations, of which 208 were excluded at abstract review. Twenty-one of the remaining 35 articles were deemed eligible for data abstraction. Sixteen of these studies addressed the effectiveness of hospital disaster drills in training hospital staff to respond to an MCI (key question 1); ^3–18 one study addressed the effectiveness of computer simulations in training hospital staff to respond to an MCI (key question 2); ¹⁹ and four studies addressed the effectiveness of tabletop or other exercises in training hospital staff to respond to an MCI (key question 3). ^20–23

The reviewed studies represented a heterogeneous body of literature. They ranged from descriptions of local drills, including transportation incidents, fires, and radiological exposures, to sophisticated telecommunication exercises, such as a large regional drill involving multiple agencies. ²² Studies also varied in terms of targeted staff, learning objectives, identified outcomes, and evaluation methods. Because of the wide range of foci for the studies, it was difficult to draw definitive conclusions about the most effective approaches for training hospital staff to respond to an MCI. However, some potentially valuable points could be identified in the literature:

• internal and external communications were the key to effective disaster response;

• a well-defined incident command center reduced confusion; conference calls were an inefficient way to manage disaster response;

• accurate phone numbers for key players were vital and regular updating was necessary;

• disaster drills appeared to be an effective way to improve clinicians' knowledge of hospital disaster procedures;

• computer simulation may be an economical method to educate key hospital decisionmakers and improve hospital disaster preparedness before implementation of a full-scale drill;

• a tabletop exercise can help to motivate hospital staff to learn more about disaster preparedness and can help to teach staff about aspects of disaster-related patient care in a way that simulates the practice setting;

• a regional exercise involving top government officials can help to increase awareness of the need for better disaster response planning;

• video demonstrations may be an inexpensive, convenient way to educate a large number of staff about disaster procedures and equipment use in a short time.

Nineteen studies included specific evaluation methods (key question 4), and 13 of these used more than one type of evaluation method. ^{4, 5, 7, 8, 10, 13, 14, 16–19, 22, 23} Group or individual debriefings were the most common, ^{5, 7–10, 12, 14, 17–20, 22} followed by “smart” observers (medical personnel). ^{4, 5, 7, 8, 10, 17} Other observers^{4–6, 8, 19, 23} and trained “smart” casualties^{4, 13, 14} were also used in several studies. Four studies used a written exam. ^{14, 16, 21, 23} Other methods of evaluating the educational intervention included

• individual interviews, ²²

• inspection and review by chemical spill specialists, ⁴

• self-assessment, ²³

• a computer-generated detailed picture of the situation, ¹⁹

• observer checklists, ¹⁰

• mock disaster patient charts, ¹¹

• victim tracking cards, ¹³

• videotaping. ¹⁶

Discussion

Hospital disaster drills, computer simulations, and tabletop and other exercises are designed to test the hospital's disaster plan and to allow employees to become familiar with disaster procedures. Based on the review of the literature, discussion with experts, and analysis of disaster response plans, ²⁴ the EPC team identified several important aspects of hospital disaster response that may be useful to evaluate. Most of the lessons learned relate to one or more of the following aspects of disaster response:

• the incident command system; communications (both internal and external);

• clinical care, including triage, patient care, patient flow, and patient tracking;

• security;

• materials and resources;

• decontamination.

Purpose of This Evidence Report

In 2000, the Agency for Healthcare Research and Quality (AHRQ) awarded the Johns Hopkins University (JHU) Evidence-based Practice Center (EPC) a task order to develop the evidence report “Training of Clinicians for Public Health Events Relevant to Bioterrorism Preparedness” as part of the Agency's bioterrorism preparedness initiative. The report was requested by AHRQ's Center for Primary Care Research. The AHRQ published the evidence report, in print and on its website, in 2002.^{8, 9}

The current evidence report updates the previous report, focusing specifically on the effectiveness of hospital disaster drills, computer simulations, and tabletop or other exercises in training hospital staff to respond to an MCI. For the purpose of this report, hospital staff refers to all levels of individuals employed by the hospital, and an MCI is defined as an incident that results in multiple casualties that overwhelm local resources and that may involve natural, biological, chemical, nuclear, or other agents. This report also reviews evidence concerning the methods or tools that have been used to evaluate the effectiveness of these training activities.

By synthesizing the existing evidence on the training of hospital staff to respond to an MCI and by determining strategies most likely to work effectively, this report will provide direction for future training of staff in hospital disaster preparedness. The premise is that a review of published literature will help hospital leaders in their efforts to formulate best practices. Because some training programs for disaster preparedness, including those carried out by the military, may not be published, it would be a daunting task to identify such programs and obtain meaningful evaluations of them. Such an undertaking is beyond the scope of this review of evidence.

Recruitment of Experts

The JHU EPC team identified 12 experts to provide input at key points during the project (see Appendix A). These included two representatives of relevant professional organizations, two experts representing government agencies, and eight experts from academic settings. The experts participated in the task of refining the key questions (see Identifying the Specific Questions, below), and they also reviewed the draft report (see Peer Review Process, below).

Target Population

The target population addressed in this evidence report consisted of hospital staff who participated in an educational intervention related to MCI response. For the purpose of this report, hospital staff included all clinical, non-clinical, and administrative staff.

Identifying the Specific Questions

The Health Resources and Services Administration (HRSA) developed the initial list of questions to be addressed. The EPC team refined the original questions through analysis of preliminary literature searches and input from the experts.

Analytic Framework

Figure 1. Analytic framework depicting the hospital's (more...)

   Figure 1. Analytic framework depicting the hospital's role in responding to an MCI

Figure 1

Literature Search Methods

The literature search consisted of several steps, including identifying sources, formulating a search strategy for each source, and executing and documenting each search.

Sources

Several literature sources were used to identify all studies potentially relevant to the key questions. Both electronic database searching and hand searching were completed. Six electronic databases were searched. The databases included PubMed^®, the Cochrane Central Register of Controlled Trials (CENTRAL), the Excerpta Medica database (EMBASE), the Educational Research Information Clearinghouse (ERIC), the specialized register of the Effective Practice and Organization of Care Cochrane Review Group (EPOC), and the Research and Development Resource Base in Continuing Medical Education (RDRB/CME). The electronic searches were conducted in December 2002, with an updated search of PubMed in February 2003, and no restrictions based on publication date were used.

EPC team members also hand searched the literature to ensure comprehensiveness. Team members reviewed the reference lists of relevant reviews, reference papers, and the eligible articles. Team members also hand searched the most recent issues of journals (through January 2003) frequently identified by the electronic search and/or identified as high priority by the team (see Appendix B).

Search Terms and Strategies

The search strategies were designed to maximize sensitivity and were developed in consultation with team members. Key articles were identified from the previous EPC project.⁸ Using these key articles determined to be eligible for review, search strategies were developed and refined in an iterative process. A strategy was first developed for PubMed^® and modified for use in the other electronic databases. The strategy used text words and controlled vocabulary words, such as mass casualty, disaster, disaster planning, and drill. All electronic database search strategies are included in Appendix C.

Organization and Tracking of Literature Search

The results of the searches were downloaded from electronic sources whenever possible or, if necessary, manually entered into a ProCite^® database (ProCite, ISI ResearchSoft, Berkeley, CA). The ProCite^® database was used to store citations and track search strategies and sources. The software was also used to track the abstract review process.

Abstract Review Process

Two members of the study team independently reviewed each abstract identified by the search. For each team of reviewers, one reviewer had training in emergency medicine and one had training in epidemiology and research methods. Team members applied the following criteria to exclude articles from further consideration:

1. not written in English;

2. did not include human data;

3. no original data;

4. meeting abstract (no full article for review);

5. did not include hospital staff;

6. did not include response to an MCI or a disaster;

7. did not include training or education;

8. no evaluation of the training or education; or

9. did not apply to any of the key questions

A copy of the abstract review form is included in Appendix D. Disagreements about the eligibility of an article were adjudicated by consensus.

Qualitative and Quantitative Data Abstraction

The EPC team developed and pilot tested two article review forms. The quality assessment form and the content abstraction form are included in Appendix D.

The quality assessment form asked questions designed to address study quality. The following areas were examined: representativeness of the targeted hospital staff, bias and confounding, description of the intervention, assessment of outcomes, and statistical quality and interpretation. The items in these categories were derived from study quality forms used in previous JHU EPC projects.^{8, 11} Items were modified to fit a focus on teaching strategies based on published criteria for evaluating an educational program.¹² The study team assigned each response level a score of zero (criteria not met), one (criteria partially met), or two (criteria fully met). The score for each category of study quality was the percentage of the total points available in each category and therefore could range from zero to 100 percent. The overall quality score was the average of the five categorical scores.

The content abstraction form was designed to collect such information as the description of the participants, the geographic location, the type of MCI, the training intervention, hospital staff targeted, and the hospital departments and other entities involved. The form also included items on the objectives of the training and the training evaluation methods. We classified objectives as knowledge, skills, behaviors, and clinical outcomes. On the form, we grouped outcomes and main conclusions of the drill by the target area involved in the exercise (e.g., incident command system, internal/external communications, patient flow and tracking, and security). We developed this grouping on the basis of discussions with experts and initial article review. This approach is consistent with the content of the job action sheets of the Hospital Emergency Incident Command System (HEICS), developed to assist the operation of a medical facility in time of crisis.¹⁰ Many hospitals have adopted the HEICS system as they develop disaster response systems.

Article Review Process

The EPC team conducted the article review in a serial fashion. The first reviewer completed the quality assessment form and the content abstraction form. The second reviewer then reviewed the article and checked each response on the forms. Any disagreements between the two reviewers were resolved by consensus. Reviewers were not masked to author or journal names because to do so is both costly and time-consuming, and previous work has shown that masking is unlikely to make a significant difference in the results of the review.¹³

Evidence Grading

For each question, the EPC team assigned evidence grades based on an established grading scheme with well-defined levels of evidence. The grading scheme, used in previous systematic reviews,^{14, 15} assigns grades as follows:

Grade A (strong): Appropriate data available for evaluating the outcomes of the training program, including at least one well-done randomized controlled trial; the population of learners is sufficiently large and well described, and adequate controls have been used; data are consistent; and the educational intervention is well described and one intervention is clearly superior, equivalent, or inferior to another for well-defined outcomes.

Grade B (Moderate): Appropriate data available for evaluating the outcomes of the training program; the population of learners is sufficiently large and well described, and adequate controls have been used; data are reasonably but not entirely consistent; and the educational intervention is well described and one intervention is superior or equivalent for well-defined outcomes, but there is insufficient evidence to make a definite conclusion of superiority of one approach over another.

Grade C (Weak): Some data for evaluating the educational intervention is available; the population is adequately large but poorly defined; there may be a trend for preference of one intervention over another for well-defined outcomes, but there is insufficient evidence to draw firm conclusions of superiority.

Grade I (Insufficient): Appropriate data not available, or there is an insufficient number of trainees to assess the intervention either alone or in comparison with alternatives.

Evidence Tables

Evidence tables were constructed to present the information addressing each key question. The evidence tables summarize the basic characteristics of each study, study quality, and results of the studies. Within each evidence table, studies are listed by type of training (i.e., hospital disaster drill, computer simulation, or tabletop and other exercises). The evidence tables are included in Appendix E.

Peer Review Process

The draft evidence report was sent to the 12 experts for peer review. Experts were asked to comment on the content of specific sections of the report according to their areas of expertise and interest. The EPC team addressed the reviewers' comments in the final report and submitted a detailed summary of the comments and responses to the AHRQ.

Literature Search and Abstract Review

The literature search process identified 243 unique, potentially relevant citations. Appendix F provides a summary of the results of the literature search and review process.

Two hundred eight articles (86 percent) were excluded from further consideration during the abstract review process. The following were grounds for exclusion: did not include hospital staff; no training or education; no original data; no evaluation; did not include a response to an MCI or a disaster; abstract only; or did not apply to any of the key questions.

Article Review

Of the 35 articles deemed eligible through abstract review, 21 (58 percent) were eligible for data abstraction. The remaining 14 articles were excluded for the following reasons: did not include hospital staff; no original data; no training or education; or did not apply to any of the key questions (see Appendix G). We were unable to locate one article (which was identified through hand searching) because of an incorrect or incomplete citation. All articles reviewed and referenced are listed in Appendix H.

Focus and Design of the Reviewed Studies

Sixteen of these studies addressed the effectiveness of hospital disaster drills in training hospital staff to respond to an MCI (key question 1);^{16–26, 28, 29, 32, 33, 35} one study addressed the effectiveness of computer simulations in training hospital staff to respond to an MCI (key question 2);³¹ and four studies addressed the effectiveness of tabletop or other exercises in training hospital staff to respond to an MCI (key question 3).^{27, 30, 34, 36} Nineteen studies described methods or tools that have been used to evaluate the effectiveness of hospital disaster drills, computer simulations, and tabletop or other exercises in training hospital staff (key question 4).^18–36

Target of the Training

Quality Scores

Results of Key Questions

Question 1: What is the effectiveness of hospital disaster drills in training hospital staff to respond to an MCI?

Table 1. Summary of Results: Incident Command System (more...)

Table 1. Summary of Results: Incident Command System

Author, year Type of training	Incident Command System
Baughman, 1990 Hospital fire and explosion in the emergency department	Confusion resulted because no single person was designated as incident commander.
Cook, 1990 Transportation accident	The drill led to less confusion in incident command.
Gretenkort, 2002 Hospital fire	The leadership concept of the Coordinating Physician of the hospital working together with other hospital executives and the incident commander proved effective.
Inglesby, 2001 Biological	It was unclear how to coordinate different operation centers set up by a variety of state and federal emergency management offices. Personnel were not familiar with language used in disaster control. Leadership roles and authorities in the crisis were uncertain; it was not clear who was in charge.
Lau, 1997 Transportation accident	The disaster plan activated successfully.
Menczer, 1968 Incendiary device and boiler explosion	No overall leader was recognized to coordinate services and agencies. No medical authority was at the scene.
Weston, 1988 Hospital fire in operating room	The absence of a senior hospital nursing officer led to command confusion. Incident flow charts were needed.

Table 4. Summary of Results: Security, Materials and (more...)

Table 4. Summary of Results: Security, Materials and Resources, Decontamination, and Other

Author, year Type of training	Security	Materials/Resource	Decontamination	Other Findings
Baughman, 1990 Hospital fire and explosion in the emergency department	Security informed fire department of situation.	Disaster charts were not available because stored in the emergency department.	Not addressed.	Not addressed.
Chobin, 1989 Chemical	Not addressed.	Fire department arrived on scene in full gear: material safety data sheet and breathing apparatus were used.	Not addressed.	Not addressed.
Classic, 2000 Radiational	Security of building and perimeter exceeded the standards.	Not addressed.	County Hazardous Materials Response Team required an hour to set up the portable decontamination facility. Only one nurse had been released to prepare the hospital decontamination facility, an activity that requires at least two people. Victims found dead on the scene were not released to local funeral homes until they had first gone to the autopsy laboratory, allowing control of the body in the event it is contaminated. County response team set up a portable decontamination facility outside the perimeter. Bodies should not be released until decontamination is done.	Disaster response personnel needed special identification. Bioassay specimen collection was needed for those with radiation exposure. Access to facilities may not occur within regulatory time frames.
Cook, 1990 Transportation accident	Not addressed.	Not addressed.	Not addressed.	At least one person in each department now has an in-depth understanding post-drill. Staff stress levels were more manageable with game approach.
Gofrit, 1997 Not specified	Not addressed.	There was a shortage of ventilators and other trauma care equipment resulting from failure to report from in-hospital storage to emergency department.	Not addressed.	Not addressed.
Gray, 1996Transportation accident	Not addressed.	Not addressed.	Not addressed.	In a video describing the use of a control room, a staff reporting station, field equipment, and protective clothing in a transportation accident, video viewers retained information significantly better than those who had read the disaster plan (72% versus 45%, p<0.01).
Gretenkort, 2002 Hospital fire	Not addressed.	Not addressed.	Not addressed.	Preparation of patient collection points did not meet the needs for the actual number of patients.
Halstead, 1993 Hospital fire in operating room	Not addressed.	Gas levers were difficult to find. Fire door in back corridor did not close. A second water hose was needed.	Not addressed.	A secondary program taught the operating room staff how to use fire extinguishers.
Inglesby, 2001 Biological	There were concerns about ability of security to create an effective “security lock-down”.	Antibiotic supplies were exhausted early in the exercise and antibiotic distribution was logistically difficult. Other resources were scarce.	Not addressed.	Serious disagreements occurred about antibiotic distribution. It as not clear which healthcare workers should be wearing protective equipment or what level of protection was appropriate.
Lau, 1997 Transportation accident	Not addressed.	Nurse wasted time to summon back staff. There were not enough wheelchairs, extra chairs in waiting room, or poles and ropes to maintain order.	Not addressed.	Not addressed.
Levi, 1998 Not specified	The exercise identified crowd control issues and other security problems.	The exercise identified specific medical equipment/medication needs and electro-mechanical failures.	Not addressed.	Not addressed.
Menczer, 1968 Incendiary device and boiler explosion	Not addressed.	First aid equipment and supplies in quantity must be taken to site as soon as the type of disaster is ascertained.	Not addressed.	Out-of-town ambulance drivers did not know hospital locations as the state highway signs were inadequate.
Tur-Kaspa, 1999 Chemical	Not addressed.	Not addressed.	Full protective equipment must be worn in the “contaminated area.” Decontamination must be directed by personnel with loudspeakers.	Adequate pre-drill instruction and training were vital for the drill's success.

The majority of studies we identified addressed the key question regarding hospital disaster drills.^{16–26, 28, 29, 32, 33, 35} The learning objectives of these exercises included various knowledge objectives (i.e., to test coordination of response and evacuation procedures),³² skill objectives (e.g., to assess knowledge of use of fire extinguishers),¹⁹ and behavioral objectives (e.g., timely contact of appropriate safety personnel)¹⁹ (see Appendix E, Evidence Table 1). This group of studies also used a number of training evaluation methods, most typically group debriefings. “Smart” (i.e., medically trained) observers, other observers, and triage and/or victim tracking cards were often used to evaluate the outcomes. The outcomes of these studies provided many lessons having an impact on command and control, communications and patient flow, and other areas (see Tables 1 to 4). All studies reported important lessons learned.

The six hospital disaster studies that addressed transportation accidents took different approaches, resulting in a variety of conclusions.^{22–26, 35} Cook et al. took a “game” approach that allowed employees to study the disaster plan prior to the drill. The author believed this approach was useful in increasing understanding of the disaster plan, identifying plan flaws, and improving coordination.²² Eisner, in a study focused on the emergency department and targeted to first responders and the triage teams, simulated an airplane crash at a nearby airfield. The study found that more than half of the notional victims experienced a serious delay in care that would result in excess deaths.²³ Paris also studied a simulated airport disaster and used “smart” casualties and triage cards to analyze the care provided to victims.²⁶ Lau, simulating an undescribed transportation accident designed to test knowledge, skills, and behaviors, found that although the disaster plan was successfully activated, there were difficulties in the areas of triage and charting.³⁵ However, Lau also found that practice drills provided clinicians with the opportunity to anticipate possible operational difficulties and find remedies to track them. Fishel and Maxwell reported difficulties in triage in their studies.^{24, 25} In addition, Maxwell reported problems with patient tracking, and Fishel reported problems in communications.

Five studies reported on a fire disaster or explosion, four of these placing the event within the hospital^{18, 19, 32, 33} and one setting the mock event at a college.²⁰ All studies targeted several groups, including physicians,^{18–20, 32, 33} nurses,^{18, 19, 32, 33} administrators,^{32, 33} and first responders.^{20, 32} Training objectives varied from evacuation assessment^{32, 33} to assessing first aid at the scene of the fire²⁰ and knowing the location of and how to use fire extinguishers.¹⁹ Similar to other hospital drills, these studies were evaluated by group interviews and de-briefings, and “smart” and other observers.

Chobin, Saxena, and Tur-Kaspa studied chemical spills.^{16, 17, 28} Chobin et al. tested the Occupational Safety and Health Administration (OSHA) requirements in responding to a chemical spill.¹⁶ This study used simulated patients and involved external and internal response and multiple departments. The authors found it very useful to test OSHA-required disaster plans before an emergency, and they concluded that the hospital may not be able to meet some OSHA regulations. Saxena evaluated the ability of a hospital to make a coordinated response to a hazardous materials incident and found that the list of chemicals involved was not correctly reported and that communication ranged from ineffective to nonexistent.¹⁷ Tur-Kaspa evaluated a hospital drill with “smart” casualties—army physicians with experience in managing chemical casualties.²⁸ This study identified lessons learned that were incorporated into the hospital deployment plan. The authors felt their study had wide implications for disaster mitigation worldwide.

The EPC team concluded that enough studies were available to suggest that hospital disaster drills were effective in training hospital staff to respond to an MCI; however, the study designs were weak, and overall the evidence was insufficient to draw firm conclusions, leading to an evidence grade of C (weak) (see Chapter 2, Methods, Evidence Grades). The published literature lacked studies addressing either bioterrorism directly or other prolonged MCIs. These incidents have a different presentation, with sporadic presentation of cases and perhaps continuing over days or weeks with a high casualty toll and large numbers of concerned and potentially exposed as well.

Outcomes of the Studies

Hospital disaster drills, computer simulations, and tabletop and other exercises are designed to test the hospital's disaster plan and to allow employees to become familiar with disaster procedures, leading to reduced chaos and improved institutional response at the onset of an actual disaster. These training exercises address many aspects of disaster response. On the basis of this review of the literature, discussion with experts, and analysis of the HEICS job action sheets,¹⁰ the EPC team identified several important aspects of hospital disaster response that include outcomes that may be useful to evaluate.

Figure 2. Hospital Disaster Response Outcome Areas (more...)

   Figure 2. Hospital Disaster Response Outcome Areas Addressed by Included Articles^a

Figure 2

Communications

Table 2. Summary of Results: Communication

Table 2. Summary of Results: Communication

Author, year Type of training	Internal Communication	External Communication
Baughman, 1990 Hospital fire and explosion in the emergency department	Considerable time delay occurred because the emergency department was immobilized.	Not addressed.
Chobin, 1989 Chemical	Hospital operator was called using established hotline. Hospital fire brigade was alerted by code. Hospital operator notified nursing administration.	Assistance was requested from local fire department. The hospital's Chief Executive Officer was asked to call a disaster code.
Classic, 2000 Radiational	Intercom system was inadequate as messages could not be understood clearly. Fire alarms worked well. Radiation call staff were contacted successfully.	Contact of radiation safety was immediate, but message was incomplete (significant deficiency).
Cook, 1990 Transportation accident	Overhead announcement was not heard. Some vital personnel had not received new disaster plan.	Not addressed.
Fishel, 1974 Transportation accident	The emergency call-up system was inadequate because names and telephone numbers were not correct.	Radio communications developed several technical and operational problems.
Gretenkort, 2002 Hospital fire	Not addressed.	The drill went smoothly and provided true interface between authorities and hospital administration.
Halstead, 1993 Hospital fire in operating room	Staff could not hear overhead announcement of fire in operating room. Staff needed printed protocol for fighting fire. Operating room representatives need to be added to hospital committee.	Not addressed.
Inglesby, 2001 Biological	A significant amount of time was spent exchanging phone, beeper, and facsimile numbers (should have been done prior to exercise).	Process of decision making by conference call was highly inefficient and led to indecision and significant delays in taking action. 800 MHz radios had efficient communication where regular phone lines were not answered or otherwise dysfunctional.
Lau, 1997 Transportation accident	Better radio training was needed.	Telephone operator preferred native language under stressful conditions.
Levy, 2000 Radiational	Not addressed.	Extensive live communication occurred among sites in 5 time zones. All sites participated in 7 live conferences within 74 hours. Proper authorities were notified in each country.
Saxena, 1986 Chemical	Not addressed.	Notification for the activation of Emergency Operating Centers among participating agencies was not effectively accomplished. The list of chemicals involved was not correctly reported to the state Emergency Operating Center. Exercise communications between the Emergency Operating Center were inefffective and in come cases nonexistent.
Tur-Kaspa, 1999 Chemical	Not addressed.	An effective communication system between different sites and the control center is essential.
Weston, 1988 Hospital fire in operating room	Communication was poor because of small number of alarm bells and low level of buzzers.	Not addressed.

Thirteen studies dealt with the issue of communications (see Table 2). Nine of the studies reported findings related to internal communications,^{16, 18, 19, 21, 22, 24, 27, 33, 35} and nine studies addressed external communications.^{16, 17, 21, 24, 27, 28, 32, 34, 35}

Internal Communications. Among the studies addressing internal communications, only one, Chobin et al., found that communication occurred smoothly.¹⁶ Eight studies illustrated breakdown of communications.^{18, 19, 21, 22, 24, 27, 33, 35} Studies cited the inadequacy of overhead intercom systems,^{19, 21, 22, 33} delay in communication because the emergency department was immobilized and unable to receive messages,¹⁸ lack of training in the use of radios,³⁵ and significant time delays spent identifying correct contact numbers.^{24, 27}

External Communications. The results of the studies reporting on external communications were mixed. Chobin and Gretenkort reported smooth and successful interfaces with outside agencies.^{16, 32} In the Levy study, participants were able to successfully carry out notification of proper authorities and extensive live communication among sites in five time zones.³⁴ Tur-Kaspa emphasized that an effective communication system between different sites and the control center is essential.²⁸ Five studies noted shortcomings including incomplete messages,²¹ ineffective activation of emergency operations centers,¹⁷ language difficulties when the operator was under pressure,³⁵ technical and operational radio communication problems,²⁴ and a highly inefficient process of decision making by conference call leading to indecision and significant delays in taking action.²⁷

Clinical Care

Table 3. Summary of Results: Clinical Care

Table 3. Summary of Results: Clinical Care

Author, year Type of training	Triage	Patient Care	Patient Flow	Patient Tracking
Baughman, 1990 Hospital fire and explosion in the emergency department	Usual triage area was not available, thus causing confusion. Relief staff was inexperienced in triage.	Treatment began in triage areas before patients were sent to treatment areas.	Lack of triage area confused patient flow.	Not addressed.
Burns, 1984 Incendiary device	Not addressed.	Participants' burn care knowledge increased 6 to 7 points out of 200 from pre-test to post-exercise. The median outcome of self-scoring was 87% with a range from 60%–96%. Leadership personnel from the emergency department working with members of the burn unit scored the highest. A team consisting of an emergency department technician and a staff nurse scored the lowest.	Not addressed.	Not addressed.
Chobin, 1989 Chemical	Not addressed.	Not addressed.	Victims were evacuated to emergency department.	Admitting personnel in the emergency department made charts and identification bracelets.
Classic, 2000 Radiational	Physical barriers to identify hot, warm, and cold zones for ambulatory victims were not used correctly.	Plan to use building exits as “choke points” for screening worked well.	30 victims were transported to emergency department.	Not addressed.
Cook, 1990 Transportation accident	Not addressed.	Not addressed.	Congestion in triage and emergency department was less since personnel reported directly to assigned areas rather than to the emergency department to ask for guidance.	Not addressed.
Eisner, 1985 Transportation accident	53% of the group that needed immediate care arrived at care location greater than 1.5 hours post-disaster.	Not addressed.	Not addressed.	Not addressed.
	85% of patients triaged to the trauma center required admission, indicating excellent correlation between action and need.
Fishel, 1974 Transportation accident	Teams of doctors and nurses were not experienced or knowledgeable in triage. A course to develop triage personnel is needed. The triage tags were not easily identifiable. Color coded tags may address this problem.	Not addressed.	Ambulance crews became exhausted moving the victims.	Not addressed.
Gofrit, 1997 Not specified	9% of patients were over-triaged. 4% of patients were under-triaged.	Simulated casualties were not examined head-to-toe. Patients with post-traumatic stress disorder were not examined fully and referred directly to psychology.	Delays were encountered in treatment due to lack of leadership and shortage of personnel. Patients were transferred from one area to another without appropriate medical escort and without properly controlled ventilation.	Medical documentation was inadequate.
Gretenkort, 2002 Hospital fire	Not addressed.	Not addressed.	Patient flow and staff allocation was greatly aided by Jaerven Rescue Drag Sheet.	Not addressed.
Halstead, 1993 Hospital fire in operating room	Not addressed.	Operating room beds were too heavy to maneuver for evacuation.	Corridors, exits, and evacuation routes were blocked with equipment. More storage space was needed for extra equipment.	Not addressed.
Inglesby, 2001 Biological	There was concern over ability to distinguish between the concerned and potentially exposed and those harboring early signs of plague.	Hospitals were beyond capacity for patients in less than 24 hours.	Plans were inadequate for disposition of patients before and after triage, and for the deceased.	Not addressed.
Lau, 1997 Transportation accident	19 patients were triaged and discharged. Patients' particulars were inadequately up-dated on the Accident and Emergency clinical records sheets.	Charting and filling out forms detracted from patient care. Staff summoned from other units were not familiar with the emergency department.	It was 45 minutes from first patient in to last patient out. Porters did not know role in drill.	Patients were given bracelets and record sheet with identification. All patients were accounted for.
Levi, 1998 Not specified	Not addressed.	Not addressed.	The drill identified bottlenecks and predicted ability to care for more casualties.	Not addressed.
Levy, 2000 Radiational	Correlations were made between clinical symptoms in emergency department and common source of exposure. Names of those exposed were identified and sent to the Departments of Public Health in participating countries.	Not addressed.	Not addressed.	Not addressed.
Maxwell, 1987 Transportation accident	6 victims were not assigned any hospital triage category.	13 of the 14 victims were judged to have received appropriate treatment.	The median time to triage was 3 minutes with a range of 0 to 10. The median time to treatment area was 10 minutes with a range from 0 to 39.	4 victims slipped through hospital triage without being tagged.
Menczer, 1968 Incendiary device and boiler explosion	There was no selection of victims for removal from the scene. Immediate establishment of an area for victim safety and treatment was an unfulfilled need. One observer found no evidence of effective triage and no follow-up.	Training of police and fire department personnel in first aid was deficient. Victims need more thorough and adequate first aid after being removed from the disaster site.	Transportation of victims from the disaster scene was done with little regard to the type or site of injury. A great deal of unnecessary handling of victims occurred. Several victims were laid on cold ground uncovered for as long as 20 to 30 minutes. Ambulance services generally provided proper handling and transportation.	Not addressed.
Paris, 1985 Transportation accident	5% of victims were never assigned to a triage category. 44% of victims were assigned to proper triage category.	All 133 patient-tracking cards were collected. 3% of victims with correctable injuries “died” as a result of necessary treatment not provided in timely manner. 6% of victims had deterioration attributed to lack of timely intervention.	Not addressed.	Not addressed.
Tur-Kaspa, 1999 Chemical	Not addressed.	Continuous care and repeated reevaluation of patients are essential during transfer and treatment. Clinicians must know dosages and side-effects of antidotes. Training should occur in full protective equipment and include “intubation dolls”, ventilation, and decontamination procedures. At each treatment site, medical personnel must be ready to handle casualties with injuries other than those of the specific type and severity for which they have been prepared. Note: The above statements were presented as results of the study.	Not addressed.	Clear labeling, identification, and record keeping were vital for efficient reception and treatment of casualties.
Weston, 1988 Hospital fire in operating room	Not addressed.	Patient casualty occurred due to evacuation from the operating room.	Patients were incorrectly moved outside the building instead of to behind the first fire door.	In the 28-minute evacuation, all patients and staff were accounted for.

Nineteen studies cited results related to clinical care of patients (see Table 3). Eleven studies described outcomes on triage,^{18, 20, 21, 23–27, 29, 34, 35} twelve studies described the impact of the drill on patient care,^{18–21, 25–29, 33, 35, 36} fourteen studies described patient flow issues,^{16, 18–22, 24, 25, 27, 29, 31–33, 35} and six studies described patient tracking issues.^{16, 25, 28, 29, 33, 35}

Triage. Levy reported success in triage based on correlation of clinical signs with radiation exposure.³⁴ All other studies in this category reported triage problems, including confusion due to unavailability of the usual triage area;¹⁸ incorrect use of physical zones for different categories of patients;²¹ inadequate updating of patient demographics;³⁵ slow arrival at triage;²³ inexperienced staff in triage;^{18, 24} inadequate selection of victims for removal from the incident scene;²⁰ patients who were either never assigned to a triage category^{25, 26} or were assigned to an incorrect triage category.²⁹ Inglesby identified concern over the ability to distinguish between the concerned and potentially exposed and those with early signs of infection.²⁷ This point is significant because Inglesby was the only study that looked at a bioterrorism event.

Patient Care. Paris, Lau, and Maxwell found that patient care drills proceeded according to plan,^{25, 26, 35} and Classic found that a plan to use building exits as a “choke point” for screening worked well.²¹ Burns found a knowledge increase in nurses' capacity to treat victims of an incendiary device.³⁶ Other studies identified a range of deficiencies. Tur-Kaspa reported that clinicians must be able to access information on dosages and side effects of antidotes to be effective when responding to a chemical event.²⁸ Baughman reported that treatment began in triage area, before patients were sent to the treatment area.¹⁸ Menczer reported a need for more thorough first aid after removal from the disaster site.²⁰ Paris reported that significant delays in patient care were noted for patients under the drill procedures, and Weston, Paris, and Halstead reported on events leading to adverse outcomes for patients, the latter due to inability to maneuver heavy operating room beds.^{19, 26, 33} Gofrit reported that simulated casualties received incomplete medical evaluations,²⁹ and Lau found a conflict for busy clinicians between documentation and giving effective patient care.³⁵ The comprehensive exercise by Inglesby found that hospitals were beyond capacity for patients in less than twenty-four hours.²⁷

Patient Flow. Several patient flow issues were identified by the studies. For example, Gretenkort studied the use of the Jaerven Rescue Drag Sheet, which greatly facilitated removal of patients from the area of the simulated hospital fire.³² Cook found that personnel reporting to assigned areas lessened congestion in the emergency department.²² Halstead and Inglesby identified needed improvements, the former finding that corridors and marked evacuation routes were blocked by equipment,¹⁹ and the latter finding inadequate plans for patient disposition, including disposition of the deceased.²⁷ Gofrit, Menczer, and Weston identified issues in transporting patients,^{20, 29, 33} and Fishel found that the ambulance crews became overwhelmed and exhausted in a planned drill.²⁴

Patient Tracking. Weston reported that all patients were accounted for within the planned time limit of the drill.³³ Other investigators found that clear labeling, identification, and record keeping were vital for efficient reception and treatment of casualties.^{28, 35} Gofrit reported medical documentation was inadequate,²⁹ and Maxwell identified patients who had arrived in treatment areas without completing triage.²⁵ Chobin identified the importance of patient identification and charting.¹⁶

Principal Findings

The extensive literature search identified 21 articles that described and evaluated an educational intervention designed to train hospital staff to respond to an MCI. The majority of these studies addressed the use of hospital disaster drills as a training tool (key question 1).

The studies represent a heterogeneous body of literature, ranging from descriptions of local drills to sophisticated telecommunications exercises. Studies also varied in terms of targeted staff, learning objectives, identified outcomes, and evaluation methods. Because of the wide range of foci for the studies, it is difficult to make definitive recommendations on the most effective approaches for training clinicians to respond to an MCI. However, some potentially valuable points can be identified in the literature.

Lessons Learned Based on Outcome Categories

Limitations

This evidence report has a number of limitations, of which the most obvious is the small number of studies that were directed to the training of hospital staff in how to respond to MCIs. In addition, the search was limited to published English language articles. There may be classified, unpublished material or studies in press that were not included in this report. An example would be material from the U.S. Department of Defense, which undoubtedly has experience at testing different scenarios, but these materials are not available in the published literature. There may be aspects of military disaster drills that have potential applications for hospitals. Another example is unpublished results of drills and exercises associated with JCAHO requirements taking place at state and local levels. It is not known if this unpublished material includes evaluation data.

Another major limitation relates to the fact that different types of disasters raise different issues for training of hospital staff. For example, the issues differ for drills that simulate a transportation incident or fire and those that simulate a biological incident. The latter most likely would evolve over an extended time period while the former would introduce a sudden influx of cases to hospitals. Since nearly all of the published studies focused only on training in how to respond to conventional types of disasters, little direct evidence exists on the effectiveness of training hospital staff in how to respond to a biological MCI.

Although many experts believe that tabletop exercises have an important role to play in disaster preparedness,^37–40 the literature search identified only one study that evaluated use of a tabletop exercise for training of hospital staff in disaster preparedness.³⁶ One other study reported on the use of a tabletop exercise to provide disaster training to emergency medical technicians (EMTs).⁴⁰ However, this study did not meet our eligibility criteria because it did not involve hospital staff. In this study, Chi assessed the attitudes of EMTs toward tabletop exercises. Survey results showed that EMTs believed that tabletops performed better than field exercises in linking the results of disaster exercises to appropriate changes in terms of training, equipment, and supplies. Other tabletop exercises have been described in the literature,^{38, 39} but none of these reported any evaluation data. Without evaluation data, one cannot draw conclusions about the effectiveness of such exercises. Although numerous tabletop exercises have been conducted as a less expensive alternative to operationalizing drill training, most of the identified studies on tabletop exercises were not focused on hospital-based activity, and among those that were, no results, i.e., no data, were given.

The quality and methodological limitations of the studies make it difficult to judge external validity of results. Furthermore, marked differences in educational interventions, objectives, targeted audience, and evaluation methods present challenges in drawing generalized conclusions relevant to bioterrorism preparedness. Another specific limitation is that the search identified only one evaluation of a tabletop exercise, one of a computer simulation, one of video training, and one of teleconferencing. Although each of these educational techniques may have distinct advantages, the evidence is insufficient to draw definitive conclusions about their effectiveness.

In addition, the financial burden of the educational interventions generally was not reported. Full-scale disaster drills are expensive.²⁷ The large-scale three-day drill described by Inglesby et al. cost $3 million (U.S.). None of the other studies identified cost figures, thereby leaving a gap in this important aspect of hospital disaster preparedness. Finally, very few studies identified the organizing or sponsoring entity (e.g., federal, state, or local agency, or hospital) for the drill or exercise, thereby precluding any conclusions about who most effectively plans and conducts drills.

Future Research

Part of the challenge in reviewing the existing literature regarding training of hospital staff to respond to an MCI arose from the numerous formats of studies and differences in evaluation methods. Creating a template for future training reports (e.g., Utstein-style guidelines)⁴¹ may facilitate the accessibility, synthesis, and interpretation of collected data. Authors of future reports should consider the merits of adopting a common nomenclature and explore establishing the incident command system among hospitals as a standard.

One major issue is the cost of conducting drills and the need for a steady stream of funding to support these activities. Although governmental funding has increased recently, many hospitals are short of flexible funding and are unable to assign high priority to disaster preparedness. Given the evidence on the potential value of drills, the recent international events that indicate an increasing likelihood of future MCIs, and the heightened focus of the government in providing funding, it has become a priority to explore the uses of drills. Evidence is needed. One approach might be to provide funding for hospital exercises that are designed to overcome the limitations identified here.

The purposes of drills are important and in general underexamined. Drills may have many different purposes and it is imperative to conduct different types of exercises to test the different operational elements involved. Valid purposes include testing communications, triage planning, evacuation or decontamination procedures, and focusing on improving familiarity with emergency protocols. Related issues in drilling include the necessity to be efficient and to incorporate continuous training to meet the needs related to turnover in the hospital workforce. To develop a drill, major preparedness issues should be identified and then tested in different types of drills; however, hospitals should remain open to learning from the unexpected that occurs during the course of a drill as well. Overall, it is important to follow the principle of learning from the experience without judging the drill as a success or a failure.

The current evidence is not definitive on the effectiveness of hospital disaster drills in training staff to respond to an MCI. Although hospital disaster drills arguably may provide the most realistic training, they also represent a resource-intensive training format for MCI preparation. To date, no evidence supports the cost effectiveness of any particular type of training intervention. Future studies addressing the costs of educational interventions will facilitate recommendations regarding training strategies. For example, the strength of a video is that it is a relatively inexpensive way to standardize training for a large group (hospital employees or others) who have different schedules and operate remotely from each other. A good tabletop exercise allows observers to see the action develop, to gain increased awareness, to build teamwork, and to test strategic scenarios. Tabletops are economical and more efficient for some purposes. Given these attributes of these different types of exercises, a logical progression to familiarize employees with a hospital disaster drill plan may be to use videos in a group setting, then demonstrate key points with a tabletop exercise, and later graduate to a fully operationalized partial or complete drill.

Another major issue that needs to be addressed is the lack of evaluation of completed drills, and an equally important issue is the dearth of published reports from individual hospital- or health department-supported drills. Very few reports of hospital disaster drills have been ublished or made available in electronic databases of relevant material. Therefore, hospitals and other agencies are denied the benefit of others' experience. In the future, agencies or institutions funding drills may chose to prioritize both evaluation and more rigorous written reports post-drill. Federal agencies might direct grantees to document their findings and submit them for publication. As drills of some nature are now mandated by JCAHO, hospitals should be able to generate an increased number of reports about what does and does not work well. JCAHO may also want to encourage hospitals to meet the drill requirements through other than standard (i.e., non-disaster-related) training exercises.

Translating the reports of drills into future activity may help to promote orderly development of capability in the field. Disaster drills might be designed to test specific elements of response as identified in this report, including incident command system, communications, triage, patient flow, tracking, security, materials and resources, and decontamination.

More attention should be given to evaluating the effectiveness of relevant training programs in a scientifically rigorous manner. The weak study designs led to insufficient strength of the evidence to draw firm conclusions. This is typical of the present literature in disaster medicine and points to a need for better_designed studies.

This synthesis of the existing evidence on the implementation and evaluation of hospital disaster drills, computer simulations, and tabletop or other exercises may help to establish criteria for assessing the effectiveness of future training exercises. Because the current evidence on tools or methods used to evaluate effectiveness of training hospital staff to respond to an MCI is insufficient, development of a modular evaluation tool to address the effectiveness of different educational interventions will be of significant importance. These modules could follow the elements of response identified above (e.g., incident command system, triage, treatment, communication, and security).

Finally, the published evidence includes very little information that directly pertains to the training of hospital staff in how to respond to a biological MCI. Although bioterrorism is a current federal priority, only one study described the response to a mock bioterrorist attack.²⁷ The majority of studies focused on more common disasters (e.g., a fire/explosion or transportation accident).^{18–20, 22, 23, 25, 26, 30, 32, 33, 35, 36} It seems reasonable to postulate that preparedness for a conventional MCI would enhance readiness for unconventional MCIs. For example, an effective response to a bioterrorist event will depend on the general training and preparedness necessary for other MCIs, including training on communications, triage, and treatment during an MCI. However, a biological event would differ from a conventional MCI in important ways, such as evolving presentation of cases over days or weeks with a potentially high casualty toll, coupled with a large number of concerned and potentially exposed victims. Future training should therefore also address biological as well as chemical, nuclear, or radiation events.

Disaster training activities at the local, state, and national level could improve our knowledge pertaining to drill training, but they have not yet been reported consistently in the peer-reviewed literature. This lack of information suggests a need for creating improved ways of sharing such training experiences. Future disaster preparedness would be facilitated by a systematic method for collecting this information and making it readily available for review and synthesis.