In Situ Simulation: Challenges and Results

Rationale

Conducting simulations in the clinical environment can be justified by a number of rationales. These include everything from training efficiency to the underpinnings of adult learning theory.

Experiential adult learning theory serves as one basis for in situ simulation. Kolb’s theory of experiential learning provides a rationale for conducting in situ simulation from the perspective of the educator and the participant. This is especially true in that this theory relies on concrete experiences, reflection on the experiences, and “active experimentation” so that “new ideas and concepts can be used in actual practice.”⁶

As an educational tool, in situ simulation promotes experiential learning by training the health care provider in the actual environment in which the provider is expected to use these skills. Experiences in simulation labs may accomplish this to some degree, but in situ simulation, by definition, is more closely aligned with the actual “work” of the health care provider and is more likely to achieve success for certain training objectives.

In situ simulation also offers the advantage of training efficiency for the health care provider and the organization. In situ simulation occurs during the actual workday, utilizing on-duty clinical providers, which alleviates the need to schedule health care workers on nonclinical days, pay overtime, or schedule additional providers to “backfill” the clinical unit while one team of clinical workers is off the unit for training. It also provides an opportunity to review at frequent intervals the skills related to high-risk or infrequent events. Frequent reinforcement of the skills needed for these types of scenarios will likely result in better retention. However, this enhanced efficiency must be balanced by the necessity of conducting in situ simulations for all shifts, not just the day shift, in order to achieve competency for the entire provider team.

For those institutions that are just beginning to develop simulation programs, in situ simulation offers an opportunity to begin to expose clinical personnel to simulation even before a “bricks-and-mortar” center is constructed. The development of an in situ simulation program potentially provides some cost savings, but it is unlikely to meet all the institution’s needs in the long run. Nevertheless, the implementation of simulation in an unoccupied treatment room, patient room, operating room, or emergency bay provides a number of opportunities to begin to realize the benefits of simulation. Aside from safety and educational benefits, this approach has the potential to stimulate providers’ interest—and ultimately that of the institutional leadership as well—in further development of a simulation program.

It is likely that the most valuable benefits of in situ simulation are related to the identification of latent hazards, knowledge gaps, and opportunities for clinical teams to rehearse infrequent and/or high-risk clinical scenarios. In this respect, in situ simulation has been compared to “crash-testing the dummy” in automobile safety testing. In order to determine the safety of a particular vehicle, the National Highway Traffic Safety Administration does not drive a car around a neighborhood for several hours. Rather, the evaluators crash the vehicle into a wall to determine how well the car withstands the crash and what damage to the “dummies” (and by extension their human counterparts) results from the impact.

In situ simulation offers a diagnostic method to identify clinical providers’ knowledge and technical proficiency gaps. This is particularly valuable for those infrequent and/or high-risk scenarios that most providers do not experience often enough to remain proficient. In situ simulation also offers a method to identify latent hazards in clinical systems. In emergency department (ED) in situ simulations at Cincinnati Children’s Hospital Medical Center (CCHMC), in situ simulations have identified resource issues related to personnel, medication, and equipment—whether missing or an inability to use—that were secondary to knowledge gaps.

Realistic but intentional equipment malfunctions, deliberate errors (especially common errors), missing information, or even the simultaneous introduction of more than one simulated patient reflect a naturalistic approach. These types of simulations also offer the opportunity to stress the system and identify those areas that are highest in risk and have the greatest need for remediation. An example of the use of this type of simulation also combines a usability-testing facet to the in situ simulation. Kobayashi et al., used simulation as a means to test the safety of a newly designed ED prior to its occupation as clinical space. As a result of this evaluation, 18 latent hazards were identified and remedied before patients used the space.⁷

Challenges

Despite the many benefits associated with in situ simulation, there are at least an equal number of challenges. These can be classified as technical issues, logistics, cultural obstacles, and medical-legal concerns.

Technical issues. The first issue related to the implementation of in situ simulations is the use of the simulator. Currently available simulators may be somewhat portable, but they require the ability to transport the simulator, compressor, and laptop computer to the selected location and a method to shield or mask the ancillary equipment from view during the simulation. This problem might be somewhat alleviated if the in situ simulations are conducted in the same clinical unit routinely, and storage space is available in the unit. However, the majority of programs are not able to devote a single simulator to a clinical unit. Thus, the need to transport the simulators and set up and dismantle the equipment is obligatory. The simulators and equipment do not perform perfectly in all circumstances, and the additional jostling related to multiple relocations can contribute to further equipment malfunctions.

During implementation of in situ simulations, the use of medical supplies and equipment required for clinical scenarios can be approached in two ways. One approach is to use the medical equipment already in place in the clinical setting, such as angiocaths, intravenous fluids, endotracheal tubes, bags, masks, monitor leads, pulse oximeters, and others. This is obviously a desirable approach, but it does beg the question of paying for the replacement of this equipment, whether through educational funds or by the clinical unit that benefits. Replacement costs can be substantial, depending on the cost of restocking and the frequency of the in situ simulations.

An alternative approach requires the simulation team to replace the equipment in the clinical unit with reusable equipment used only for simulations. For example, equipment carts stocked exactly like those used in a particular unit can be placed and removed before and after an in situ simulation. This does require additional storage space and transport, but all the reusable equipment needs to be clearly labeled to indicate that it is to be used for training purposes only. A safety risk posed by this method is that equipment intended to be used for training could inadvertently be used on actual patients.

Infection control is another concern raised by the transport of simulators and medical equipment into a clinical setting. Although in situ simulation implementation in the ED might not be expected to raise these issues, it might do so in an operating room or critical care unit. The storage of simulators and other necessary equipment also can raise these issues.

Finally, in situ simulations can be extremely labor-intensive for the simulation team. The resources and time necessary to transport, set up, conduct, and dismantle an in situ simulation can be substantial. It might require half an hour on either side of the in situ simulation to account for the transport, set up, and take down. If the simulation is video taped, three people might be necessary to conduct the simulation: one to videotape, one to run the equipment, and one to observe and facilitate. This kind of resource commitment could strain even large centers and needs to be scheduled with respect to other ongoing simulation activities.

Logistics. The units that often derive the greatest benefits from in situ simulations are—at baseline—high-acuity, high-census areas. These include critical care units, operating rooms, and EDs, all of which are areas that are subject to large surges of patients and seasonal variations in census and acuity. The conduct of in situ simulation should stress the system, but those conducting the simulations also need to be sensitive to the system stressors already in place. This might mean that the simulation team consults with the charge nurse and/or physician prior to conducting an in situ simulation during a particularly busy time.

On the other hand, it is important that in situ simulations be conducted on a continuing and relatively frequent basis. At CCHMC, we have found that 10 to 15 percent of in situ simulations are cancelled due to volume and/or acuity. This number might fluctuate, particularly during times of extremely high census. When simulations typically occur in empty patient rooms, these rooms might be occupied by actual patients, calling for the creative use of treatment rooms, trauma bays, or other spaces. In situ simulations might be more acceptable at these times by imposing a time limit on them and the debriefings that follow to minimize any negative clinical impact.

In the ED at CCHMC, in situ simulations are limited to 10 minutes and the debriefing to 7 minutes. This sometimes necessitates initiating a simulation mid-scenario or presenting a patient that has deteriorated in another area of the ED or hospital. Standardized debriefing formats ensure that all the critical areas are covered during an abbreviated debriefing. A follow up survey requests any additional observation or comments from the participants.

A significant concern of health care providers is the delay or perception of delay in actual patient care during the conduct of an in situ simulation. Again, limiting the length of the simulation and the debriefing is helpful for preventing a negative impact on clinical care. At CCHMC, we addressed this question with our ED family advocacy group. The family advocacy group is composed of family members with children who have been patients in the CCHMC ED or who are patients with chronic illness and require ED resources several times per year. Despite providers’ anxieties, these families reported that they were supportive of in situ simulations. They stated that they were glad the health care teams were practicing for high-risk situations and that the additional time spent waiting because of a simulation was not significant in the context of an ED visit. For these families, a brief delay in care was outweighed by the value of the team’s training

Cultural Obstacles

Patient and family perceptions. Family-centered care has become a legitimate focus for many health care delivery systems. Some health care providers express concerns that in situ simulation might be perceived as either disruptive or intimidating to patients and their families. For example, mock emergency rehearsals and observation by patients and families of their trusted health care providers performing interventions, making mistakes, receiving coaching and debriefing, and asking questions (i.e., appearing less than perfect)could theoretically undermine patient and family confidence, and it might serve to remind families of the worst potential problems the patients will face.

Other health care providers express the opposite opinion. They acknowledge that in situ rehearsals might be stressful for families, but with proper support and explanation, the family can be reassured that their health care providers have recently been refreshed on the interventions and procedures that might be needed. In particular, they are aware of and prepared for the worst emergency that could happen to the patient.

Experience at the Children’s Hospital of Philadelphia (CHOP) has reflected the latter attitude. When patients’ families and staff have been exposed to an extensive daily bedside “rolling refresher” CPR and defibrillator update and mock emergency for the five sickest patients in the pediatric ICU, the response has been positive. Families have expressed appreciation for the training and preparation (much as is done on a daily basis in the airline industry on flights every day).This suggests that, with proper attention and provision of information to patients and families, integration of in situ simulation (even at the bedside) can be accomplished with respect to family-centered care.

On-Duty Clinical Practitioners as Study Subjects

Medical research involving human subjects has traditionally assumed the subject to be a patient with a medical ailment. However, patient safety research is performance research and studies the clinical care system. In this context, clinicians serve as study subjects. The clinicians, the local clinical context, and the organizational institutional culture represent psychosocial dimensions that can become evident when performing in situ simulation, whether for training or for assessment. In our different institutions when first implementing in situ exercises, we all underestimated: (1) the psychology of being a videotaped professional in a simulation exercise looking at performance, (2) the complexity of classifying “whole system” investigations appropriately and how to approach health care worker participation, and (3) the medical-legal/risk management dimensions of exposing care systems’ vulnerabilities during training and assessment exercises. The analysis that follows is based on our experience with psychosocial/cultural barriers; similar problems should be anticipated by involved in patient safety research. The expression of these issues and the solutions will vary depending on factors specific to the host organization, State law, safety culture, and other local factors. In addition, the shifting regulatory environment and Federal mandates regarding human subject research also affect this type of program.

These psychosocial factors we have categorized as:

• Voluntary research vs. mandatory training.
• Video recording performance anxiety.
• Video recording privacy issues.
• Motivation of clinicians.

Understanding these psychosocial factors prior to beginning in situ simulation exercises allows the use of a proactive approach to balance the training and assessment goals against opposing psychosocial factors.

Voluntary Research vs. Mandatory Training or Something Else?

Nationally, many institutional review boards (IRBs) struggle with definitions that differentiate research, quality assurance, quality improvement, and training. The way in situ simulation projects are framed can affect their classification and how issues—such as subject enrollment, informed consent, and privacy issues for participants—are approached. Consultation with hospital risk management, legal counsel, and the Committee for the Protection of Human Subjects (CPHS) experts suggests that in situ simulation is a hybrid. To paraphrase the consensus opinion, in situ simulation is a complex, nontraditional piece of research that arguably includes a quality assurance (QA) component in the conduct of the research and might yield significant training benefits, all in the service of quality improvement. There are good reasons to consider this activity a combination of research, QA, training, and quality improvement (QI).

Research. In situ simulation is not consistent with classical clinical research (e.g., a study of a new chemotherapy protocol for a specific form of cancer).Traditional clinical research requires a full CPHS review. The Federal Common Rule: 45 CFR §46.102; defines research as “…a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge.”⁸ Thus, this work is, technically, human subject research. But it is not traditional “human subject research” on patients, of the kind the Federal Common Rule was designed to regulate and the IRB (CPHS) is normally asked to approve. This is human factors research into the performance of clinicians under (simulated) stress, rather than an inquiry into the effects of an experimental treatment on patients themselves. Still, we believe CPHS review should be sought, and requirements must be met if academic funding and publication are anticipated.

Training. In contrast, pure training is typically an expectation of employees and requires no approval by the CPHS. Indeed, performance in mandatory training is often linked to performance evaluations and might be linked to employment. Consequences for nonparticipation in required training are standard in medicine and other industries (i.e., firemen cannot refuse to participate in “fire drills”). We have framed standardized exercises as mandatory training once a validated intervention has been developed and deployed. Many institutions are developing assessment simulations as a prerequisite to privileging clinicians for specific clinical activity. This strategy allows regulatory requirements for a demonstration of competency to be met.

Quality assurance and quality improvement. Between these two extremes of classical clinical research on informed patients and pure employee mandatory training lie quality assurance investigation and quality improvement activity. Quality assurance (QA) activities and records are defined by State law. In New Hampshire, RSA 151:13-a defines QA as the activities and records of a hospital committee organized to “evaluate matters relating to the care and treatment of patients, or to reduce morbidity and mortality.”⁹ In situ exercises, such as mock codes, do not legally qualify under the first part of this definition because care and treatment are given not to patients but to a mannequin. However, the activity clearly is organized to reduce morbidity and mortality, and the performance of providers during the mock codes is peer reviewed, so arguably it could qualify as QA if one desired to keep the videotapes and performance confidential.

Unlike QA, quality improvement (QI) records do not have any clearly defined legal meaning and are not given any particular legal privilege, nor should they be, because while QA only works well if its confidentiality is protected, QI materials should be disseminated as widely as possible in the hope that the materials are used to educate providers and improve the quality of care. The purpose of in situ simulation is to improve the quality of care, and the research and training aspects of this activity are completely consistent with this end. Furthermore, if participants are deidentifed before using the videotapes with subsequent generations of students/trainees or publishing the results—and thus removing any cause for professional anxiety—there would be no reason not to use the tapes as widely as QI training tools.

Unfortunately, consensus on definitions for research, QA, QI, and training do not exist. The general themes described here should provide guidance for how to frame in situ simulation to meet local goals and objectives. Each activity needs to be assessed with regard to current Federal and State laws, institutional bylaws and local CPHS policies and procedures. Navigating the complex hybrid aspects associated with in situ simulation is a significant challenge.

Video Recording Performance Anxiety

The Hawthorne effect,¹⁰ in which performance while being studied tends to be superior to usual performance, has been well described. However, performance anxiety is also clearly associated with the use of videotape in medicine and can significantly degrade observed performance and affect participants’ motivation. Being observed while working, even by those with no expertise, can be stressful.¹¹ In a survey that reviewed family-witnessed resuscitation, medicolegal concerns and performance anxiety affecting the CPR team were listed by 24 and 27 percent of respondents, respectively, as the reasons they disapproved of this practice.

The magnitude of videotaping’s effect on clinical performance has not been well studied. However, performance studies acknowledge that video has the potential to produce anxiety in the subjects of the recording. For example, in a study of emergency medicine resident communication skills, the authors note, “… considerable initial resistance to videotaping … among the ED nursing staff”; and “Our residents reported an initial discomfort with the possibility of being videotaped, but this diminished with time.”¹² This finding—that one limitation of videotape recording is its influence on behavior—has been described by psychiatric, behavioral medicine, and other experts.¹³ The phenomenon of social anxiety is aggravated by infrequent and unpredictable exposure to the stressful stimulus (as occurs with some study protocols; e.g., random once per month mock codes that are videotaped and reviewed for errors in performance).The more predictable and frequent a stress or is, the more rapidly social anxiety dissipates.^{14, 15} In addition, informing subjects that the focus of the videotape review is directed toward team and systemic performance rather than individual error can further reduce anxiety.

Video Recording Privacy Issues

Although video is a powerful vehicle for capturing performance in complex, high-hazard domains, privacy issues emerge when it is used for in situ simulation. When patients and family are in the clinical domains of interest, it is common to accidentally record them. This is especially true in exercises in which transports and handoffs are of interest. ASTAT cesarean section simulation requires organizing a patient for transport, movement through potentially crowded hallways, and transfer to an operating room table. It is important that the videographer be aware of privacy regulations and avoid taping by standers. In some systems, it might be possible to obscure participants’ faces.

Since simulation training involves forced error in the service of learning, it is important that the video be used only with the individuals participating in the exercise. Trust is a critical factor for all simulations and is especially hard to protect when the simulation is in situ. If any permanent record of the video is maintained, explicit permission from each individual as to the purpose and use of the tape must be obtained. Use of the video for education of others without the consent of the participants should not occur. If video is to be used for public presentation at academic conferences, explicit permission from each individual visible on the video should be obtained. Often, it is more acceptable to re-enact the interesting segment with “actors.” If privacy is not respected from the perspective of the participants, trust will be lost, and motivation will be a major barrier to continued exercise.

Motivation of Clinicians

Since in situ simulation is performed in the clinical domain with clinicians on duty, motivation to participate can be a major obstacle. While groups of individuals can be motivated through edict and threat of punitive action, such as one might use for mandatory training, self-motivation is preferable. The lack of motivation is not always due to performance anxiety. Models of individual motivation suggest an important first step is helping potential participants to move from the pre-contemplative state (“There is no problem. Why should I waste my time on this?”) to the contemplative state (“There is a significant problem with the status quo, and change is needed.”). Although originally described in relation to motivating individuals to change behaviors like smoking, this model is robust.^{16, 17}

Education on the rationale for crisis training in situ should provide sufficient information for clinicians to understand the problem with current practice and see the benefits of participating in such exercises. Further steps toward individual motivation include determination, action, and finally relapse vs. maintenance of the new behavior. In situ simulation is a powerful motivator because it reveals less than optimal adverse event management due to flaws in the status quo.

After participating in an exercise, clinicians universally request feedback on their performance. Although immediate debriefing can be difficult, the information provided is ideal for helping individuals determined to change their behavior. Debriefing in the form of video feedback is a well-established methodology in simulation centers to facilitate and motivate learning, but the efficacy of this approach is difficult to demonstrate. In a multicenter trial attempting to show that videotape feedback improved learning compared to a matched group of trainees not given feedback, the measured differences in performance were not statistically different.¹⁸ However, since performance is difficult to measure, it may be that the impact of video feedback was simply obscured by the variability of the measures and a relatively small sample size.

Video feedback is also used in “real world” settings to facilitate learning (e.g., videotaping of trauma resuscitations in emergency rooms).A multi-State survey of videotaping practices for major trauma resuscitations found that of 45 hospitals that videotape, 75 percent used the tapes for teaching, and 45 percent used them in morbidity and mortality (M&M) conferences.¹⁹

Sustaining motivation to participate in in situ exercises over time requires feedback that system failures are being addressed by the organization. If debriefing includes individual, team, and systemic contributors to failure, the participants know that these latent conditions will exist until corrected. Individuals tend to normalize deviance and, with repeated exposure over time, fail to notice all the “accidents waiting to happen.” In situ simulation makes these conditions explicitly visible and can lead to frustration and anger that the organization is committed only to patient safety on the backs of hypervigilant individuals, rather than to robust supporting systems

In addition, immediate rewards for participation often include small local incentives (e.g., food, coffee, chocolate, badges, trinkets) and more professional outcomes (e.g., documentation of mandatory competencies, fulfillment of unit based CEUs).This latter reward system can serve to promote the utility of in situ simulation for both initial orientation training and refresher training documentation, either of minimal competency or level of excellence.

Overcoming Barriers to In Situ Simulation

As mentioned earlier, numerous potential or perceived barriers to in situ simulations exist. Strategies for success often require tailoring simulations to the local environment and multidisciplinary staff, particularly when using bedside rather than adjoining (but physically separated) simulation areas for mission rehearsal.

Some consistent themes that emerge for facilitating successful in situ simulation experiences include garnering dedicated equipment storage and equipment space in or near the site of simulation (rather than requiring laborious transport of equipment and support). In addition, the facility to rapidly put up and take down the simulation equipment to convert simulation space to usable patient care space can add to the realism of rehearsals and to the functionality of the dedicated space. Simulation working groups are helpful when they are unit-based, multidisciplinary in nature, highly integrated into the continuous quality improvement (CQI) fabric of the unit, linked to incident reporting and patient safety initiatives in the unit, and conducted in conjunction with efforts for mandatory nursing education within a hospital.

Key to successful implementation of in situ simulation is integration of in situ simulation with the involvement of those who are responsible for credentialing and orientation of unit staff and familiarization for rarely used or new equipment. At CHOP, prospectively designated standards and limits set by staff leadership consensus for conducting in situ simulation based on volume and acuity of patient care needs have been helpful for obtaining support for the program and avoiding concern about interference with unit function. These processes further serve to engage health care providers in the process of simulation, and they solicit objectives and training outcomes that help meet the needs of providers and the administrative network.

Outcome #1: In Situ Training for Technical Proficiency and Measurable Changes in Clinical Competency

Tracheal intubation. Appropriate airway management is the initial and most critical step for pediatric resuscitation: prevention, initial stabilization, and optimized resuscitation from acute respiratory failure and cardiac arrest. It is one of the core skills to be learned by pediatric residents during their pediatric intensive care unit (ICU) rotation as defined by the Accreditation Council for Graduate Medical Education (ACGME).²²

Studies have shown that psychomotor skills start to decay as soon as 3 months after the training and decay much faster than knowledge-based skills.^{23, 24} The effectiveness of high fidelity simulations was evaluated in achieving and measuring competence in initial airway management skills in medicine interns (PGY-1) using a computer-simulated training process involving a respiratory arrest scenario in an adult patient. The investigators developed essential and nonessential steps of initial airway management and showed improved performance in those steps in the training group. Although they also suggested an excellent clinical performance in actual airway events in this intern group, they could not evaluate the effect of simulation-based training on the clinical practice directly.²⁵

An evaluation of the reliability and validity of simulation-based acute care skills in medical students and residents found that the reliability was most strongly influenced by the choice and number of simulated encounters. The validity of the simulation scores was supported through comparisons of the performance of students and residents in relation to their clinical backgrounds and experience.²⁶

Refresher training based only on cognitive knowledge review does not improve psychomotor skills,²³ and it remains a challenge for pediatric residents to learn and retain this critical skill. NEAR-4-KIDS (National Emergency Airway Registry for children) has been implemented in our pediatric intensive care unit (PICU) to capture the unit’s airway management practices.²⁷ Since 2004, NEAR-4-KIDS has successfully captured more than 200 intubation events (97 percent of all intubation events). It describes characteristics of intubation, operator factors, patient factors, and associated events that might be related to patient safety issues. We came to know that our residents did not participate in the majority of orotracheal intubations (28 percent), and the success rate of the selected cases was still significantly lower than that of our fellows (38 percent).^{27, 28} Based on this knowledge, just-in-time training on psychomotor skills—such as resuscitation and airway management—immediately before the work makes sense. However, no study has yet tested the efficacy of just-in-time resuscitation/procedure training.

To improve the operational performance (i.e., first attempt success rate) and safety (i.e., minimal associated events that could lead to adverse events), a clinical study was conducted to test the association between simulation training and improvement in patient outcomes and patient safety and the impact of “fidelity” on the effectiveness of simulation training.

With IRB approval, experienced nonanesthesia pediatric providers with airway management responsibility serially refreshed orotracheal intubation skills in six simulated infant trauma airway scenarios involving cervical spine stabilization. Time (T) to successful completion of key actions was measured by computerized mannequin and debriefing. The ratio of time to successful tracheal intubation from end of bag-mask ventilation to confirmation of correct tracheal placement was recorded for each scenario.

A total of 26 skilled providers (pediatric transport nurses, pediatric emergency medicine fellows, pediatric critical care fellows) performed 156 intubations. Overall, time to successful tracheal intubation (mean ± SD) was T1 = 33.8 ± 9.4 seconds for the first scenario. For subsequent scenarios: T2 = 29 ± 6.4 seconds, T3 = 27.4 ± 5.6 seconds, T4 = 29.8 ± 9.2 seconds, T5 = 28 ± 5.4 seconds, and T6 = 25.6 ± 5.1 seconds. Immediate effectiveness of refresher training assessed by the individual provider’s ratio of T2/T1 was associated with recent intubation training within 3 months (P = 0.025) but not with provider clinical experience >3 years (P = 0.93) or discipline (P = 0.40). Recent intubation training (<3 months) remained significant (P = 0.017) in multivariable linear regression adjusted for years of previous experience and discipline.

Recent tracheal intubation training, but not clinician years of experience or training discipline, was significantly associated with the ability of refresher training to improve provider excellence and patient safety in simulated advanced pediatric airway management.

CPR training. Another example of successful in situ simulation training involves “training to excellence” in cardiopulmonary resuscitation (CPR).It is widely recognized that high-quality CPR skill retention is poor. We hypothesized that “just-in-time” and “just-in-place” refresher training programs would be effective and well-accepted to maintain CPR skills among pediatric ICU staff. Each day, five ICU patients at highest risk for cardiac arrest were identified by clinical staff, and “rolling refreshers” were implemented to multidisciplinary bedside providers.

A trainer and portable mannequin/defibrillator system with force transducer and accelerometer provided individualized review of CPR skills with automated corrective feedback to optimize CPR skills. Each provider practiced CPR with corrective feedback until he/she had attained skill success, defined prospectively as >90 percent proficiency for a chest compression (CC) rate of 90 to 120/min and CC depth of 38 to 51 mm during 60 seconds continuous CPR. Providers who completed two or more refreshers per month were compared to providers completing less than two refreshers per month in terms of time to achieve prospectively defined CPR skill success. Also, following real-life cardiac arrests in the ICU, resuscitation participants were surveyed for subjective feedback on the efficacy of the training approach, using a 5-point Likert scale.

Over 15 weeks, 420 multidisciplinary PICU staff were “refreshed,” including 340 nurses, 34 physicians, and 46 respiratory therapists; 100 percent of participants met CPR skill success targets during refreshers. A convenience sample of 20 PICU staff was assessed before subsequent refresher sessions. The time to achieve CPR skill success was significantly lower in those refreshed more than twice per month (median, 21 seconds; range 7–30 seconds), compared to those who were refreshed less often (median, 67 seconds; range 30 to 116 seconds; P <0.001).

Following real-life resuscitations, participants rated “rolling refresher” in situ training as effective (mean = 4.2 ± 0.67). From this experience, the staff concluded that this novel “rolling refresher” CPR skill training approach using “just-in-time” and “just-in-place” simulation could be effective and well-received by a multidisciplinary pediatric ICU staff. More frequent refreshers resulted in significantly shorter times to achieve proficient CPR skills.

Outcome #2: Teamwork and Desirable Communication Behaviors Reinforced

In situ simulation might stand alone or complement center-based simulation. One of the primary advantages of in situ simulation is the opportunity to review and reinforce teamwork and communication behaviors that are critical to a high-functioning, high-reliability health care team. The conduct of an in situ simulation allows for the partial scripting of deliberate errors, authority gradient issues, and equipment malfunctions. For example, in the CCHMC ED, one partially scripted simulation involved the attending physician deliberately failing to follow new Pediatric Advanced Life Support (PALS) algorithms for ventricular fibrillation. The facilitator observed that the body language of members of the health care team indicated discomfort with the actions of the attending physician, but no one spoke up during the simulation.

During the debriefing, it emerged that four team members knew that the algorithm had been violated but did not question the attending physician. Subsequently, several alternative methods of addressing these concerns were explored. During the debriefing, team members were encouraged to explore and practice these methods of questioning the team leader. In addition, the health care resuscitation team was observed referencing and researching the most recent PALS algorithms during lulls for the remainder of the day.

Outcome #3: Identification of Active and Latent Conditions

The identification of latent conditions using standardized assessment type in situ simulation has been reported previously.²⁹ When used to test pediatric sedation rescue capability, in situ exercises exposed substantial variation between different care settings and different clinical teams in caring for the same simulated patient. In addition, these exercises identified 5 to 10 care-management problems and 20 to 40 contributory factors (i.e., latent conditions predisposing systems to fail).

Active failures by clinicians due to knowledge deficits and technical incompetence were also readily identified. For example, clinicians had little know ledge of how to administer narcotic reversal agents (e.g., indications, contraindications, and dose) during an overdose scenario. Technical proficiency in effective positive pressure ventilation and use of bag-mask was lacking in all clinicians—except for respiratory therapists and anesthesiologists—studied in this hospital, which cares for pediatric and adult patients. The opportunities for improvement exposed by just 10 in situ simulations has led to several years’ work to implement corrective actions to the sedation rescue systems.

In situ simulations also can be presented with malfunctioning or missing equipment. Sometimes this occurs by design, but on other occasions, it might occur serendipitously. For example, a foreign body airway obstruction scenario was carried out in the CCHMC ED. During the course of this simulation, it was discovered that the McGill forceps (used for removing foreign bodies from the posterior pharynx or airway) was missing from the trauma bay. This forced the team to quickly locate backup equipment. However, it also emerged during the debriefing that staff from a particular subspecialty was known to remove these tools from the shock trauma bay, thereby putting critical patients at risk.

Outcome #4: Opportunity to Improve Clinical Outcomes

The empiric research performed in the pediatric sedation domain has been used to design a high-reliability sedation/analgesia care microsystem. The design consisted of specifying the components that support sedation, analgesia, and movement control and respiratory depression control (i.e., rescue).The resource specification included the people, tools, and environmental components that afford control (also known as an affordance map, Tables 1, 2, and 3).These tables demonstrate the specificity of the microsystem that can be created using simulation as a tool to develop and test systems. Some of the implemented changes included:

Table 1

Best/safe practice specifications for sedation/analgesia.

Table 2

Best/safe practice specifications for rescue.

Table 3

Measured competencies for rescue.

• Establishing a specialty service for pediatric deep sedation and anesthesia.
• Establishing standard rescue equipment.
• Establishing privileges for sedation with explicit competencies.
• Correcting deficits in the emergency response communication paging system.
• Establishing a specific simulation-based sedation rescue training intervention.
• Establishing better metrics of performance and outcomes.

Within-hospital variation of pediatric procedural sedation has led to benchmark performance relative to other sedation services. These changes resulted in improvements in:

• Pain/sedation management.
• Safety from the life-threatening side effects of sedatives and pain medications.
• Conditions for diagnostic or therapeutic procedures.
• Patient experience.
• Efficiency and access to procedures.

The program was reviewed in the Joint Commission Journal on Quality and Safety in a series describing the implementation of high-reliability clinical microsystems.³⁰ While we do not claim to have Class A evidence that clinical outcomes have been improved, the face validity is very strong. Specifically, use patterns by physicians practicing at our hospital support a strong belief that the changes have improved the safety and efficacy of the care provided.