A Safety Evaluation of the Impact of Maternity-Orientated Human Factors Training on Safety Culture in a Tertiary Maternity Unit

Supplemental digital content is available in the text.

H uman factors have long been recognized by other safety critical industries as key contributing factors in adverse events. 1 These include breakdown in communications, poor decisionmaking, lack of clear leadership, and team working. In medicine, human factors, also referred to as nontechnical skills, are widely acknowledged as contributing to medical harm. 2 These factors have risen to national attention in maternity care through high-profile cases, such as the Kirkup report, and confidential enquires by the Royal College of Obstetricians and Gynaecologists, such as Each Baby Counts and Mothers and Babies Reducing Risk through Audits and Confidential Enquiries across the United Kingdom. [3][4][5] These have provided invaluable information on the cause of intrapartum and neonatal death, with data analysis focusing on individual and organizational factors, instead of simply clinical events.
Despite the national recommendation for multidisciplinary training, there is no formal midwifery or medical curriculum or training and consequently a lack of standardization between units and specialities. 6 Formal training, equivalent to crew resource management used in aviation, has been proposed as the solution for healthcare. 7 Safety critical industries, such as nuclear and aviation, often experience similar organizational and therefore cultural issues, which allows for direct transfer of some training across all work settings. 8,9 Although this addresses the basic principles of human factors, such as team-working, leadership, communication, decision-making, and situational awareness, it is essential to acknowledge that flying a plane and delivering babies are different. 10 In the National Health Service (NHS), and specifically maternity, there is more diversity in work settings (with midwifery standalone units and tertiary teaching hospitals) culture and even organizational aspirations. This means that the "one-size-fits-all" training model is ineffective and problematic. There is a growing consensus that human factors training should be specific and relatable to healthcare to elicit the benefits achieved in aviation. 11 Adverse events in maternity are rare but often complex and multifactorial. Therefore, it is difficult to attribute causality of a potential reduction in these events simply because of human factors training.
Safety culture is defined as "the product of individual and group values, attitudes, perceptions, competencies, and patterns of behavior that determine the commitment to, and the style and proficiency of, an organization's health and safety management." 12 It measures team-working, leadership, handover, and communication, which are all key elements in human factors. Positive patient safety culture is associated with fewer adverse events, which can be used as a surrogate marker of patient safety. 13 The overall aim of this study was to determine whether the implementation of a maternity-orientated human factors training

Study Design
This was a prospective observational cohort study conducted for a 6-month period from August 1, 2017 to January 31, 2018. The study period commenced after maternity staff had rotated into new areas to reduce potential loss to follow-up but acknowledge that some staff were recruited to the trust after this time. A confounding factor is that some maternity staff had already received training in human factors. This was limited to three consultant obstetricians and three consultant anesthetists.

Population
The study was conducted at the Norfolk and Norwich University Hospital NHS Trust, a tertiary maternity and neonatal unit with an average birth rate of 5600 deliveries per year. All healthcare professionals who regularly worked on delivery suite involved in high-risk intrapartum care 14 during the study period were eligible to participate, including obstetricians, midwives, healthcare assistants, anesthetists, theater practitioners, and neonatologists. Staff working outside intrapartum care were excluded from the study to maintain a specific cohort to establish association. Nonpermanent staff were also excluded because of lack of exposure to unit culture.
A cohort of participants received basic human factors training and the opportunity to become trainers. This was advertised through posters, e-mails, social media pages, and announcements during shift handover. All staff eligible to participate including key senior staff, such as consultant obstetricians, anesthetists, and coordinating midwives were invited to self-nominate. This resulted in a cohort of 61 participants, a mixed cross-section of the population covering all range of shift patterns. Table 1 represents the total number of staff with corresponding professional role. The cohort of participants who received human factors training were a mixture of midwives, theater staff, midwifery care assistants, neonatologists, anesthetists, and obstetricians-who composed of both consultant obstetricians and speciality trainees in obstetrics and gynecology.

Educational Intervention
The cohort received human factors training in theoretical concepts over a 2-day course delivered by an external consultancy firm, Atrainability. The company specializes in human factors training as the cofounder contributed to the introduction of the subject into British Airways as a method of reducing "pilot error." All facilitators are experienced in human factors from civil and military aviation backgrounds. The authors preassessed Atrainability's materials to ensure that they were in reflection of themes highlighted as contributory factors in intrapartum stillbirth, early neonatal death, and hypoxic brain injury-shown in reports such as the Royal College of Obstetricians and Gynaecologists Each Baby Counts Report 2015. The course curriculum covered the following topics: safety culture, migration of boundaries, cognition, human limitations, stress & team dynamics, situational awareness, self-awareness, decision-making, dealing with difficult behavior, communication, leadership, handover/safety huddles, and debriefing/feedback. Educational methods included interactive lectures and small group exercises. Participants were then encouraged to self-nominate to create a faculty of 17 multidisciplinary trainers from this initial cohort of 61 to deliver the training to the remaining population.

Theoretical Training
The training was disseminated to the population using a blended learning approach. Basic concepts of human factors were introduced through classroom teaching in monthly mandatory drills and skills sessions. This consisted of short cognitive activities to illustrate a human factor element, with maternity examples to demonstrate relevance in clinical practice. Social media-delivered content, for example "The Little Voice Inside" video from the University Hospital of Leicester, promoted identification, reflection, and group discussion of concepts, such as situational awareness, task fixation, confirmation bias, and mental models. Forum theater 15 and behavioral simulations demonstrated by the faculty explored more complex communication issues, such as conflict resolution and transactional analysis, and provided an opportunity for participants to generate solutions in a psychologically safe space. The training specifically focused on empowering staff to raise concerns through structure communication tools for graded assertiveness, for example, PACE or Probe, Alert, Challenge, and Emergency. This was complimented with the briefing of senior staff to acknowledge and appropriately manage the safety concern, rather than dismiss the challenge.

Simulation Training
The focus of training was primarily through impromptu in situ simulations on the delivery suite. This ranged from large-scale obstetric emergencies to small one-on-one "micro" simulations, for example, distractions and interruptions while taking a blood sample. These micro simulations clearly demonstrate the human factors relevant in clinical practice and more importantly provide opportunity for participants to practice behaviors and management of these situations to reduce future error. The maternity unit did not have a designated area for training. The simulations were specifically designed to only last 15 to 20 minutes, including the set-up and debriefing to prevent disruption to patient care and the maternity service. The simulations were delivered by faculty members and occurred at least once a week during the day, night, and weekend shift patterns to include all staff groups. The project devised strict criteria for when an impromptu simulation would not be appropriate (Appendix 1, http://links.lww.com/JPS/A233).

Control
The study did not have a control group because of resource limitation and service demand. Specifically, the lack of maternity staff would reduce the cohort and influence the power of the results.

Outcome
The primary outcome measure was the Hospital Survey of Patient Safety Culture Agency for Healthcare Research and Quality, 16 a psychometrically validated tool to assess safety culture specifically in acute care. The survey contains 42 items measured on five-point Likert response scale of agreement (strongly agree to strongly disagree) and frequency (never to always). These items were categorized into the 12 domains, which constitute safety culture. Compared with other safety assessment tools, the survey has multiple subscales measuring each dimension to increase reliability of results. 17 Secondary outcome measures included frequency of reporting adverse outcomes in maternity as suggested by Maternity Review. 18 This is difficult to interpret as improvement in patient safety can be correlated to an increase in reporting, rather than a reduction. Error reporting is a key component of any organization to understand the patient safety issues. As organizations address and improve safety culture, the type of events reported change from large events that result in significant harm to multiple, "minor" errors. Higher reporting rates are associated with more positive safety culture. 19 Training quality was assessed using the Kirkpatrick model of training evaluation. 20 The model is widely used for its simplistic and pragmatic approach and allows for evaluation of different training styles. 21

Ethical Consideration
Ethical approval was not deemed necessary because this was an evaluation of the efficacy of this particular quality improvement program before wider rollout.

Survey Administration
The survey was conducted at three set points at 2 monthly intervals representing the beginning, mid, and end points of the project. Each data set spanned a 3-day period to include day, night, and weekend staff perceptions. This consisted of a total of 18 handovers from which data were collected from participants. The survey was administered through the use of portable electronic tablets to improve participant response rate. Individual identifiers were not required in the survey because there was a single individual for data collection to prevent duplication and to allow for anonymity and thus reduced bias. The survey asked participants to define their healthcare role for each data set. There were two participants in the January data set who did not answer this question, represented as "skipped" in Table 2.

Statistical Analysis
The answer response was aggregated into broader categories of "disagree," "neither" and "agree," or "never"/"rarely," "sometimes," and "often"/"always." This was required for the statistical analysis because often there was no response in the stronger preference categories, such as strongly agree or strongly disagree. Preintervention and postintervention group comparison was conducted using χ 2 test for nonordered category variables with statistical significance at the 5% level (P < 0.05). A pilot study in May 2017 demonstrated poor participant engagement because of concerns about anonymity of responses. The electronic survey collection tool resolved this issue; however, this meant that the Cronbach α to assess internal reliability was not applicable in this context.

Participant Characteristics
The demographic characteristics are presented in Table 2. There were more obstetricians in the second and third data set due to multiple obstetrician handovers and increased attendance at daily cesarean section operating lists because of complex cases. Neonatology involvement was lower than expected because of time limitations on the delivery suite and workload. Statistical analysis performed using χ 2 test for comparison of group composition demonstrated no significant difference between the groups (P = 0.09).

Unit-Level Aspects of Patient Safety Culture
The unit-level domains assess the safety culture in relation to the labor ward. The participants responded positively to the domain "teamwork" representing cooperation and respect, which was unchanged after the intervention. For "supervisor/manager expectations and actions promoting patient safety," participant response was marginally positive but did not increase with the intervention. There was an overall improvement in "organizational learning" with 83% of participants in agreement with the statement "we are actively doing things to improve patient safety" compared with 78% preintervention. However, this was not deemed statistically significant. The "Overall perception of safety" was positive, with statistical significance at the 5% level. This was evident in the response to the statement "we have a patient safety problem in this unit" with 45% of participants actively disagreeing compared with 30% (P < 0.01). There was evidence of a trend that participants felt more informed of errors and were invited for discussion about prevention postintervention in the domain "feedback and communication about error." However, this finding was not statistically significant. In the domain "communication openness," there was evidence of a trend that participants felt more positive and willing to raise concerns and question the actions of those in authority, which might affect patients. The responses to these statements were statistically significant. Staffing levels were a key issue from the response of the participants, which remained unchanged during the study period. There was a positive response to the domain "nonpunitive response to error." Fewer participants were in agreement with the statement "when an event is reported, it feels like the person is being written up, not the problem" 32% postintervention compared with 42%, statistically significant at the 5% level. Participants were required to grade the unit on patient safety, with an increase in "excellent" or "good" response from 45% to 73% postintervention, a finding that was statistically significant.

Hospital-Level Aspects of Patient Safety Culture
There are three domains to assess safety culture at hospital level. Overall, there was an increase in positive response in all domains after the training intervention. The domain "hospital management support for patient safety" indicates managerial influence on work climate. In particular, fewer participants actively disagree with the statement "hospital management seems interested in patient safety only after an adverse event" 56% decreased to 49%, which was statistically significant at the 5% level (P < 0.05). With regard to the domain "teamwork across units," there was an increase in positive response, especially in the statement "hospital units work well together to provide the best care for patients. " After intervention, 68% of participants agree with the statement compared with 53% (P < 0.05).
Medical errors and adverse events have been attributed to poor communication at handover and shift change. 22 There was an increase in positive response in all statements regarding handover correlating with statistical significance at the 5% level (P < 0.05).

Frequency of Event Reporting
Fifty percent of participants responded "most of the time" or "always" to reporting a mistake, regardless of the consequences to patient safety. This remained unchanged during the study period.
There was evidence of an increase of participants who had not reported a single event for the last 12 months; however, there was also an increase in those reporting at higher rates (11-20 reports) ( Table 3).

Level 1 -Reaction
The participants who attended the initial foundation course completed a questionnaire, using a five-point Likert-type scale to assess individual reactions and thoughts to the training. The response rate was 93% (57 participants). Seventy-two percent of participants thought that the course was "very enjoyable," 25% "enjoyable," and only 3% "enjoyed slightly." There were no responses for "neutral" or "did not enjoy." Eighty-one percent of the participants felt that the course was very relevant to the work environment, and 19% relevant. One-hundred percent of participants would recommend this course to a colleague.

Level 2 -Learning
The questionnaire encouraged the cohort to provide key learning points from the training session. Qualitative analysis generated common themes. The first was communication, specifically the use of structured tools, such as SBAR (Situation, Background, Assessment and Recommendation), and the impact or influence of different forms of communication with regard to leading questions. Other themes identified include the concept of "Just Culture," situational awareness and briefings, and finally the effect of stress and fatigue on performance.

Level 3 -Behavior
The Hospital Survey of Patient Safety Culture assessed if training had influenced the behavior of participants and the impact on unit culture. Participants were more likely to raise patient safety concerns and question those in authority. Participants perceived

Level 4 -Results
The survey demonstrated an overall improvement in safety culture in the unit. The impact of the training on frequency of adverse outcomes will require further evaluation and research over a prolonged duration.

DISCUSSION
This study demonstrates an overall improvement in safety culture after the maternity orientated human factors training intervention, specifically with regard to communication and openness. Participants were more likely to raise patient safety concerns and question those in authority to increase the situational awareness and reduce harm. The training included education about the importance of handover as a concept, rather than "another tick-box exercise" and tools to facilitate this, which improved participant's response to patient safety and handover, improving patient care.
Shift patterns and staffing levels did not change during the study, and therefore as expected, there was no change in response to this section of the survey.
There was good team work in the unit and the training emphasized the importance of this, which remained positive throughout the study. Teamwork itself was not formally assessed based on patient outcome but rather how human factors training impacted safety culture within the maternity unit.
As previously discussed, the number of events reported by participants is a difficult measure of safety. Over the study period, there seemed to be an increased percentage of participants not reporting a single event for a 12-month period; however, an increase in those reporting at higher rates (11-20 reports). This supports the rationale that adverse outcomes in isolation are not an appropriate measure of safety culture. It does signify the NHS is still lagging behind other safety critical industries and more work is required for improvement.

Strengths
The study was conducted in a large tertiary teaching hospital with multiple team members together with a wide variety of maternal and neonatal complex cases. There was a robust evaluation of the impact of safety culture and the quality of training delivered to inform future programs. The study demonstrates proof of concept for a specific human factors training program and therefore would benefit from further evaluation through cluster interventions in other maternity trusts.

Limitations
The study is observational and confounding factors cannot be completely controlled for. The study team made a pragmatic assessment of the training activity and cultural influences happening at the same time as our program and did not find any which were considered likely to have an effect on culture.
A strong confounding factor is the introduction of strong local leadership in human factors, as well as the educational intervention per se. This, however, should be considered as a factor when attempting to reproduce these results.
The study incorporated perceptions from all maternity staff groups during different shift patterns; however, only a proportion of staff received the training and the overall cohort size was small and susceptible to bias because of loss to follow-up. The demographic characteristics of the population also lack ethnic diversity because only 8% are nonwhite.
The Hospital Survey of Patient Safety Culture was developed in the United States of America with a notably different healthcare system, and there is some speculation on the relevance to the NHS. Another criticism of the survey is the length and time it takes for completion with the potential effect on validity and reliability of participant response. The survey has been psychometrically analyzed by a Scottish NHS data set, which replicated the 12 original domains of the survey. It concluded that no modifications are required and use of the survey would allow for crossnational comparisons. 23 The anonymity of data was prioritized to encourage participation and honest opinions to generate an accurate representation of safety culture. However, this meant the authors could not calculate the Cronbach α to assess the internal reliability.

Conclusions and Recommendations
The study suggests that multidisciplinary human factors training in maternity can positively influence patient safety culture to improve patient safety and consequently reduce adverse outcomes.
Future studies of cluster implementation of the training program are required to establish if this improvement in safety culture can be replicated in other maternity unit and specialities in the NHS.