The impact of COVID‐19 workflow changes on radiation oncology incident reporting

Abstract Background The Ottawa Hospital's Radiation Oncology program maintains the Incident Learning System (ILS)—a quality assurance program that consists of report submissions of errors and near misses arising from all major domains of radiation. In March 2020, the department adopted workflow changes to optimize patient and provider safety during the COVID‐19 pandemic. Purpose In this study, we analyzed the number and type of ILS submissions pre‐ and postpandemic precautions to assess the impact of COVID‐19‐related workflow changes. Methods ILS data was collected over six one‐year time periods between March 2016 and March 2021. For all time periods, the number of ILS submissions were counted. Each ILS submission was analyzed for the specific treatment domain from which it arose and its root cause, explaining the impetus for the error or near miss. Results Since the onset of COVID‐19‐related workflow changes, the total number of ILS submissions have reduced by approximately 25%. Similarly, there were 30% fewer ILS submissions per number of treatment courses compared to prepandemic data. There was also an increase in the proportion of “treatment planning” ILS submissions and a 50% reduction in the proportion of “decision to treat” ILS submissions compared to previous years. Root cause analysis revealed there were more incidents attributable to “poor, incomplete, or unclear documentation” during the pandemic year. Conclusions COVID‐19 workflow changes were associated with fewer ILS submissions, but a relative increase in submissions stemming from poor documentation and communication. It is imperative to analyze ILS submission data, particularly in a changing work environment, as it highlights the potential and realized mistakes that impact patient and staff safety.

are encouraged to submit reports of errors and near misses arising from all major domains of radiation medicine including treatment simulation, planning, delivery, and patient care. Incidents are submitted to and logged in an electronic ILS, which has been specifically tailored for radiation oncology. The system has been in use since 2009. Incidents are investigated, categorized, and discussed by a multidisciplinary committee made up of medical physicists, physicians, dosimetrists, nurses, radiation therapists, and administrative personnel. The committee recommends and tracks corrective actions, including escalation of incidents,where appropriate.The system also reports relevant incidents to the national incident reporting system. 2 A successful quality assurance program should promote a culture of learning and continuous improvement. 4 The success of an ILS within that context requires that individuals recognize the importance of the system and feel comfortable reporting errors and near misses. This relies on the system being designed such that investigations are conducted in a nonpunitive way, using a just-culture approach. 7,8 One indicator of a strong safety culture is the number of incidents submitted to the ILS, with a higher number showing more individuals engaged and willing to bring forward potential events. 9,10 In March 2020, the COVID-19 pandemic forced rapid changes to healthcare workflows throughout the world. Early evidence shows that in addition to the direct impact of COVID-19 on patients and staff, the increased workload and changing work environment created a more stressful work environment. 11 Rapidly changing directives, high stress environments, and fatigue are all factors that can contribute to an increased risk of error. 12,13 As part of the hospital-wide COVID-19 preparedness plan, the Radiation Oncology Department adopted workflow changes to reduce the number of personnel required to be in hospital to perform their duties. For physicians, this meant a drastically reduced number of in-person patient assessments, even for new consults. For other program staff, including physicists and dosimetrists, many carried out their duties remotely, away from the hospital. Many employees found themselves working in a new, high-stress environment with new methods of communications and less face-to-face interaction with their colleagues. With unprecedented changes to work flow and working environment, our group sought to analyze the number and nature of reports submitted to the ILS for review. For each ILS submission, the incident type, origin domain, and root causes were extracted from the database. Each of these parameters had been previously assigned at the time of the incident investigation. The incident type is classified as either actual or potential. An actual incident is when an unwanted or unexpected change from a normal system behavior causes adverse effect to the patient or equipment. A potential incident or "near miss" is an event that could have resulted in harm to a patient but did not, either by chance or through timely intervention. The origin domain represents the specific treatment domain from which the incident arose and is categorized as one of: decision to treat, treatment simulation, treatment planning, treatment delivery, quality assurance, or patient discharge. Each incident had also previously been assigned a root cause, selected from a pre-defined list of causes built into the ILS framework. The collected data was analyzed for the number and type of ILS submissions and compared across the time periods.

Total and normalized number of incidents
Incident totals and normalized data calculated against the total number of fractions are represented in Figure 1. In addition, the number of treatment plans for each time period can be found in Table 1

Categories and root causes
As the number of ILS submissions originating from the treatment planning domain increased during the pandemic years, ILS submissions from this domain were further characterized. During the first pandemic year (2020-2021), 48.5% of treatment planning submissions were from the "transfer of treatment plan to record and verify" category compared with a prepandemic average of 19.7% (SD 3.1%). This category refers to the any part of the treatment plan preparation process where the plan documentation or treatment instructions from the treatment planning software are transferred into the record and verify system used to drive the patient treatment. Examples of these submissions include missing plan documents or approvals, incorrect shifts entered into the record and verify software, and incorrect setup fields. During the second pandemic year (2021-2022), 57.1% of treatment planning submissions were from the "creation of dose distribution and calculation of dose" category compared with a prepandemic average of 27.2% (SD 3.4%). An additional notable change during the pandemic years is the proportion of "review and approval of treatment plan" submissions. The prepandemic average for this category was 30.7% (SD 13.2%), Note: Normalized data is represented as the number of ILS submissions per treatment course initiated, as the majority of the ILS submissions originated from the decision to treat and planning stages of treatment, therefore not related to the total number of fractions while the pandemic values were 9.1% and 4.8% for the 2020-2021 and 2021-2022 pandemic years, respectively. There were no other notable differences in category representation across the years studied ( Figure 3). As the number of ILS submissions originating from the decision to treat domain were reduced during the pandemic years, ILS submissions from this domain were further characterized. On root cause analysis, there were no clear trends in the "policy not followed" cause, accounting for 29%, 12%, 21%, 15%, 25%, and 11% of all decision to treat-specific ILS submissions for the 6 years considered. "Policy not followed"could include any submission where the cause was determined to be a deviation from a policy already in place.Furthermore,the "poor, incomplete, unclear, missing documentation" root cause showed a relative increase in the first pandemic year as a percentage of all decision to treat ILS submissions. Within the "decision to treat" category, examples of incomplete documentation may include missing prescription information, poorly communicated changes to patient scheduling, or unclear instructions. The percentage of decision to treat ILS submissions due to poor or missing documentation for the 6 years starting in 2016-2017 and ending with the 2021-2022 year were 8.9%, 3.8%, 6.2%, 11.5%, 20%, and 5.6%. Finally, the "plan forgotten in progress" or "loss of attention" root causes accounted for 15% and 16% of the decision to treat-specific ILS submission for the two pandemic years 2020-2021 and 2021-2022, respectively, but only 6.7%, 4.8%, 4.2%, and 5.8% for the four preceding years. These root causes are assigned when the investigation shows that there were competing priorities or environmental distractions that led to the staff member being unable to devote sufficient focus to the task at hand.  While it may be possible that pandemic workflow changes led to fewer ILS-worthy events, this seems unlikely as changes to long-established workflow norms are more commonly associated with an increase in near misses and potential events as team members become acclimated to their new work environment and modified work conditions. 10,14,15 Therefore, it is possible that the decrease in ILS submissions is indicative of decreased engagement with ILS submissions as team members worked to balance their duties with rapidly changing hospital policy, patient care restrictions, and workflow changes. 16 To address this potential reason for the decreased ILS usage, the ILS committee sought to increase staff engagement through increased communications and improved feedback with the hope of increasing submissions since engagement in the ILS is essential during times of significant workflow changes.Further monitoring of the ILS submissions will be required to determine if the rate of submissions returns to prepandemic levels. While the number of radiotherapy treatment courses was similar when comparing prepandemic years to the two pandemic periods, the number of fractions delivered was much lower. The average number of fractions delivered over the 12-month periods prepandemic was 71,902 (SD 1594), whereas for the two time periods considered following the onset of the pandemic, the average number of fractions over 12 months was 60,570. Comparable numbers of treatment courses but fewer fractions is reflective of the department attempt to employ hypofractionated radiotherapy regimens where feasible in order to limit pandemic exposures to both patients and staff. With fewer fractions delivered during the pandemic years, we might expect there to be a smaller proportion of ILS submissions originating from the treatment delivery domain. Interestingly, however, the proportion of treatment delivery ILS submissions was higher during the pandemic years-18.5% compared with an average of 14.7% (SD 2.1%) during the prepandemic years. During the pandemic, on-treatment workflows were disrupted by time-consuming infection control requirements (including PPE requirements for radiation therapists and patients alike) combined with limited numbers of radiation therapists. This amounts to increased treatment complexity, which has been shown to be associated with increased error rates and is one potential explanation for the observed increase in treatment planning submissions during the pandemic years. 14,15

Breakdown by origin domain
During the prepandemic years, the treatment planning domain accounted for an average of 16.3% of ILS submissions (SD 2.2%), while there was a comparative increase in treatment planning submissions during the pandemic years, 28% and 27% for the 2020-2021 and 2021-2022 years, respectively. On analysis of the specific category of treatment planning submissions, it was found that there were two noteworthy changes in submission patterns during the pandemic years. During the first pandemic year, 48.5% of treatment planning submissions were from the "transfer of treatment plan to record and verify" category, more than twice the prepandemic average. Transfer of treatment plan to record and verify ILS submissions often arise from events that require multiple team members (physicists, dosimetrists, and physicians) to coordinate treatment plan amendments. Amid pandemic precautions, team members had less face-to-face interaction, relying instead on electronic forms of communication, which may not be as effective. Increased workload and communication breakdown are known sources of error in the radiation oncology work flow. 14,17 In addition, our institution initiated manual entry of couch movements just before initiation of pandemic precautions. This workflow alteration may also account for the increase in the proportion of "transfer of treatment plan to record and verify" submissions during the first pandemic year. The observed increase then returned to baseline during the second pandemic year, as the workflow alteration was no longer novel, and the affected team members acclimated to the change. During the second pandemic year (2021-2022), 57.1% of treatment planning submissions were from the "creation of dose distribution and calculation of dose" category, again, more than twice the prepandemic average to determine the etiology. Finally, for both pandemic years, the proportion of "review and approval of treatment plan" submissions was observed to be less than a quarter of the prepandemic average for this category. These ILS submissions typically arise from changes made to the treatment plan or prescription that were not clearly communicated or documented. Prior to these years, changes were made to the electronic workflow that automated more of the process when a change is made, which has led to a reduction in the number of submissions related to the review and approval of treatment plans. Therefore, it is thought that this change in relative frequency is due to intentional process changes rather than effects of pandemic-related issues.
During the pandemic year, there were an average of 50% fewer decision to treat ILS submissions compared to previous years. Root cause analysis showed no difference in the proportion of "policy not followed" submissions, suggesting good compliance with rapidly changing hospital and radiation oncology-specific workflow changes. Conversely, the relative number of "poor, incomplete, unclear, and missing documentation" decision to treat ILS reports more than doubled during the first pandemic year. With more people working from home, clear documentation becomes paramount to effective workflow. This relative increase may be reflective of a change in the mode of communication during the pandemic, with a greater reliance on electronic documentation, and fewer in-person interactions. This relative increase was not seen during the second pandemic year, which could be the result of a greater number of staff returning to on-site work so that communication patterns more closely resembled the prepandemic environment and it could also be indicative of the adjustments made by staff to the increased reliance on electronic forms of communication. Alternatively, the increase in the proportion of poor, incomplete, unclear, and missing documentation reports may represent poorer staff documentation under the weight of rapidly changing clinical and clerical duties during the first pandemic year. Within only 2 years of pandemic data, it is difficult to draw conclusions as to why there was a spike in the relative incidence of poor documentation as a root cause. Regardless of why the changes occurred, it has been well documented that poorer communication leads to a greater frequency of ILS events, and our data is consistent with this finding. 14,15 Decision to treat ILS submissions were significantly increased for the "plan forgotten in progress" or "loss of attention" root causes during the pandemic year-more than seven times that from preceding years, on average. With fewer team members working in person, a greater number of competing interests including personal and patient PPE, infection control precautions, and a change to the type of radiotherapy courses (greater number of hypofractionated courses), it is reasonable to posit that such factors contributed to lapses in maintaining focus on the intended radiotherapy plan.
Finally, the "quality assurance" domain consistently accounted for < 15% of ILS submissions across all time periods studied. This is understandable, as quality assurance processes are designed to detect ILS-worthy events and are themselves less likely to account for a significant number of ILS submissions.

CONCLUSION
Safe and effective patient care is the goal of any radiation oncology program. During the pandemic years, workflow and staffing changes were associated with a decrease in the number of ILS submissions, indicating less engagement with a vital component of quality assurance and therefore patient safety. In addition, significant changes observed regarding the types of submissions reported during the pandemic years are reflective of the unique challenges encountered during pandemic precautions. Continued engagement with ILS reporting is essential to the continued safety of the radiation oncology program, particularly during the dynamic COVID-19 pandemic or other periods of rapid change within a program. Incident learning committees should be aware of potential competing priorities during such times and actively work to ensure that the ILS continues to play a key role in promoting learning and quality improvements.