DOE Safety Analysis Methods

DOE uses a “Safety Analysis” process to develop controls (safety interventions) for non-reactor nuclear facilities. Briefly, this Safety Analysis method includes four basic steps that involve several substeps:

1. Identifying hazards by the participating institutions prior to the grouping of risk assessments.
2. Performing hazard evaluations using failure events from existing risk assessments.
3. Selecting candidate accidents by:
   1. “Binning” failure events into accident categories and according to other relevant criteria.
   2. Selecting representative cases (emblematic case scenarios representing a particular contour of risk) to further evaluate and identify risks and risk contributors.
4. Identifying safety controls or interventions to reduce risk.

Adapting the Method to Health Care

Terminology, contexts, and processes in health care differ from other high-risk industries, and these differences must be accommodated for effective application of the Leveraging Existing Assessments of Risk Now (LEARN) Safety Analysis method to health care. The following sections describe the initial modifications of the process and criteria necessary for this method to be applied effectively to health care.

To be clear, this LEARN Safety Analysis method is not intended to provide an epidemiology of types of events. The LEARN Safety Analysis method is more akin to a meta-analysis of risk assessment data to glean and combine the results of many studies. These types of data qualitatively analyze existing risks inherent in medical care processes and cannot adequately generate rates of events.

FMEA prospective risk assessments are intended to specify the particular way that processes fail. Risk assessments are conducted on processes thought to need improvement. The results of these risk assessments are failure modes—i.e., how the system fails. The risks related to these fail points, common to a number of institutions’ risk assessments (generic risks), are identified as generic risks. The contributors to these risks that are common to a number of institutions’ risk assessments are generic risk contributors and are identified and described. The processes and criteria described here will be further modified iteratively through use.

The DOE method has been adapted from evidence-based criteria found in the available patient safety literature related to performance-shaping factors and child-specific risk factors by applying different accident category/event types appropriate for health care. The extent of an increase or decrease in the frequency of risk and resulting consequence(s) are considered based on these criteria.

Step 1: Identify Hazards

The hazards are identified in the risk assessments that are subsequently grouped.

Step 2: Hazard Evaluations

This step is also completed during the risk assessments (i.e., FMEA, root cause analysis [RCA], probabilistic risk assessment [PRA]). The following questions are typically asked (Table 5):

Table 5

Risk bin categories.

• “What can go wrong?” — to identify fail points.
• “How likely is it?” — to identify frequency.
• “What are the consequences?” — to identify harm.

Table 1 provides an example the types of information received in the risk assessments for analysis.⁴

Table 1

Example FMECA worksheet for correct blood transfusion.

Step 3: Selecting a Candidate Accident

Selecting the candidate accident requires multiple substeps. The fail points are categorized according to the type of event, using the categories, performance-shaping factors, and patient characteristics (e.g., child-specific risk factors) described below and underlying thematic similarities (i.e. handoffs, verification of task completion). This set of categories, developed through a review of patient safety events, is appropriate for health care. The categories are effective for designating medical care processes in both hospital-based and ambulatory medical care.⁵

Step 3.1: Accident categories/event types (Table 2).

Table 2

Accident categories/event types.

Step 3.2: Criteria – Performance-shaping factors. The fail points are categorized by performance-shaping factors (Table 3). The performance-shaping factors follow the framework presented by Charles Vincent as “factors that influence clinical practice.”⁶

Table 3

Performance-shaping factors in health care.

Step 3.3: Criteria – Child-specific risk factors. There is growing evidence that the epidemiology of errors and patient safety risk differs in pediatrics from that of adult medical care.^{7, 8, 9} Specific characteristics of children—“child-specific risk factors”—have been shown to lead to patient safety risk.^{5, 9, 10} Nevertheless, the literature establishing high-priority targets and effective interventions for pediatric patient safety is relatively limited.^{11, 12} Further study is needed to improve the safety of children’s medical care. Most inpatient and emergency medical care for children is delivered in institutions that primarily treat adults and may have only a small pediatric service.

These institutions are unlikely to have a pediatric emergency medicine physician on staff, and they may lack basic pediatric equipment and skills.^{12, 13} They are also unlikely to conduct proactive risk assessments that focus on children’s medical care and might not have the requisite personnel or expertise to conduct such an analysis. Research on pediatric patient safety has established “child-specific risk factors” (Table 4) and has demonstrated how these factors contribute to patient safety risk. Studies are emerging that demonstrate the need for pediatric customization of safety interventions to prevent increases in morbidity and mortality when safety interventions are implemented.^{14, 15} The application of these factors can add one step to a risk assessment to specify particular risks to child patients from the results of a general risk assessment.

Table 4

Child-specific risk factor categories.

For example, the context of pediatric medication ordering requires the consideration of two child-specific factors: (1) physical characteristics (e.g., variable size) and (2) physiologic development (e.g., limited or variable physiologic development). Medication dosage must be customized based on weight and physiologic state (e.g., kidney function). These additional steps in the medication ordering process can result in increased risk of error. What might seem to be a relatively minor misplacement of a decimal point in calculating the medication dose can result in a 10-fold error, often with serious consequences for pediatric patients. Additional criteria could be applied for other subpopulations of patients. Figure 1 shows further classification based on child-specific factors.

Figure 1

Additional analysis using child-specific risk factors.

In applying the method to medicine, a representative case of this type of failure would be further assessed to identify the potential impact of the performance-shaping factors of emergency medicine. For example, the influence of performance-shaping factors, such as frequent interruptions and time pressures on medication ordering, would be assessed to estimate the frequency of risk and determine the risk bin.

Through the application of these child-specific risk factor criteria, an additional analysis could be conducted to further specify particular risks specific to children’s health care to inform safety improvement of the subpopulation of pediatric patients. Additional criteria could be applied for other subpopulation of patients. Figure 1 show further classification based on child-specific factors.

Step 3A. Risk-binning analysis protocol to identify significant risks. The categories of frequency and consequence of each fail point are used in combination to identify the significant (relatively higher) risks through a process called “risk binning.” Table 5 illustrates a matrix for the application of these sets of criteria. In this schema, risks assessed in Risk Bin IV are low to moderate consequence and extremely unlikely to beyond extremely unlikely. The highest risks with the highest consequence are in Risk Bin I, the next highest in Risk Bin II, and so forth. Failure events in Risk Bins I and II become the focus for targeted attention.

Identified failure events in the assessed processes are reviewed for frequency of occurrence, the resulting consequence, and the safeguard effectiveness (Table 6) based on the results of the risk assessment. They are then modified iteratively through the review of a dedicated risk assessment team, according to the child-specific risk factors and “medicine performance-shaping factors,” to evaluate the potential need for adjustment to increase or decrease the assessment of frequency, and consequence. The combination of frequency and consequence categories are used to calculate a level of patient risk that is defined as High, Medium, or Low. Safeguard effectiveness is dropped out of this analysis as safeguards usually refer to institution specific safety features. A separate analysis of safeguards can generate particularly effective safeguards that may warrant broader adoption.

Table 6

Frequency and consequence of failure mode categories and safeguard effectiveness categories.

Step 3B. Selection of representative cases. In the adapted LEARN Safety Analysis methodology, after “risk binning” the failures, representative cases are selected for each represented accident category that falls into Risk Bins I or II, the highest risk categories. Representative cases are those that are emblematic of the risk scenarios identified. For example, the selection of representative cases in the nuclear industry is based primarily on grouping failure events by causes, physical characteristics, and the potential for severity of consequences for particular relevant phenomena, such as fire. In this study, these groups are called “failure categories.” Representative cases are then selected from the failure categories.

Typically, in the DOE Safety Analysis process for important topics—such as facility handling or processing of nuclear material—the types of representative cases would include such cases as fires, leaks, and load drops. Representative cases provide an embedded context of risk that enables exploration of the complexity of the context.

Selection of representative cases for this adapted methodology uses the child-specific factors (e.g., medication ordering, accident category, variable size and weight, and immature physiology of a young child), the performance-shaping factors (e.g., distractions and noise, verbal communication of orders, and lack of staff with pediatric training), and the underlying thematic failure similarities as categories. The cases themselves determine the need for refinement. This categorization results in generic themes of risks and specific contributors to these risks and the selection of potential risk-reducing interventions.

Step 4: Identify Risk Contributors

Examination of representative cases identifies generic risks and risk contributors. Through iterative analysis, these risks and risk contributors should be listed and grouped to consolidate and direct attention toward high-impact contexts of risk. Risk assessments can be reviewed to examine specific risks found, based on specific organizational or process features called “institutional artifacts,” risks or safeguards due to the specific method, process, or a specific organizational structure present. Potential examples of institutional artifacts we may encounter include computerized physician order entry (CPOE) or inclusion of a pediatric pharmacist in rounds on a particular service.

Step 5: Design of Safety Interventions and Controls

In the analysis of representative cases, the hierarchy of safety improvement strategies can be applied toward development of potential safety interventions aimed at mitigating the generic risks and risk contributors. We present here a modified hierarchy of interventions based initially on that developed by Vaida and The Institute for Safe Medication Practices.¹⁶ The design of potential safety interventions can be informed by the error preventing or mitigating strength of the intervention. The modified hierarchy of interventions includes:

1. Forcing functions.
2. Automation, computerization, and technology.
3. Standardization and protocols.
4. Staffing organization.
5. Policies, rules, and expectations.
6. Checklists & double-checks.
7. Risk assessment and communication errors.
8. Education and information.
9. Personal initiative – vigilance.

It is important to remember that the resulting safety interventions require further testing before they can contribute new or additional fail-points. This LEARN Safety Analysis method can then become another method for prospective assessment of implementation of safety interventions across institutions.