Chapter  43:  Making Health Care Safer: A Critical Analysis of Patient Safety Practices: Evidence Report/Technology Assessment Number 43

Overview

Patient safety has become a major concern of the general public and of policymakers at the State and Federal levels. This interest has been fueled, in part, by news coverage of individuals who were the victims of serious medical errors and by the publication in 1999 of the Institute of Medicine's (IOM's) report To Err is Human: Building a Safer Health System. In its report, IOM highlighted the risks of medical care in the United States and shocked the sensibilities of many Americans, in large part through its estimates of the magnitude of medical-errors-related deaths (44,000 to 98,000 deaths per year) and other serious adverse events. The report prompted a number of legislative and regulatory initiatives designed to document errors and begin the search for solutions. But Americans, who now wondered whether their next doctor's or hospital visit might harm rather than help them, began to demand concerted action.

Three months after publication of the IOM report, an interagency Federal government group, the Quality Interagency Coordination Task Force (QuIC), released its response, Doing What Counts for Patient Safety: Federal Actions to Reduce Medical Errors and Their Impact. That report, prepared at the President's request, both inventoried on-going Federal actions to reduce medical errors and listed more than 100 action items to be undertaken by Federal agencies.

An action promised by the Agency for Healthcare Research and Quality (AHRQ), the Federal agency leading efforts to research and promote patient safety, was "the development and dissemination of evidence-based, best safety practices to provider organizations." To initiate the work to be done in fulfilling this promise, AHRQ commissioned the University of California at San Francisco (UCSF) - Stanford University Evidence-based Practice Center (EPC) in January 2001 to review the scientific literature regarding safety improvement. To accomplish this, the EPC established an Editorial Board that oversaw development of this report by teams of content experts who served as authors.

Defining Patient Safety Practices

Working closely with AHRQ and the National Forum for Quality Measurement and Reporting (the National Quality Forum, or NQF) -- a public-private partnership formed in 1999 to promote a national health care quality agenda the EPC began its work by defining a patient safety practice as a type of process or structure whose application reduces the probability of adverse events resulting from exposure to the health care system across a range of diseases and procedures.

This definition is consistent with the dominant conceptual framework in patient safety, which holds that systemic change will be far more productive in reducing medical errors than will targeting and punishing individual providers. The definition's focus on actions that cut across diseases and procedures also allowed the research team to distinguish patient safety activities from the more targeted quality improvement practices (e.g., practices designed to increase the use of beta-blockers in patients who are admitted to the hospital after having a myocardial infarction). The editors recognize, however, that this distinction is imprecise.

This evidence-based review also focuses on hospital care as a starting point because the risks associated with hospitalization are significant, the strategies for improvement are better documented there than in other health care settings, and the importance of patient trust is paramount. The report, however, also considers evidence regarding other sites of care, such as nursing homes, ambulatory care, and patient self-management.

The results of this EPC study will be used by the NQF to identify a set of proven patient safety practices that should be used by hospitals. Identification of these practices by NQF will allow patients throughout the nation to evaluate the actions their hospitals and/or health care facilities have taken to improve safety.

Reporting The Evidence

As is typical for evidence-based reviews, the goal was to provide a critical appraisal of the evidence on the topic. This information would then be available to others to ensure that no practice unsupported by evidence would be endorsed and that no practice substantiated by a high level of proof would lack endorsement. Readers familiar with the state of the evidence regarding quality improvement in areas of health care where this has been a research priority (e.g., cardiovascular care) may be surprised and even disappointed, by the paucity of high quality evidence in other areas of health care for many patient safety practices. One reason for this is the relative youth of the field. Just as there had been little public recognition of the risks of health care prior to the first IOM report, there has been relatively little attention paid to such risks - and strategies to mitigate them - among health professionals and researchers.

Moreover, there are a number of methodologic reasons why research in patient safety is particularly challenging. Many practices (e.g., the presence of computerized physician order entry systems, modifying nurse staffing levels) cannot be the subject of double-blind studies because their use is evident to the participants. Second, capturing all relevant outcomes, including "near misses" (such as a nurse catching an excessive dosage of a drug just before it is administered to a patient) and actual harm, is often very difficult. Third, many effective practices are multidimensional, and sorting out precisely which part of the intervention works is often quite challenging. Fourth, many of the patient safety problems that generate the most concern (wrong-site surgery, for example) are uncommon enough that demonstrating the success of a "safety practice" in a statistically meaningful manner with respect to outcomes is all but impossible.

Finally, establishing firm epidemiologic links between presumed (and accepted) causes and adverse events is critical, and frequently difficult. For instance, in studying an intuitively plausible "risk factor" for errors, such as "fatigue," analyses of errors commonly reveal the presence of fatigued providers (because many health care providers work long hours and/or late at night). The question is whether or not fatigue is over-represented among situations that lead to errors. The point is not that the problem of long work-hours should be ignored, but rather that strong epidemiologic methods need to be applied before concluding that an intuitive cause of errors is, in fact, causal.

Researchers now believe that most medical errors cannot be prevented by perfecting the technical work of individual doctors, nurses, or pharmacists. Improving patient safety often involves the coordinated efforts of multiple members of the health care team, who may adopt strategies from outside health care. The report reviews several practices whose evidence came from the domains of commercial aviation, nuclear safety, and aerospace, and the disciplines of human factors engineering and organizational theory. Such practices include root cause analysis, computerized physician order entry and decision support, automated medication dispensing systems, bar coding technology, aviation-style preoperative checklists, promoting a "culture of safety," crew resource management, the use of simulators in training, and integrating human factors theory into the design of medical devices and alarms. In reviewing these practices, the research team sought to be flexible regarding standards of evidence, and included research evidence that would not have been considered for medical interventions. For example, the randomized trial that is appropriately hailed as the "gold standard" in clinical medicine is not used in aviation, as this design would not capture all relevant information. Instead, detailed case studies and industrial engineering research approaches are utilized.

Methodology

To facilitate identification and evaluation of potential patient safety practices, the Editorial Board divided the content for the project into different domains. Some cover "content areas," including traditional clinical areas such as adverse drug events, nosocomial infections, and complications of surgery, but also less traditional areas such as fatigue and information transfer. Other domains consist of practices drawn from broad (primarily nonmedical) disciplines likely to contain promising approaches to improving patient safety (e.g., information technology, human factors research, organizational theory). Once this list was created -- with significant input from patient safety experts, clinician-researchers, AHRQ, and the NQF Safe Practices Committee -- the editors selected teams of authors with expertise in the relevant subject matter and/or familiarity with the techniques of evidence-based review and technology appraisal.

The authors were given explicit instructions regarding search strategies for identifying safety practices for evaluation (including explicit inclusion and exclusion criteria) and criteria for assessing each practice's level of evidence for efficacy or effectiveness in terms of study design and study outcomes. Some safety practices did not meet the inclusion criteria because of the paucity of evidence regarding efficacy or effectiveness but were included in the report because an informed reader might reasonably expect them to be evaluated or because of the depth of public and professional interest in them. For such high profile topics (such as bar coding to prevent misidentifications), the researchers tried to fairly present the practice's background, the experience with the practice thus far, and the evidence (and gaps in the evidence) regarding the practice's value.

For each practice, authors were instructed to research the literature for information on:

• prevalence of the problem targeted by the practice;

• severity of the problem targeted by the practice;

• the current utilization of the practice;

• evidence on efficacy and/or effectiveness of the practice;

• the practice's potential for harm;

• data on cost, if available; and

• implementation issues.

The report presents the salient elements of each included study (e.g., study design, population/setting, intervention details, results), and highlights any important weaknesses and biases of these studies. Authors were not asked to formally synthesize or combine the evidence across studies (e.g., perform a meta-analysis) as part of their task.

The Editorial Board and the Advisory Panel reviewed the list of domains and practices to identify gaps in coverage. Submitted chapters were reviewed by the Editorial Board and revised by the authors, aided by feedback from the Advisory Panel. Once the content was finalized, the editors analyzed and ranked the practices using a methodology summarized below.

Summarizing the Evidence and Rating the Practices

Because the report is essentially an anthology of a diverse and extensive group of patient safety practices with highly variable relevant evidence, synthesizing the findings was challenging, but necessary to help readers use the information. Two of the most obvious uses for this report are: 1) to inform efforts of providers and health care organizations to improve the safety of the care they provide, and 2) to inform AHRQ, other research agencies, and foundations about potential fruitful investments for their research support. Other uses of the information are likely. In fact, the National Quality Forum plans to use this report to help identify a list of patient safety practices that consumers and others should know about as they choose among the health care provider organizations to which they have access.

In an effort to assist both health care organizations interested in taking substantive actions to improve patient safety and research funders seeking to spend scarce resources wisely, AHRQ asked the EPC to rate the evidence and rank the practices by opportunity for safety improvement and by research priority. This report, therefore, contains two lists.

To create these lists, the editors aimed to separate the practices that are most promising or effective from those that are least so on a range of dimensions, without implying any ability to calibrate a finely gradated scale for those practices in between. The editors also sought to present the ratings in an organized, accessible way while highlighting the limitations inherent in their rating schema. Proper metrics for more precise comparisons (e.g., cost-effectiveness analysis) require more data than are currently available in the literature.

Three major categories of information were gathered to inform the rating exercise:

• Potential Impact of the Practice: based on prevalence and severity of the patient safety target, and current utilization of the practice;

• Strength of the Evidence Supporting the Practice: including an assessment of the relative weight of the evidence, effect size, and need for vigilance to reduce any potential negative collateral effects of the practice; and

  • Implementation: considering costs, logistical barriers, and policy issues.

For all of these data inputs into the practice ratings, the primary goal was to find the best available evidence from publications and other sources. Because the literature has not been previously organized with an eye toward addressing each of these areas, most of the estimates could be improved with further research, and some are informed by only general and somewhat speculative knowledge. In the summaries, the editors have attempted to highlight those assessments made with limited data.

The four-person editorial team independently rated each of the 79 practices using general scores (e.g., High, Medium, Low) for a number of dimensions, including those italicized in the section above. The editorial team convened for 3 days in June, 2001 to compare scores, discuss disparities, and come to consensus about ratings for each category.

In addition, each member of the team considered the totality of information on potential impact and support for a practice to score each of these factors on a 0 to 10 scale (creating a "Strength of the Evidence" list). For these ratings, the editors took the perspective of a leader of a large health care enterprise (e.g., a hospital or integrated delivery system) and asked the question, "If I wanted to improve patient safety at my institution over the next 3 years and resources were not a significant consideration, how would I grade this practice?" For this rating, the Editorial Board explicitly chose not to formally consider the difficulty or cost of implementation in the rating. Rather, the rating simply reflected the strength of the evidence regarding the effectiveness of the practice and the probable impact of its implementation on reducing adverse events related to health care exposure. If the patient safety target was rated as "High" impact and there was compelling evidence (i.e., "High" relative study strength) that a particular practice could significantly reduce (e.g., "Robust" effect size) the negative consequences of exposure to the health care system (e.g., hospital-acquired infections), raters were likely to score the practice close to 10. If the studies were less convincing, the effect size was less robust, or there was a need for a "Medium" or "High" degree of vigilance because of potential harms, then the rating would be lower.

At the same time, the editors also rated the usefulness of conducting more research on each practice, emphasizing whether there appeared to be questions that a research program might have a reasonable chance of addressing successfully (creating a "Research Priority" list). Here, they asked themselves, "If I were the leader of a large agency or foundation committed to improving patient safety, and were considering allocating funds to promote additional research, how would I grade this practice?" If there was a simple gap in the evidence that could be addressed by a research study or if the practice was multifaceted and implementation could be eased by determining the specific elements that were effective, then the research priority was high. (For this reason, some practices are highly rated on both the "Strength of the Evidence" and "Research Priority" lists.) If the area was one of high potential impact (i.e., large number of patients at risk for morbid or mortal adverse events) and a practice had been inadequately researched, then it would also receive a relatively high rating for research need. Practices might receive low research scores if they held little promise (e.g., relatively few patients are affected by the safety problem addressed by the practice or a significant body of knowledge already demonstrates the practice's lack of utility). Conversely, a practice that was clearly effective, low cost, and easy to implement would not require further research and would also receive low research scores.

In rating both the strength of the evidence and the research priority, the purpose was not to report precise 0 to 10 scores, but to develop general "zones" or practice groupings. This is important because better methods are available for making comparative ratings when the data inputs are available. The relative paucity of the evidence dissuaded the editors from using a more precise, sophisticated, but ultimately unfeasible, approach.

Clear Opportunities for Safety Improvement

The following 11 patient safety practices were the most highly rated (of the 79 practices reviewed in detail in the full report and ranked in the Executive Summary Addendum, AHRQ Publication No. 01-E057b) in terms of strength of the evidence supporting more widespread implementation. Practices appear in descending order, with the most highly rated practices listed first. Because of the imprecision of the ratings, the editors did not further divide the practices, nor indicate where there were ties.

• Appropriate use of prophylaxis to prevent venous thromboembolism in patients at risk;

• Use of perioperative beta-blockers in appropriate patients to prevent perioperative morbidity and mortality;

• Use of maximum sterile barriers while placing central intravenous catheters to prevent infections;

• Appropriate use of antibiotic prophylaxis in surgical patients to prevent perioperative infections;

• Asking that patients recall and restate what they have been told during the informed consent process;

• Continuous aspiration of subglottic secretions (CASS) to prevent ventilator-associated pneumonia;

• Use of pressure relieving bedding materials to prevent pressure ulcers;

• Use of real-time ultrasound guidance during central line insertion to prevent complications;

• Patient self-management for warfarin (Coumadin™) to achieve appropriate outpatient anticoagulation and prevent complications;

• Appropriate provision of nutrition, with a particular emphasis on early enteral nutrition in critically ill and surgical patients; and

• Use of antibiotic-impregnated central venous catheters to prevent catheter-related infections.

This list is generally weighted toward clinical rather than organizational matters, and toward care of the very, rather than the mildly or chronically ill. Although more than a dozen practices considered were general safety practices that have been the focus of patient safety experts for decades (i.e., computerized physician order entry, simulators, creating a "culture of safety," crew resource management), most research on patient safety has focused on more clinical areas. The potential application of practices drawn from outside health care has excited the patient safety community, and many such practices have apparent validity. However, clinical research has been promoted by the significant resources applied to it through Federal, foundation, and industry support. Since this study went where the evidence took it, more clinical practices rose to the top as potentially ready for implementation.

Clear Opportunities for Research

Until recently, patient safety research has had few champions, and even fewer champions with resources to bring to bear. The recent initiatives from AHRQ and other funders are a promising shift in this historical situation, and should yield important benefits.

In terms of the research agenda for patient safety, the following 12 practices rated most highly, as follows:

• Improved perioperative glucose control to decrease perioperative infections;

• Localizing specific surgeries and procedures to high volume centers;

• Use of supplemental perioperative oxygen to decrease perioperative infections;

• Changes in nursing staffing to decrease overall hospital morbidity and mortality;

• Use of silver alloy-coated urinary catheters to prevent urinary tract infections;

• Computerized physician order entry with computerized decision support systems to decrease medication errors and adverse events primarily due to the drug ordering process;

• Limitations placed on antibiotic use to prevent hospital-acquired infections due to antibiotic-resistant organisms;

• Appropriate use of antibiotic prophylaxis in surgical patients to prevent perioperative infections;

• Appropriate use of prophylaxis to prevent venous thromboembolism in patients at risk;

• Appropriate provision of nutrition, with a particular emphasis on early enteral nutrition in critically ill and post-surgical patients;

• Use of analgesics in the patient with an acutely painful abdomen without compromising diagnostic accuracy; and

• Improved handwashing compliance (via education/behavior change; sink technology and placement; or the use of antimicrobial washing substances).

Of course, the vast majority of the 79 practices covered in this report would benefit from additional research. In particular, some practices with longstanding success outside of medicine (e.g., promoting a culture of safety) deserve further analysis, but were not explicitly ranked due to their unique nature and the present weakness of the evidentiary base in the health care literature.

Conclusions

This report represents a first effort to approach the field of patient safety through the lens of evidence-based medicine. Just as To Err is Human sounded a national alarm regarding patient safety and catalyzed other important commentaries regarding this vital problem, this review seeks to plant a seed for future implementation and research by organizing and evaluating the relevant literature. Although all those involved tried hard to include all relevant practices and to review all pertinent evidence, inevitably some of both were missed. Moreover, the effort to grade and rank practices, many of which have only the beginnings of an evidentiary base, was admittedly ambitious and challenging. It is hoped that this report provides a template for future clinicians, researchers, and policy makers as they extend, and inevitably improve upon, this work.

In the detailed reviews of the practices, the editors have tried to define (to the extent possible from the literature) the associated costs -- financial, operational, and political. However, these considerations were not factored into the summary ratings, nor were judgments made regarding the appropriate expenditures to improve safety. Such judgments, which involve complex tradeoffs between public dollars and private ones, and between saving lives by improving patient safety versus doing so by investing in other health care or non-health care practices, will obviously be critical. However, the public reaction to the IOM report, and the media and legislative responses that followed it, seem to indicate that Americans are highly concerned about the risks of medical errors and would welcome public and private investment to decrease them. It seems logical to infer that Americans value safety during a hospitalization just as highly as safety during a transcontinental flight.

1.1. General Overview

The Institute of Medicine's (IOM) report, To Err is Human: Building a Safer Health System,¹ highlighted the risks of medical care in the United States. Although its prose was measured and its examples familiar to many in the health professions (for example, the studies estimating that up to 98,000 Americans die each year from preventable medical errors were a decade old), the report shocked the sensibilities of many Americans. More importantly, the report undermined the fundamental trust that many previously had in the health care system.

The IOM report prompted a number of legislative and regulatory initiatives designed to document errors and begin the search for solutions. These initiatives were further catalyzed by a second IOM report entitled Crossing the Quality Chasm: A New Health System for the 21^st Century,² which highlighted safety as one of the fundamental aims of an effective system. But Americans, who now wondered whether their next health care encounter might harm rather than help them, began to demand concerted action.

Making Health Care Safer represents an effort to determine what it is we might do in an effort to improve the safety of patients. In January 2001, the Agency for Healthcare Research and Quality (AHRQ), the Federal agency taking the lead in studying and promoting patient safety, commissioned the UCSF-Stanford Evidence-based Practice Center (EPC) to review the literature as it pertained to improving patient safety. In turn, the UCSF-Stanford EPC engaged 40 authors at 11 institutions around the United States to review more than 3000 pieces of literature regarding patient safety practices. Although AHRQ expected that this evidence-based review would have multiple audiences, the National Quality Forum (NQF) -- a public-private partnership formed in the Clinton Administration to promote a national quality agenda -- was particularly interested in the results as it began its task of recommending and implementing patient safety practices supported by the evidence.

1.2. How to Use this Report

1.3. Acknowledgments

We are grateful to our authors, whose passion for the evidence overcame the tremendous time pressure driven by an unforgiving timeline. They succumbed to our endless queries, as together we attempted to link the literature from a variety of areas into an evidence-based repository whose presentation respected an overarching framework, notwithstanding the variety of practices reviewed. Our Managing Editor, Amy J. Markowitz, JD, edited the manuscript with incredible skill and creativity. Each part was reviewed and improved multiple times because of her unending dedication to the project. Susan B. Nguyen devoted countless hours assisting the team in every imaginable task, and was critical in keeping the project organized and on track. Invaluable editorial and copyediting assistance was provided by Kathleen Kerr, Talia Baruth, and Mary Whitney. We thank Phil Tiso for helping produce the electronic version of the Report. We also thank A. Eugene Washington, MD, MSc, Director of the UCSF-Stanford Evidence-based Practice Center, for his guidance and support.

We were aided by a blue-ribbon Advisory Panel whose suggestions of topics, resources, and ideas were immensely helpful to us throughout the life of the project. Given the time constraints, there were doubtless many constructive suggestions that we were unable to incorporate into the Report. The responsibility for such omissions is ours alone. We want to recognize the important contribution of our advisors to our thinking and work. The Advisory Panel's members are:

• David M. Gaba, MD, Director, Patient Safety Center of Inquiry at Veterans Affairs Palo Alto Health Care System, Professor of Anesthesia, Stanford University School of Medicine

• John W. Gosbee, MD, MS, Director of Patient Safety Information Systems, Department of Veterans Affairs National Center for Patient Safety

• Peter V. Lee, JD, President and Chief Executive Officer, Pacific Business Group on Health

• Arnold Milstein, MD, MPH, Medical Director, Pacific Business Group on Health; National Health Care Thought Leader, William M. Mercer, Inc.

• Karlene H. Roberts, PhD, Professor, Walter A. Haas School of Business, University of California, Berkeley

• Stephen M. Shortell, PhD, Blue Cross of California Distinguished Professor of Health Policy and Management and Professor of Organization Behavior, University of California, Berkeley

We thank the members of the National Quality Forum's (NQF) Safe Practices Committee, whose co-chairs, Maureen Bisognano and Henri Manasse, Jr., PhD, ScD, and members helped set our course at the outset of the project. NQF President and Chief Executive Officer Kenneth W. Kizer, MD, MPH, Vice President Elaine J. Power, and Program Director Laura N. Blum supported the project and coordinated their committee members' helpful suggestions as the project proceeded. We appreciate their patience during the intensive research period as they waited for the final product.

Finally, we are indebted to the Agency for Healthcare Research and Quality (AHRQ), which funded the project and provided us the intellectual support and resources needed to see it to successful completion. Jacqueline Besteman, JD of AHRQ's Center for Practice and Technology Assessment, and Gregg Meyer, MD, and Nancy Foster, both from AHRQ's Center for Quality Improvement and Patient Safety, were especially important to this project. John Eisenberg, MD, MBA, AHRQ's Administrator, was both a tremendous supporter of our work and an inspiration for it.

The Editors San Francisco and Palo Alto, California June, 2001

Definition and Scope

For this project the UCSF-Stanford Evidence-based Practice Center (EPC) defined a patient safety practice as "a type of process or structure whose application reduces the probability of adverse events resulting from exposure to the health care system across a range of conditions or procedures." Examples of practices that meet this definition include computerized physician order entry (Chapter 6), thromboembolism prophylaxis in hospitalized patients (Chapter 31), strategies to reduce falls among hospitalized elders (Chapter 26), and novel education strategies such as the application of "crew resource management" to train operating room staff (Chapter 44). By contrast, practices that are disease-specific and/or directed at the underlying disease and its complications (eg, use of aspirin or beta-blockers to treat acute myocardial infarction) rather than complications of medical care are not included as "patient safety practices." Further discussion of these issues can be found in Chapter 1.

In some cases, the distinction between patient safety and more general quality improvement strategies is difficult to discern. Quality improvement practices may also qualify as patient safety practices when the current level of quality is considered "unsafe," but standards to measure safety are often variable, difficult to quantify, and change over time. For example, what constitutes an "adequate" or "safe" level of accuracy in electrocardiogram or radiograph interpretation? Practices to improve performance to at least the "adequate" threshold may reasonably be considered safety practices because they decrease the number of diagnostic errors of omission. On the other hand, we considered practices whose main intent is to improve performance above this threshold to be quality improvement practices. An example of the latter might be the use of computer algorithms to improve the sensitivity of screening mammography.¹

We generally included practices that involved acute hospital care or care at the interface between inpatient and outpatient settings. This focus reflects the fact that the majority of the safety literature relates to acute care and the belief that systems changes may be more effectively achieved in the more controlled environment of the hospital. However, practices that might be applicable in settings in addition to the hospital were not excluded from consideration. For example, the Report includes practices for preventing decubitus ulcers (Chapter 27) that could be applied in nursing homes as well as hospitals.

The EPC team received input regarding the scope of the Report from the Agency for Healthcare Research and Quality (AHRQ), which commissioned the report. In addition, the EPC team participated in the public meeting of the National Quality Forum (NQF) Safe Practices Committee on January 26, 2001. The NQF was formed in 1999 by consumer, purchaser, provider, health plan, and health service research organizations to create a national strategy for quality improvement. Members of the Safe Practices Committee collaborated with the EPC team to develop the scope of work that would eventually become this Report.

Organization by Domains

To facilitate identification and evaluation of potential patient safety practices, we divided the content for the project into different domains. Some cover "content areas" (eg, adverse drug events, nosocomial infections, and complications of surgery). Other domains involve identification of practices within broad (primarily "non-medical") disciplines likely to contain promising approaches to improving patient safety (eg, information technology, human factors research, organizational theory). The domains were derived from a general reading of the literature and were meant to be as inclusive as possible. The list underwent review for completeness by patient safety experts, clinician-researchers, AHRQ, and the NQF Safe Practices Committee. For each domain we selected a team of author/collaborators with expertise in the relevant subject matter and/or familiarity with the techniques of evidence-based review and technology appraisal. The authors, all of whom are affiliated with major academic centers around the United States, are listed on page 9-13.

Identification of Safety Practices for Evaluation

Search Strategy

Table 3.1. Search strategy recommended by coordinating (more...)

Table 3.1. Search strategy recommended by coordinating team to participating reviewers

a)	Electronic bibliographic databases. All searches must include systematic searches of MEDLINE and the Cochrane Library. For many topics it will be necessary to include other databases such as the Cumulative Index to Nursing & Allied Health (CINAHL), PsycLit (PsycINFO), the Institute for Scientific Information's Science Citation Index Expanded, Social Sciences Citation Index Arts & Humanities Citation Index, INSPEC(physics, electronics and computing), and ABI/INFORM (business, management, finance, and economics).
b)	Hand-searches of bibliographies of retrieved articles and tables of contents of key journals.
c)	Grey literature. For many topics it will necessary to review the "grey literature," such as conference proceedings, institutional reports, doctoral theses, and manufacturers' reports.
d)	Consultation with experts or workers in the field.

Table 3.2. Inclusion/Exclusion criteria for practices (more...)

Table 3.2. Inclusion/Exclusion criteria for practices

Inclusion Criteria
1.	The practice can be applied in the hospital setting or at the interface between inpatient and outpatient settings AND can be applied to a broad range of health care conditions or procedures.
2.	Evidence for the safety practice includes at least one study with a Level 3 or higher study design AND a Level 2 outcome measure. For practices not specifically related to diagnostic or therapeutic interventions, a Level 3 outcome measure is adequate. (See Table 3.3 for definition of "Levels").
Exclusion Criterion
1.	No study of the practice meets the methodologic criteria above.

The UCSF-Stanford Evidence-based Practice Center Coordinating Team ("The Editors") provided general instructions (Table 3.1) to the teams of authors regarding search strategies for identifying safety practices for evaluation. As necessary, the Editors provided additional guidance and supplementary searches of the literature.

To meet the early dissemination date mandated by AHRQ, the Editors did not require authors to search for non-English language articles or to use EMBASE. These were not specifically excluded, however, and authoring teams could include non-English language articles that addressed important aspects of a topic if they had translation services at their disposal. The Editors did not make recommendations on limiting database searches based on publication date. For this project it was particularly important to identify systematic reviews related to patient safety topics. Published strategies for retrieving systematic reviews have used proprietary MEDLINE interfaces (eg, OVID, SilverPlatter) that are not uniformly available. Moreover, the performance characteristics of these search strategies is unknown.^2,3 Therefore, these strategies were not explicitly recommended. The Editors provided authors with a search algorithm (available upon request) that uses PubMed, the freely available search interface from the National Library of Medicine, designed to retrieve systematic reviews with high sensitivity without overwhelming users with "false positive" hits.⁴

The Editors also performed independent searches of bibliographic databases and grey literature for selected topics.⁵ Concurrently, the EPC collaborated with NQF to solicit information about evidence-based practices from NQF members, and consulted with the project's Advisory Panel (page 31), whose members provided additional literature to review.

Inclusion and Exclusion Criteria

The EPC established criteria for selecting which of the identified safety practices warranted evaluation. The criteria address the applicability of the practice across a range of conditions or procedures and the available evidence of the practices' efficacy or effectiveness.

Practices that have only been studied outside the hospital setting or in patients with specific conditions or undergoing specific procedures were included if the authors and Editors agreed that the practices could reasonably be applied in the hospital setting and across a range of conditions or procedures. To increase the number of potentially promising safety practices adapted from outside the field of medicine, we included evidence from studies that used less rigorous measures of patient safety as long as the practices did not specifically relate to diagnostic or therapeutic interventions. These criteria facilitated the inclusion of areas such as teamwork training (Chapter 44) and methods to improve information transfer (Chapter 42).

Table 3.3. Hierarchy of study designs*

Table 3.3. Hierarchy of study designs*

Level 1.	Randomized controlled trials - includes quasi-randomized processes such as alternate allocation
Level 2.	Non-randomized controlled trial - a prospective (pre-planned) study, with predetermined eligibility criteria and outcome measures.
Level 3.	Observational studies with controls - includes retrospective, interrupted time series (a change in trend attributable to the intervention), case-control studies, cohort studies with controls, and health services research that includes adjustment for likely confounding variables
Level 4.	Observational studies without controls (eg, cohort studies without controls and case series)

*Systematic reviews and meta-analyses were assigned to the highest level study design included in the review, followed by an "A" (eg, a systematic review that included at least one randomized controlled trial was designated "Level 1A")

Table 3.4. Hierarchy of outcome measures

Table 3.4. Hierarchy of outcome measures

Level 1.	Clinical outcomes - morbidity, mortality, adverse events
Level 2.	Surrogate outcomes - observed errors, intermediate outcomes (eg, laboratory results) with well-established connections to the clinical outcomes of interest (usually adverse events).
Level 3.	Other measurable variables with an indirect or unestablished connection to the target safety outcome (eg, pre-test/post-test after an educational intervention, operator self-reports in different experimental situations)
Level 4.	No outcomes relevant to decreasing medical errors and/or adverse events (eg, study with patient satisfaction as only measured outcome; article describes an approach to detecting errors but reports no measured outcomes)

Each practice's level of evidence for efficacy or effectiveness was assessed in terms of study design (Table 3.3) and study outcomes (Table 3.4). The Editors created the following hierarchies by modifying existing frameworks for evaluating evidence^6-8 and incorporating recommendations from numerous other sources relevant to evidence synthesis.^9-25

Implicit in this hierarchy of outcome measures is that surrogate or intermediate outcomes (Level 2) have an established relationship to the clinical outcomes (Level 1) of interest.²⁶ Outcomes that are relevant to patient safety but have not been associated with morbidity or mortality were classified as Level 3.

Exceptions to EPC Criteria

Some safety practices did not meet the EPC inclusion criteria because of the paucity of evidence regarding efficacy or effectiveness, but were included in the Report because of their face validity (ie, an informed reader might reasonably expect them to be evaluated; see also Chapter 1). The reviews of these practices clearly identify the quality of evidence culled from medical and non-medical fields.

Evaluation of Safety Practices

For each practice, authors were instructed to research the literature for information on:

• prevalence of the problem targeted by the practice

• severity of the problem targeted by the practice

• the current utilization of the practice

• evidence on efficacy and/or effectiveness of the practice

• the practice's potential for harm

• data on cost if available

• implementation issues

These elements were incorporated into a template in an effort to create as much uniformity across chapters as possible, especially given the widely disparate subject matter and quality of evidence. Since the amount of material for each practice was expected to, and did, vary substantially, the Editors provided general guidance on what was expected for each element, with particular detail devoted to the protocol for searching and reporting evidence related to efficacy and/or effectiveness of the practice.

The protocol outlined the search, the threshold for study inclusion, the elements to abstract from studies, and guidance on reporting information from each study. Authors were asked to review articles from their search to identify practices, and retain those with the better study designs. More focused searches were performed depending on the topic. The threshold for study inclusion related directly to study design. Authors were asked to use their judgment in deciding whether the evidence was sufficient at a given level of study design or whether the evidence from the next level needed to be reviewed. At a minimum, the Editors suggested that there be at least 2 studies of adequate quality to justify excluding discussion of studies of lower level designs. Thus inclusion of 2 adequate clinical trials (Level 1 design) were necessary in order to exclude available evidence from prospective, non-randomized trials (Level 2) on the same topic.

Table 3.5. Ten Required Abstraction Elements

Table 3.5. Ten Required Abstraction Elements

1.	Bibliographic information according to AMA Manual of Style: title, authors, date of publication, source
2.	Level of study design (eg, Level 1-3 for studies providing information for effectiveness; Level 4 if needed for relevant additional information) with descriptive material as follows: Descriptors For Level 1 Systematic Reviews Only (a) Identifiable description of methods indicating sources and methods of searching for articles. (b) Stated inclusion and exclusion criteria for articles: yes/no. (c) Scope of literature included in study. For Level 1 or 2 Study Designs (Not Systematic Reviews) (a) Blinding: blinded, blinded (unclear), or unblinded (b) Describe comparability of groups at baseline - ie, was distribution of potential confounders at baseline equal? If no, which confounders were not equal? (c) Loss to follow-up overall: percent of total study population lost to follow-up. For Level 3 Study Design (Not Systematic Reviews) (a) Description of study design (eg, case-control, interrupted time series). (b) Describe comparability of groups at baseline - ie, was distribution of potential confounders at baseline equal? If no, which confounders were not equal? (c) Analysis includes adjustment for potential confounders: yes/no. If yes, adjusted for what confounders?
3.	Description of intervention (as specific as possible)
4.	Description of study population(s) and setting(s)
5.	Level of relevant outcome measure(s) (eg, Levels 1-4)
6.	Description of relevant outcome measure(s)
7.	Main Results: effect size with confidence intervals
8.	Information on unintended adverse (or beneficial) effects of practice
9.	Information on cost of practice
10.	Information on implementation of practice (information that might be of use in whether to and/or how to implement the practice - eg, known barriers to implementation)

We present the salient elements of each included study (eg, study design, population/setting, intervention details, results) in text or tabular form. In addition, we asked authors to highlight weaknesses and biases of studies where the interpretation of the results might be substantially affected. Authors were not asked to formally synthesize or combine (eg, perform a meta-analysis) the evidence across studies for the Report.

Review Process

Authors submitted work to the Editors in 2 phases. In the first phase ("Identification of Safety Practices for Evaluation"), which was submitted approximately 6 weeks after authors were commissioned, authoring teams provided their search strategies, citations, and a preliminary list of patient safety practices to be reviewed. In the subsequent phase ("Evaluation of Safety Practices"), due approximately 12 weeks after commissioning, authors first submitted a draft chapter for each topic, completed abstraction forms, and -- after iterative reviews and revisions -- a final chapter.

Background

Errors in medical care are discovered through a variety of mechanisms. Historically, medical errors were revealed retrospectively through morbidity and mortality committees and malpractice claims data. Prominent studies of medical error have used retrospective chart review to quantify adverse event rates.^1,2 While collection of data in this manner yields important epidemiologic information, it is costly and provides little insight into potential error reduction strategies. Moreover, chart review only detects documented adverse events and often does not capture information regarding their causes. Important errors that produce no injury may go completely undetected by this method.^3-6

Computerized surveillance may also play a role in uncovering certain types of errors. For instance, medication errors may be discovered through a search for naloxone orders for hospitalized patients, as they presumably reflect the need to reverse overdose of prescribed narcotics.^7,8 Several studies have demonstrated success with computerized identification of adverse drug events.^9-11

Complex, high-risk industries outside of health care, including aviation, nuclear power, petrochemical processing, steel production, and military operations, have successfully developed incident reporting systems for serious accidents and important "near misses."⁶ Incident reporting systems cannot provide accurate epidemiologic data, as the reported incidents likely underestimate the numerator, and the denominator (all opportunities for incidents) remains unknown.

Given the limited availability of sophisticated clinical computer systems and the tremendous resources required to conduct comprehensive chart reviews, incident reporting systems remain an important and relatively inexpensive means of capturing data on errors and adverse events in medicine. Few rigorous studies have analyzed the benefits of incident reporting. This chapter reviews only the literature evaluating the various systems and techniques for collecting error data in this manner, rather than the benefit of the practice itself. This decision reflects our acknowledgment that incident reporting has clearly played a beneficial role in other high-risk industries.⁶ The decision also stems from our recognition that a measurable impact of incident reporting on clinical outcomes is unlikely because there is no standard practice by which institutions handle these reports.

Practice Description

Flanagan first described the critical incident technique in 1954 to examine military aircraft training accidents.¹² Critical incident reporting involves the identification of preventable incidents (ie, occurrences that could have led, or did lead, to an undesirable outcome¹³) reported by personnel directly involved in the process in question at the time of discovery of the event. The goal of critical incident monitoring is not to gather epidemiologic data per se, but rather to gather qualitative data. Nonetheless, if a pattern of errors seems to emerge, prospective studies can be undertaken to test epidemiologic hypotheses.¹⁴

Incident reports may target events in any or all of 3 basic categories: adverse events, "no harm events," and "near misses." For example, anaphylaxis to penicillin clearly represents an adverse event. Intercepting the medication order prior to administration would constitute a near miss. By contrast, if a patient with a documented history of anaphylaxis to penicillin received a penicillin-like antibiotic (eg, a cephalosporin) but happened not to experience an allergic reaction, it would constitute a no harm event, not a near miss. In other words, when an error does not result in an adverse event for a patient, because the error was "caught," it is a near miss; if the absence of injury is owed to chance it is a no harm event. Broadening the targets of incident reporting to include no harm events and near misses offers several advantages. These events occur 3 to 300 times more often than adverse events,^5,6 they are less likely to provoke guilt or other psychological barriers to reporting,⁶ and they involve little medico-legal risk.¹⁴ In addition, hindsight bias¹⁵ is less likely to affect investigations of no harm events and near misses.^6,14

Barach and Small describe the characteristics of incident reporting systems in non-medical industries.⁶ Established systems share the following characteristics:

• they focus on near misses

• they provide incentives for voluntary reporting;

• they ensure confidentiality; and

• they emphasize systems approaches to error analysis.

The majority of these systems were mandated by Federal regulation, and provide for voluntary reporting. All of the systems encourage narrative description of the event. Reporting is promoted by providing incentives including:

• immunity;

• confidentiality;

• outsourcing of report collation;

• rapid feedback to all involved and interested parties; and

• sustained leadership support.⁶

Incident reporting in medicine takes many forms. Since 1975, the US Food and Drug Administration (FDA) has mandated reporting of major blood transfusion reactions, focusing on preventable deaths and serious injuries.¹⁶ Although the critical incident technique found some early applications in medicine,^17,18 its current use is largely attributable to Cooper's introduction of incident reporting to anesthesia in 1978,¹⁹ conducting retrospective interviews with anesthesiologists about preventable incidents or errors that occurred while patients were under their care. Recently, near miss and adverse event reporting systems have proliferated in single institution settings (such as in intensive care units (ICUs)^20,21), regional settings (such as the New York State transfusion system²²), and for national surveillance (eg, the National Nosocomial Infections Surveillance System administered by the Federal Centers for Disease Control and Prevention.)²³

Table 4.1. Examples of events reported to hospital (more...)

Table 4.1. Examples of events reported to hospital incident reporting systems^25-27

Adverse Outcomes	Procedural Breakdowns	Catastrophic Events
Unexpected death or disability Inpatient falls or "mishaps" Institutionally-acquired burns Institutionally-acquired pressure sores	Errors or unexpected complications related to the administration of drugs or transfusion Discharges against medical advice ("AMA") Significant delays in diagnosis or diagnostic testing Breach of confidentiality	Performance of a procedure on the wrong body part ("wrong-site surgery") Performance of a procedure on the wrong patient Infant abduction or discharge to wrong family Rape of a hospitalized patient Suicide of a hospitalized patient

In 1995, hospital-based surveillance was mandated by the Joint Commission on the Accreditation of Healthcare Organizations (JCAHO)²⁶ because of a perception that incidents resulting in harm were occurring frequently.²⁸ JCAHO employs the term sentinel event in lieu of critical incident, and defines it as follows: An unexpected occurrence involving death or serious physical or psychological injury, or the risk thereof. Serious injury specifically includes loss of limb or function. The phrase "or the risk thereof" includes any process variation for which a recurrence would carry a significant chance of a serious adverse outcome. ²⁶

As one component of its Sentinel Event Policy, JCAHO created a Sentinel Event Database. The JCAHO database accepts voluntary reports of sentinel events from member institutions, patients and families, and the press.²⁶ The particulars of the reporting process are left to the member health care organizations. JCAHO also mandates that accredited hospitals perform root cause analysis (see Chapter 5) of important sentinel events. Data on sentinel events are collated, analyzed, and shared through a website,²⁹ an online publication,³⁰ and its newsletter Sentinel Event Perspectives.³¹

Another example of a national incident reporting system is the Australian Incident Monitoring Study (AIMS), under the auspices of the Australian Patient Safety Foundation.¹³ Investigators created an anonymous and voluntary near miss and adverse event reporting system for anesthetists in Australia. Ninety participating hospitals and practices named on-site coordinators. The AIMS group developed a form that was distributed to participants. The form contained instructions, definitions, space for narrative of the event, and structured sections to record the anesthesia and procedure, demographics about the patient and anesthetist, and what, when, why, where, and how the event occurred. The results of the first 2000 reports were published together, following a special symposium.³²

The experiences of the JCAHO Sentinel Event Database and the Australian Incident Monitoring Study are explored further below.

Prevalence and Severity of the Target Safety Problem

The true prevalence of events appropriate for incident reporting is impossible to estimate with any accuracy, as it includes actual adverse events as well as near misses and no harm events. The Aviation Safety Reporting System (ASRS), a national reporting system for near misses in the airline industry,^33,34 currently processes approximately 30,000 reports annually, 35 exceeding by many orders of magnitude the total number of airline accidents each year.³⁴ The number of reports submitted to a comparable system in health care would presumably number in the millions if all adverse events, no harm events, and near misses were captured.

By contrast, over 6 years of operation, the JCAHO Sentinel Event Database has captured only 1152 events, 62% of which occurred in general hospitals. Two-thirds of the events were self-reported by institutions, with the remainder coming from patient complaints, media stories and other sources.²⁹ These statistics are clearly affected by underreporting and consist primarily of serious adverse events (76% of events reported resulted in patient deaths), not near misses. As discussed in the chapter on wrong-site surgeries (Subchapter 43.2), comparing JCAHO reports with data from the mandatory incident reporting system maintained by the New York State Department of Health³⁶ suggests that the JCAHO statistics underestimate the true incidence of target events by at least a factor of 20.

Opportunities for Impact

Most hospitals' incident reporting systems fail to capture the majority of errors and near misses.²⁴ Studies of medical services suggest that only 1.5% of all adverse events result in an incident report³⁷ and only 6% of adverse drug events are identified through traditional incident reporting or a telephone hotline.²⁴ The American College of Surgeons estimates that incident reports generally capture only 5-30% of adverse events.³⁸ A study of a general surgery service showed that only 20% of complications on a surgical service ever resulted in discussion at Morbidity and Mortality rounds.³⁹ Given the endemic underreporting revealed in the literature, modifications to the configuration and operation of the typical hospital reporting system could yield higher capture rates of relevant clinical data.

Study Designs

We analyzed 5 studies that evaluated different methods of critical incident reporting. Two studies prospectively investigated incident reporting compared with observational data collection^24,39 and one utilized retrospective chart review.³⁷ Two additional studies looked at enhanced incident reporting by active solicitation of physician input compared with background hospital quality assurance (QA) measures.^40,41 In addition, we reviewed JCAHO's report of its Sentinel Event Database, and the Australian Incident Monitoring Study, both because of the large sizes and the high profiles of the studies.^13,26 Additional reports of critical incident reporting systems in the medical literature consist primarily of uncontrolled observational trials^42-44 that are not reviewed in this chapter.

Study Outcomes

In general, published studies of incident reporting do not seek to establish the benefit of incident reporting as a patient safety practice. Their principal goal is to determine if incident reporting, as it is practiced, captures the relevant events.⁴⁰ In fact, no studies have established the value of incident reporting on patient safety outcomes.

The large JCAHO and Australian databases provide data about reporting rates, and an array of quantitative and qualitative information about the reported incidents, including the identity of the reporter, time of report, severity and type of error.^13,26 Clearly these do not represent clinical outcomes, but they may be reasonable surrogates for the organizational focus on patient safety. For instance, increased incident reporting rates may not be indicative of an unsafe organization,⁴⁵ but may reflect a shift in organizational culture to increased acceptance of quality improvement and other organizational changes.^3,5

None of the studies reviewed captured outcomes such as morbidity or error rates. The AIMS group published an entire symposium which reported the quantitative and qualitative data regarding 2000 critical incidents in anesthesia.¹³ However, only a small portion of these incidents were prospectively evaluated.¹⁴ None of the studies reviewed for this chapter performed formal root cause analyses on reported errors (Chapter 5).

Evidence for Effectiveness of the Practice

Table 4.2. Sentinel event alerts published by JCAHO (more...)

Table 4.2. Sentinel event alerts published by JCAHO following analysis of incident reports⁴⁶

Medication Error Prevention - Potassium Chloride Lessons Learned: Wrong Site Surgery Inpatient Suicides: Recommendations for Prevention Preventing Restraint Deaths Infant Abductions: Preventing Future Occurrences High-Alert Medications and Patient Safety Operative and Postoperative Complications: Lessons for the Future Fatal Falls: Lessons for the Future Infusion Pumps: Preventing Future Adverse Events Lessons Learned: Fires in the Home Care Setting Kernicterus Threatens Healthy Newborns

The first 2000 incident reports to AIMS from 90 member institutions were published in 1993.¹³ In contrast to the JCAHO data, all events were self-reported by anesthetists and only 2% of events reported resulted in patient deaths. A full 44% of events had negligible effect on patient outcome. Ninety percent of reports had identified systems failures, and 79% had identified human failures. The AIMS data were similar to those of Cooper¹⁹ in terms of percent of incidents with reported human failures, timing of events with regard to phase of anesthesia, and type of events (breathing circuit misconnections were between 2% and 3% in both studies).^13,19 The AIMS data are also similar to American "closed-claims" data in terms of pattern, nature and proportion of the total number of reports for several types of adverse events,¹³ which lends further credibility to the reports.

The AIMS data, although also likely to be affected by underreporting because of its voluntary nature, clearly captures a higher proportion of critical incidents than the JCAHO Sentinel Event Database. Despite coming from only 90 participating sites, AIMS received more reports over a similar time frame than the JCAHO did from the several thousand accredited United States hospitals. This disparity may be explained by the fact that AIMS institutions were self-selected, and that the culture of anesthesia is more attuned to patient safety concerns.⁴⁷

The poor capture rate of incident reporting systems in American hospitals has not gone unnoticed. Cullen et al²⁴ prospectively investigated usual hospital incident reporting compared to observational data collection for adverse drug events (ADE logs, daily solicitation from hospital personnel, and chart review) in 5 patient care units of a tertiary care hospital. Only 6% of ADEs were identified and only 8% of serious ADEs were reported. These findings are similar to those in the pharmacy literature,^48,49 and are attributed to cultural and environmental factors. A similar study on a general surgical service found that 40% of patients suffered complications.³⁹ While chart documentation was excellent (94%), only 20% of complications were discussed at Morbidity and Mortality rounds.

Active solicitation of physician reporting has been suggested as a way to improve adverse event and near miss detection rates. Weingart et al⁴¹ employed direct physician interviews supplemented by email reminders to increase detection of adverse events in a tertiary care hospital. The physicians reported an entirely unique set of adverse events compared with those captured by the hospital incident reporting system. Of 168 events, only one was reported by both methods. O'Neil et al³⁷ used e-mail to elicit adverse events from housestaff and compared these with those found on retrospective chart review. Of 174 events identified, 41 were detected by both methods. The house officers appeared to capture preventable adverse events at a higher rate (62.5% v. 32%, p=0.003). In addition, the hospital's risk management system detected only 4 of 174 adverse events. Welsh et al⁴⁰ employed prompting of house officers at morning report to augment hospital incident reporting systems. There was overlap in reporting in only 2.6% of 341 adverse events that occurred during the study. In addition, although the number of events house officers reported increased with daily prompting, the quantity rapidly decreased when prompting ceased. In summary, there is evidence that active solicitation of critical incident reports by physicians can augment existing databases, identifying incidents not detected through other means, although the response may not be durable.^37,40,41

Potential for Harm

Users may view reporting systems with skepticism, particularly the system's ability to maintain confidentiality and shield participants from legal exposure.²⁸ In many states, critical incident reporting and analysis count as peer review activities and are protected from legal discovery.^28,50 However, other states offer little or no protection, and reporting events to external agencies (eg, to JCAHO) may obliterate the few protections that do exist. In recognition of this problem, JCAHO's Terms of Agreement with hospitals now includes a provision identifying JCAHO as a participant in each hospital's quality improvement process.²⁸

Costs and Implementation

Few estimates of costs have been reported in the literature. In general, authors remarked that incident reporting was far less expensive than retrospective review. One single center study estimated that physician reporting was less costly ($15,000) than retrospective record review ($54,000) over a 4-month period.³⁷ A survey of administrators of reporting systems from non-medical industries reported a consensus that costs were far offset by the potential benefits.⁶

Comment

The wide variation in reporting of incidents may have more to do with reporting incentives and local culture than with the quality of medicine practiced there.²⁴ When institutions prioritize incident reporting among medical staff and trainees, however, the incident reporting systems seem to capture a distinct set of events from those captured by chart review and traditional risk management^40,41 and events captured in this manner may be more preventable.³⁷

The addition of anonymous or non-punitive systems is likely to increase the rates of incident reporting and detection.⁵¹ Other investigators have also noted increases in reporting when new systems are implemented and a culture conducive to reporting is maintained.^40,52 Several studies suggest that direct solicitation of physicians results in reports that are more likely to be distinct, preventable, and more severe than those obtained by other means.^8,37,41

The nature of incident reporting, replete with hindsight bias, lost information, and lost contextual clues makes it unlikely that robust data will ever link it directly with improved outcomes. Nonetheless, incident reporting appears to be growing in importance in medicine. The Institute of Medicine report, To Err is Human,⁵³ has prompted calls for mandatory reporting of medical errors to continue in the United States.^54-57 England's National Health Service plans to launch a national incident reporting system as well, which has raised concerns similar to those voiced in the American medical community.⁵⁸ While the literature to date does not permit an evidence-based resolution of the debate over mandatory versus voluntary incident reporting, it is clear that incident reporting represents just one of several potential sources of information about patient safety and that these sources should be regarded as complementary. In other industries incident reporting has succeeded when it is mandated by regulatory agencies or is anonymous and voluntary on the part of reporters, and when it provides incentives and feedback to reporters.⁶ The ability of health care organizations to replicate the successes of other industries in their use of incident reporting systems⁶ will undoubtedly depend in large part on the uses to which they put these data. Specifically, success or failure may depend on whether health care organizations use the data to fuel institutional quality improvement rather than to generate individual performance evaluations.

Background

Historically, medicine has relied heavily on quantitative approaches for quality improvement and error reduction. For instance, the US Food and Drug Administration (FDA) has collected data on major transfusion errors since the mid-1970s.^1,2 Using the statistical power of these nationwide data, the most common types of errors have been periodically reviewed and systems improvements recommended.³

These epidemiologic techniques are suited to complications that occur with reasonable frequency, but not for rare (but nonetheless important) errors. Outside of medicine, high-risk industries have developed techniques to address major accidents. Clearly the nuclear power industry cannot wait for several Three Mile Island-type events to occur in order to conduct valid analyses to determine the likely causes.

A retrospective approach to error analysis, called root cause analysis (RCA), is widely applied to investigate major industrial accidents.⁴ RCA has its foundations in industrial psychology and human factors engineering. Many experts have championed it for the investigation of sentinel events in medicine.^5-7 In 1997, the Joint Commission on the Accreditation of Healthcare Organizations (JCAHO) mandated the use of RCA in the investigation of sentinel events in accredited hospitals.⁸

The most commonly cited taxonomy of human error in the medical literature is based on the work of James Reason.^4,9,10 Reason describes 2 major categories of error: active error, which represents failures of system design. RCA is generally employed to uncover latent errors underlying a sentinel event.^6,7

RCA provides a structured and process-focused framework with which to approach sentinel event analysis. Its cardinal tenet is to avoid the pervasive and counterproductive culture of individual blame.^11,12 Systems and organizational issues can be identified and addressed, and active errors are acknowledged.⁶ Systematic application of RCA may uncover common root causes that link a disparate collection of accidents (ie, a variety of serious adverse events occurring at shift change). Careful analysis may suggest system changes designed to prevent future incidents.¹³

Despite these intriguing qualities, RCA has significant methodologic limitations. RCAs are in essence uncontrolled case studies. As the occurrence of accidents is highly unpredictable, it is impossible to know if the root cause established by the analysis is the cause of the accident.¹⁴ In addition, RCAs may be tainted by hindsight bias.^4,15,16 Other biases stem from how deeply the causes are probed and influenced by the prevailing concerns of the day.^16,17 The fact that technological failures (device malfunction), which previously represented the focus of most accident analyses, have been supplanted by staffing issues, management failures, and information systems problems may be an example of the latter bias.¹⁷ Finally, RCAs are time-consuming and labor intensive.

Despite legitimate concerns about the place of RCA in medical error reduction, the JCAHO mandate ensures that RCA will be widely used to analyze sentinel events.⁸ Qualitative methods such as RCA should be used to supplement quantitative methods, to generate new hypotheses, and to examine events not amenable to quantitative methods (for example, those that occur rarely).¹⁸ As such, its credibility as a research tool should be judged by the standards appropriate for qualitative research, not quantitative.^19,20 Yet, the outcomes and costs associated with RCA are largely unreported. This chapter reviews the small body of published literature regarding the use of RCA in the investigation of medical errors.

Practice Description

To be credible, RCA requires rigorous application of established qualitative techniques. Once a sentinel event has been identified for analysis (eg, a major chemotherapy dosing error, a case of wrong-site surgery, or major ABO incompatible transfusion reaction), a multidisciplinary team is assembled to direct the investigation. The members of this team should be trained in the techniques and goals of RCA, as the tendency to revert to personal biases is strong.^13,14 Multiple investigators allow triangulation or corroboration of major findings and increase the validity of the final results.¹⁹ Based on the concepts of active and latent error described above, accident analysis is generally broken down into the following steps^6,7:

1. Data collection: establishment of what happened through structured interviews, document review, and/or field observation. These data are used to generate a sequence or timeline of events preceeding and following the event;

2. Data analysis: an iterative process to examine the sequence of events generated above with the goals of determining the common underlying factors:

   1. Establishment of how the event happened by identification of active failures in the sequence;

   2. Establishment of why the event happened through identification of latent failures in the sequence which are generalizable.

In order to ensure consideration of all potential root causes of error, one popular conceptual framework for contributing factors has been proposed based on work by Reason.⁷ Several other frameworks also exist.^21,22 The categories of factors influencing clinical practice include institutional/regulatory, organizational/management, work environment, team factors, staff factors, task factors, and patient characteristics. Each category can be expanded to provide more detail. A credible RCA considers root causes in all categories before rejecting a factor or category of factors as non-contributory. A standardized template in the form of a tree (or "Ishikawa") may help direct the process of identifying contributing factors, with such factors leading to the event grouped (on tree "roots") by category. Category labels may vary depending on the setting.²³

At the conclusion of the RCA, the team summarizes the underlying causes and their relative contributions, and begins to identify administrative and systems problems that might be candidates for redesign.⁶

Prevalence and Severity of the Target Safety Problem

JCAHO's 6-year-old sentinel event database of voluntarily reported incidents (see Chapter 4) has captured a mere 1152 events, of which 62% occurred in general hospitals. Two-thirds of the events were self-reported by institutions, with the remainder coming from patient complaints, media stories and other sources.²⁴ These statistics are clearly affected by underreporting and consist primarily of serious adverse events (76% of events reported resulted in patient deaths), not near misses. The number of sentinel events appropriate for RCA is likely to be orders of magnitude greater.

The selection of events for RCA may be crucial to its successful implementation on a regular basis. Clearly, it cannot be performed for every medical error. JCAHO provides guidance for hospitals about which events are considered "sentinel,"⁸ but the decision to conduct RCA is at the discretion of the leadership of the organization.¹²

If the number of events is large and homogeneous, many events can be excluded from analysis. In a transfusion medicine reporting system, all events were screened after initial report and entered in the database, but those not considered sufficiently unique did not undergo RCA.²⁵

Opportunities for Impact

While routine RCA of sentinel events is mandated, the degree to which hospitals carry out credible RCAs is unknown. Given the numerous demands on hospital administrators and clinical staff, it is likely that many hospitals fail to give this process a high profile, assigning the task to a few personnel with minimal training in RCA rather than involving trained leaders from all relevant departments. The degree of underreporting to JCAHO suggests that many hospitals are wary of probationary status and the legal implications of disclosure of sentinel events and the results of RCAs.^12,26

Study Designs

As RCA is a qualitative technique, most reports in the literature are case studies or case series of its application in medicine.^6,27-30 There is little published literature that systematically evaluates the impact of formal RCA on error rates. The most rigorous study comes from a tertiary referral hospital in Texas that systematically applied RCA to all serious adverse drug events (ADEs) considered preventable. The time series contained background data during the initial implementation period of 12 months and a 17-month follow-up phase.¹³

Study Outcomes

Published reports of the application of RCA in medicine generally present incident reporting rates, categories of active errors determined by the RCA, categories of root causes (latent errors) of the events, and suggested systems improvements. While these do not represent clinical outcomes, they are reasonable surrogates for evaluation. For instance, increased incident reporting rates may reflect an institution's shift toward increased acceptance of quality improvement and organizational change.^5,21

Evidence for Effectiveness of the Practice

The Texas study revealed a 45% decrease in the rate of voluntarily reported serious ADEs between the study and follow-up periods (7.2 per 100,000 to 4.0 per 100,000 patient-days, p<0.001).¹³ Although there were no fatal ADEs in the follow-up period, the small number of mortalities in the baseline period resulted in extremely wide confidence intervals, so that comparing the mortality rates serves little purpose.¹³

The authors of the Texas study attribute the decline in serious ADEs to the implementation of blame-free RCA, which prompted important leadership focus and policy changes related to safety issues. Other changes consisted of improvements in numerous aspects of the medication ordering and distribution processes (eg, the application of "forcing" and "constraining" functions that make it impossible to perform certain common errors), as well as more general changes in organizational features, such as staffing levels.

The significance of the decline in ADEs and its relationship to RCA in the Texas study is unclear. As the study followed a highly publicized, fatal ADE at the hospital, other cultural or systems changes may have contributed to the measured effect. The authors were unable to identify a control group, nor did they report data from serious ADEs in the year preceding the study. Their data may reflect underreporting, as there is no active surveillance for ADEs at the study hospital, leaving the authors to rely on voluntary reports. The decline in reported ADEs may actually call into question the robustness of their reporting system as other studies have found that instituting a blame-free system leads to large increases in event reporting.⁵ On the other hand, it seems unlikely that serious ADEs would be missed in a culture of heightened sensitivity to error.

In a separate report, an event reporting system for transfusion medicine was implemented at 2 blood centers and 2 transfusion services.²⁵ Unique events were subjected to RCA, and all events were classified using a model adapted from the petrochemical industry.²¹ There were 503 events reported and 1238 root causes identified. Human failure accounted for 46% of causes, 27% were due to technical failures, and 27% were from organizational failures. This distribution was very similar to that seen in the petrochemical industry, perhaps an indication of the universality of causes of error in complex systems, regardless of industry.

The potential for harm with the use of RCA has received only passing mention in the literature, but might result from flawed analyses.³¹ The costs of pursuing absolute safety may be the implementation of increasingly complex and expensive safeguards, which in themselves are prone to systems failures.^4,16 Ill-conceived RCAs which result in little effective systems improvement could also dampen enthusiasm for the entire quality improvement process. Arguably the harm caused by pursuit of incorrect root causes must be offset by the costs of not pursuing them at all.

Costs and Implementation

No estimates of costs of RCA have appeared in the literature, but as it is a labor-intensive process they are likely significant. Counterproductive cultural norms and medico-legal concerns similar to those seen in incident reporting may hinder implementation of RCA.^12,26 The authors of the Texas study note the importance of clear expressions of administrative support for the process of blame-free RCA.¹³ Other studies note the receptiveness of respondents to blame-free investigation in the name of quality improvement, with one health system reporting a sustained 10-fold increase in reporting.^25,27

Comment

Root cause analyses systematically search out latent or system failures that underlie adverse events or near misses. They are limited by their retrospective and inherently speculative nature. There is insufficient evidence in the medical literature to support RCA as a proven patient safety practice, however it may represent an important qualitative tool that is complementary to other techniques employed in error reduction. When applied appropriately, RCA may illuminate targets for change, and, in certain health care contexts, may generate testable hypotheses. The use of RCA merits more consideration, as it lends a formal structure to efforts to learn from past mistakes.

Background

Medication errors and adverse drug events (ADEs) are common, costly, and clinically important problems.^1-7 Two inpatient studies, one in adults and one in pediatrics, have found that about half of medication errors occur at the stage of drug ordering,^2,7 although direct observation studies have indicated that many errors also occur at the administration stage.⁸ The principal types of medication errors, apart from missing a dose, include incorrect medication dose, frequency, or route.² ADEs are injuries that result from the use of a drug. Systems-based analysis of medication errors and ADEs suggest that changes in the medication ordering system, including the introduction of computerized physician order entry (CPOE) with clinical decision support systems (CDSSs), may reduce medication-related errors.⁹

Despite growing evidence and public mandates, implementation of CPOE has been limited. The Leapfrog Group, a consortium of companies from the Business Roundtable, has endorsed CPOE in hospitals as one of the 3 changes that would most improve patient safety in America (see also Chapter 55).¹⁰ A Medicare Payment Advisory Commission report suggested instituting financial incentives for CPOE implementation.¹¹ US Senators Bob Graham (D-Fla.) and Olympia Snowe (R-Maine) recently introduced a bill, entitled the "Medication Errors Reduction Act of 2001," to establish an informatics system grant program for hospitals and skilled nursing facilities.¹² In addition, California recently enacted legislation stipulating that acute care hospitals implement information technology, such as CPOE to reduce medication-related errors.¹³

Practice Description

CPOE refers to a variety of computer-based systems of ordering medications, which share the common features of automating the medication ordering process. Basic CPOE ensures standardized, legible, complete orders by only accepting typed orders in a standard and complete format. Almost all CPOE systems include or interface with CDSSs of varying sophistication. Basic clinical decision support may include suggestions or default values for drug doses, routes, and frequencies. More sophisticated CDSSs can perform drug allergy checks, drug-laboratory value checks, drug-drug interaction checks, in addition to providing reminders about corollary orders (eg, prompting the user to order glucose checks after ordering insulin) or drug guidelines to the physician at the time of drug ordering (see also Chapter 53).

At times, CDSSs are implemented without CPOE. Isolated CDSSs can provide advice on drug selection, dosages, and duration. More refined CDSSs can incorporate patient-specific information (for example recommending appropriate anticoagulation regimens), or incorporate pathogen-specific information such as suggesting appropriate anti-infective regimens. After viewing such advice, the physician proceeds with a conventional handwritten medication order.

Prevalence and Severity of the Target Safety Problem

It is estimated that over 770,000 people are injured or die in hospitals from ADEs annually.^{4, 5, 14} The few hospitals that have studied incidence rates of ADEs have documented rates ranging from 2 to 7 per 100 admissions.^{2, 4, 15, 16} A precise national estimate is difficult to calculate due to the variety of criteria and definitions used by researchers.¹⁷ One study of preventable inpatient ADEs in adults demonstrated that 56% occurred at the stage of ordering, 34% at administration, 6% at transcribing, and 4% at dispensing.² In this study, the drug class most commonly associated with preventable ADEs was analgesics, followed by sedatives and antibiotics. Even fewer studies have been conducted in the outpatient setting. One recent cross-sectional chart review and patient care survey found an ADE rate of 3% in adult primary care outpatients.¹⁸

Opportunities for Impact

Clear data do not exist about the prevalence of CPOE with CDSSs or isolated CDSSs. One survey of 668 hospitals indicated that 15% had at least partially implemented CPOE.¹⁹ A slightly more recent survey of pharmacy directors at 1050 acute care hospitals in the United States (51% response rate) reported that 13% of hospitals had an electronic medication order-entry system in place, while an additional 27% stated they were in the process of obtaining such a sysytem.²⁰

Study Designs

Table 6.1. Studies of computerized physician order (more...)

Table 6.1. Studies of computerized physician order entry (CPOE) with clinical decision support systems (CDSSs)*

Study	Study Design	Study Outcomes	Results
Overhage, 1997.21 Impact of faculty and physician reminders (using CPOE) on corollary orders for adult inpatients in a general medical ward at a public teaching hospital affiliated with the Indiana University School of Medicine	Level 1 (RCT with physicians randomized to receive reminders or not)	Levels 2 & 3 (errors of omission in corollary orders)	25% improvement in ordering of corollary medications by faculty and residents (p<0.0001)
Bates, 1998.22 CPOE with CDSSs for adult inpatients on medical, surgical, and intensive care wards at BWH, a tertiary care center affiliated with Harvard University	Levels 2 & 3 (two study designs)	Level 1 (ADE rates) and Level 2 (serious medication errors)	55% decrease in non-intercepted serious medication errors (p=0.01) 17% decrease in preventable ADEs (p=0.37)
Bates, 1999.23 CPOE with CDSSs for adult inpatients in 3 medical units at BWH	Level 3 (retrospective time series)	Level 1 (ADEs) and Level 2 (main outcome measure was medication errors)	81% decrease in medication errors (p<0.0001) 86% decrease in non-intercepted serious medication errors (p=0.0003)
Teich, 2000.24 CPOE with CDSSs for all adult inpatients at BWH	Level 3 (retrospective before-after analysis)	Levels 2 & 3 (changes in 5 prescribing practices)	Improvement in 5 prescribing practices (p<0.001 for each of the 5 comparisons)

*ADE indicates adverse drug event; BWH, Brigham and Women's Hospital; and RCT, randomized controlled trial.

Table 6.2. Studies of clinical decision support systems (CDSSs)* (more...)

Table 6.2. Studies of clinical decision support systems (CDSSs)*

Study	Study Design	Study Outcomes	Results
Hunt, 1998.25 Use of CDSSs by healthcare practitioners in multiple inpatient and outpatient settings	Level 1A (systematic review of RCTs)	Levels 1 & 2 (a variety of measures related to patient outcomes and physician practice, not just ADEs and processes of care related to medication use.	6 of 14 studies showed improvement in patient outcomes 43 of 65 studies showed improvement in physician performance
Walton, 2001.26 Use of CDSSs for drug dosage advice by healthcare practitioners for 1229 patients in multiple inpatient settings	Levels 1A-3A (systematic review of RCTs, interrupted time series analyses, and controlled before-after studies)	Level 1 (one main outcome measure was adverse drug reactions	Absolute risk reduction with CDSSs: 6% (95% CI: 0-12%)
Evans, 1994.27 Use of a computerized antibiotic selection consultant for 451 inpatients at Salt Lake City's LDS Hospital, a 520-bed community teaching hospital and tertiary referral center	Level 1 (RCT with crossover design)	Level 2 (one of 5 primary outcomes was pathogen susceptibility to prescribed antibiotic regimens	17% greater pathogen susceptibility to an antibiotic regimen suggested by computer consultant versus physicians (p<0.001)
Evans, 1998.28 Computer-based anti-infective management program for 1136 intensive care unit patients from a 12-bed ICU at LDS Hospital	Level 2 (prospective before-after analysis)	Level 2 (one primary outcome was ADEs due to anti-infective agents	70% decrease in ADEs caused by anti-infectives (p=0.02)

* ADE indicates adverse drug event; CI, confidence interval; ICU, intensive care unit; and RCT, randomized controlled trial.

Study Outcomes

Adverse drug events (ADEs), (injuries that result from the use of drugs) by definition constitute clinical outcomes (Level 1). ADEs that are associated with a medication error are considered preventable, while those not associated with a medication error (eg, known medication side effects) are considered non-preventable. An example of a preventable ADE is the development of rash after the administration of ampicillin to a known penicillin-allergic patient. In contrast, a non-preventable ADE would be development of an ampicillin-associated rash in a patient with no known drug allergies. Non-intercepted serious medication errors include non-intercepted potential ADEs and preventable ADEs (ie, medication errors that either have the potential or actually cause harm to a patient).

Medication errors refer to errors in the processes of ordering, transcribing, dispensing, administering, or monitoring medications, irrespective of the outcome (ie, injury to the patient). One example is an order written for amoxicillin without a route of administration. Other medication errors have a greater potential for patient harm and so are often designated as "serious medication errors" or "potential ADEs" - eg, an order for amoxicillin in a patient with past anaphylaxis to penicillin.

Potential ADEs may or may not be intercepted before reaching the patient. An example of an intercepted potential ADE would be an order written for an acetaminophen overdose that is intercepted and corrected by a nurse before reaching the patient. An example of a non-intercepted potential ADE would be an administered overdose of acetaminophen to a patient who did not suffer any sequelae.

Medication errors include a mixture of serious medication errors with a significant potential for patient injury (Level 2) and other deviations from recommended practice that do not have a clear or established connection to adverse events (Level 3). Examples of the latter include failure to choose the optimal dosing schedule for a medication and many standards related to monitoring serum drug levels and routine electrolytes.

Only 2 studies (from a single institution) evaluating CPOE with CDSSs included ADEs as a secondary outcome (Level 1),²² and even these studies primary outcomes were serious medication errors (Level 2) and non-intercepted medication errors.²³ The other studies reported a variety of other errors often involving mixtures of Level 2 and Level 3 outcomes - eg, "prescribing practices"²⁴ and "corollary orders."²¹ Corollary orders refer to orders required to detect or ameliorate adverse reactions that may result from the trigger order ‐ eg, checking the serum creatinine a minimum of 2 times per week in a patient receiving a nephrotoxic agent such as amphotericin. Many corollary orders capture Level 3 outcomes, as failure to prescribe these orders would in most cases have no clear impact on clinical outcomes - eg, failure to order daily tests for fecal occult blood in patients on heparin or screening audiometry for patients receiving vancomycin.²¹

The predominance of Level 2 and 3 outcomes in these studies is understandable, given the much higher frequency of these outcomes compared to actual ADEs and the enormous costs of conducting these studies.

Similarly, the studies evaluating CDSSs reported a mixture of outcomes from Levels 1-3. Hunt et al reviewed articles to determine changes in patient outcomes (Level 1) or physician performance (Levels 2 and 3, depending on the practice).²⁵ Walton et al evaluated a range of outcomes (Levels 1-3), including reductions in "adverse reactions and unwanted effects" (Level 1).²⁶ In one study, Evans et al determined rates of pathogen susceptibility to an antibiotic regimen²⁷ (Level 2); another study by the same authors evaluated adverse drug events caused by anti-infectives as a main outcome (Level 1).²⁸

Evidence for Effectiveness of the Practice

Of the 2 studies at BWH that addressed the impact of CPOE with CDSSs on medication errors and ADEs, the first demonstrated a 55% decrease in serious medication errors.²² As a secondary outcome, this study found a 17% decrease in preventable ADEs, which was not statistically significant. The second study, a time series analysis, demonstrated marked reductions in all medication errors excluding missed dose errors and in non-intercepted serious medication errors.²³ The number of ADEs in this study was quite small - 25 in the baseline period and 18 in the third of the 3 periods with CPOE and CDSS. Correcting for the number of opportunities for errors, the total number of ADEs/1000 patient days decreased from 14.7 to 9.6 (p=0.09). For the sub-category of preventable ADEs, the reduction (from 5 to 2) achieved borderline statistical significance (p=0.05).

Overhage et al and Teich et al studied more focused aspects of the medication system. Overhage et al²¹ studied computerized reminders for corollary orders (eg, entering a laboratory order to check electrolytes when ordering potassium for a patient) and showed a greater than 100% improvement in the rate of corollary orders (p<0.0001). Teich et al²⁴ studied a broad range of computerized clinical decision support tools utilized at BWH and demonstrated 5 prescribing improvements in types, doses, and frequencies of drug usage.

In summary, these studies provide some evidence that CPOE with CDSSs can substantially decrease medication errors in broad as well as in more focused areas. Despite the significant impact on medication errors, the reduction in ADEs did not achieve statistical significance in one study,²² and achieved only borderline significance in one of the outcomes in the other study.²³ Furthermore, the systems evaluated in this relatively small literature were developed internally rather than purchased and installed, so the potential impact of commercially available systems remains somewhat speculative.

In the studies evaluating CDSSs, 2 were systematic reviews.^{25, 26} In Hunt's review, which emphasized clinical performance, 6 of 14 studies reported improvements in patient outcomes and 43 of 65 studies showed improvement in physician performance.²⁵ This study concludes that CDSSs can enhance clinical performance for drug dosing, preventive care, and other aspects of medical care, but that the impact of CDSSs on patient outcomes remains unclear (see also Chapter 53). Walton's systematic review evaluated computerized drug dosage advice and found a 6% decrease in adverse drug reactions.²⁶ The authors concluded that there is some limited evidence that CDSSs for drug dosing are effective, however there are relatively few studies, many of which are of sub-optimal quality. They also suggest that further research is needed to determine if the CDSS benefits realized with specialist applications can be realized by generalist use. Evans et al performed the remaining 2 studies.^{27, 28} The 1994 study evaluated the use of a computerized antibiotic selection consultant, and demonstrated a 17% greater pathogen susceptibility to an antibiotic regimen suggested by a computer consultant versus a physician (p<0.001).²⁷ The second study demonstrated a 70% decrease in ADEs caused by anti-infectives (p=0.018) through use of a computer based anti-infective management program. As with CPOE, these CDSSs studies demonstrate improvements in medication errors with statistical significance. In addition, both Walton's systematic review²⁶ and the latter study by Evans et al²⁸ demonstrated significant decreases in ADEs. Importantly, each of these CDSSs studies only addressed focal aspects of the medication system. In addition, relatively little information is available about the differences between systems.

Potential for Harm

Faulty decision support data, for example an incorrect default dosing suggestion, can lead to inappropriate ordering choices by physicians. The BWH time series analysis demonstrated an initial rise in intercepted potential ADEs due to the structure of the ordering screen for potassium chloride.²³ This structural error was identified and easily rectified, but underscores the importance of close scrutiny of all aspects of CPOE screens, both initially and on an ongoing basis.²³

Also, analogously to writing an order in the wrong patient chart in a conventional system, a physician can electronically write an order in the wrong patient's record - eg, after walking away from the terminal, opening the wrong record from a personalized rounding list. In addition, it is critical that the trigger level for warnings appropriately balances alarm sensitivity and specificity. These systems must have thresholds set so that physicians receive warnings in situations with a potential for significant harm, without being overwhelmed by "false alarms." Another potential risk is hardware outage or software instability. For example, the reliability that is needed with CPOE is much higher than that required for systems that simply report laboratory results.

Costs and Implementation

Six of the studies described in this review evaluated "home-grown" rather than "off-the-shelf" systems. The present costs for purchasing commercial systems are substantially more than the previous costs of developing such systems. For BWH, the cost of developing and implementing CPOE in 1992 was estimated to be $1.9 million, with maintenance costs of $500,000 per year. In comparison, the cost of purchasing and implementing large commercial systems varies substantially, but may be on the order of tens of millions of dollars. Several studies demonstrate that only minimal resources are needed to introduce and/or maintain decision support programs into existing order entry programs.^{21, 24, 30}

Relatively few data are available regarding the financial benefits of CPOE, although they extend well beyond medication-related events. The net savings of the BWH system are estimated at $5-10 million per year.³¹ It is estimated that the costs to BWH for preventable ADEs are $2.8 million annually.³² Evans et al reported a $100,000 per year cost avoidance with a computer-assisted antibiotic dosing program largely attributable to decreased antibiotic use and avoided ADEs.³³

Importantly, healthcare systems must garner both financial and organizational support before introducing CPOE with CDSSs. CPOE requires a very large up-front investment with more remote, albeit substantial returns. In addition, CPOE impacts clinicians and workflow substantially. Its complexity requires close integration with multiple systems, such as the laboratory and pharmacy systems. Failure to attend to the impact of such a large-scale effort on organizational culture and dynamics may result in implementation failure.³⁴ Therefore, it is essential to have organizational support and integration for its successful implementation and use.

Comment

The literature supports CPOE's beneficial effect in reducing the frequency of a range of medication errors, including serious errors with the potential for harm. Fewer data are available regarding the impact of CPOE on ADEs, with no study showing a significant decrease in actual patient harm. Similarly, isolated CDSSs appear to prevent a range of medication errors, but with few data describing reductions in ADEs or improvements in other clinical outcomes. Finally, the studied CDSSs address focused medication use (for example, antibiotic dosing) rather than more general aspects of medication use.

Further research should be conducted to compare the various types of systems and to compare "home-grown" with commercially available systems. Such comparisons are particularly important since the institutions that have published CPOE outcomes have generally been those with strong institutional commitments to their systems. Whether less committed institutions purchasing "off the shelf" systems will see benefits comparable to those enjoyed by "pioneers" with home-grown systems remains to be determined. Studying the benefits of such complex systems requires rigorous methodology and sufficient size to provide the power to study ADEs. Further research also needs to address optimal ways for institutions to acquire and implement computerized ordering systems.

Background

A large literature documents the multiple roles clinical pharmacists can play in a variety of health care settings.^1-8 Much of this literature focuses on measures of impact not directly relevant to this Report - eg, economic benefits,^4,8 patient compliance,⁶ and drug monitoring.^2,3 More recently, systems-based analyses of medication errors and adverse drug events (ADEs) have drawn attention to the impact clinical pharmacists can exert on the quality and safety of medication use.^9-12 In this chapter, we review the evidence supporting the premise that direct participation of pharmacists' in clinical care reduces medication errors and ADEs in hospitalized and ambulatory patients.

Practice Description

Clinical pharmacists may participate in all stages of the medication use process, including drug ordering, transcribing, dispensing, administering, and monitoring. The specific activities of clinical pharmacists vary substantially in the studies reviewed in this chapter. In the hospital setting, one study evaluated the role of a senior pharmacist participating fully in intensive care unit rounds and available throughout the day in person or by page for questions.¹³ Another study evaluated a ward pharmacy service that examined order sheets for new therapies and carried out checks that were formerly performed in the pharmacy.¹⁴

Pharmacists may also play a role at the time of discharge. One study reported the impact of clinical pharmacists' consultation for geriatric patients at the time of discharge,¹⁵ with pharmacists serving as consultants to physicians and reinforcing patients' knowledge of their medication regimen. The roles of clinical pharmacists are similarly diverse in studies of ambulatory settings. Here they include the provision of consultative services,^16-18 patient education,^16-18 therapeutic drug monitoring,³ and even follow-up telephone calls to patients.^3,16-18

Prevalence and Severity of the Target Safety Problem

It is estimated that over 770,000 people are injured or die in hospitals from adverse drug events (ADEs) annually.^19-21 The few hospitals that have studied incidence rates of ADEs have documented rates ranging from 2 to 7 per 100 admissions.^11,19,22,23 A precise national estimate is difficult to calculate due to the variety of criteria and definitions used by researchers.²⁴ One study of preventable inpatient ADEs in adults demonstrated that 56% occurred at the stage of ordering, 34% at administration, 6% at transcribing, and 4% at dispensing.²² In this study, the drug class most commonly associated with preventable ADEs was analgesics, followed by sedatives and antibiotics. Even fewer studies have been conducted in the outpatient setting. One recent cross-sectional chart review and patient care survey found an ADE rate of 3% in adult primary care outpatients.²⁵

Opportunities for Impact

Although many hospital pharmacy departments offer clinical pharmacy consultation services,²⁶ the degree to which these services include rounding with clinicians¹³ is unclear. Hospital pharmacies provide support to a variable degree in different ambulatory settings (clinics, nursing homes, adult daycare),²⁶ but the precise nature of clinical pharmacists' activities in these settings is not uniformly characterized.

Study Designs

We identified 3 systematic reviews of clinical pharmacists in the outpatient setting.^5,17,18 We included only the most recent review,¹⁸ as it followed the most rigorous methodology and included the bibliographies of the previous reviews.^5,17 We identified only one study¹⁶ of the impact of clinical pharmacists on patient outcomes in the ambulatory setting that had not been included in this systematic review.¹⁸ This study, a randomized controlled trial evaluating clinical pharmacists' participation in the management of outpatients with congestive heart failure,¹⁶ was therefore also included.

For studies of clinical pharmacists in the hospital setting, an older review² did not meet the characteristics of a systematic review, but was a very thorough summary of the relevant literature. It included 8 studies evaluating pharmacists' roles in detecting and reporting ADEs in the hospital setting. Preliminary review of these studies revealed that the measured outcomes were Level 3 (detection of ADEs as an end in itself). Consequently, we did not include them. One older retrospective before-after analysis (Level 3)¹⁴ did meet our inclusion criteria, as did 2 more recent studies of clinical pharmacists' roles in the inpatient setting: a prospective, controlled before-after study (Level 2)¹³ and a randomized controlled trial (Level 1).¹⁵

We included an additional meta-analysis focusing on therapeutic drug monitoring by clinical pharmacists in the hospital and ambulatory settings.³ The studies included in this meta-analysis consisted predominantly of controlled observational studies (Level 3) and non-randomized clinical trials (Level 2), but one randomized controlled trial was also included (Level 1-3A).

Table 7.1. Studies of clinical pharmacists' impact (more...)

Table 7.1. Studies of clinical pharmacists' impact on ADEs and medication errors*

Stydy	Study Design	Study Outcomes	Results
Beney, 2000.18 Systematic review of the roles and impacts of pharmacists in ambulatory settings; reviewed studies included 16,000 outpatients and 40 pharmacists	Level 1A (systematic review)	Levels 1-3 (variety of patient outcomes, surrogate outcomes, impacts on physician prescribing practices and measures of resource use)	Improvement in outcomes for patients with hypertension, hypercholesterolemia, chronic heart failure, and diabetes
Gattis, 1999.16 181 patients with heart failure due to left ventricular dysfunction followed in a general cardiology clinic	Level 1 (RCT)	Level 1 (mortality and other clinical outcomes related to heart failure)	16 versus 4 deaths or other heart failure events (p<0.005)
Leape, 1999.13 Medical and cardiac intensive care unit patients at Massachusetts General Hospital, a tertiary care hospital in Boston	Level 2 (prospective before-after study with concurrent control)	Level 1 (ADEs)	66% decrease in the rate of preventable ADEs (p<0.001)
Leach, 1981.14 315 patients at Queen Elizabeth Hospital in Birmingham, England	Level 3 (retrospective before-after analysis)	Level 2 (various types of medication errors)	40-50% overall reduction in medication errors
			All 8 of the targeted error types decreased (results achieved statistical significance for 5 error types)
Lipton, 1992.15 236 geriatric patients discharged from the hospital on three or more medications	Level 1	Levels 2 & 3 ("prescribing problems")	Less likely to have a "prescribing problem" (p=0.05)
Ried, 1989.3 Pooled patient population not reported, but review of articles indicates predominance of studies of (mostly adult) hospitalized patients	Level 1A-3A (meta-analysis predominantly included controlled observational studies and non-randomized trials)	Levels 2 & 3 (measures of peak, trough and toxic serum drug concentrations for a variety of medications)	More likely to have therapeutic peak and trough and less likely to have toxic peak and trough, but modest effect sizes (results achieved statistical significance for only 2 measures)

* ADE indicates adverse drug event; and RCT, randomized controlled trial

Study Outcomes

Level 1 outcomes reported in the included studies consisted of ADEs¹³ and clinical events related to heart failure, including mortality.¹⁶ One study used telephone interviews to solicit patient self-reports of adverse drug reactions.^13,27 This study was excluded since the systematic review of clinical pharmacists' roles in ambulatory settings¹⁸ incorporated its findings in several of its analyses.

The distinction between Level 2 and 3 outcomes can be somewhat ambiguous in studies of prescribing practice, as the exact point at which choices of therapeutic agents or dosing patterns become not just sub-optimal but actually represent "errors" is difficult to define precisely. Even for objective outcomes, such as serum drug concentrations, the connection to patient outcomes is weak in some cases (eg, monitoring vancomycin levels^28,29), and therefore more appropriately designated as Level 3 rather than Level 2. Acknowledging the subjectivity of this judgment in some cases, we included studies that contained a mixture of Level 2 and 3 outcomes, including "prescribing problems," (which included inappropriate choice of therapy, dosage errors, frequency errors, drug-drug interactions, therapeutic duplications, and allergies¹⁵), and serum drug concentrations for a broad range of medications.³

Evidence for Effectiveness of the Practice

In the inpatient setting, Leape's study¹³ demonstrated a statistically significant 66% decrease in preventable ADEs due to medication ordering. The study of geriatric patients at the time of discharge demonstrated clinically and statistically significant decreases in medication errors.¹⁵ The meta-analysis of the effect of clinical pharmacokinetics services on maintaining acceptable drug ranges indicated only modest effect sizes for the outcomes measured, and only 2 of the main results achieved statistical significance.³

The comprehensive review of clinical pharmacist services in ambulatory settings reported positive impacts for patients with hypertension, hypercholesterolemia, chronic heart failure, and diabetes.¹⁸ However, the authors identify important limitations: these studies are not easily generalizable, only 2 studies compared pharmacist services with other health professional services, and both studies had important biases. Consequently, they emphasized the need for more rigorous research to document the effects of outpatient pharmacist interventions.¹⁸ The additional study of outpatients demonstrated significant decreases in mortality and heart failure events,¹⁶ but these results may reflect closer follow-up and monitoring (including telemetry) for the intervention group or the higher doses of angiotensin-converting enzyme (ACE) inhibitors the patients received. Generalizing this benefit to other conditions is difficult since most conditions do not have a single medication-related process of care that delivers the marked clinical benefits as do ACE inhibitors in the treatment of congestive heart failure.³⁰

Potential for Harm

Introducing clinical pharmacists might potentially disrupt routine patient care activities. However, the 2 studies that assessed physician reactions to clinical pharmacists^13,27 found excellent receptivity and subsequent changes in prescribing behavior.

Costs and Implementation

Two studies examined resource utilization and cost savings in the inpatient setting. The intensive care unit study indicated that there could be potential savings of $270,000 per year for this hospital if the intervention involved re-allocation of existing pharmacists' time and resources.¹³ McMullin et al studied all interventions made by 6 hospital pharmacists over a one-month period at a large university hospital and estimated annual cost savings of $394,000.³¹

A systematic review of outpatient pharmacists indicated that pharmacist interventions could lead to increased scheduled service utilization and decreased non-scheduled service utilization, specialty visits, and numbers and costs of drugs.¹⁸

Comment

At present, one study provides strong evidence for the benefit of clinical pharmacists in reducing ADEs in hospitalized intensive care unit patients.¹³ One additional study provides modest support for the impact of ward-based clinical pharmacists on the safety and quality of inpatient medication use.¹⁴ The evidence in the outpatient setting is less substantial, and not yet convincing. Given the other well-documented benefits of clinical pharmacists and the promising results in the inpatient setting, more focused research documenting the impact of clinical pharmacist interventions on medication errors and ADEs is warranted.

Background

Adverse drug events (ADEs) occur in both inpatient and outpatient settings.¹,² Most institutions use spontaneous incident reporting to detect adverse events in general, and ADEs in particular. Spontaneous incident reporting relies exclusively on voluntary reports from nurses, pharmacists and physicians focused on direct patient care. However, spontaneous reporting is ineffective, identifying only one in 20 ADEs.³ Efforts to increase the frequency of spontaneous reporting have had only a minor impact.

Several studies demonstrate the effectiveness of using computerized detection and alert systems (referred to as computer "monitors") to detect ADEs. In 1991, Classen et al⁴ published information about a computerized ADE monitor that was programmed to identify signals -- in effect mismatches of clinical information -- that suggested the presence of an ADE. The signals included sudden medication stop orders, antidote ordering, and certain abnormal laboratory values.⁴ The computerized signals were then evaluated by a pharmacist who determined whether an ADE had occured. Based on the rules of this study, Jha et al developed a similar monitor that identified approximately half the ADEs identified by chart review, at much lower cost.⁵ Similarly, Bowman and Carlstedt used the Regenstrief Medical Record System to create a computerized inpatient ADE detection system.⁶ Compared to the "gold standard" of chart review, the monitor had 66% sensitivity and 61% specificity, with a positive predictive value of 0.34. Finally, one community hospital implemented an ADE monitor with triggers that were reviewed by pharmacists who then contacted physicians to make appropriate regimen changes. This study identified opportunities to prevent patient injury at a rate of 64/1000 admissions.⁷ These studies and others demonstrate the potential value of computer monitors, especially when linked to effective integrated information systems. While monitors are not yet widely used, they offer an efficient approach for monitoring the frequency of ADEs on an ongoing basis, and the Health Care Financing Administration is considering mandating them.⁸

Practice Description

Computerized ADE alert monitors use rule sets to search signals that suggest the presence of adverse drug events. The most frequently studied rule sets (or "triggers") are those that search for drug names (eg, naloxone, kayexalate), drug-lab interactions (eg, heparin and elevated PTT) or lab levels alone (eg, elevated digoxin levels) that frequently reflect an ADE. Simple versions can be implemented with pharmacy and laboratory data alone, although the yield and positive predictive value of signals is higher when the 2 databases are linked.

Further refinements include searches for International Classification of Diseases (ICD-9) codes, and text-searching of electronic nursing bedside charting notes or outpatient notes for drug-symptom combinations (eg, medication list includes an angiotensin converting enzyme inhibitor and the patient notes mention "cough"). Although these refinements do increase the yield of monitors, they require linkage to administrative databases or electronic medical records.

The information captured with computer monitors is used to alert a responsible clinician or pharmacist, who can then change therapy based on the issue in question. Systems are designed to alert the monitoring clinician in various ways. Alerts can go to one central location (eg, hospital pharmacist) or to individual physicians. Monitoring pharmacists typically review the alert and contact the appropriate physician if they determine that the alert has identified a true event. The alert modality also varies based on the available technology, from printed out reports, to automatic paging of covering physicians, to display of alerts on computer systems (either in results or ordering applications). It should be emphasized that computerized physician order entry (Chapter 6) is not a requirement for these monitors. Thus, a simple version of this approach could be implemented in most US hospitals.

Prevalence and Severity of the Target Safety Problem

It is estimated that over 770,000 people are injured or die in hospitals from ADEs annually,³,⁹,¹⁰ but variations in study criteria and definitions prevent precise national estimates.¹¹ Fewer data address the epidemiology of ADEs in the outpatient setting. One recent study found an ADE rate of 3% in adult primary care outpatients,¹² while an older study reported a similar rate of 5% among ambulatory patients attending an internal medicine clinic over a 1-year period.²

Detection and alerting interventions primarily target ADEs related to the medication ordering process. One study of preventable inpatient ADEs in adults demonstrated that 56% occurred at the stage of ordering.¹³ Among the 6.5 ADEs per 100 admissions in this study, 28% were judged preventable, principally by changing the systems by which drugs are ordered and administered.¹⁴ In one study of computerized ADE detection, the ADEs identified by computerized monitoring were significantly more likely to be classified as severe than those identified by chart review (51% vs. 42%, p=0.04).⁵ Thus, monitors may capture a subset of events with the most potential for patient injury.

Injuries due to drugs have important economic consequences. Inpatients that suffer ADEs have increased lengths of stay of nearly 2 days and added hospital costs of more than $2000.^9,15 Bootman has estimated the annual cost of drug-related morbidity and mortality within the United States at $76.6 billion, with the majority ($47 billion) related to hospital admissions due to drug therapy or the absence of appropriate drug therapy.¹⁶

Opportunities for Impact

Unfortunately, there are no good data as to how many hospitals have integrated lab and medication systems, which are required for many of the triggers used in computerized ADE monitors.

Study Designs

Table 8.1. Included studies of computerized systems (more...)

Table 8.1. Included studies of computerized systems for ADE detection and alerts*

Study Intervention	Study Design, Outcomes	Results**
Kuperman, 199919 Computerized alerts to physicians via paging system	Level 1, Level 1	Median time until initial treatment ordered: 1 vs. 1.6 hours (p=0.003) Median time until condition resolved: 8.4 vs. 8.9 hours (p=0.11) Number of adverse events: no significant difference
McDonald, 197621 Alerts to outpatient physicians with suggested responses to medication related events	Level 1, Level 2	Physicians performed recommended testing: 36% vs. 11% (p<0.00001) Physicians made changes in therapy: 28% vs. 13% (p<0.026)
Evans, 199418 Computerized monitor to detect ADEs (including drug/lab monitors and searches of nursing notes) and then computerized alerts to physicians	Level 3, Level 1	Type B ADEs (allergic or idiosyncratic) and severe ADEs: 0.1 vs. 0.5 per 1000 patient-days (p<0.002) Severe ADEs: 0.1 vs. 0.4 per 1000 patient-days (p<0.001)
Rind, 199420 Computerized alert system to physicians about rising serum creatinine values in patients receiving potentially nephrotoxic medications	Level 3, Level 1	Serious renal impairment: RR 0.45 (95% CI: 0.22-0.94) Mean serum creatinine lower after an event (0.16 mg/dL lower on Day 3, p<0.01) Dose adjusted or medication discontinued an average of 21.6 hours sooner after an event (p<0.0001)
Bradshaw, 198917 Computerized alerts integrated into result review and alerts by flashing light	Level 3, Level 2	Response time to alert condition: 3.6 (±6.5) vs. 64.6 (±67.1) hours

* ADE indicates adverse drug event; CI, confidence interval; and RR, relative risk.

** Results reported as rates with intervention vs. control (Level 1 study designs) or after intevention vs. before intervention (Level 3 study designs).

Study Outcomes

All of the studies reported Level 1 or 2 outcomes. Level 1 outcomes were the rate of ADEs^18,19 and renal impairment (as reflected by rises in creatinine).²⁰ Level 2 outcomes included percent of time recommended actions were taken and time to respond to an event.^17-20

Evidence for Effectiveness of the Practice

Potential for Harm

None of the studies discuss any potential for harm associated with the monitor and alerts. It is certainly possible that alerts could be erroneous, but it is doubtful that this would lead to any direct patient harm. As in studies of safety in other industries, one possible source of harm could be too many false positives. Large numbers of clinically insignificant warnings for patients would interfere with routine care, and might result in providers ignoring all warnings, even clinically meaningful ones.

Costs and Implementation

In general, implementation of alert monitors requires computer systems that can link laboratory and medication information. Integrating this information requires the creation of interface between the drug and laboratory databases, with costs that will vary depending on the nature of the existing information systems. In addition, alert methods vary, with some applications directly contracting physicians (which requires further integration of coverage and pager databases) and others using pharmacist intermediaries. The cost of pharmacist review of triggers was less than 1 FTE per hospital in 2 studies^5,7; one of them reported an annual cost savings of up to 3 million dollars by reducing preventable ADEs with this alerting technique.⁷

Studies thus far suggest that physicians view computerized alert systems favorably. Forty-four percent of physician-respondents receiving alerts indicated that the alerts were helpful and 65% wished to continue receiving them (although these alerts went to many physicians because it was unclear who the responsible doctor was). In another study in which alerts were sent only to the responsible physician, 95% of physician-respondents were pleased to receive them.¹⁹

The systems in these studies were all "homegrown" and contained idiosyncrasies that might undermine their implementation elsewhere. Clearly it is important that systems track which physicians are responsible for which patients. In addition, all 4 of the inpatient studies were conducted at tertiary care hospitals and the outpatient study was done at clinics affiliated with a tertiary care center. The translation of this alerting approach to community settings may be difficult. One community teaching hospital has reported success in detecting opportunities to prevent injury (64/1000 admissions) using computerized detection and alerting. This report had only a Level 4 design (no control group), so it was not included in the Table.⁷

Comment

Computerized real-time monitoring facilitates detection of actual and potential ADEs and notification of clinicians. This in turn may aid in the prevention of ADEs or decrease the chances that ADEs will cause harm. The monitors also yield improvements in secondary measures relating to the length of time until response and the quality of response.

The applications in these studies were all "homegrown." Future applications should be evaluated and refined further. In particular, it is important to quantify the yield of collecting these data in terms of patient outcomes, since the start-up costs are significant. If monitors do lead to important clinical benefits, they should become standard features of commercially available hospital computer systems. As this occurs, careful attention will need to be paid to optimizing the response process.

In addition, little has been done to translate these monitoring systems to the outpatient setting, largely because outpatient clinical information is often not computerized or resides in disparate systems. As integrated computerized outpatient records become more common, the systems developed in the inpatient setting should be translatable to the outpatient setting.

Background

Published studies of adverse drug events and multiple case reports have consistently identified certain classes of medications as particularly serious threats to patient safety.^1-3 These "high risk" medications include concentrated electrolyte solutions such as potassium chloride, intravenous insulin, chemotherapeutic agents, intravenous opiate analgesics, and anticoagulants such as heparin and warfarin. Analyses of some of the adverse events involving these mediations have led to important recommendations regarding their administration. Examples include the use of order templates for chemotherapeutic agents, removal of intravenous electrolyte solutions from general ward stock, and protocols for reviewing the settings of intravenous pumps delivering continuous or frequent doses of opiates.^2,4,5 While these recommendations have high face validity, they have generally not been subject to formal evaluation regarding their impact in reducing the targeted adverse events. By contrast, several practices relating to the management of patients receiving anticoagulants have been evaluated quite extensively, and therefore constitute the focus of this chapter.

Heparin and warfarin are medications whose use or misuse carry significant potential for injury. Subtherapeutic levels can lead to thromboembolic complications in patients with atrial fibrillation or deep venous thrombosis (DVT), while supratherapeutic levels can lead to bleeding complications. These medications are commonly involved in ADEs for a variety of reasons, including the complexity of dosing and monitoring, patient compliance, numerous drug interactions, and dietary interactions that can affect drug levels. Strategies to improve both the dosing and monitoring of these high-risk drugs have potential to reduce the associated risks of bleeding or thromboembolic events.

Practice Description

The practices reviewed in this chapter are all intended to reduce dosing and/or monitoring errors for heparin and warfarin, as follows:

• Heparin dosing protocols ("nomograms") typically involve a standard initial bolus and infusion rate, instructions for when to draw the first partial thromboplastin time (PTT), and orders for dosing adjustments in response to this and subsequent values (so nurses can adjust doses automatically). In some cases, the initial bolus and infusion rates are based on patient weight.

• Inpatient anticoagulation services for both heparin and warfarin (with or without dosing nomograms) typically consist of pharmacist-run services that provide daily pharmacy input on dosing and monitoring for patients on heparin and/or warfarin. (We excluded studies focusing solely on warfarin prophylaxis in orthopedic patients.⁶)

• Outpatient anticoagulation clinics provide coordinated services for managing outpatient warfarin therapy. Services typically include anticoagulation monitoring and follow-up, warfarin dose adjustment, and patient education. These clinics are usually run by pharmacists or nurses operating with physician back-up, and sometimes following specific dosing nomograms.

• Patient self-monitoring using a home finger-stick device and self-adjustment of warfarin dosages using a nomogram. (The accuracy of these devices and the comparability of patients' and professional readings have been extensively evaluated.^7-11)

Prevalence and Severity of the Target Safety Problem

Intravenous heparin and oral warfarin are commonly used medications for cardiac disease and thromboembolism in the inpatient and outpatient settings. While in the aggregate they are highly beneficial (see Chapter 31), these drugs can have significant morbidities unless they are dosed and monitored appropriately. For example, inadequate therapeutic dosing of heparin can lead to increased length of stay and the potential for clot formation and/or propagation.¹² The risk of recurrent thromboembolism is reduced if the therapeutic effect of heparin is achieved quickly.¹² In addition, Landefeld et al¹³ showed that the frequency of fatal, major, and minor bleeding during heparin therapy was twice that expected without heparin therapy. The effect with warfarin therapy was even more pronounced - approximately 5 times that expected without warfarin therapy. Consistent with this finding, anticoagulants accounted for 4% of preventable ADEs and 10% of potential ADEs in one large inpatient study. Finally, careful drug monitoring in hospitals can reduce ADEs, suggesting that some events are due to inadequate monitoring of therapies and doses.¹⁴ These studies highlight the clear need for safety-related interventions with respect to both the dosing and monitoring of these high-risk drugs in order to prevent thromboembolic and bleeding complications.

Opportunities for Impact

The number of hospitals using weight-based heparin nomograms, or that have established anticoagulation clinics or services is unknown. Although common in some European countries,¹⁵ patient self-management of long-term anticoagulation with warfarin is unusual in the United States as many payers, including Medicare, do not currently cover the home testing technology.¹⁵

Study Designs

Heparin nomograms were evaluated in one randomized controlled trial (Level 1),¹⁶ one prospective cohort comparison (Level 2)¹⁷ and 4 controlled observational studies (Level 3).^18-21 Two of these studies involved weight-based nomograms.^16,21 A third study involving a weight-based nomogram²² was included with the studies of anticoagulation services (see below), as clinical pharmacists actively managed the dosing protocol. We excluded one retrospective before-after analysis of a weight-based heparin protocol for cardiac intensive care patients,²³ because the method of selecting charts for review was never stated. Moreover, when the authors found an increase in the number of patients with excessive anticoagulation in the intervention group, they chose a second group of control patients (again with an unspecified selection method) for review, and in the end concluded that the difference was not significant.

Table 9.2. Inpatient anticoagulation services and outpatient anticoagulation (more...)

Table 9.2. Inpatient anticoagulation services and outpatient anticoagulation clinics*

Study	Study Design, Outcomes	Results
Ansell, 199630 Pooled comparison of anticoagulation clinics and routine medical care	Pooled results from 6 Level 3 study designs comparinganticoagulation clinics with routine medical care31-36 (Level 3A) Major bleeding and thromboembolic events (Level 1)	Major bleeding events per patient-year: anticoagulation clinic, 0.028 (95% CI: 0-0.069) vs. routine care, 0.109 (95% CI: 0.043-0.268) Thromboembolic events per patient-year: anticoagulation clinic, 0.024 (95% CI: 0-0.08) vs. routine care, 0.162 (95% CI: 0.062-0.486)
Hamby, 200029 Analysis of adverse events related to outpatient warfarin therapy among 395 patients followed at a Veterans Affairs Hospital, with 306 enrolled in an anticoagulation clinic and 89 patients receiving usual care	Case-control study (Level 3) Adverse events related to under- or over-anticoagulation (Level 1)	Among the 12 patients with preventable adverse events related to anticoagulation, 8 were not enrolled in the anticoagulation clinic Patients receiving usual care had 20 times the relative risk (95% CI: 6-62) of an adverse event compared with patients in the anticoagulation clinic.
Lee, 199626 Comparison of pharmacist-managed anticoagulation clinic with patient receiving usual care	Retrospective cohort comparison (Level 3) Hospital admissions related to under- or over-anticoagulation - ie, thromboembolic or bleeding events (Level 1)**	Patients in anticoagulation clinic had non-significant reductions in hospital admissions related to thromboembolic or bleeding events compared with control group***
Ellis, 199237 Pharmacy-managed inpatient anticoagulation service (flow sheet for monitoring, but no nomogram) for monitoring patients receiving warfarin for a variety of indications	Retrospective before-after analysis (Level 3) Anticoagulation "stability" at discharge and odds of therapeutic anticoagulation at first outpatient visit (Level 2)	Patients receiving the intervention were more likely to have PT "stability" at discharge: 61.5% vs. 42.3% (p=0.02) Odds of having therapeutic PT at first outpatient clinic visit with intervention: OR 5.4 (95% CI: 1.87-15.86)
Gaughan, 200024 Anticoagulation clinic for outpatients receiving warfarin for atrial fibrillation (managed by nurse practitioner using warfarin dosing nomogram)	Retrospective before-after analysis (Level 3) Percentage of patients in the desired range for anticoagulation (Level 2) was evaluated as a secondary outcome	Minor increase in percentage of patients with INR in desired range: 53.7% vs. 49.1% (p<0.05, but questionable clinical significance)
Radley, 199527 Performance of pharmacist-run hospital-based outpatient anticoagulation clinic in England compared with historical control (management by rotating physician trainees)	Retrospective before-after analysis (Level 3) Proportions of INR measurements "in" or "out" of the therapeutic range	No significant difference for patients with stable INR in the baseline period, but patients with an INR result "out" of range were more likely to return to "in" range under anticoagulation clinic management compared with routine physician management
Rivey, 199322 Pharmacy-managed inpatient anticoagulation service (using weight-based heparin protocol) for medicine inpatients compared with older fixed-dose protocol without any active management by pharmacists	Before-after analysis (Level 3) Time to therapeutic PTT (Level 2)	Time to therapeutic PTT was less with nomogram protocol: 40 vs. 20 hours (p<0.05) Fewer supra-therapeutic PTTs with protocol: 1.7 vs. 5.5 (p<0.05) Bleeding rates: no difference but numbers were small

* CI indicates confidence interval; INR, international normalized ratio; OR, odds ratio; PT, prothrombin time; and PTT, partial thromboplastin time.

** We counted this outcome as Level 1, but it is important to note that authors did not capture all of the designated clinical events, just those that resulted in admissions to the study hospital.

*** Using the results reported in the study, we calculated the 95% CIs for admissions related to thromboembolic events (intervention, 0.2-18.5%; usual care, 12.7-42.5%) and bleeding events (inervention, 1.1-22.8%; usual care, 7-33.4%).

Two studies evaluated the impact of a coordinated inpatient anticoagulation service (with or without nomograms for dosing).^22,37

Patient self-management of warfarin therapy has been evaluated in at least 3 randomized controlled trials^38-40 (Level 1) and one non-randomized clinical trial.⁴¹ Because a number of higher-level studies exist, we did not include retrospective cohort analyses (Level 3) addressing this topic.^42-45

Study Outcomes

Table 9.1. Studies focused primarily on heparin or (more...)

Table 9.1. Studies focused primarily on heparin or warfarin nomograms*

Study	Study Design, Outcomes	Results**
Raschke, 199316 Weight-based heparin nomogram for patients with venous thromboembolism or unstable angina	Randomized controlled trial (Level 1) Various markers of adequate anticoagulation (Level 2)	PTT in therapeutic range within 24 hours: 97% vs. 77% (p<0.002) Mean time to therapeutic PTT: 8.2 vs. 20.2 hours (p<0.001) PTT exceeding the therapeutic range: at 24 hours, 27% vs. 7% (p<0.001) at 48 hours, 18% vs. 8% (p<0.001)
Elliott, 199417 Use of heparin nomogram for patients with acute proximal deep venous thrombosis	Non-randomized clinical trial (Level 2) Time to therapeutic PTT (Level 2)	Time to therapeutic PTT: less with use of nomogram (values not given, p=0.025)
Brown, 199721 Weight-based heparin nomogram for ICU patients requiring acute anticoagulation with unfractionated heparin	Retrospective before-after analysis (Level 3) Time to therapeutic PTT (Level 2)	Mean time to therapeutic PPT: 16 vs. 39 hours (p<0.05) Supratherapeutic PTTs were more common after implementation of the nomogram, but there was no observed increase in bleeding
Cruickshank, 199118 Heparin nomogram for patients with acute venous thromboembolism	Retrospective before-after analysis (Level 3) Time to first therapeutic PTT, time to correct subsequent PTTs, time outside the therapeutic range (Level 2)	PTT in therapeutic range at 24 hours, 66% vs. 37% (p<0.001) PTT in therapeutic range at 48 hours, 81% vs. 58% (p<0.001)
Hollingsworth, 199519 Heparin nomogram for hospitalized patients with acute venous thromboembolism	Retrospective before-after analysis (Level 3) Primary outcome of the study was length of hospital stay (Level 3) but time to therapeutic PTT was a secondary outcome (Level 2)	Time to therapeutic PTT: 17.9 vs. 48.8 hours (p<0.001) PTTs were sub-therapeutic less often: 28% vs. 56% (p<0.001) Proportion of patients with supra-therapeutic PTTs was significantly increased in the intervention group. There was no increase in bleeding complications associated with this finding, but the study was underpowered to detect such a difference.
Phillips, 199720 Inpatient heparin and warfarin nomograms and monitoring charts	Retrospective before-after analysis (Level 3) Measures of under- and over-anticoagulation (Level 2)	Heparin nomogram Time spent under-anticoagulated: 18.5% vs, 32.7% (p<0.0001) Time spent above the therapeutic range: 35.6% vs. 24.4% (p<0.01) Warfarin nomogram: Time spent over-anticoagulated: 5.4% vs. 2.7% (p<0.001, but questionable clinical significance)

* PTT indicates partial thromboplastin time.

** Results reported as rates with intervention vs. control (Level 1 & 2 study designs) or after intevention vs. before intervention (Level 3 study designs).

Table 9.3. Outpatient self-management using home testing (more...)

Table 9.3. Outpatient self-management using home testing devices and dosing nomograms*

Study	Study Design, Outcomes	Results
Cromheecke, 200038 Oral anticoagulation self-management with home monitoring and dose adjustment compared with anticoagulation clinic (Netherlands)	Randomized trial with crossover comparison (Level 1) Adequacy of anticoagulation (Level 2)	Percent of self-managed measurements within 0.5 INR units of therapeutic target did not differ (55% vs. 49%, p=0.06). However, 29 patients (60%) during self-management spent >50% of time in target range, compared with 25 (52%) during clinic management (p<0.05).
Sawicki, 199939 Oral anticoagulation self-management with home monitoring and dose adjustment compared with routine care (Germany)	Single blind, multicenter randomized controlled trial (Level 1) Adequacy of anticoagulation (Level 2)	Intervention group more often had INRs within target range (p<0.01), and had significantly fewer deviations from target range and 6 months
White, 198940 Oral anticoagulation self-management with home monitoring and dose adjustment compared with anticoagulation clinic (United States)	Randomized prospective comparison (Level 1) Adequacy of anticoagulation (Level 2)	Self-management group had significantly greater proportion of patients in target INR range (93% vs. 75%, p<0.01)
Watzke, 200041 Self-management compared with anticoagulation clinic (Austria)	Prospective cohort comparison (Level 2) Various measures of adequacy of anticoagulation (Level 2)	Non-significant trends towards more INR values within the therapeutic range for self-management group compared with anticoagulation clinic, both for standard therapeutic range of INR 2.0-3.0 (82.2% vs. 68.9%) and for more intense anticoagulation targeted to INR range of 2.5-4.5 (86.2% vs. 80.1%)

* INR indicates international normalized ratio.

Evidence for Effectiveness of the Practice

• Heparin nomograms: As shown in Table 9.1, all studies showed a significant decrease (ie, improvement) in time to achievement of a therapeutic PTT and/or an increase in the proportion of patients in the therapeutic range.

• Inpatient anticoagulation services: As shown in Table 9.2, both Level 3 studies evaluating this practice showed significant improvements in relevant measures of anticoagulation.^{22, 37}

• Outpatient anticoagulation services for warfarin (with and without dosing nomograms): the multiple Level 3 studies of this practice showed improvements in relevant measures of anticoagulation, with one exception.²⁸ This study took place in a semi-rural region of England, and the hospital-based anticoagulation clinic was staffed mainly by junior physician trainees rotating through the clinic. The one study that focused primarily on Level 1 outcomes²⁵ showed significant reductions in adverse events related to under- or over-anticoagulation.

• Patient self-management: Patient self-management achieved superior measures of anticoagulation on one Level 1 comparison with routine care.^22,37 More impressive is that two Level 1 studies^38,46 and one Level 2 study⁴¹ reported equivalent or superior measures of anticoagulation for self-management compared with anticoagulation clinics.

Potential for Harm

Heparin nomograms are primarily intended to achieve PTT values within the therapeutic range as quickly as possible. Although none of the studies showed increased bleeding as a result of aggressive anticoagulation, it is important to note that 4 of the 6 studies showed a significant increase in the proportion of patients with PTTs above the target range.^16,19-21

Anticoagulation clinics carry the usual theoretical risk that increased fragmentation of care will introduce new hazards, but no study showed any significant cause for concern.

Patient self-monitoring clearly carries with it risks relating to the possibilities of patient misunderstanding of, or non-compliance with dosing and monitoring protocols. No increases in adverse events were observed in the studies reviewed, but the patients evaluated in these studies, even if randomized, were still chosen from a group of relatively compliant and motivated patients.

Costs and Implementation

For anticoagulation clinics, one study showed reduced costs of $162,058 per 100 patients annually, primarily through reductions in warfarin-related hospitalizations and emergency room visits.²⁵ Other studies indicate potential cost-savings due to reduced hospitalizations from anticoagulation-related adverse events, or show that the anticoagulation was revenue neutral.^19,24,29 Considering without these offsetting potential savings, however, anticoagulant clinics often require institutional subsidy since professional fee or laboratory payments do not fully cover costs.

Heparin nomograms may increase lab costs due to more frequent monitoring, but one study calculated that lab costs were offset by the need for fewer heparin boluses.²²

For patient self-management of warfarin, one study showed that the cost of self-monitoring was $11/international normalized ratio (INR) value and postulated that this would be cost-effective if it reduced the number of clinic visits.³⁹ Other studies have suggested that the capillary blood testing devices themselves⁴⁷ and the overall practice of patient self-management are cost-effective.^48,49 In the United States, the home monitoring devices sell for approximately $1000. Factoring in the price of cartridges and assuming the devices operate without requiring repair for 5 years, one source estimated an annual cost of approximately $600.⁴⁰

Implementation of a heparin nonogram appears feasible, and was well received by physicians and nurses.¹⁸ Physician/staff education about the protocols was important to its success.^23,24 One study showed a high level of physician and patient satisfaction with an anticoagulation clinic.²⁴ In addition, multiple studies reveal that patients who self-manage warfarin have significantly higher levels of satisfaction and experience less anxiety.^9,10,38,39

Comment

The primary purpose of heparin nomograms is the timely achievement of therapeutic anticoagulation, and their superiority in this regard (compared with routine care) has been convincingly established. While none of the studies showed adverse consequences of this focus on timely anticoagulation, the trend toward increases in excessive anticoagulation presents safety concerns. Studies powered to detect significant differences in bleeding complications in patients being managed with heparin dosing protocols may be warranted.

The literature on anticoagulation clinics consists entirely of observational studies with important possible confounders. Nonetheless, with one exception²⁸ they are consistently shown to achieve superior measures of anticoagulation, and in one study,²⁵ superior clinical outcomes.

Among the practices reviewed in this chapter, the literature on patient self-management is perhaps the most impressive. Three randomized trials and one non-randomized clinical trial show that patient control of anticoagulation is at least equivalent, if not superior, to management by usual care or an anticoagulation clinic. Additional observational studies reach the same results.^42-45

Thus, a relatively substantial literature supports patient self-management for outpatient warfarin therapy for motivated patients able to comply with the monitoring and dosing protocols. These studies clearly involved select groups of patients,⁹ so that a larger randomized trial with intention-to-treat analysis would be helpful.

Many insurance carriers in the United States, including Medicare, do not currently subsidize the home testing technology or provide only partial coverage.¹⁵ Despite the relatively high cost of the home testing devices, this practice may nonetheless be cost-effective due to reduced use of other clinical services.^48,49 A larger US study or detailed cost-effectiveness analysis appears warranted, especially given the higher level of patient satisfaction with this approach as compared with outpatient anticoagulation.

Background

Medication errors are especially likely when health professionals engage in multiple tasks within a short time span. This situation occurs repeatedly in hospitals when pharmacists and technicians load unit-dose carts, and when nurses administer medications. Unit-dose carts are prepared daily, often manually, by technicians and then checked by pharmacists. These carts, containing thousands of patient-specific dosages of drugs, are sent to the wards daily, for nurses to administer medications to patients. Dosing frequencies vary widely, ranging from regular intervals around the clock to "stat" doses given to control acute pain or other symptoms. Medication administration alone is an enormous task for nurses, and one in which they are repeatedly interrupted. It is not surprising that the administration phase of medication use is particularly vulnerable to error.¹

Unit-dose dispensing of medication was developed in the 1960s to support nurses in medication administration and reduce the waste of increasingly expensive medications. Most of the investigations of medication errors and unit-dose dispensing took place from 1970 to 1976. Now, unit-dose dispensing of medications is a standard of practice at hospitals in the United States. This chapter reviews the evidence supporting this practice.

Practice Description

In unit-dose dispensing, medication is dispensed in a package that is ready to administer to the patient.² It can be used for medications administered by any route, but oral, parenteral, and respiratory routes are especially common. When unit-dose dispensing first began, hospital pharmacies equipped themselves with machines that packaged and labeled tablets and capsules, one pill per package. They also purchased equipment for packaging liquids in unit-doses. As the popularity of this packaging increased, the pharmaceutical industry began prepackaging pills in unit-of-use form. Many hospitals now purchase prepackaged unit-dose medications. However, it is still common for hospital pharmacies to purchase bulk supplies of tablets and capsules from manufacturers and repackage them in the central pharmacy into unit-dose packages.² It is important to note that hospitals vary in the proportion of their wards covered by a unit-dose system.

There are many variations of unit-dose dispensing. As just one example, when physicians write orders for inpatients, these orders are sent to the central pharmacy (by pharmacists, nurses, other personnel, or computer). Pharmacists verify these orders and technicians place drugs in unit-dose carts. The carts have drawers in which each patient's medications are placed by pharmacy technicians -- one drawer for each patient. The drawers are labeled with the patient's name, ward, room, and bed number. Before the carts are transported to the wards, pharmacists check each drawer's medications for accuracy. Sections of each cart containing all medication drawers for an entire nursing unit often slide out and can be inserted into wheeled medication carts used by nurses during their medication administration cycles. A medication administration recording form sits on top of the cart and is used by the nurse to check-off and initial the time of each administration of each medication. The next day, the carts are retrieved from the wards and replaced by a fresh and updated medication supply. Medications that have been returned to the central pharmacy are credited to the patient's account.

A 1999 national survey of drug dispensing and administration practices indicated that three-fourths of responding hospitals had centralized pharmacies, 77% of which were not automated.² Larger hospitals and those affiliated with medical schools were more likely to have some component of decentralized pharmacy services. About half of the surveyed hospitals reported drug distribution "systems" that bypassed the pharmacy, including hospitals that reported using floor stocks, borrowing other patients' medications, and hidden drug supplies.

Studies often compare unit-dose dispensing to a ward stock system. In this system, nurses order drugs in bulk supplies from the pharmacy; the drugs are stored in a medication room on the ward. Nurses prepare medication cups for each patient during medication administration cycles. The correct number of pills must be taken out of the correct medication container for each cycle and taken to the patient for administration. Liquids must be poured by the nurse from the appropriate bottle and each dose carefully measured. Nurses are responsible for any necessary labeling. Any medications taken from stock bottles and not administered to patients are generally disposed of.

Prevalence and Severity of the Target Safety Problem

The targets of the safety problem for unit-dosing are drug dispensing³ and administration.^4,5 Improving these stages probably carries the greatest opportunity to reduce medication errors.

Bates et al⁶ identified 530 medical errors in 10,070 written orders for drugs (5.3 errors/100 orders) on 3 medical units observed for 51 days. Of the 530 errors, 5 (0.9%) resulted in an adverse drug event. The most common reason for an error was a missing dose of medication, which occurred in 53% of orders. In a systems analysis of 334 errors causing 264 adverse drug events over 6 months in 2 tertiary care hospitals, 130 errors (39%) resulted from physician ordering, 40 (12%) involved transcription and verification, 38 (11%) reflected problems with pharmacy dispensing, and 126 (38%) were from nursing administration.⁴ In other words, 164 (49%) of the errors in the above-cited study⁴ were in the dispensing and administration stages. In further research, the investigators found that errors resulting in preventable adverse drug events were more than likely to be those in the administration stage (34%) than those in the dispensing stage (4%) of the medication use process.¹

Opportunities for Impact

Because unit-dose dispensing now constitutes a standard for approval by the Joint Commission for Accreditation of Healthcare Organizations (JCAHO)^7,8 and is closely linked to the increasingly common use of automated dispensing devices (see Chapter 11), there is likely little opportunity for further implementation of this practice in US hospitals. In a 1994 survey of pharmacy directors, 92% of acute care hospitals reported using unit-dose dispensing.⁷ Use of unit-dose dispensing is extremely common on general medical and surgical wards, but less so in other locations such as intensive care units, operating rooms, and emergency departments. In these areas, bulk medication stock systems are still found. In a 1999 survey of pharmacy directors, 80% reported that unit-dose dispensing was used for 75% or more of oral doses, and 52% of injectable medications dispensed in their hospitals.⁹

Study Designs

Table 10.1. Studies evaluating the impact of unit-dose (more...)

Table 10.1. Studies evaluating the impact of unit-dose dispensing on medication errors*

Study	Study Design, Outcomes**	Results: Error Rates (95% CI)
Hynniman, 197023	Cross-sectional comparison between study hospital and non-randomly selected "comparison" hospitals (Level 3) Errors of commission and omission (Level 2) among doses ordered	Unit-dose system: 3.5% (3.1-4.0%) Conventional distribution systems at 4 hospitals: 8.3% (7.1-9.7%) 9.9% (8.0-12.2%) 11.4% (9.9-13.2%) 20.6% (18.4-22.9%)
Means, 197513 Simborg, 197514 ***	Cross-sectional comparison of 2 wards within a single hospital over a 60-day period (Level 3) Errors of commission (Level 2) among doses administered during randomly chosen observation periods	Unit-dose ward: 1.6% (1.0-2.5%) Multi-dose ward: 7.4% (6.1-8.9%)§
Schnell, 197624	Prospective before-after study (Level 2) at four Canadian hospitals Errors observed during medication preparation and administration (Level 2)	Before vs. after implementation of unit-dose system: 37.2 vs. 38.5%; 42.9 vs. 23.3%; 20.1% vs. 7.8%; 38.5% vs. 23.1%¶
Dean, 199522	Cross-sectional comparison (Level 3) of US and UK hospitals with different pharmacy distribution systems Errors observed during medication administration (Level 2)	84 errors among 2756 observations in UK hospital using traditional ward stock system: 3.0% (2.4-3.7%) 63 errors among 919 observations in US hospital using unit-doses and automated dispensing: 6.9% (5.2-8.5%) Absolute difference: 3.9% (2.1-5.7%)
Taxis, 199825	Cross-sectional comparison (Level 3) of 2 hospitals in Germany and one hospital in the UK Errors observed during medication administration	UK hospital using traditional ward stock system: 8.0% (6.2-9.8%) German hospital using traditional ward stock system: 5.1% (4.4-5.8) German hospital using unit-dose system: 2.4% (2.0-2.8%) Omission was the most common type of error

* CI indicates confidence interval.

** Errors of commission include administration of wrong dose or wrong or unordered drug, whereas errors of omission include missed doses for inclusion in a patient's unit-dose drawer or a dose not administered.

*** As outlined in the text, the similarities in study setting, time, design and results suggest that these 2 references contain data from the same study; information from these references was therefore combined and treated as a single study.

§ The 95% CIs shown in the table were calculated using the reported data: 20 errors in 1234 observed doses on the unit-dose ward vs. 105 errors in 1428 observed doses on the multidose ward.

¶ When wrong time errors were omitted, the above results changed so that the change to a unit-dose was associated with a significant increase in errors at the first hospital, a non-significant decrease in errors at the second hospital, and significant decreases in errors at the other two hospitals.

• Read¹⁰ conducted a study of a unit-dose system applied to respiratory therapy solutions that was focused primarily on measures of efficiency. Volume/concentration errors in preparing respiratory solutions were also discussed, but the method of detection was not stated nor were specific error data provided for the study period.

• Reitberg et al¹¹ conducted a prospective before-after study with a concurrent comparison floor as a control (Level 2 design) in which medication errors represented the main outcomes (Level 2). During period 1 when both wards used the ward stock system, total errors were 4.7% on the ward that remained on ward stock and 34.2% on the ward that used the modified dispensing system. During period 2, the ward stock system had 5.1% error, while the intervention ward (unit-dose) exhibited a significantly increased 18% error rate. Excluding administration-time errors resulted in error rates of 5.1% (conventional system) and 4.8% (unit-dose), which are not significantly different. The greater number of errors on the modified dispensing ward before and after the intervention were attributed by the authors to factors other than the dispensing systems. Because of this problem, and the fact that this study took place in a skilled nursing facility rather than an acute care hospital, we excluded this study.

• Shultz et al¹² conducted a prospective before-after study (Level 2) involving a somewhat complicated comparison. In the baseline period, approximately 50% of wards used a unit-dose system. During the study period, some wards switched to a unit-dose system in which nurses still administered the medications, while other wards adopted an experimental system, in which pharmacists and pharmacy technicians handled all aspects of the mediation dispensing and administration process. The authors report an error rate of 0.64% for complete unit-dose plus pharmacy technician medication system compared to 5.3% in the more conventional unit-dose system in which nurses continue to administer medications (p<0.001 for this comparison). The authors repeated their observations 2 years later, and show a persistent marked reduction in the "complete unit-dose system" compared to the nurse-administered unit-dose system. Unfortunately, nowhere do the authors report the error rate in the baseline period in the wards without a unit-dose system. Thus, none of the results reported by the authors relate specifically to the conversion from a multi-dose to a unit-dose system.

Lastly, we were unable to obtain one of the original studies of the unit-dose system within the timeline of the project. The data from this study appears to have been published only as a technical report.¹⁵ Other publications related to this study and which we were able to obtain^16-18 did not provide sufficient detail on the aspects of the study relating to medication errors to permit abstraction or inclusion in this chapter. The published reports of a study conducted in a private hospital similarly did not contain sufficient detail about the study methods or results to permit abstraction and inclusion.^19,20

Study Outcomes

Studies reported errors measured by direct observation using a methodology that was first described by Barker.²¹ All of these studies involved Level 2 outcomes.

Evidence for Effectiveness of the Practice

Potential for Harm

Unit-dosing shifts the effort and distraction of medication processing, with its potential for harm, from the nursing ward to central pharmacy. It increases the amount of time nurses have to do other tasks but increases the volume of work within the pharmacy. Like the nursing units, central pharmacies have their own distractions that are often heightened by the unit-dose dispensing process itself, and errors do occur.³

Overall, unit-dose appears to have little potential for harm. The results of most of the Level 2 observational studies seem to indicate that it is safer than other forms of institutional dispensing. However, the definitive study to determine the extent of harm has not yet been conducted.

A major advantage of unit-dose dispensing is that it brings pharmacists into the medication use process at another point to reduce error.³ Yet, as pointed out in the Practice Description above, about half of the hospitals in a national survey bypass pharmacy involvement by using floor stock, borrowing patients' medications, and hiding medication supplies.²

Costs and Implementation

The cost considerations of unit-dose dispensing are mainly a trade-off between pharmacy and nursing personnel. The pharmacy personnel involved are mainly technicians who load unit-dose ward carts for the pharmacists to check and may package some medications that are commercially unavailable in unit-dose form. The pharmacist must verify that the correct medication and dosage of each medication is sent to the ward for the nurse to administer. Nursing time to maintain a drug inventory is reduced, allowing more time for other nursing activities. A variable cost of unit-dose dispensing is the cost of equipment and supplies to those hospitals that wish to do much of the packaging themselves instead of purchasing medications pre-packaged as unit-doses.

Comment

The studies evaluating the practice of unit-dosing contain important methodologic problems and, although yielding somewhat heterogeneous results, are overall relatively consistent in showing a positive impact on error reduction. In contrast to other practices related to medication use, none of the studies evaluated Level 1 outcomes, such as actual adverse drug events. Nonetheless, unit-dose dispensing or some form of automated dispensing of unit-doses (see Chapter 11) has become ubiquitous in American hospitals and a standard of care in the delivery of pharmacy services. Consequently, it is unlikely that more rigorous studies could now be conducted.

Background

Table 11.1. Six studies reviewing automated drug dispensing systems* (more...)

Table 11.1. Six studies reviewing automated drug dispensing systems*

Study	Study Design	Study Outcomes	N	Results
Barker, 19845	Prospective controlled clinical trial (Level 2)	Errors of omission and commission among number of ordered and unauthorized doses. (Level 2)	1775	96 errors among 902 observations (10.6%) using the McLaughlin dispensing system vs. 139 errors among 873 observations (15.9%) using unit-dose dispensing (control)
Klein, 19946	Prospective comparison of two cohorts (Level 2)	Dispensing errors in unit-dose drawers to be delivered to nursing units (Level 2)	7842	34 errors found among 4029 doses (0.84%) filled manually by technicians vs. 25 errors among 3813 doses (0.66%) filled by automated dispensing device
Borel, 19957	Prospective before-after study (Level 2)	Errors observed during medication administrationin medications administered (Level 2)	1802	148 errors among 873 observations (16.9%) before vs. 97 errors among 929 observations (10.4%) after Medstation Rx (p<0.001). Most errors were wrong time errors.
Schwarz, 19958	Prospective before-after study (Level 2)	Errors in medications administered (Level 2)	NA**	Medication errors decreased after automated dispensing on the cardiovascular surgery unit but increased on the cardiovascular intensive care unit.
Dean, 199511	Cross-sectional comparison (Level 3) of US and UK hospitals with different pharmacy distribution systems	Errors in medications administered (Level 2)	3675	63 errors among 919 observations (6.9%, 95% CI: 5.2-8.5%) in the US hospital using unit doses and automated dispensing vs. 84 errors among 2756 observations (3.0%; 95% CI, 2.4-3.7%) in the UK hospital using ward stock. The absolute difference in error rates was 3.9% (95%CI: 2.1-5.7%).

* CI indicates confidence interval.

** Study used various denominator data.

Practice Description

Automated dispensing systems are drug storage devices or cabinets that electronically dispense medications in a controlled fashion and track medication use. Their principal advantage lies in permitting nurses to obtain medications for inpatients at the point of use. Most systems require user identifiers and passwords, and internal electronic devices track nurses accessing the system, track the patients for whom medications are administered, and provide usage data to the hospital's financial office for the patients' bills.

These automated dispensing systems can be stocked by centralized or decentralized pharmacies. Centralized pharmacies prepare and distribute medications from a central location within the hospital. Decentralized pharmacies reside on nursing units or wards, with a single decentralized pharmacy often serving several units or wards. These decentralized pharmacies usually receive their medication stock and supplies from the hospital's central pharmacy.

More advanced systems provide additional information support aimed at enhancing patient safety through integration into other external systems, databases, and the Internet. Some models use machine-readable code for medication dispensing and administration. Three types of automated dispensing devices were analyzed in the studies reviewed here, the McLaughlin dispensing system, the Baxter ATC-212 dispensing system, and the Pyxis Medstation Rx. Their attributes are described below.

• The McLaughlin dispensing system⁵ includes a bedside dispenser, a programmable magnetic card, and a pharmacy computer. It is a locked system that is loaded with the medications prescribed for a patient. At the appropriate dosing time, the bedside dispenser drawer unlocks automatically to allow a dose to be removed and administered. A light above the patient's door illuminates at the appropriate dosing time. Only certain medications fit in the compartmentalized cabinet (such as tablets, capsules, small pre-filled syringes, and ophthalmic drops).

• The Baxter ATC-212 dispensing system⁶ uses a microcomputer to pack unit-dose tablets and capsules for oral administration. It is usually installed at the pharmacy. Medications are stored in calibrated canisters that are designed specifically for each medication. Canisters are assigned a numbered location, which is thought to reduce mix-up errors upon dispensing. When an order is sent to the microcomputer, a tablet is dispensed from a particular canister. The drug is ejected into a strip-packing device where it is labeled and hermetically sealed.

• The Pyxis Medstation, Medstation Rx, and Medstation Rx 1000 are automated dispensing devices kept on the nursing unit.^7-9 These machines are often compared to automatic teller machines (ATMs). The Medstation interfaces with the pharmacy computer. Physicians' orders are entered into the pharmacy computer and then transferred to the Medstation where patient profiles are displayed to the nurse who accesses the medications for verified orders. Each nurse is provided with a password that must be used to access the Medstation. Pharmacists or technicians keep these units loaded with medication. Charges are made automatically for drugs dispensed by the unit. Earlier models had sufficient memory to contain data for about one week, and newer models can store data for longer periods.

Studies reviewed did not include the automated dispensing systems manufactured by Omnicell, which produces point-of-use systems that can be integrated into a hospital's information system.¹⁰ Omnicell systems are also capable of being integrated into external support systems that support machine-readable code, drug information services, and medication error reporting systems.

Prevalence and Severity of the Target Safety Problem

Medication errors within hospitals occur with 2% to 17% of doses ordered for inpatients.⁵,⁷,^11-14 It has been suggested that the rate of inpatient medication errors is one per patient per inpatient day.¹⁵ The specific medication errors targeted by automated dispensing systems are those related to drug dispensing and administration. Even with the use of unit-doses (see Chapter 11) errors still occur at the dispensing¹⁶ and administration stages^3-17 of the medication use process. For instance, in one large study of 530 medical errors in 10,070 written orders for drugs (5.3 errors/100 orders),¹⁸ pharmacy dispensing accounted for 11% of errors and nursing administration 38%.³

Opportunities for Impact

Automated dispensing devices have become increasingly common either to supplement or replace unit-dose distribution systems in an attempt to improve medication availability, increase the efficiency of drug dispensing and billing, and reduce errors. A 1999 national survey of drug dispensing and administration practices indicated that 38% of responding hospitals used automated medication dispensing units and 8.2% used machine-readable coding with dispensing.¹⁹ Three-fourths of respondents stated that their pharmacy was centralized and of these centralized pharmacies, 77% were not automated. Hospitals with automated centralized pharmacies reported that greater than 50% of their inpatient doses were dispensed via centralized automated systems. Half of all responding hospitals used a decentralized medication storage system. One-third of hospitals with automated storage and dispensing systems were linked to the pharmacy computer. Importantly, about half of the surveyed hospitals reported drug distributions that bypassed the pharmacy including floor stock, borrowing patients' medications, and hidden drug supplies.

Study Designs

Table 12.1. Fourteen studies of practices to improve (more...)

Table 12.1. Fourteen studies of practices to improve handwashing compliance*

Study Setting; Practice	Study Design, Outcomes	Handwashing Compliance (unless otherwise noted)**
All medical staff in a neurologic ICU and a surgical ICU in a 350-bed tertiary care teaching hospital in Washington, DC, 1983-84; multifaceted intervention (education, automatic sinks, feedback)16	Level 2, Level 2	69% vs. 59% (p=0.005)
Medical staff in 2 ICUs in a university teach hospital in Philadelphia; increase number of available sinks17	Level 2, Level 2	76% vs. 51% (p<0.01)
Medical staff in a 6-bed post-anesthesia recovery room and a 15-bed neonatal ICU in a tertiary care hospital in Baltimore, 1990; automatic sink compared with standard sink14	Level 2, Level 2	Mean handwashes per hour: 1.69 vs. 1.21 on unit 1;á 2.11 vs. 0.85 on unit 2; (p<0.001)
All staff at a large acute-care teaching hospital in France, 1994-97; hand hygiene campaign including posters, feedback, and introduction of alcohol-based solution18	Level 3, Level 1	Noscomial infections: 16.9% vs. 9.9% Handwashing: 66.2% vs. 47.6% (p<0.001)
Medical staff in a 6-bed pediatric ICU in a large academic medical center in Virginia, 1982-83; mandatory gowning19	Level 3, Level 2	29.6% vs. 30.7%
Medical staff in 2 ICUs in a community teaching hospital in Tennessee, 1983-84; sequential interventions of lectures, buttons, observation, and feedback24	Level 3, Level 2	29.9% vs. 22% (p=0.071)
Medical staff in an 18-bed ICU in a tertiary care hospital in Australia; introduction of chlorhexidine-based antiseptic handrub lotion9	Level 3, Level 2	45% vs. 32% (p<0.001)
12 nurses in a 12-bed ICU in Mississippi, 1990; education/feedback intervention31	Level 3, Level 2	92% vs. 81%
Medical staff in an 18-bed pediatric ICU in a children's teaching hospital in Melbourne, 1994; 5-step behavioral modification program25	Level 3, Level 2	Handwashing rates after patient contact: 64.8% vs. 10.6%
Medical staff in a 3000-bed tertiary care center in France, 1994-95; 13-step handwashing protocol13	Level 3, Level 2	18.6% vs. 4.2% (p<0.0001)
Medical staff in two ICUs at a teaching hospital in Virginia, 1997; 6 education/feedback sessions followed by introduction of alcohol antiseptic agent22	Level 3, Level 2	Baseline 22%; Education/feedback 25%; Alcohol antiseptic 48%; (p<0.05)
Medical staff in a 14-bed ICU in a tertiary careá hospital in France, 1998; introduction of alcohol-based solution21	Level 3, Level 2	60.9% vs. 42.4% (p=0.0001)
All staff in a medical ICU and step-down unit in a large teaching hospital in Virginia; installation of alcohol-based solution23	Level 3, Level 2	52% vs. 60% (p=0.26)
Medical staff on 2 general inpatient floor at each of 4 community hospitals in New Jersey; patient education intervention20	Level 3, Level 3	Soap usage (as an indicator of handwashing) increased by 34% (p=0.021)

*ICU indicates intensive care unit.

**Results are reported as intervention group vs. control group.

Study Outcomes

All studies measured rates of medication errors (Level 2 outcome). Four studies ⁵,⁷,⁸,¹¹ detected errors by direct observation using a methodology that was first described by Barker.⁵ Direct observation methods have been criticized because of purported Hawthorne effect (bias involving changed behavior resulting from measurements requiring direct observation of study subjects). However, proponents of the method state that such effects are short-lived, dissipating within hours of observation.¹⁵ Dean and Barber have recently demonstrated the validity and reliability of direct observational methods to detect medication administration errors.²⁰ Another study, a Level 3 design, determined errors by inspecting dispensed drugs.⁶

Evidence for Effectiveness of the Practice

The evidence provided by the limited number of available, generally poor quality studies does not suggest that automated dispensing devices reduce medication errors. There is also no evidence to suggest that outcomes are improved with the use of these devices. Most of the published studies comparing automated devices with unit-dose dispensing systems report reductions in medication errors of omission and scheduling errors with the former.^7-9 The studies suffer from multiple problems with confounding, as they often compare hospitals or nursing care units that may differ in important respects other than the medication distribution system.

Potential for Harm

Human intervention may prevent these systems from functioning as designed. Pharmacists and nurses can override some of the patient safety features. When the turn around time for order entry into the automated system is prolonged, nurses may override the system thereby defeating its purpose. Furthermore, the automated dispensing systems must be refilled intermittently to replenish exhausted supplies. Errors can occur during the course of refilling these units or medications may shift from one drawer or compartment to another causing medication mix-ups. Either of these situations can slip past the nurse at medication administration.

The results of the study of the McLaughlin dispensing system indicated that though overall errors were reduced compared to unit-dose (10.6% vs. 15.9%), errors decreased for 13 of 20 nurses but increased for the other 7 nurses.⁵ In a study of Medstation Rx vs. unit-dose,⁸ errors decreased in the cardiovascular surgery unit, where errors were recorded by work measurement observations. However, errors increased over 30% in 6 of 7 nurses after automated dispensing was installed in the cardiovascular intensive care unit, where incident reports and medication error reports were both used for ascertaining errors, raising the question of measurement bias. Finally, in a study primarily aimed at determining differences in errors for ward and unit-dose dispensing systems,¹¹ a greater error prevalence was found for medications dispensed using Medstation Rx compared with those dispensed using unit-dose or non-automated floor stock (17.1% vs. 5.4%).

Costs and Implementation

The cost of automated dispensing mainly involves the capital investment of renting or purchasing equipment for dispensing, labeling, and tracking (which often is done by computer). A 1995 study revealed that the cost of Medstation Rx to cover 10 acute care units (330 total beds) and 4 critical care units (48 total beds) in a large referral hospital would be $1.28 million over 5 years. Taking into account costs saved from reduced personnel and decreased drug waste, the units had the potential to save $1 million over 5 years. Most studies that examine economic impact found a trade-off between reductions in medication dispensing time for pharmacy and medication administration time for nursing personnel. A common complaint by nurses is long waiting lines at Pyxis Medstations if there are not enough machines. Nurses must access these machines using a nurse-specific password. This limited access to drugs on nursing units decreases drug waste and pilferage.

Comment

Although the implementation of automated dispensing reduces personnel time for medication administration and improves billing efficiency, reduction in medication errors have not been uniformly realized. Indeed, some studies suggest that errors may increase with some forms of automation. The results of the study of the McLaughlin Dispensing System by Barker et al⁵ showed considerable nurse-to-nurse variability in the error rate between the automated system and conventional unit dose. Qualitative data aimed at determining the reason for this variability would be useful. The study by Klein et al⁶ indicated little difference in the accuracy of medication cart filling by the Baxter ATC-212 (0.65%) versus filling by technicians (0.84%). Borel and Rascati found that medication errors, largely those related to the time of administration, were fewer after implementation of the Pyxis Medstation Rx (10.4%) compared with the historical period (16.9%).⁷ These results are consistent with a more recent study by Shirley, that found a 31% increase in the on-time administration of scheduled doses after installation of the Medstation Rx 1000.⁹ In contrast, errors were greater after Medstation Rx in the study by Schwarz and Brodowy,⁸ increasing on 6 of 7 nursing units by more than 30%. Finally, Dean et al found half the errors in a ward-based system without automation in the United Kingdom (3.0%, 95% CI: 2.4-3.7%) compared with an automated unit-dose medication distribution system in the United States (6.9%, 95% CI: 5.2-8.5%).¹¹

The practical limitations of the systems were illustrated by a variety of process deviations observed by Borel and Rascati.⁷ These included nurses waiting at busy administration times, removal of doses ahead of time to circumvent waiting, and overriding the device when a dose was needed quickly. These procedural failures emphasize an often-raised point with the introduction of new technologies, namely that the latest innovations are not a solution for inadequate or faulty processes or procedures.²

Although automated dispensing systems are increasingly common, it appears they may not be completely beneficial in their current form. Further study is needed to demonstrate the effectiveness of newer systems such as the Omnicell automated dispensing devices. If the standard, namely unit-dose dispensing, is to be improved, such improvements will likely derive from robotics and informatics. To document impact of automated dispensing devices on patient safety, studies are needed comparing unit-dose dispensing with automated dispensing devices. Until the benefits of automated dispensing devices become clearer, the opportunities for impact of these devices is uncertain.

Background

Hospital-acquired infections exact a tremendous toll, resulting in increased morbidity and mortality, and increased health care costs.^1,2 Since most hospital-acquired pathogens are transmitted from patient to patient via the hands of health care workers,³ handwashing is the simplest and most effective, proven method to reduce the incidence of nosocomial infections⁴ Indeed, over 150 years ago, Ignaz Semmelweis demonstrated that infection-related mortality could be reduced when health care personnel washed their hands.⁵ A recent review summarized the 7 studies published between 1977 and 1995 that examined the relationship between hand hygiene and nosocomial infections.⁶

Most of the reports analyzed in this study reveal a temporal relation between improved hand hygiene and reduced infection rates.⁶ Despite this well-established relationship, compliance with handwashing among all types of health care workers remains poor.^7-11 Identifying effective methods to improve the practice of handwashing would greatly enhance the care of patients and result in a significant decrease in hospital-acquired infections.

Practice Description

This chapter focuses on practices that increase compliance with handwashing, rather than the already proven efficacy of handwashing itself.⁴ The term "handwashing" defines several actions designed to decrease hand colonization with transient microbiological flora, achieved either through standard handwashing or hand disinfection.⁴ Standard handwashing refers to the action of washing hands in water with detergent to remove dirt and loose, transient flora. Hand disinfection refers to any action where an antiseptic solution is used to clean the hands (ie, medicated soap or alcohol). Handwashing with bland soap (without disinfectant) is inferior to handwashing with a disinfecting agent.¹² Hygienic hand rub consists of rubbing hands with a small quantity (2-3mL) of a highly effective and fast acting antiseptic agent. Because alcohols have excellent antimicrobial properties and the most rapid action of all antiseptics, they are the preferred agents for hygienic hand rub (also called waterless hand disinfection). Also, alcohols dry very rapidly, allowing for faster hand disinfection.⁴

Prevalence and Severity of the Target Safety Problem

Nosocomial infectionsáoccur in about 7-10% of hospitalized patients¹ and account for approximately 80,000 deaths per year in the United States.¹⁵ Although handwashing has been proven to be the single most effective method to reduce nosocomial infections, compliance with recommended hand hygiene practices is unacceptably low.^7-9 Indeed, a recent review of 11 studies noted that the level of compliance with basic handwashing ranged from 16% to 81%.⁴ Of these 11 studies, only 2 noted compliance levels above 50%.⁴ One reason for poor handwashing compliance may be that the importance of this simple protocol for decreasing infections is routinely underestimated by health care workers.² Recent surveys demonstrate that although most health care workers recognize the importance of handwashing in reducing infections, they routinely overestimate their own compliance with this procedure.¹⁰ A survey of approximately 200 health care workers noted that 89% recognized handwashing as an important means of preventing infection.¹⁰ Furthermore, 64% believed they washed their hands as often as their peers, and only 2% believed that they washed less often than their peers did.¹⁰

Opportunities for Impact

Given these findings, opportunities for improvement in current practice are substantial, and efforts to improve current practice would have vast applicability. Many risk factors for non-compliance with hand hygiene guidelines have been identified, including professional category (eg, physician, nurse, technician), hospital ward, time of day or week, and type and intensity of patient care.⁸ These results suggest that interventions could be particularly targeted to certain groups of health care workers or to particular locations, to increase the likelihood of compliance. Importantly, this study demonstrates that the individuals with the highest need for hand hygiene (ie, those with the greatest workloads) were precisely the same group least likely to wash their hands. Finally, another recent study noted that approximately 75% of health care workers surveyed reported that rewards or punishments would not improve handwashing, but 80% reported that easy access to sinks and availability of hand washing facilities would lead to increased compliance.¹⁰

Study Designs

Study Outcomes

Evidence for Effectiveness of the Practice

Since many different risk factors have been identified for non-compliance with handwashing, it is not surprising that a variety of different interventions have been studied in an effort to improve this practice. While most of the reviewed studies demonstrated significant improvement in handwashing compliance,^{9,13,17,18,20-22} some did not.^14,19,23,24 No single strategy has consistently been shown to sustain improved compliance with handwashing protocols.¹¹ In fact, of the studies which assessed longer-term results following intervention,^16,21,25 all 3 found that compliance rates decreased from those immediately following the intervention, often approaching pre-intervention levels.

Potential for Harm

While no harm is likely to befall a patient as a result of handwashing, one potential adverse effect of handwashing for health care workers is skin irritation. Indeed, skin irritation constitutes an important barrier to appropriate compliance with handwashing guidelines.²⁷ Soaps and detergents can damage the skin when applied on a regular basis. Alcohol-based preparations are less irritating to the skin, and with the addition of emollients, may be tolerated better.⁶

Another potential harm of increasing compliance with handwashing is the amount of time required to do it adequately. Current recommendations for standard handwashing suggest 15-30 seconds of handwashing is necessary for adequate hand hygiene.²⁸ Given the many times during a nursing shift that handwashing should occur, this is a significant time commitment that could potentially impede the performance of other patient care duties. In fact, lack of time is one of the most common reasons cited for failure to wash hands.¹¹ Since alcohol-based handrubs require much less time, it has been suggested that they might resolve this concern. In fact, a recent study which modeled compliance time for handwashing as compared with alcoholic rubs, suggested that, given 100% compliance, handwashing would consume 16 hours of nursing time per standard day shift, while alcohol rub would consume only 3 hours.²⁹

Costs and Implementation

Interventions designed to improve handwashing may require significant financial and human resources. This is true both for multifaceted educational/feedback initiatives, as well as for interventions that require capital investments in equipment such as more sinks, automated sinks, or new types of hand hygiene products. The costs incurred by such interventions must be balanced against the potential gain derived from reduced numbers of nosocomial infections. Only one study addressed the cost implications of handwashing initiatives.²⁰ The implementation of a patient education campaign, when compared to the estimated $5000 per episode cost of each nosocomial infection, would result in an annual savings of approximately $57,600 for a 300-bed hospital with 10,000 admissions annually.²⁰ As others have estimated that the attributable cost of a single nosocomial bloodstream infection is approximately $40,000 per survivor,³⁰ the potential cost savings of interventions to improve handwashing may be even greater.

Comment

While many studies have investigated a variety of interventions designed to improve compliance with handwashing, the results have been mixed. Even when initial improvements in compliance have been promising, long-term continued compliance has been disappointing. Future studies should focus on more clearly identifying risk factors for non-compliance, and designing interventions geared toward sustainability. Some investigators postulate that better understanding of behavior theory, and its application to infection control practices, might result in more effectively designed interventions.²⁶ In addition, any intervention must target reasons for non-compliance at all levels of health care (ie, individual, group, institution) in order to be effective. A more detailed study of the cost (and potentially cost savings) of handwashing initiatives would also foster greater enthusiasm among health care institutions to support such initiatives.