• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of jamiaJAMIA - The Journal of the American Medical Informatics AssociationInstructions for authorsCurrent TOC
J Am Med Inform Assoc. 2006 Sep-Oct; 13(5): 547–556.
PMCID: PMC1561794

Types of Unintended Consequences Related to Computerized Provider Order Entry

Emily M. Campbell, RN, MS, a Dean F. Sittig, PhD, a , b Joan S. Ash, PhD, a , * Kenneth P. Guappone, MD, a and Richard H. Dykstra, MD a , c

Abstract

Objective

To identify types of clinical unintended adverse consequences resulting from computerized provider order entry (CPOE) implementation.

Design

An expert panel provided initial examples of adverse unintended consequences of CPOE. The authors, using qualitative methods, gathered and analyzed additional examples from five successful CPOE sites.

Methods

Using a card sort method, the authors developed a categorization scheme for the 79 unintended consequences initially identified and then iteratively modified the scheme to categorize 245 additional adverse consequences resulting from fieldwork. Because the focus centered on consequences requiring prevention or remedial action, the authors did not further analyze reported unintended beneficial (positive) consequences.

Results

Unintended adverse consequences (UACs) fell into nine major categories (in order of decreasing frequency): 1) more/new work for clinicians; 2) unfavorable workflow issues; 3) never ending system demands; 4) problems related to paper persistence; 5) untoward changes in communication patterns and practices; 6) negative emotions; 7) generation of new kinds of errors; 8) unexpected changes in the power structure; and 9) overdependence on the technology. Clinical decision support features introduced many of these unintended consequences.

Conclusion

Identifying and understanding the types and in some instances the causes of unintended adverse consequences associated with CPOE will enable system developers and implementers to better manage implementation and maintenance of future CPOE projects.

Introduction

Computerized provider order entry (CPOE), narrowly defined, is the process by which physicians or their surrogates (but not intermediaries) directly enter medical orders into a computer application. CPOE systems commonly exist as one of many integrated clinical applications in larger institutions' information systems; the other applications offer complementary functionality such as real-time clinical decision support, on-line clinical documentation, and electronic message transmission. This study refers to CPOE systems as containing, at a minimum, electronic order entry capabilities, whether or not this functionality is part of a larger, more complex information system.

Health care organizations often implement CPOE as part of their approach to improve medication safety and reduce health care costs. 1-3 Yet, several studies indicate that unpredictable, emergent problems, or unintended adverse consequences (UACs) can surround CPOE implementation and maintenance. 4,5 Careful identification, description, and categorization of UACs can provide insight into the unexpected outcomes of placing CPOE systems into complex health care work environments.

The purpose of this study was to identify and describe the major types of UACs related to CPOE implementation. Because CPOE implementations affect many different types of personnel in the health care environment, their evaluation must encompass multiple, discrete perspectives. The current study focuses not on the impact of CPOE on clinical outcomes for patients, but instead on impacts affecting health care personnel who use, maintain, or manage CPOE systems. Specifically, we gathered perspectives regarding CPOE from three groups: clinical end-users, IT staff, and administrators. We broadly define clinical end-users as those health care providers and other clinical staff (e.g., physicians, pharmacists, nurses, ward secretaries, etc.) who work with CPOE systems. IT staff includes those who implement, configure, maintain, and support CPOE systems, whether or not their primary professional background is technical or clinical in nature. Finally, administrative staff refers to those who manage organizational implementation of CPOE, through establishing policies and procedures, assuring compliance with local and federal guidelines, and making high-level CPOE-related resource allocation decisions.

Background

Theoretical Framework

Diffusion of Innovations (DOI) theory served as the framework for this study. Diffusion has been defined by Everett Rogers as “the process by which an innovation is communicated through certain channels over time among the members of a social system,” and an innovation is “an idea, practice, or objective perceived as new by an individual, a group, or an organization.” 6 While adoption of any innovation inevitably generates consequences, Rogers notes that such consequences constitute the least studied aspect of diffusion of innovation. DOI theory suggests that consequences can be desirable or undesirable, and anticipated or unanticipated. In this study, unintended consequences refer to events that are neither anticipated or the specific goals of the associated CPOE project. Although unintended most often connotes consequences that are both unanticipated and undesirable, we have found, in our prior work, numerous unintended desirable positive, beneficial consequences of CPOE as well. 7 While the current study focuses on unintended adverse consequences of CPOE adoption, it is important to remember that unintended consequences are not uniformly errors or mistakes: they are simply surprises that can span a spectrum from lucky to unfortunate. Errors and adverse events comprise a subset of all consequences.

Methods

Selection of Sites

The initial expert panel helped the authors to identify five hospitals in three different organizations where successful CPOE implementation occurred (See [triangle]). To study a range of successful CPOE implementations, the project selected both academic and non-academic hospitals of differing sizes and geographic locations. Each site had used distinct CPOE products (either developed “in house,” or purchased commercially) for varied implementation durations. All had high overall percentages of orders directly entered by clinicians.

Table 1
Table 1. Description of Sites Studied

Data Collection Methods

Institutional Review Board approval for the study was granted by the Oregon Health & Science University and all study sites. Initial expert group discussions focusing on the unintended consequences surrounding the implementation of CPOE led to subsequent fieldwork using participant observation and semi-structured oral history interviews. In April 2004, at the rural Menucha Conference Center in Corbett, OR, the project team and informatics leaders (technical and clinical experts) participated in a two-day conference focused on identifying and understanding unintended adverse consequences of CPOE implementation. The conference discussions were recorded and transcribed. 8

The project team's multidisciplinary informatics researchers (two physicians, a nurse, a pharmacist, a librarian, a public health researcher, and a technically-oriented informaticist) carried out participant observation at the five field sites (CPOE hospitals) by accompanying hospital staff (resident physicians, pharmacists, nurses and allied health care providers) during the course of their daily activities. Researchers unobtrusively observed the hospital staff as they interacted with the CPOE systems. Project members documented the staff's activities and comments; only rarely did researchers interrupt the staff to ask for clarifications. Project members recorded observations in field journals that were later transcribed into formal field notes. Occasionally, circumstances allowed informal interviews of the observed clinicians. If taped, these interviews were subsequently transcribed. The project selected certain clinicians, and hospital and IT administrators, based on their long history and involvement with both the institution and the implementation of the CPOE systems, to undergo debriefing through formal oral history interviews with open-ended questioning. Rather than observing these clinicians interacting with CPOE systems, project members instead asked them to describe the development and adoption of these systems in their respective organizations. Interviews were audiotaped and transcribed by experienced oral-history transcriptionists.

During nine months of field data collection (August, 2004 through April, 2005) project team members spent 390 total hours observing roughly 95 clinical providers interacting with CPOE systems in various settings. The 32 semi-structured interviews totaled approximately 43 hours. Transcripts from these interviews and the Menucha conference, and field notes comprised 1,894 single spaced, typed pages. The project team collected and compiled the field notes and interview transcriptions using qualitative research software (N6, QSR International Pty. Ltd., Melbourne, Australia, 2002).

Data Analysis

The project research team of six individuals met 36 times to analyze data. Using a card sort method, 9 researchers developed a categorization scheme for the 79 unintended adverse consequences identified by the expert panel. Individual team members identified UACs in specifically assigned transcripts. During team meetings, consensus developed regarding which quotes represented UACs and how these UACs could be categorized. Using a grounded theory approach, the categories emerged from the data, rather than from preconceived expectations. 10 Researchers then iteratively modified the initial categorization scheme as they reviewed the 245 additional unintended consequences identified during fieldwork. After several months of analysis, common themes emerged. The team formalized its list of UAC categories. The final list was both simple (consisting of only nine categories) and comprehensive (the categories directly covered all observed UACs).

Results

Introduction

Nine major types of unintended consequences emerged from the data. [triangle] lists the UACs and their frequencies of occurrence. A detailed discussion of each UAC type follows below, including direct quotations from speakers who articulated an issue particularly well. Table 3 (available as a JAMIA on-line supplement at www.jamia.org) includes additional quotes relevant to each type of UAC. Because study subjects were promised confidentiality, researchers edited statements to protect confidentiality whenever original statements potentially identified either speakers or the observation site.

Table 2
Table 2. Unintended Consequences and Their Frequencies of Occurrence

Types of Unintended Adverse Consequences

Type 1: More/New Work for Clinicians

Clinical systems can potentially create new work for all staff members (e.g., both clinical and non-clinical staff). The present UAC focuses on the ever-increasing workload of clinicians. Despite the common CPOE implementation goal of providing a better “patient overview” to the clinician, many CPOE systems make clinicians do more work to get this overview than before CPOE implementation. The CPOE systems may engender new work by requiring that clinicians: (a) enter new information (e.g., justification for a treatment selection) not previously required; (b) respond to excessive alerts that may contain non-helpful information (e.g., non-specific medication interactions with no application to the current patient); or (c) expend extra time in completing non-routine, complex orders (e.g., selecting among differing doses and types of insulin to be administered at different times for a diabetic patient).

Many CPOE systems slow the speed at which clinicians can carry out the clinical documentation and ordering processes. 11 This loss of efficiency often recovers over time. 12 Simply learning to use CPOE takes time and attention away from demanding schedules. If their patient loads are not decreased temporarily during training periods, clinicians work longer hours to complete their combined electronic and clinical work. 13 The indiscriminant, excessive generation of clinical alerts by CPOE systems can also slow clinicians as they pause to decipher alerts, deliberate on whether and how to respond, and potentially document reasons for not complying with alerts.

Administrators and researchers commonly leverage CPOE to collect information not directly related to patient care. The time burden for doing so usually falls on clinicians. One noted: “It seems like every new organizational mandate filters down to the… fingertips…of our primary care physicians in the form of something else that needs to be entered through the computer and the feeling is ‘Well, they have a computer, so it's easy for them to do that' but the cumulative effect [on the physicians] of all those tasks is not fully appreciated.”

When CPOE systems are poorly integrated with other clinical information systems, clinicians find it time-consuming to log in to different systems using different account names and passwords. In some cases, data from one system must be entered manually into another, doubling the work. In addition, built-in functionality such as “cut and paste” may proliferate redundant text in electronic records that clinicians must navigate in order to have a complete picture of the patient. One physician said: “There is no way for me to really know what's new, but I keep seeing chunks of the same text over and over so I have to read every word. Most of it isn't useful.”

Type 2: Unfavorable Workflow Issues

Clinical information systems (CIS) in general, and some CPOE systems in particular, by rigidly modeling work processes according to the “letter of the law” (as set forth in organizational policy and procedure) can dramatically highlight mismatches between intended and actual work processes in real-world clinical settings. These systems can add to ineffective or dysfunctional workflows when CPOE developers' computational models do not reflect actual clinical practices. Such failures can shed light on issues related to clinical role boundaries (due to confusion, misunderstanding, or duplication), and uncover misconceptions among clinical team members regarding what specific work processes actually entail. Furthermore, if CPOE designers have not considered the appropriate range of workflow perspectives (e.g., those of the nursing or clerical staff, as well as of physicians), the resulting technological system cannot accommodate comprehensive, fully integrated clinical workflows. When the system fails to support all the individual role-players who must interact with it, work-shifting to others may occur, leading to resentment and ineffective work activity synchronization. The following quote from a physician illustrates these combined workflow issues: “We found that [the labor and delivery area] was one of the most complex places in the hospital because the patients are going from the screening room to the pod room to a labor room to the delivery room to postpartum, and each of those [is] a different level of care and so orders need to be rewritten…and although nurses are good about blending the orders as necessary…the computer is trying to execute JCAHO rules about changing orders for every level of care [and they aren't] nearly so flexible.”

The clinical ordering process might appear to follow predictable steps: e.g., a clinician places (enters on the computer) an order, the system routes it to the desired destination, the order is processed, and the requested action occurs. However, in actual clinical practice the process is much more adaptable, and includes a variety of checks, balances, interventions, and exceptions. 14 This non-computerized process of placing an order is multi-threaded, and consists of several concurrent and asynchronous steps, each of which may modify, terminate, or intervene in the processing of a given medical order. Many CPOE implementations change or eliminate these multiple interdependent steps, resulting in fewer process reviews and greater potential difficulties.

The project team noted many instances where fewer process reviews led to problems. In one case, x-ray orders were unnecessarily duplicated: “We probably underestimated the gatekeeper function that the clerical staff [provided]…One of the first symptoms is that patients had daily chest x-ray orders in many units, and the clerk had sort of provided a function of questioning after a certain amount of time, based on what he or she knew about the patient, [if] this was still appropriate. Once we automated those daily chest x-ray orders, [they] went on ad infinitum until we came up with an intervention to address that.” In another instance, a double check was eliminated: “The process before wasn't just the clerk writing down the allergies. The process was the clerk writing down the allergy and then the physician reviewing it before anyone took any action on it…that physician review was not kept in the [new CPOE] system.”

One of the most often cited benefits of CPOE is the ability for clinicians to enter orders from anywhere in the hospital, or even from home. However, such new workflows can cause unexpected duplications or contradictions among orders, to the point of endangering patient care. For example, we heard…“We had a lot more instances of within thirty seconds of each other, two, sometimes three providers would enter the same order at approximately the same time [from different locations] and so it really forced us to go back and really do more education on being careful to look and see what's active before you enter a new order.”

Type 3: Never-Ending Demands for System Changes

Never-ending system demands arise regarding hardware and software purchases, implementation tasks, and maintenance issues. They represent UACs from the perspective of both administration and IT staff. Implementation of CPOE requires advanced hardware platforms that can support clinical software. Purchasing or upgrading hardware is not a one-time event, as future technology advances make this an ongoing need. As clinical software systems increase in scope and capabilities, more users require more computer access time, via more computers. Clinicians complain when contention for access to computers interferes with accomplishing clinical tasks, particularly during the busiest hours (e.g., after morning rounds).

Software application demands are also never-ending. One clinical development group moved upgrade releases from a weekly to monthly schedule because “the testing requirements… [were] becoming unbearable.” In another instance, JCAHO recommendations to eliminate the use of common abbreviations meant that “there [were] over 4000 occurrences of the abbreviation ‘QD' in various order entry templates that would have to be manually changed,” and this was only one of a list of about 20 changes. “I can't imagine how much work it is going to take to review all of the screens to find them, and what the incidence of new errors might be during the fix, such as eliminating an element in a pick list by accident, or making a typo in some drug name.” Overhead in maintaining systems and data increases regularly. When CPOE systems allow clinicians to create their own order sets, disparate single-user sets proliferate. It becomes progressively difficult to standardize, update, or maintain these over time. It is also difficult to reconcile old order sets with new institutional initiatives to streamline care processes and to follow the most recent evidence-based clinical practice guidelines.

CPOE adoption transforms some acute problems into subtle and insidious ones. One author calls these the “revenge effects” of technology. 15 During system implementation, the organization necessarily focuses on system go-live activities. But after go-live, ongoing work begins. The system must be tuned, upgraded, tested, interfaced with other systems, and backed up regularly. As the clinical staff increasingly depends on the technology for their daily work, pressure to keep the system operational 24×7×365 increases. Round-the-clock help-desk support becomes necessary. All employees must be trained in system use, and retrained after substantive system changes. Backup systems must operate if the primary system fails. The burden on the technical support staff rarely levels off. Although these consequences can be anticipated, their extent is typically underestimated. One study participant referred to CPOE maintenance as “repairing a jet engine in flight” because the consequences of making mistakes with these systems are “orders of magnitude” greater than for less-integrated, less closely-coupled clinical systems.

As a CPOE system evolves, users rely more completely on the software and demand ever more sophisticated functionality for clinical support. As medical practices evolve, corresponding new features must be added to the original implementation. Over time, complex interactions among the numerous software features can make the installation both unmanageable and outdated, such that the system needs to be replaced with a newer (and “cleaner”) version: “The fact that you develop a critical mass of code, doing anything radically different becomes extremely difficult when you have an installed user base that you are supporting. So a lot of the early rapid flexibility and leeway you had in the early years of implementation you get stuck with … and it isn't easy to sort of wipe things clean and start over.”

Type 4: Problems Related to Paper Persistence

Many CPOE vendors advertise products as helping an organization to “go paperless.” While eliminating the paper-based medical record has clear advantages, including “improved legibility; simultaneous, remote access; and integration with other information sources” 16 one should not confuse this concept with eliminating the use of paper by clinicians in their efforts to take care of patients. Instead, the key issue is to decrease or eliminate the dependency on ineffective, paper-based processes that form barriers to optimal health care delivery, and in this regard, CPOE systems can be particularly effective.

Use of paper was endemic in the five institutions we visited. Use was especially pronounced when paper interfaces substituted for lack of electronic CPOE system integration with other clinical systems (e.g., medication administration recording, pharmacy dispensing, or laboratory ordering). In hospitals where CPOE and ancillary system integration was incomplete, we observed computerized orders being printed out in the processing departments, then re-entered into the local department's clinical information system. We typically saw nurses manually transcribing allergy, blood type, and medication information from the CPOE system to paper-based medication administration records.

We observed providers using paper for temporary, handwritten data storage for later entry into the computer and conversely as a portable, disposable, computer output display medium for quick reference use during their workdays. In hospitals where CPOE systems generated and printed patient summary sheets, we noticed some providers documenting patient progress notes on these printouts. Despite an explicit directive on CPOE printouts not to do so, and counter to current recommendations by the American Hospital Information Management Association, 17 clinicians placed annotated printouts in the patient's chart as formal documentation. These are but a few examples of paper persistence and proliferation in all areas of patient care.

By contrast, paper often serves as a necessary, sometimes superior, cognitive memory aid. As one clinician noted: “I like to have the information on paper where I can hold onto it.“ Furthermore, paper remains the most malleable, flexible, and easily transportable data medium available. Organizations are understandably hard pressed to limit its use. Personnel need only point and click to print ”hard copies“ of stored information. Paper-based clinical record storage will become obsolete, but use of paper in the clinical setting will not. One leader indicated that his institution uses roughly ”…1.6 million pieces of paper per month—printed or copied—and we think half is related to clinical care…we print and destroy 40% of that paper.”

Type 5: Untoward Changes in Communication Patterns and Practices

CPOE systems often dramatically alter traditional communication patterns among care providers, ancillary services, and clinical departments. Installation of CPOE replaces the nexus of previously interpersonal conversations regarding provision of care with a computer system. Some describe CPOE as providing an “illusion of communication” 18 because it promotes the belief that entry of an order into the system ensures that the proper people will see it and act upon it. This unfounded belief is especially problematic for “stat” (emergency) orders, because their execution in a timely fashion should depend on interpersonal communication (even post CPOE implementation), but many CPOE users assume that electronic transmission will be efficient, and do not understand that fast computer or network transmission does not guarantee fast, or accurate, notification of the person who must take care of the order. We observed many instances where emergency orders were not only placed in the CPOE system but were also (redundantly) phoned in to assure they took place immediately.

Doctors, nurses and other providers consistently report that clinical systems like CPOE can cause unsatisfactory reductions in face-to-face communication regarding patient care. The providers further suggest that reduction in communication increases the likelihood of errors due to miscommunication, delayed initiation and execution of orders, and fewer team-wide discussions regarding planning and coordination of care. For example, order entry sessions may precede or remotely follow ICU rounding sessions, when the attending and consulting physicians, the respiratory therapist, and the nurse are together to discuss patient care. Rapid and significant changes in the patient's condition in the time interval between order discussion and order entry may lead to omission or delayed entry of some relevant orders. CPOE systems can exacerbate problems related to the use of verbal orders in conjunction with system entry; some institutions have gone to the extreme of banning verbal orders except in the case of emergencies: “It is not uncommon for a physician to enter an order which has also been verbally stated to the nurse. The nurse, acting on that verbal order, then goes back to, say, mix an IV. The doctor in the meantime changes his mind [about the IV admixture, then places the order] and does not tell the nurse until the bag has been hung, resulting in a waste of a $100 bag of IV fluid.”

Type 6: Negative Emotions

Organizational change is never easy, and shifting clinical practices and workflows can engender enormous emotional resentment in end-users. Sittig and colleagues have suggested that “a specific event or series of events that either cause the person to succeed or fail in reaching his or her goal(s)” triggers many emotions, and that such emotions can affect one's ability to carry out complex physical and cognitive tasks. 19 Shifting from paper-based order generation to CPOE is bound to evoke strong emotional responses as users struggle to adapt to the new technology.

We noted a wide variety of emotional responses to CPOE, including both strongly negative and highly positive emotions. Negative comments predominated. The amount of time a CPOE system had been in use strongly correlated with the level of positive emotions the system elicited. For example, one nurse described her first impression of a CPOE system in this manner: “At first we hated every second of it. I mean we were all like 'I have sick patients here. I'm busy. I don't have time to sit here [at the computer] for twenty minutes.' It was a pain.” Most agree that the high level of negative emotions decreases over time: “It gets better.”

Type 7: Generation of New Kinds of Errors

Studies have indicated that CPOE adoption can generate new kinds of health care–practice related errors, while others have described roles for CPOE in both preventing and causing medication errors.( 4,20–25 ) Here, we focus on new types of errors that emerge when CPOE replaces paper-based ordering.

New CPOE-related errors result from: problematic electronic data presentations; confusing order option presentations and selection methods; inappropriate text entries; misunderstandings related to test, training, and production versions of the system; and workflow process mismatches. System designs (including poor data organization, data omissions, etc.), and end-user confusion about system functionality contribute to new forms of errors. When users make data entry selections from pick lists (drop down lists), a new class of “juxtaposition errors” results from making a wrong selection without realizing it. For example, long, dense pick lists predispose a provider to selecting a patient name adjacent to the intended name. The system should provide adequate feedback on who was selected (e.g., displaying the selected name in large letters on the next screen). If this does not occur, the user may proceed to enter an entire set of orders on the wrong patient. “Backing out” such erroneous orders before they are executed can be problematic. Similar errors occur whenever pick lists facilitate selection of other order parameters.

CPOE systems manage massive amounts of clinical information. However, CPOE workstation screens cannot display large amounts of data simultaneously. Thus, clinicians must learn to navigate serially through CPOE interface screens to perform their work. When busy clinicians cannot readily find the “correct” data entry location, they tend to enter data where it might fit, such as in a “miscellaneous” section. Although such information resides in the system, it may be stored in a manner that makes categorizing, cross-checking, processing, and acting upon it more difficult. Furthermore, improper data placement may impede other clinicians from finding important information: “The biggest problem with orders [is that] people get frustrated finding the right spot to put something, or don't see what they need immediately, then end up entering orders in the miscellaneous section. This makes it easy to miss things and hard to capture data on the orders being entered.”

Poor coordination in deploying test, training, and production versions of CPOE systems can create new kinds of errors. For example, unless safeguards are in place that only allow obviously artificial patient names in “test” and “train” modes, it may be possible for a clinician to use a test system for a patient name that by chance matched an actual patient. The user would not know that the orders entered will not be acted upon, because they were outside the production system. Similarly, without appropriate safeguards, a provider might, during a training session, enter orders for a “test” patient who is actually a “live” patient in the production system. Resulting “test” orders will be processed and have consequences. Finally, problems may emerge when test patients are only identified by a cute, simple name (e.g., “Tom E. Test,” rather than a safer “ZZZTest, ZZZTom”), especially when an actual patient with a last name of “Test” is admitted.

Type 8: Unexpected and Unintended Changes in Institutional Power Structure

As CPOE systems enforce specific clinical practice patterns, while at the same time monitoring clinicians' behaviors, they may induce changes in the power structure and culture of an organization. Power, whether formally or informally delegated, plays an extremely important role in any work environment. 26 This is especially true in health care, where lines of authority emanate from a tradition based on educational hierarchies, various providers' roles, differences between general practitioners and specialists, and more. 27 For these reasons, CPOE systems may subtly bring important power issues to the surface.

CPOE system configurations control who may do what (and when) through the use of clinical, role-based authorizations. While narrowly defined authorizations may lead to much needed role standardizations that reduce unnecessary clinical practice overlaps, the constraints may also redistribute work in unexpected ways, causing frustration. Physicians may resent the need to enter orders directly into a computer, especially when they view this work as a clerical task. Nurses may refuse to take verbal orders except in cases of emergency, or insist that the physician enter orders into the CPOE system before any order will be carried out. We heard many nurses express comments such as: “It really isn't a nursing job to convince the doctor to use the system—and if we must use it, they should too.”

Physicians report loss of professional autonomy when CPOE systems prevent them from ordering the types of tests or medications they prefer, or force them to comply with clinical guidelines they may not embrace, or limit their narrative flexibility through structured rather than free-text clinical documentation. It is irritating when the system directly rewords clinical orders to standardize them: “I didn't realize how important nomenclature was in ordering…Our system has a very robust synonym process where you can use synonyms; we have synonyms for…common misspellings. The doctor puts [an order] in and [it] goes into the patient's chart with the kind of ‘accepted' name. They'll come back later and look at that and say, ‘I didn't order that. It doesn't look like anything I ordered.' In the preceding example, the clinician might object to the forced use of awkward terminology. Such terminology not only disrupts clinicians' workflows, but might also compromise patient care. While order standardization may benefit the organization, it may confuse clinicians. This represents a non-trivial domain in which clinicians' professional autonomy may be circumvented.

Whether the power base is centralized or decentralized plays an important role in occurrence of UACs. Centralized power structures use top-down, hierarchical formats to mandate compliance with organizational rules and to enforce procedure standardization. Decentralized arrangements lead to greater variations in CPOE system configuration and utilization, and increased competition and conflict among departments. Conflicts create significant problems for IT departments, and lead to problems with the application consistency, clinical coordination, and evaluation of impact on patient care.

When many departments participate in CPOE implementation, significant unanticipated power shifts occur. Viewed as the new enforcer of standards, the IT department gains power, even when other departments mandate the standards. This can be frustrating to others: “CPOE should be a clinical project, not an IT project, but it's still amazing how much I think it comes out of the IT department.” Administration and quality assurance departments gain power by requiring users to comply with CPOE-based directives: “The doctor can't place that order unless he fills in that field. They're [administration] really happy and I think that's really a problem for the doctors.”

Type 9: Overdependence on Technology

As CPOE technology diffuses and becomes entrenched within organizations, clinical care delivery becomes inextricably dependent upon it. 28,29 System failures increasingly wreak havoc when paper backup systems are not in place: “It's funny now. When the system goes down, we don't remember how to work with paper.” Prolonged system failures (lasting hours) can so dramatically halt the flow of clinical information that outpatient activities may be curtailed or canceled and emergency rooms at trauma centers may divert admissions until vital systems are restored. The more widely and deeply diffused the technology, the more difficult it becomes to work without it.

Although limited experience exists, embedding clinical decision support (CDS) within CPOE systems may increase clinician-users' access to educational material 30,31 and may affect learning and retention. 32 Clinicians who have only worked with CPOE systems using CDS technologies face new and interesting problems when they transfer to work settings without this technology. Important knowledge gaps might emerge for the clinician who relied on CDS to provide real-time information and/or error prevention. The clinician might have trouble remembering standard dosages, hospital formulary recommendations, and medication contraindications. On the other hand, prior use of CDS may actually promote learning, through repeated and consistent presentation of the sorts of information mentioned in real-time alerts and reminders. Thus, depending on an individual's learning style and the type and amount of available decision support, CDS may actually enrich clinical training.

Finally: “Our society is geared so that if it's in the computer, it must be accurate and complete, and as we know, it just isn't so.” For example, certain free-text fields cannot be processed by CDS components of CPOE. Allergies to medications, entered as atypical abbreviations (e.g., “PCN”, “SMTX”), are a common example. Clinicians assume the computer “knows” the information. This can be especially worrisome for clinically vital information such as allergies.

Decision Support Systems

During analysis of unintended consequences, the researchers learned that clinical decision support generates a disproportionately large number (over 25%) of UACs, spanning all 9 types above. Clinical decision support functionality, viewed as necessary to make CPOE beneficial, often lacks relevance to many specific clinical situations. Accordingly, CPOE-based CDS is not consistently useful for clinicians, is impractical to maintain, and is less than fully reliable in complex situations. 19,24,33–38

Discussion

This study identified types of unintended, negative (adverse) consequences resulting from CPOE implementation. Project team members observed end-users interacting with CPOE, interviewed key players involved in implementations, and analyzed transcripts of meetings designed to elicit UAC-related information. The project identified hundreds of UACs and grouped them into nine categories. The project team came to the realization that the degree of undesirability of each UAC depends to a great extent on one's perspective. We found that the nine types of unintended consequences emerged with surprising regularity at the sites visited. Many UACs originated with attempts to implement clinical decision support.

The project team learned significant lessons. First, UACs occur during all CPOE implementations, though not all institutions experience all of the types of UACs. Second, the nine types of UACs occur in a widespread manner. The UACs pose significant consequences for clinicians, technical staff, and organizations as a whole. Finally, the project's typology establishes a framework for systematic approaches to address these issues.

  • More work/new work: CPOE systems can significantly increase clinician workload, and improved system design may not reduce the amount of new work such systems require. Great care must be taken to balance the risks of over-alerting with not alerting. Developers should rework clinical system interfaces to: (a) reduce collection of redundant information; (b) display relevant information in logical locations; and (c) reduce the amount of required typing. The lesson is that more work for the clinician is inevitable, and must be addressed in the planning process. Successful implementations balance required new work with system-based reductions in old work to make use of the systems by clinicians tolerable.
  • Workflow: Clinical workflows are complex, and clinical computer technology integration significantly impacts health care workflows.12,38–52 Modeling clinical workflows is difficult because clinical practice is so inherently complex, interruption-driven, and constantly changing. No CPOE system fits all workflows of a given hospital perfectly. Even if a system initially did so, it would not eliminate the need for constant system adaptation to changing workflows in the future. Whenever there are adjustments, there will be unintended consequences.
  • Never-ending system demands: CPOE systems evolve (i.e., are reconfigured, enhanced, or replaced) over time, making hardware and software upgrades a necessity. With each change, implementers should expect unintended consequences. As changes occur, users must be retrained and quality assurance measures must be reassessed. The lesson is that planning must allocate adequate resources for ongoing improvements.
  • Paper persistence: While electronic medical record systems trend toward “going paperless,” health care organizations, as a whole, do not. Vendors and administrators alike must understand differences between having a paperless record system and a paperless office. Organizations must delineate what constitutes legal documentation in the presence of an electronic medical record.53 A likely reduction in paper use will follow full integration of disparate clinical information systems and widespread deployment of clinical computing workstations or wireless, handheld devices. Paper is here to stay for utilitarian, as opposed to permanent record-keeping purposes, and attempts to limit its pragmatic use in health care are often misguided.
  • Changes in communication patterns and practices: Computerized systems are unlikely to replicate the richness of face-to-face communication, but computer-based communication systems must improve. Improvements in system interface design must pay special attention to the communication needs of health care providers. The lesson is that CPOE implementation changes clinical communication patterns. In addition, a comprehensive communication plan that reaches all levels of the organization must be part of any CPOE project management plan.
  • Emotions: Emotional responses to change are inevitable. These responses can point out significant problems with the system design, and can lead to solutions. Training and open communication can help to promote better understanding, which may reduce the negative emotional responses to CPOE.
  • New kinds of errors: CPOE systems prevent some types of errors while creating or propagating new ones.4,5,21,22,24,25,33,54–61 Many new errors result from straightforward system interface design problems, such as dense pick lists that cause juxtaposition problems. Recognizing current unintended consequences should encourage system designers to optimize human computer interface design, and to exert caution when implementing new alerts.
  • Changes in the power structure: Because CPOE-related power changes affect organizational and personal autonomy, they often cause significant UACs for end-users. Most often it is the physician who loses power: this must be recognized and dealt with explicitly during the CPOE planning process.
  • Overdependence on technology: Health care is increasingly dependent on technology and this is unlikely to change. Dependence on technology must never become so great that basic medical care cannot be provided in its absence. Planning for management of unexpected downtime is critical.

Decision Support Systems and CPOE

The project identified decision-support-based clinical alerts as a common source of UACs. There are optimal and less optimal approaches to designing and implementing decision support. The suboptimal approaches to CDS can have widespread negative impacts on clinical practices. Poorly designed alerts constantly interrupt providers, often with trivial, redundant, or already known information. Alerts present a major workflow process issue, adding to the steps required to enter an order. Clinicians report inappropriate alerts as highly frustrating nuisances. Alerts can beneficially and adversely change the power structure, and present challenges to professional expertise or autonomy. Because alerts appear to be from “the system,” they may be viewed as correct when they are erroneous. Conversely, when appropriate alerts are ignored, error prevention may not occur, and redundant or unnecessary test avoidance diminishes. Building and maintaining an appropriate set of CPOE alerts based on an up-to-date evidence base is an onerous, never-ending task.

Looking to the Future

Improvements are both warranted and attainable with respect to unintended consequences. Currently, CPOE technology is immature and rapidly evolving. The field must prioritize ongoing efforts to better understand the nature of clinical work. Designers must build systems that respect health care providers' burdens, and constant feedback from providers should guide system implementation and evolution. Workflow improvements require experience and time. As the adoption of CPOE systems increases, so will the wealth of knowledge about how to use the systems to improve care. Improved interface designs may eliminate or reduce the possibility of juxtapositions and other related errors. As technology evolves, dramatic improvements in providing clinicians with pertinent data will occur. Such technological advances first and foremost require clinical involvement to better support clinical work. Only through a careful combination of CPOE-related research, design, feedback, and understanding will many of the unintended consequences be reduced or eliminated.

Conclusion

It is important to view UACs as the result of a constellation of factors. Their causative agents almost never occur in isolation. UACs emerge in all aspects of health care, whether or not the task from which they emerge is technological. Not all types of UACs may occur with every CPOE implementation. The goal is to discover and understand causative factors leading to UACs to allow future CPOE developers and implementers to mediate or eliminate UACs that are preventable or remediable. Doing so will reduce the negative impacts CPOE systems have on providers, patients, and administrators. It is only through careful evaluation of UACs that we can gain better insight into how to best approach these problems.

Footnotes

This research was funded by research grant LM06942 and training grant ASMM10031 from the U.S. National Library of Medicine, National Institutes of Health.

The authors thank all the individuals who allowed us to observe or interview them and the experts who participated in the Menucha Conference. Special thanks go to the site principal investigators J. Marc Overhage, MD, PhD, Eric G. Poon, MD, MPH, and Rainu Kaushal, MD. We also thank Cody Curtis, MBA, the reviewers, and the JAMIA Editorial Board for assistance with this manuscript.

References

1. Committee on Quality Health Care in America To err is human. Building a safer health system. Washington, DC: Institute of Medicine; 1999.
2. Committee on Quality Health Care in America Using information technology. Crossing the quality chasm: A new health system for the 21st century. Washington, DC: Institute of Medicine; 2001.
3. The Leapfrog Group Factsheet. Computer physician order entry. 2004. Available at: http://www.leapfroggroup.org/for_hospitals/leapfrog_safety_practices/cpoe. Accessed: Sep 16, 2005.
4. Koppel R, Metlay JP, Cohen A, et al. Role of computerized physician order entry systems in facilitating medication errors J Am Med Inform Assoc 2005;293(10):1197-1203. [PubMed]
5. Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health carethe nature of patient care information system-related errors. J Am Med Inform Assoc 2004;11(2):104-112. [PMC free article] [PubMed]
6. Rogers EM. Diffusion of Innovations. 5th ed. New York: Free Press; 1998.
7. Ash J, Sittig D, Dykstra R, Guappone K, Carpenter J, Seshadri V. Categorizing the unintended sociotechnical consequences of computerized physician order entry Int J Med Inform 2006. in press. [PubMed]
8. Ash JS, Stavri PZ, Kuperman GJ. A consensus statement on considerations for a successful CPOE implementation J Am Med Inform Assoc 2003;10(3):229-234. [PMC free article] [PubMed]
9. Lincoln Y, Guba E. Naturalistic inquiry. Newbury Park, CA: Sage; 1985. pp. 346-351.
10. Crabtree BF, Miller, William L. Doing qualitative research. 2nd ed. Thousand Oaks, CA: Sage Publications, Inc; 1999.
11. Poissant L, Pereira J, Tamblyn R, Kawasumi Y. The impact of electronic health records on time efficiency of physicians and nursesa systematic review. J Am Med Inform Assoc 2005;12(5):505-516. [PMC free article] [PubMed]
12. Overhage JM, Perkins S, Tierney WM, McDonald CJ. Controlled trial of direct physician order entryeffects on physicians' time utilization in ambulatory primary care internal medicine practices. J Am Med Inform Assoc 2001;8(4):361-371. [PMC free article] [PubMed]
13. Scott JT, Rundall TG, Vogt TM, Hsu J. Kaiser Permanente's experience of implementing an electronic medical recorda qualitative study. Br Med J 2005;331(7528):1313-1316. [PMC free article] [PubMed]
14. Hazlehurst B, McMullen C, Gorman PN, Sittig DF. How the ICU follows ordersCare delivery as a complex activity system. AMIA Ann Symp 2003:284-288. [PMC free article] [PubMed]
15. Tenner E. Why things bite back. Technology and the revenge of unintended consequences. New York: Random House, Inc; 1996.
16. Powsner SM, Wyatt JC, Wright P. Opportunities for and challenges of computerisation Lancet 1998;352(9140):1617-1622. [PubMed]
17. AHIMA e-HIM Work Group on Health Information in a Hybrid Environment. The Complete Medical Record in a Hybrid EHR Environment. Part III. Authorship of and Printing the Health Record (AHIMA Practice Brief). Available at: http://library.ahima.org/xpedio/groups/public/documents/ahima/bok1_021583.hcsp?dDocName=bok1_021583. Accessed Apr 29, 2006..
18. Dykstra R. Computerized physician order entry and communicationreciprocal impacts. Proc AMIA Ann Symp 2002:230-234. [PMC free article] [PubMed]
19. Sittig DF, Krall M, Kaalaas-Sittig J, Ash JS. Emotional aspects of computer-based provider order entryA qualitative study. J Am Med Inform Assoc 2005;12(5):561-567. [PMC free article] [PubMed]
20. Bates DW, Kuperman GJ, Rittenberg E, et al. A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests Am J Med 1999;106(2):144-150. [PubMed]
21. Bates DW, Teich JM, Lee J, et al. The impact of computerized physician order entry on medication error prevention J Am Med Inform Assoc 1999;6(4):313-321. [PMC free article] [PubMed]
22. Bobb A, Gleason K, Husch M, Feinglass J, Yarnold PR, Noskin GA. The epidemiology of prescribing errorsthe potential impact of computerized prescriber order entry. Arch Intern Med 2004;164(7):785-792. [PubMed]
23. Horsky J, Kuperman GJ, Patel VL. Comprehensive analysis of a medication dosing error related to CPOE J Am Med Inform Assoc 2005;12(4):377-382. [PMC free article] [PubMed]
24. Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safetya systematic review. [see comment] Arch Intern Med 2003;163(12):1409-1416. [PubMed]
25. Oren E, Shaffer ER, Guglielmo BJ. Impact of emerging technologies on medication errors and adverse drug events Am J Health-System Pharm 2003;60(14):1447-1458. [PubMed]
26. Robbins SP. Organizational behavior. 11th ed. Upper Saddle River, NJ: Pearson Prentice Hall; 2005.
27. Massaro TA. Introducing physician order entry at a major academic medical center: 1. Impact on organizational culture and behavior Academ Med 1993;68(1):20-25. [PubMed]
28. Kilbridge P. Computer crash—lessons from a system failure N Eng J Med 2003;348(10):881-882. [PubMed]
29. Oppenheim MI, Vidal C, Velasco FT, et al. Impact of a computerized alert during physician order entry on medication dosing in paitents with renal impairment AMIA Ann Symp 2002:577-581. [PMC free article] [PubMed]
30. Rosenbloom S, Geissbuhler A, Dupont W, et al. Effect of CPOE user interface design on user-initiated access to educational and patient information during clinical care J Am Med Inform Assoc 2005;12(4):458-473. [PMC free article] [PubMed]
31. Bochicchio GV, Smit PA, Moore R, et al. Pilot Study of a Web-Based Antibiotic Decision Management Guide J Am Coll Surg 2006;202(3):459-467. [PubMed]
32. Knight A, Kravet S, Harper G, Leff B. The effect of computerized provider order entry on medical student clerkship experiences J Am Med Inform Assoc 2005;12(5):554-560. [PMC free article] [PubMed]
33. Galanter WL, Polikaitis A, DiDomenico RJ. A trial of automated safety alerts for inpatient digoxin use with computerized physician order entry J Am Med Inform Assoc 2004;11(4):270-277. [PMC free article] [PubMed]
34. Handler JA, Feied CF, Coonan K, et al. Computerized physician order entry and online decision support Acad Emerg Med 2004;11(11):1135-1141. [PubMed]
35. Hsieh TC, Kuperman GJ, Jaggi T, et al. Characteristics and consequences of drug allergy alert overrides in a computerized physician order entry system J Am Med Inform Assoc 2004;11(6):482-491. [PMC free article] [PubMed]
36. Kawamoto K, Lobach DF. Clinical decision support provided within physician order entry systemsa systematic review of features effective for changing clinician behavior. Proc AMIA Ann Symp 2003:361-365. [PMC free article] [PubMed]
37. Neilson EG, Johnson KB, Rosenbloom ST, et al. The impact of peer management on test-ordering behavior Ann Intern Med 2004;141(3):196-204. [PubMed]
38. Sengstack PP, Gugerty B. CPOE systemssuccess factors and implementation issues. J Healthcare Inform Manag 2004;18(1):36-45. [PubMed]
39. Berg M. Medical work and the computer-based patient recorda sociological perspective. Methods Inform Med 1998;37(3):294-301. [PubMed]
40. Berg M. Patient care information systems and health care worka sociotechnical approach. Int J Med Inform 1999;55(2):87-101. [PubMed]
41. Beuscart-Zephir MC, Pelayo S, Degoulet P, Anceaux F, Guerlinger S, Meaux JJ. A usability study of CPOE's medication administration functionsimpact on physician-nurse cooperation. Medinfo 2004;11(Pt 2):1018-1022. [PubMed]
42. Briggs B. The top 10 CPOE challenges Health Data Management 2004;12(7):20-22. [PubMed]
43. Cheng CH, Goldstein MK, Geller E, Levitt RE. The effects of CPOE on ICU workflowan observational study. Proc AMIA Ann Symp 2003:150-154. [PMC free article] [PubMed]
44. Cordero L, Kuehn L, Kumar RR, Mekhjian HS. Impact of computerized physician order entry on clinical practice in a newborn intensive care unit J Perinatol 2004;24(2):88-93. [PubMed]
45. Dourish P, Bellotti V. Awareness and coordination in shared work spaces ACM Conference on Computer-Supported Cooperative Work (CWSW '92); 1992. Toronto, Canada: ACM Press; 1992. pp. 107-114.
46. Eisenberg F, Barbell AS. Computerized physician order entryeight steps to optimize physician workflow. J Healthcare Inform Manag 2002;16(1):16-18. [PubMed]
47. Gaillour FR. Why do health systems flop with CPOE? Ask Yoda Physic Exec 2004;30(2):28-29. [PubMed]
48. Levick D, Lukens HF, Stillman PL. You've led the horse to water, now how do you get him to drinkmanaging change and increasing utilization of computerized provider order entry. J Healthcare Inform Manag 2005;19(1):70-75. [PubMed]
49. Poon EG, Blumenthal D, Jaggi T, Honour MM, Bates DW, Kaushal R. Overcoming barriers to adopting and implementing computerized physician order entry systems in U.S. hospitals Health Affairs 2004;23(4):184-190. [PubMed]
50. Ali NA, Mekhjian HS, Kuehn PL, et al. Specificity of computerized physician order entry has a significant effect on the efficiency of workflow for critically ill patients Crit Care Med 2005;33(1):110-114. [PubMed]
51. Shane R. CPOEthe science and the art. Am J Health-System Pharm 2003;60(12):1273-1276. [PubMed]
52. Shu K, Boyle D, Spurr C, et al. Comparison of time spent writing orders on paper with computerized physician order entry Medinfo 2001;10(Pt 2):1207-1211. [PubMed]
53. AHIMA e-HIM Work Group on the Legal Health Record UpdateGuidelines for Defining the Legal Health Record for Disclosure Purposes. J AHIMA 2005;76(8):64A-64G. [PubMed]
54. Berger RG, Kichak JP. Computerized physician order entryhelpful or harmful?. J Am Med Inform Assoc 2004;11(2):100-103. [PMC free article] [PubMed]
55. Hagland M. Safe ways. Hospitals looking to improve patient safety are turning to CPOE, bar coding and e-prescribing Healthcare Inform 2004;21(8):20-25. [PubMed]
56. King WJ, Paice N, Rangrej J, Forestell GJ, Swartz R. The effect of computerized physician order entry on medication errors and adverse drug events in pediatric inpatients Pediatrics 2003;112(3 Pt 1):506-509. [PubMed]
57. Kremsdorf R. CPOEnot the first step toward patient safety. Health Manag Technol 2005;26(1):66. [PubMed]
58. Nebeker JR, Hoffman JM, Weir CR, Bennett CL, Hurdle JF. High rates of adverse drug events in a highly computerized hospital Arch Intern Med 2005;165(10):1111-1116. [PubMed]
59. Potts AL, Barr FE, Gregory DF, Wright L, Patel NR. Computerized physician order entry and medication errors in a pediatric critical care unit Pediatrics 2004;113(1 Pt 1):59-63. [PubMed]
60. Scanlon M. Computer physician order entry and the real worldwe're only humans. Joint Comm J Quality Safety 2004;30(6):342-346. [PubMed]
61. Wears RL, Berg M. Computer technology and clinical workstill waiting for Godot. JAMA 2005;293(10):1261-1263. [PubMed]

Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of American Medical Informatics Association
PubReader format: click here to try

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

  • Cited in Books
    Cited in Books
    PubMed Central articles cited in books
  • MedGen
    MedGen
    Related information in MedGen
  • PubMed
    PubMed
    PubMed citations for these articles

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...