• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of bumcprocBaylor University Medical Center ProceedingsAbout the JournalBaylor Health Care SystemSubmit a Manuscript
Proc (Bayl Univ Med Cent). Jul 2004; 17(3): 357–361.
PMCID: PMC1200672

A baseline study of medication error rates at Baylor University Medical Center in preparation for implementation of a computerized physician order entry system

Christina E. Seeley, MPH, MT(ASCP),corresponding author1 David Nicewander, MS,2 Robert Page, MPA,1 and Peter A. Dysert, II, MD1,3

Abstract

Objective: To determine baseline levels of medication errors and their root causes so as to highlight areas of potential process improvements and serve as a ruler against which to measure future improvements.

Design: A prospective pharmacist intervention study determining errors in 1014 medication orders at Baylor University Medical Center. Only errors in the process of medication ordering were documented; errors in drug administration were not considered. Root causes of errors were examined.

Results: The baseline medication error rate was 111.4 per 1000 orders (n = 1014). Most common were dosing errors (43.4 per 1000 orders), followed by frequency errors (19.7 per 1000 orders) and unavailable drug errors (12.8 per 1000 orders). Of the 113 total errors found, 52 (46%) had a transcription-based cause, i.e., an error in inputting the handwritten physician order into a computer system. System- or process-related root causes (such as duplicate orders or lack of crossover from one information system to another) accounted for 35.4% of the errors, and prescribing based causes (such as wrong dosage or nonformulary drugs) accounted for 18.6% of errors.

Conclusions: Implementing a computerized physician order entry (CPOE) system would eliminate order-entry transcription-based errors. Its ability to resolve system/process-based and prescribing-based root causes of error is not as clear. Furthermore, the modification of processes due to implementation of CPOE could lead to new types of errors. Present processes must be redesigned according to evidence-based medicine, and future processes must be anticipated as technological changes occur. Such efforts—rather than outright reliance on technology—are more likely to lead to an error-free environment after CPOE is implemented.

As Baylor Health Care System embarks on a 7-year odyssey of clinical transformation that will include implementation of a computerized physician order entry (CPOE) system, leaders needed baseline data on medication errors and their root causes. Such data could highlight areas of potential process improvements and serve as a ruler against which to measure future improvements.

Since 1991, many studies have attempted to quantify iatrogenie errors, including medication errors, in the USA (16). All studies have found “a substantial amount of injury to patients from medical management” (1) and have agreed that “ADRs [adverse drug reactions] represent an important clinical issue” (2). In one study, adverse drug events made up 19.4% of the disabling adverse events, and 45% of the adverse drug events were due to medication errors (3).

A number of studies have specifically addressed the frequency of medication errors (79), and these frequencies have varied. The heterogeneity of the studies' definitions, indicators, and techniques cannot be underestimated. The diversity of methodologies, the differences of opinion as to what should be counted and what is not worthy of being counted, and the use of a variety of indicators as a determinant of the presence of error (from the administration of so-called antidote drugs to the use of pharmacist interventions) have had an impact on the numbers reported in the studies, and the importance attached to these numbers is affected by the reader's perspective. However, what cannot be ignored is that medication errors are occurring during so-called “time-honored manual systemsrdquo; processes (7) and that CPOE is being considered as a means of reducing medication errors. It would be foolhardy to believe that replacing one process with another will solve a problem without fully understanding the causes of the problem.

At Baylor Health Care System, earlier studies attempted to assess the level of medication errors; these studies focused either on voluntarily reported errors or on retrospective chart reviews documenting adverse drug events discovered after the fact (8). In embarking on this baseline study, the research team made several fundamental decisions about design:

  • Medication errors were defined as “errors in the process of ordering, dispensing, or administering a medication, regardless of whether an injury occurred or whether the potential for injury was present” (4).
  • A prospective observational study of pharmacists at the time of medication order verification was chosen. While a prospective observational study at the point of care would have been ideal, it was cost-prohibitive. The research group decided against using a voluntary reporting system or a retrospective chart review for the baseline study. In voluntary systems, fears of recrimination or heavy workloads might prevent the reporting of information (10), whereas retrospective chart reviews about adverse events have had poor interrater reliability (11).
  • The study considered 2 key process junctures at which medication errors occur: prescribing by the physician (example of error: ordering a drug that is contraindicated or miscalculating dosage or frequency) and transcription and order entry (example of error: “Q/D” being entered into the system as “QID”). Because the evaluation took place at the pharmacy rather than at the point of care, 2 other processes were not analyzed: dispensing of medication (example of error: patient getting wrong drug) and administration of medication (example of error: a missed or duplicate dose). The Figure summarizes how these 4 key processes are implemented beginning at one of Baylor University Medical Center's 57 remote nursing stations.
    Figure
    Medication order process at Baylor University Medical Center. PIS indicates pharmacy information system;HIS, hospital information system.

Baylor plans to compare the results of this baseline study to results of a follow-up study after implementation of a CPOE system.

METHODS

A sample size estimate was calculated with the following considerations:

  1. The baseline rate for medication errors was assumed to be 30 per 1000 orders.
  2. The intervention would reduce the medication error rate by 67% to 10 per 1000 orders.
  3. Type I error rate = 0.05, type II error rate = 0.20 (power = 0.80).

With these assumptions, the sample size estimate was calculated to be 800. To make the estimates more precise, the research team decided to collect 1000 orders. Data collection took place between March 10 and April 14, 2003. To minimize bias, data collection times (between 7:00 AM and 9:00 PM) and verification stations (5 possible stations) were randomly assigned to the day of collection. Data were collected 7 days a week until approximately 1000 cases were in the data file. Microsoft Access was used to store the data.

Each medication order verification by the pharmacist was observed and documented. If the order required the pharmacist's intervention, the pharmacist brought it to the attention of the data collector. Two broad questions were raised:

  1. Was there a medication error and, if so, what type of error was it?
  2. What type of “rework” occurred on the part of the pharmacist (to help determine the root cause)?

For the purposes of this study, only errors in the process of medication ordering were measured. The pharmacist who verified the medication order was the sole determinant of whether such an error had occurred. Interventions to accommodate regulatory requirements, billing issues, and change of medication administration requests were not included in the error counts. Table Table11 defines the medication error categories used, as proposed in the work of Bates et al(4).

Table 1
Definitions of medication errors*

Determinations of root cause of error were retrospective and based on free-text descriptors written at the time of observation that described the type of error and the steps the pharmacist took to correct it. The research team divided root causes into 3 broad groups. Prescribing-based root causes are those associated with physician-prescribing mistakes, oversights, substitutions, and omissions and include both obvious errors such as entering a wrong dosage and less obvious errors such as prescribing nonformulary drugs. Transcription-based root causes can be traced back to the order entry action of the unit assistant or nurse. Such errors are detected when the pharmacist notes a discrepancy between the physician's written order and the order placed into the hospital information system. System/process-based causes cover a variety of issues and frequently are related to communication failures, as when patient information did not cross over between systems for verification.

RESULTS

The baseline medication error rate was 111.4 per 1000 orders (n=1014). The most common errors were dosing errors (43.4 per 1000 orders), followed by frequency errors (19.7 per 1000 orders) and unavailable drug errors (12.8 per 1000 orders) (Table (Table22).

Table 2
Medication error rates and their root causes in a study of 1014 medication orders at Baylor University Medical Center

When errors were grouped by root cause, the most common cause was transcription-based (46% of interventions) (Table (Table2).2). Transcription causes were more likely to result in dose errors, frequency errors, and route errors than other types of errors. In this study, transcription-based causes did not result in unavailable drug errors.

The second most common cause of pharmacist intervention was system/process-based root causes (35.4% of interventions). Again, dose errors were the most frequently occurring error, followed by avoidable delay errors. Unavailable drug errors, route, and inappropriate drug errors tied for third. System/process-based errors were quite diverse. Some mistakes occurred because of a variation in the customary process (e.g., a telephoned medication order). Others occurred when interfaces between the hospital information system and pharmacy information system failed to deliver the same message or when limitations in one of the information systems prevented the successful transmission of the medication order. Sometimes, errors occurred when procedures were changed and the ordering physician was unaware of the change.

The least common cause of pharmacist intervention was prescribing-based root causes (18.6% of interventions). The most common errors for this category were unavailable drug errors. Dose errors and frequency errors occurred less frequently.

DISCUSSION

The medication error rate determined by this study was 111.4 per 1000 orders. Because of differences in processes and levels of technology at different centers, it is difficult, if not impossible, to find studies that are identical or even similar to ours and thus validate our error rate.

During the baseline study of Bates et al, of 10,070 medication orders, there were 242 non—missed-dose medication errors, an error rate of 24.0 per 1000 orders (4). However, that study differed in several ways from our study. We used the same definition for medication errors as Bates et al. Our study, however, focused specifically on the ordering process and did not consider other sources of medication errors such as dispensing and administering medication. Because Bates et al included these other sources, one would expect their numbers to be higher. However, the majority of medication errors in their baseline study were discovered after reviewing medication sheets, thus focusing more specifically on prescribing-based causes of errors. Bates et al conceded that their study “differentially detected ordering errorsrdquo; (4), and because at the time of their study there was no interface between a hospital information system and a pharmacy information system and all orders were handwritten in order books, there was no such thing as a transcription-based cause for a medication error, as described in our study, nor any discussion about system/processbased root causes. In our study, 20.7 errors per 1000 orders were due to prescribing-based causes, a figure that is almost the same as that of Bates et al. It is noteworthy that the introduction of technology (an interface between a hospital information system and pharmacy information system) introduces new sources of errors (transcription- and system/process-based errors), even as it makes a paper-based process more efficient.

In another study based on clinical pharmacy interventions in an Australian teaching hospital (12), the percentage of errors related to improper dosing was 54%; in our study the percentage was 39%. In the Australian study, however, only interventions “considered to be of major significance” (12) were submitted for analysis; interventions thought to represent “minor to moderate significance” were excluded. Our study was more inclusive and did not make a distinction between potential major and minor outcomes.

Our study design presented some limitations. Because all medication errors were determined as a result of pharmacist intervention at the time of verification of the medication order, any personal biases the pharmacist might have had as to what constituted an intervention and/or medication error would be reflected in our reporting of the data. Similarly, root cause determinations were dependent upon the data collector's interpretation of the pharmacists' interventions and may have been affected by personal biases and knowledge-based limitations. No assessment of reliability was made. Interventions that may have occurred at times other than the verification of the medication order were not included. Examples of such interventions included pharmacist-physician consults on the patient floors and by telephone. Because this study was conducted entirely in the pharmacy and because many of Baylor's intensive care units have verifying stations on site, there was an underrepresentation of medication orders generated from the intensive care units. Furthermore, at the time of this study, a pharmacy pilot study that involved order entry by pharmacists dedicated to specific patient care units was in place, and these orders were also not considered. Finally, this evaluation was performed at only one medical center, so results might be different elsewhere.

Implications for the CPOE system

These baseline data will be compared with data after implementation of the CPOE system at Baylor. It is clear from the literature that Baylor cannot expect pharmacist interventions to cease after the new system is used. Senholzi et al analyzed a control and study patient care unit for a 6-week period before and after CPOE implementation and again after allowing time for physicians to become familiar with the new system (13). They showed that despite a reduction in incomplete orders, wrong dosage/route/frequency, and therapeutic interchange orders, the number of pharmacist interventions increased significantly after the CPOE was implemented (13). The authors explained that the system created inconsistent orders and/or duplicate orders and did not prevent physicians from ordering the wrong drug.

In an accompanying editorial, pharmacy directors from Tufts, Cedars'Sinai, and the University of Illinois medical centers enumerated 11 specific “realities and challenges” to be addressed in CPOE implementations (7), including the fact that CPOE systems “take more time and are far more complex than the time-honored manual systems that they replace” and that in comparison with pharmacists who have many years of experience with medication order entry, “order entry by physicians and other prescribers is a new skill and will be associated with a significant learning curve, which may result in new sources of medication errors” (7). Their observations that “CPOE systems are not ‘plug and play’ and require a systematic approach to process redesign” echo many of the sentiments strongly held by our team.

A CPOE system will have different effects on different types of errors. The system would immediately eliminate all order-entry errors that occurred as a result of transcription-based root causes, but its ability to resolve errors occurring as a result of system/process-based root causes and prescribing-based root causes is not as clear.

For example, the most common type of prescribing-based medication error in our study was “unavailable drug,rdquo; which generally occurred when the physician ordered a nonformulary drug. The end result was a delay in the patient's therapy. Maintaining a formulary and communicating accepted prescribing practices to physicians are dynamic processes. The issue at hand is not whether a data entry person can properly build and maintain a list that accurately reflects the formulary's contents but whether physicians can agree on prescribing practices and incorporate them into a best practices model that is evaluated and modified as evidence-based practices evolve.

The next most common types of prescribing-based medication errors were dose and frequency errors. Some of these errors were omissions, and a properly designed CPOE system can make it impossible for the physician to complete the medication order until all of the critical information, such as the dose and frequency, has been entered. Nevertheless, under normal circumstances, a “forcing function” such as this is a nuisance. It can be difficult to determine when “expected” information can be realistically left out.

Medication errors caused by system/process-based causes accounted for 35.4% of all errors in this study. A CPOE is unable to eliminate these errors; as mentioned earlier, any new system is likely to increase process errors. Nevertheless, some of the errors noted in our study—such as 14 duplicate medication orders— should have been captured by current technology but weren't; CPOE should decrease the frequency of such errors.

Those who understand the process of medication ordering must be able to design a system that works with the physician in an intuitive manner, prevents the wrong thing from happening, and provides feedback without delaying the process. The “rules” in the system should be carefully considered. If too many alerts are given for situations such as an inappropriate dose for a particular drug and patient type, the alerts will ultimately be ignored, as Gouveia et al warned (7). Further, system developers should ensure that when alerts and notices are given, the physician does not have to start the ordering process from scratch but can quickly resolve the issue and move on. Senholzi et al also warned that when physicians were stymied or frustrated when presented with certain options, they developed “workarounds” that resulted in new medication errors so that they could complete an order (13). While physician-based causes of medication errors accounted for only 18.6% of errors in our study, any system that can ultimately challenge a physician order requires a process that is thorough, trusted, dynamic, and quickly and easily responsive to change.

The value of pharmacist verification

Whether a medication order system is paper-based or electronic, errors will occur, and the verification of medications by a pharmacist seems both necessary and cost-efficient. Although most errors are of little consequence, some can result in a bad outcome for the patient. Classen et al estimated the average cost per adverse drug event at $2000 (14). We do not know what proportion of these medication errors would have resulted in adverse drug events. If we assume that one of our study's 113 medication errors could have culminated as an adverse drug event, pharmacist interventions during the verification process of 1014 medication orders would save Baylor $2000. If we further assume that the numbers from the baseline study are representative of the 1 million medication orders at Baylor each year, then pharmacists-interventions would save Baylor $1,972,386 annually.

Conclusion

A twofold opportunity exists. First, technology has a role in helping health care providers assimilate large amounts of information for their patients' care. A CPOE system with full decision support, to include warning systems, rules and Cross-checks, and real-time notifications of potential problems with accompanying information, if embraced by the medical staff, could bring tremendous opportunities for improving the delivery of appropriate medication orders. At the same time, however, process redesign is needed—derived from evidence-based clinical guidelines; implemented, reviewed, and revised consistently; and appropriately communicated to all stakeholders. These opportunities present important challenges.

References

1. Brennan TA, Leape LL, Laird NM, Hebert L, Localio AR, Lawthers AG, Newhouse JP, Weiler PC, Hiatt HH. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med. 1991;324:370–376. [PubMed]
2. Lazarou J, Pomeranz BH, Corey PN. Incidence of adverse drug reactions in hospitalized patients: a meta-analysis of prospective studies. JAMA. 1998;279:1200–1205. [PubMed]
3. Leape LL, Brennan TA, Laird N, Lawthers AG, Localio AR, Barnes BA, Hebert L, Newhouse JP, Weiler PC, Hiatt H. The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study II. N Engl J Med. 1991;324:377–384. [PubMed]
4. Bates DW, Teich JM, Lee J, Seger D, Kuperman GJ, Ma'Luf N, Boyle D, Leape L. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc. 1999;6:313–321. [PMC free article] [PubMed]
5. Thomas EJ, Studdert DM, Burstin HR, Orav EJ, Zeena T, Williams EJ, Howard KM, Weiler PC, Brennan TA. Incidence and types of adverse events and negligent care in Utah and Colorado. Med Care. 2000;38:261–271. [PubMed]
6. Committee on Quality of Health Care in America, Institute of Medicine. To Err Is Human: Buildinga Safer Health System. Washington, DC: National Academy Press; 2000. p. 31.
7. Gouveia WA, Shane R, Clark T. Computerized prescriber order entry: power, not panacea. Am J Health Syst Pharm. 2003;60:1838. [PubMed]
8. Ballard DJ. Indicators to improve clinical quality across an integrated health care system. Int J Qual Health Care. 2003;15(Suppl 1):113–123. [PubMed]
9. Kaushal R, Bates DW, Landrigan C, McKenna KJ, Clapp MD, Federico F, Goldmann DA. Medication errors and adverse drug events in pediatric inpatients. JAMA. 2001;285:2114–2120. [PubMed]
10. Walsh T. Beaty PCW Human factors error and patient monitoring. Physiological Measurement. 2002;23:R111–R132. [PubMed]
11. Thomas EJ, Lipsitz SR, Studdert DM, Brennan TA. The reliability of medical record review for estimating adverse event rates. Ann Intern Med. 2002;136:812–816. [PubMed]
12. Alderman CP, Farmer C. A brief analysis of clinical pharmacy interventions undertaken in an Australian teaching hospital. J Qual Clin Pract. 2001;21:99–103. [PubMed]
13. Senholzi C, Gottlieb J. Pharmacist interventions after implementation of computerized prescriber order entry. Am J Health Syst Pharm. 2003;60:1880–1882. [PubMed]
14. Classen DC, Pestotnik SL, Evans RS, Lloyd JF, Burke JP. Adverse drug events in hospitalized patients. Excess length of stay, extra costs, and attributable mortality. JAMA. 1997;277:301–306. [PubMed]

Articles from Proceedings (Baylor University. Medical Center) are provided here courtesy of Baylor Health Care System
PubReader format: click here to try

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

  • MedGen
    MedGen
    Related information in MedGen
  • PubMed
    PubMed
    PubMed citations for these articles

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...