Although much attention has been focused on finding ways to identify medical errors and thereby reduce harm in hospital settings, few efforts have been directed at these issues in ambulatory settings. Duke University’s Department of Community and Family Medicine has developed and implemented a practical, voluntary reporting system with classification and tracking of types of errors. Initially created in the Family Medicine Center, this system is now used in all of the department’s wide variety of clinical operations. By reporting errors, analyzing error patterns, and addressing them, the clinical practices have become better able to identify faulty systems and error-prone areas and to change processes to prevent future errors.
Efforts to increase patient safety within hospital settings are numerous and have resulted in many organizational, cultural, and systemic environmental changes that have reduced harm.1, 2, 3, 4, 5, 6, 7 However, less attention has been paid to outpatient settings. This limited research scope risks missing the everyday errors that occur where the largest proportion of care is delivered—in primary care. Understanding and reducing errors in primary care practices could potentially have a wide range of positive effects, including better clinical outcomes, decreased hospitalizations, improved patient-physician trust, reduced costs, and lower malpractice claims.
Few research studies have examined errors in primary care, and even fewer have addressed how relevant data can be collected in a real-life clinical setting and used to reduce errors and build a culture of safety. This paper describes how the Duke University Medical Center Department of Community and Family Medicine (CFM) developed a customized error reporting and classification system for its outpatient clinics and changed its culture to encourage reporting and quality improvement.
Previous studies have shown that frequencies of error types recorded vary between countries and regions. Some of these differences may be real, while others may simply be due to different reporting routines and classification systems. The top five types of errors reported by U.S. family physicians are:
- Errors in prescribing medication.
- Errors in getting the right laboratory tests done for the right patient at the right time.
- Errors related to filing systems.
- Errors in dispensing medications.
- Errors in responding to abnormal laboratory test results.8
Studies have found that most errors in primary care practice are preventable. In 1998, Bhasale, et al.,9 found that of 805 incidents reported in general practice settings in Australia, 76 percent were preventable. Fischer, et al.,10 reviewed incident reports from eight primary care clinics affiliated with an academic medical center in the Midwestern United States and found that 83 percent of the events were preventable.
Of particular importance in studying errors in primary care is the question of how small- and medium-sized practices can develop patient safety systems within the time and cost constraints under which they function. Elder, et al.,11 suggested that barriers to establishing systems in family medicine offices include the burden associated with the effort to report, a lack of clarity regarding the information requested in an error report, the perceived benefit (and risk) to the reporter, and the properties of the error (e.g., severity, responsibility). Although the same study found less agreement in identifying motivating factors, the most commonly cited factor in encouraging reporting was perceived benefit, particularly the idea that reporting will improve the overall process.
Overcoming these barriers requires a reporting system that involves minimum time and effort and a process that emphasizes that information gathered is directly used to improve the practice environment. It is also important to recognize that the close interactions between staff in a small office will discourage reporting if the system is perceived as designed to “punish” individuals.
Development of the Duke CFM Quality Improvement System
Initial quality improvement (QI) efforts in the Department of Community and Family Medicine focused on the Family Medicine Center. This practice currently has a volume of approximately 40,000 visits per year and a patient population that closely mirrors the racial and ethnic composition of its surrounding community: 46 percent African American, 41 percent white, 6 percent Latino, and 7 percent other. Management of chronic disease is a major focus. One-third of patients have Medicare or Medicaid coverage, 40 percent are Duke employees covered under a Duke health insurance plan, and the remainder are covered by a mix of other private payers. At any given time, there are approximately 10 full-time providers—including physicians and mid-level practitioners—supported by an interdisciplinary staff of nurses, pharmacists, social workers, dieticians, psychologists, and laboratory, radiology, and clerical staff.
This academic practice serves as the primary continuity site for family medicine residents, as well as other “providers in training,” such as medical/physician assistants and pharmacy students. The clinic’s population, volume, staffing mix, and funding sources were roughly the same a decade ago, when the QI efforts were begun. The only structural change is that the practice is now an outpatient facility of Duke Hospital.
The State of North Carolina has no mandatory reporting laws for primary care providers, and other than the standard requirement by The Joint Commission to report sentinel events, there have never been mandatory reporting requirements outside of those a medical center sets for itself. As part of the hospital peer review system, the Duke Family Medicine Center had a well established voluntary reporting system by the late 1980s. The quality efforts at that time centered on individual error and correction in a traditional “quality assurance” approach, in which the focus was on meeting preset criteria.
In 1996, in conjunction with an institutional movement to QI, patient safety efforts refocused on systems analysis, process improvement, and care improvement. With the understanding that humans are imperfect and will make errors, attention was directed to systems that would support health professionals in care delivery and could help identify errors before they affected patients. The departmental voluntary reporting system was redesigned and expanded beyond the hospital requirements. All members of the practice were encouraged to report problems and near misses of all types within a confidential, protected peer-review system.
It is notable that those reporting were not asked to determine that an error had occurred—only that there was concern about a possible error. All significant adverse events (e.g., unexpected deaths and hospitalizations) were to be reported for peer review to help determine if the event might have been prevented. Also of note, the system was not anonymous; no attempt was made to blind reviewers to the identities of reporters or involved providers.
To make the reporting system simple and yet thorough, a one-page form was created that asked the reporter to identify the issue and actions taken, if any. The form included patient identification information, the concern, and a description of any actions taken so far, along with clear labeling for confidentiality and instructions against copying or distribution. Reporters were asked to describe in free text the concern or incident and any actions taken. Attachments could be included with the sheet if available.
Refinements to the form were made based on user and reviewer requests. A severity assessment was added, in which reporters circled one of five outcomes ranging from “no adverse outcome” to “death.” As technology changed and to encourage the widest possible reporting, reports were accepted in other forms if these were more convenient at the time. For example, a provider could send a brief e-mail, or a staff member could forward a copy of a patient complaint letter.
All reports were logged into a secure, confidential database and forwarded by a staff coordinator for case review. Initial reviews were done by a nurse practitioner or physician assistant on the QI team or were requested from involved providers in order to get their perspective. All cases were routed in clearly marked confidential packets, tracked closely, and hand delivered to maintain peer review protection. Reviewers filled out a peer review sheet to indicate their opinion as to whether standard of care had been met, as well as their recommendations. Specific questions on the review form asked whether the error was an individual provider issue, a systems issue, or indicated a pattern or trend. Reviewers could identify cases for committee discussion, QI projects, and/or divisional morbidity and mortality (M & M) conferences. No case was closed until action had been taken to rectify all issues identified. Systems were developed as needed to prevent recurrences.
M & M conferences, held roughly monthly in the Family Medicine Center as part of the peer-review program, were converted in 1996 into dialogues about how systems of care could be improved. These conferences included all Family Medicine Center providers, as well as interested staff. The traditional inquisition to identify a scapegoat who might have made a mistake was eliminated.
As the practice group realized that investigations resulted in systems improvements, and that human error was not punished or ridiculed, rates of reporting increased. The rapid volume growth—from 15 reported concerns in 1996 to 113 in 2003—made it difficult to keep track of the types of problems observed (Figure 1). In 2003, to enable better tracking and analysis of error patterns, we sought a coherent and comprehensive coding system by which to classify the cases.
A search of the existing literature revealed two potentially applicable taxonomies for medical errors. Pace, et al.,12 with the Applied Strategies for Improving Patient Safety (ASIPS) Collaborative in Colorado, had created a comprehensive and thoroughly researched coding system. We found many good ideas in the system, but realized that, in a busy practice setting of our size, it would be too cumbersome and require too much time for efficient usage. Dovey, et al.,13 had published a taxonomy based on work in family medicine offices in the United States. Although this taxonomy was much more usable in a clinical environment, initial attempts at application were problematic. In particular, Dovey’s division of knowledge and process errors was found to be difficult to apply. In practice, many incidents included aspects of both, and the distinction was difficult to make based on medical records.
Building on this previous research and our reported cases, we developed a simplified taxonomy around seven main error clusters: (1) communication, (2) studies, (3) diagnosis, (4) medication, (5) other treatment or followup, (6) records, and (7) administration. Each category included several subcategories. For example, communication errors were divided into problems in communication with patients or families and between providers and/or staff. See the Appendix for a full listing. Subsequently, as adoption of the system and form progressed, we identified the need for an eighth category: (8) no error (e.g., adverse outcome despite appropriate care, unhappy patient). Since all unexpected deaths and quality-related patient complaints were reviewed, this category was needed for those cases in which no error was identified.
After each incident had been analyzed and peer review completed, the QI director assigned a primary error code. All reported errors were coded, regardless of whether they reached or harmed a patient (e.g., a “near miss” of a dosing error that was caught by a pharmacist and corrected was coded as a dosing error). The draft taxonomy was revised and refined based on experience with the cases of 2002 and 2003. To assess for agreement among multiple raters, a sample of cases was independently coded by three senior faculty members. Substantial variability was found in the assignment of subcategories, but agreement on assignment of each case into a major cluster was better.
In 2005, the confidential QI Concern Report form was revised to ask reporters to indicate their impression of the type of error according to the new classification, in addition to describing the concern or incident and corrective actions taken (see Appendixa). This initial code assignment was for consideration by the reviewers as they completed further investigation. In practice, reporters have been much less likely to complete this step (30/133 in 2006). Cases and final codes assigned are registered and tracked in a password-protected, peer review database with access strictly limited to the QI director and QI staff only.
Beginning in 2004, the system developed for the Family Medicine Center was rolled out and adapted for all clinical operations in the CFM. This included a wide variety of units, such as Community Health, Student Health, Occupational and Environmental Medicine, and the residential Diet and Fitness Center. Each unit or division made local modifications to the reporting format and initial review process to fit their setting and personnel, while maintaining the very stringent requirements of a peer review process.
Reviews by each of these units or divisions went to the CFM QI director for further review and coding. If all identified issues had not been sufficiently addressed, the QI director could send the cases back to the unit or division for further analysis and/or corrective action.
When a case was closed, it was cross-referenced in the peer review database of involved providers. This cross-reference was checked on an annual basis as part of the recredentialing process. In order to identify patterns of similar issues, cases for each individual were re-reviewed. Patterns discovered led to provider feedback, action plans, and followup reviews as needed.
In the 11 years since the transition to QI, the voluntary reporting system has facilitated the development of a culture of safety. Numbers of reported concerns have increased 10-fold since the initiation of the system (Figure 1), while the number of patient encounters has remained relatively constant.
Figure 2 shows the distribution of error types in 2005 and 2006. In contrast to earlier reports,8 problems with studies are by far the most frequent. The most common issues were errors in reporting results to providers and errors in responding to results.
By analyzing error patterns and addressing them, the clinical practices have been able to identify faulty systems and error-prone areas and change processes to prevent recurrences. For example, reported inconsistencies in communication and recommended followup for abnormal Pap tests led to the development of a weekly batch reporting process from pathology, which feeds into a departmental database to track individuals from “index abnormal Pap” through “treatment/resolution.”
Another example was difficulty with electronic reporting of cardiac studies to supervising physicians only, bypassing the mid-level providers who had ordered them. This was a serious problem because the mid-level providers function quite independently. Fortunately, a call to the cardiac studies laboratory resolved the problem within a week.
Within each clinical unit, the program described has been supported by a small degree of effort from one clinician (up to 20 percent) and staff (up to 30 percent). Budgeted amounts range from $0 to $30,000, depending on the size of the clinical practice, or up to 1 percent of the clinical revenue of each unit. This is considered part of the cost of clinical operations. Additional physician and staff time are needed for coordination and department-level review.
Primary care settings vary widely. Academic medical centers, urban clinics, rural physician offices, and suburban practices all differ in their mix of types of providers, patients, payment sources, and administrative resources. When developing patient safety reporting systems, providers should take care to learn from successful practices elsewhere but also keep in mind the individual needs of their environment. The program we describe, which is still evolving, is not offered as a one-size-fits-all solution for primary care settings, but rather as a practical guideline that can assist others.
Key to any patient safety endeavor is the creation of a reporting system that takes into account limited time, confusion, and blame. In a discussion of common barriers to error reporting, Elder, et al., identify the most significant barrier as time.11 The program described here attempts to alleviate this by making reporting as convenient as possible by accepting both paper and electronic formats. Drop boxes for confidential concern reports are available in all clinic workrooms. Reporters are not criticized if their reports are brief—a simple notation of the patient ID, date, and less than a full sentence on the concern can take under a minute—although some may spend several minutes describing relevant issues and attaching details.
A second common barrier to reporting is fear of “betraying” colleagues. By restructuring the activity to one of reporting “concerns” rather than “errors,” the burden of judgment is removed from the reporter. The process is reframed as helping the practice to identify systems that are not working because “we can’t fix problems we don’t know about.” Reporting may be done anonymously, if desired, through an institutional online system, but this method has been used rarely. Most reporters identify themselves, and many participate in the investigation of a concern. In fact, many providers report themselves when they have questions or concerns about their own actions (“Did I miss something?”), a behavior that is strongly encouraged.
Another barrier to high levels of reporting is an apparent lack of benefit to the individual provider. In this program, at the request of the providers, feedback is given directly to reporters at the completion of each case review, indicating the systems changes that resulted from their report. (No specific feedback is given on individual errors or performance of individuals other than the reporter.) For example, a provider who reported a delay in treatment because of failed notification of an abnormal study result might be told that his or her report was one of several that led to collaboration with radiology for a revised notification system. Providers who report an unexpected death of one of their patients might be notified that the peer review identified no flaws in their care. On the other hand, they might be given some concrete suggestions. Reporters are explicitly thanked for their contributions, even when the analysis reveals no error.
Identification of patterns of problems is critical. It is rarely cost effective or desirable to change systems in response to every single event. Some near-miss events have high enough risk that they must be addressed immediately; other minor glitches may not warrant action unless they recur. When numbers of reports are small, it is easy for those involved to identify clusters around similar issues. Once volume increases, a classification system becomes more helpful. The taxonomy developed here is functional for this clinical setting. It enables a practice to identify and quantify its most frequent problems.
The most common errors reported in one setting may not match other settings. Indeed, our most common errors—those relating to diagnostic studies—are different from the category most commonly cited in prior studies—i.e., medication errors.8 Some of the issues with diagnostic studies have related to the complexities of a teaching practice and the difficulty of ensuring communication with both trainees and their supervisors. Medication errors may occur less frequently due to an emphasis in the electronic medical record on medication lists, although electronic prescribing is not yet in use in this setting. Other factors may include supervisors rechecking trainees’ prescriptions, involvement of the pharmacist on staff, or the system not functioning successfully in capturing these errors in concern reports.
In some cases, neither reporting the patterns nor creating quality improvement initiatives will easily solve the problem. This is particularly true for more systemic problems involving multiple stakeholders, both within and outside our own clinics. For example, Kripalani, et al.,14 wrote about a problem common for many primary care settings: poor communication with hospitals and specialists outside the primary care practice. They found that results of tests pending at discharge from hospitals often do not make it back to primary care settings, resulting in delayed or missed care. This matches our experience. One effort to address this problem involved changing the discharge summary format to highlight pending items and abnormalities needing followup.
A delicate consideration is the use of case reviews in recredentialing. As mentioned above, cases are cross-referenced to individual providers and reviewed at time of recredentialing. This raises the specter of potential punitive consequences. No provider has been penalized for having large numbers of cases. In fact, as reporting has been encouraged, high-volume providers have often had sizeable numbers. Indeed, those who frequently self-report often have the highest numbers. In these instances, cases may be considered again looking for patterns. In the vast majority of recredentialing reviews (65 per year, on average), the results show a small number of understandable human errors, for which feedback has been given and appropriately received. In some cases, a pattern may be detected (perhaps of unhappy patients, suggesting issues with interpersonal style) and trigger some feedback. Only clear outliers with an unusual pattern and severity of issues may receive a formal corrective action.
Over the decade since this program was started, the CFM has experienced decreased liability claims. While a direct causal link cannot be proven and multiple factors are certainly involved, the culture change produced by this improvement program was likely a contributing factor.
The limitations of this report are two-fold. As a description of a system developed real-time in an active clinical practice, there is no rigorous study design, and the data are not suitable for statistical analysis. In addition, legal and confidentiality issues surrounding patient information restrict what information can be shared. As more small- and medium-size primary care environments share best practices and qualitative information in the literature, it is hoped that more structured, larger scale studies will be undertaken and offer more scientific rigor to patient safety initiatives in primary care.
Error reporting systems in primary care have the potential of assisting in continuous practice redesign so that patient safety is improved. The system described here has the advantages of being simple and usable in a busy practice setting, while providing enough detail to permit practice redesign and meaningful feedback. Use of simple error tracking systems in primary care sites is encouraged so that primary care, with all its complexities, can be the safest possible medical home.
We thank Lawrence Wu, MD, for his assistance in the initial design of our system, and Kate Holeman and Jenny Cook for their assistance in preparing this paper.
DUKE FAMILY MEDICINE CENTER
DUKE UNIVERSITY MEDICAL CENTER
OFFICE USE ONLY:
□ Pt Related
QI Concern Report Form
Please use this form to report QI concerns/issues.
✏ PLEASE DESCRIBE IN DETAIL THE CONCERN/INCIDENT BEING REPORTED:
(Attach separate page if necessary)
✏ PLEASE INDICATE ACTION(S) TAKEN:
✏ PLEASE INDICATE LEVEL OF PATIENT OUTCOME:
0 = No adverse outcome
1 = Minor adverse outcome (non-serious effect, not requiring treatment)
2 = Moderate adverse outcome (significant effect, requiring treatment)
3 = Serious adverse outcome (permanent adverse effects)
4 = Death
□ Chart Notes
□ Encounter Form
✏ IS URGENT ACTION NEEDED?
(e.g., patient waiting for reply)
□ No, routine
PLEASE COMPLETE REVERSE SIDE ALSO!
TYPE OF ERROR
Please check one or more of the following categories to identify the primary thing you believe went wrong in this case.
- □ Communication
- problems in interaction with patients or their relatives
- problems in interaction between health care providers/staff
- □ Studies
- ordered incorrectly (wrong test or not indicated)
- not ordered when indicated
- not done as ordered
- error in reporting results to provider
- error in responding to results
- □ Diagnosis
- insufficient evaluation for diagnosis
- wrong and/or missed diagnosis based on available data
- □ Medication
- ordered incorrectly (wrong medication, wrong dose, or not indicated)
- no medication ordered when indicated
- not delivered as ordered
- □ Other treatment or follow-up (excluding diagnostic studies and medications)
- ordered incorrectly (wrong treatment, wrong timing, or not indicated)
- not ordered when indicated
- not delivered or completed as ordered
- □ Records
- □ Administration
- errors in handling/transmission of messages
- errors in appointment scheduling
- other administrative error
- □ No error
- adverse outcome, no error
- unhappy patient, no adverse outcome, no error
- no error, patient choice
Please fold this form in half and staple it, or place in an envelope with relevant materials, and deliver to any of the QI concern boxes in workrooms.
- Baker DP, Gustafson S, Beaubien JM, et al. Medical teamwork and patient safety: The evidence-based relation. Washington, DC: American Institute for Research; 2003.
- Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events and negligence in hospitalized patients: Results of the Harvard Medical Practice Study. N Engl J Med. 1991;324:370–376. [PubMed: 1987460]
- Bates DW, Cullen DJ, Laird N, et al. Incidence of adverse drug events and potential adverse drug events: Implications for prevention. JAMA. 1995;274:29–34. [PubMed: 7791255]
- Phillips D, Christenfield N, Glynn I. Increase in U.S. medication error deaths between 1983 and 1993. Lancet. 1998;351:643–644. [PubMed: 9500322]
- Gaba DM. Human error in anesthetic mishaps. Int Anesthesiol Clin. 1989;27:137–147. [PubMed: 2670768]
- Bates D, Leape L, Petrycki S. Incidence and preventability of adverse drug events in hospitalized patients. J Gen Intern Med. 1993;8:289–294. [PubMed: 8320571]
- McGuire HH, Horsely JS, Salter DR. Measuring and managing quality of surgery. Statistical vs. incidental approaches. Arch Surg. 1992;127:733–737. [PubMed: 1596176]
- Elder N, Dovey S. Classification of medical errors and preventable adverse events in primary care: A synthesis of the literature. J Fam Pract. 2002;51:927–932. [PubMed: 12485545]
- Bhasale AL, Miller GC, Reid S, et al. Analysing potential harm in Australian general practice: An incident-monitoring study. Med J Aust. 1998;169:73–76. [PubMed: 9700340]
- Fischer G, Fetters MD, Munro AP, et al. Adverse events in primary care identified from a risk-management database. J Fam Pract. 1997;45:40–46. [PubMed: 9228913]
- Elder NC, Graham D, Brandt E, Hickner J. Barriers and motivators for making error reports from family medicine offices: A report from the American Academy of Family Physicians National Research Network (AAFP NRN). J Am Board Fam Med. 2007;20:115–123. [PubMed: 17341747]
- Pace WD, Staton EW, Higgins GS, et al. Database design to ensure anonymous study of medical errors: A report from the ASIPS collaborative. J Am Med Inform Assoc. 2003;10:531–540. [PMC free article: PMC264430] [PubMed: 12925548]
- Dovey SM, Meyers DS, Phillips RL Jr, et al. A preliminary taxonomy of medical errors in family practice. Qual Saf Health Care. 2002;11:233–238. [PMC free article: PMC1743626] [PubMed: 12486987]
- Kripalani S, LeFevre F, Phillips CO, et al. Deficits in communication and information transfer between hospital-based and primary care physicians: Implications for patient safety and continuity of care. JAMA. 2007;297.8:831–841. [PubMed: 17327525]
Form may be used freely with attribution as to source: Duke University Medical Center, Duke Family Medicine Center.
Victoria Kaprielian, MD, Truls Østbye, MD, PhD, Samuel Warburton, MD, Devdutta Sangvai, MD, MBA, and Lloyd Michener, MD.*
Agency for Healthcare Research and Quality, Rockville (MD)
Kaprielian V, Østbye T, Warburton S, et al. A System to Describe and Reduce Medical Errors in Primary Care. In: Henriksen K, Battles JB, Keyes MA, et al., editors. Advances in Patient Safety: New Directions and Alternative Approaches (Vol. 1: Assessment). Rockville (MD): Agency for Healthcare Research and Quality; 2008 Aug.