Abstract

Background: Approximately 70 percent of pediatric care occurs in ambulatory settings, yet there has been little research on errors and harm in these settings. Given the importance of understanding harm in ambulatory pediatrics, this study was funded by the Agency for Healthcare Research and Quality (AHRQ) as part of the University of North Carolina (UNC) Center for Education and Research on Therapeutics (CERTs), in partnership with the American Academy of Pediatrics (AAP) Pediatric Research in Office Settings (PROS) Network. Purpose: Learning from Errors in Ambulatory Pediatrics (LEAP) was designed to (1) develop a secure, Web-based tool for reporting errors; (2) identify the types and range of errors; and (3) identify errors that can be generalized across multiple practices. Methods: Data collection was pilot-tested in five pediatric practices in March 2003, using a secure, Web-based tool. After revising the tool, 14 sites collected data from June to September 2003. Three members of the research team (one pediatrician and two patient safety researchers) independently coded the qualitative error reports using the constant comparative method. Reports were coded by medical domain, problem types, and child-specific factors. Coding discrepancies were reconciled by consensus. Results: Study participants reported 136 errors. Data collection via the Web-based tool was very successful; participating practitioners reported a high degree of satisfaction and a minimal number of problems. Errors were reported in several domains: prevention, diagnosis, treatment, patient identification, communication, falls, equipment, and administration. For example, one reported treatment error was “prescription changed from liquid to capsule form of anticonvulsant. Mom misunderstood directions and gave both meds for one week. Child developed blurred vision, stuttering, and ataxia.” Conclusions and implications: Physicians reported errors, yet various members of the care team (parents, nurses, pharmacists) discovered the errors. This suggests that everyone has a role in preventing errors from reaching the child. Information learned from this study will be instrumental in the subsequent design of interventions to reduce errors and improve pediatric patient safety. The success of the Web-based, data collection tool points the way for future online data collection efforts. Further research will clarify the categories of harm observed in ambulatory settings, and explore venues for presenting errors and collaboratively designing and testing solutions.

Introduction

An estimated 70 percent of pediatric care takes place in ambulatory settings. This involves preventive, acute, and chronic illness care. Prior patient safety studies in ambulatory and in-patient adult populations have demonstrated that errors are common, costly, and often serious or fatal. ^{1, 2} It seems likely that this would also be the case with errors in the pediatric outpatient setting, but researchers are faced with a significant lack of published research on the nature, frequency, and causes of ambulatory pediatric errors.

Even with the lack of data documenting error and harm in ambulatory pediatrics, there is reason to believe that providing safe care to children poses unique challenges. Because of their limited ability to communicate, children may be unable to indicate whether they are experiencing adverse events, and those events may go unnoticed until serious harm results. Medications are a special source of concern for pediatric patients. The specific risks described in the ambulatory setting include the need for rapid dose calculation as well as parental administration, which depends on clear communication between the clinician and the parent. Additionally, the need for weight-based dosing, when the patient's weight may not be known precisely, increases the risk of either under- or overdosing a pediatric patient. Pharmacists must often dilute stock medications before they can be administered to pediatric patients, providing opportunities for errors that can lead to a range of outcomes in the inpatient environment, ranging from no harm to temporary or permanent disability to death. Because of the age of pediatric patients, when an error leads to injury it may have an impact on the development and life course of the child.

A study conducted by the American Academy of Family Physicians (AAFP) confirms that errors occur in the outpatient care of children. ³ Of the errors observed by family physicians in their own practice settings, 17.5 percent occurred during the treatment of children under the age of 14 years. Avoidable errors in ambulatory settings may often appear to be trivial, but the study found that approximately 5 percent of errors directly precipitated a hospital admission. The researchers suggested error reduction strategies aimed at the system of care delivery, such as improved chart management, more effective communication within the primary care team, and more effective communication between the primary care provider and other health care providers. ³

The Institute of Medicine report, To Err Is Human: Building a Safer Health System, suggests that medication errors are the most frequent type of adverse events. In one of the few studies documenting the epidemiology of medication errors in a children's hospital setting, Kaushal and colleagues ^{5, 6} found that serious pediatric medication errors (potential adverse drug events and preventable adverse drug events) occurred at a three-fold higher rate in children than in adults. This study provided important confirmation of the unique epidemiology of medication errors in pediatric inpatient populations and suggests that hospitalized children are at a greater risk of serious medication errors than adults.

More recently, articles in the January/February 2004 issue of Ambulatory Pediatrics described the state of existing patient safety knowledge in pediatric ambulatory care. ^6–8 This group of articles focused on general pediatric ambulatory care literature, the role of information technology in improving pediatric patient safety, the patient safety risks in the Emergency Department, and the state of our knowledge regarding medication errors in pediatrics. While the authors did make inferences about particular risks based on the characteristics of children and the features of the ambulatory context, these articles underscored the limited research that is focused on the ambulatory care environment and, in particular, on pediatric ambulatory care.

To begin an investigation of errors in ambulatory pediatric practice, a focus group with approximately 50 pediatricians who were chapter coordinators (State representatives) for the practice-based research network of the American Academy of Pediatrics—Pediatric Research in Office Settings (PROS)—was conducted in October 2000. During the focus group session, many pediatricians acknowledged that errors with the potential to harm patients happened in their primary care practices. They felt that they underestimate the prevalence of errors in ambulatory settings, in part, because they often do not see the consequences of the errors because those consequences occur after the child leaves the office. They commonly mentioned medication errors, including prescribing and administering the wrong dosages; prescribing medication despite a known allergy; giving inaccurate telephone advice about dosage; and using the wrong dosage or wrong technique to give immunizations. Furthermore, these providers acknowledged that infants and children are likely to depend on one or more adult caregivers to administer medications, which also may increase the likelihood of an error. Finally, the pediatricians said they did not have a mechanism for documenting or reporting errors in their practice settings.

Methods

The reporting tool developed for this project is shown in Appendix A.^* Because this study was intended to be an exploratory look at errors and “near misses,” the reporting tool included several open-ended questions. The reporting tool was developed based on initial conversations with the AAFP, and then revised and adapted to meet the needs of the PROS practitioners. The structure of the PROS Network provided an opportunity for chapter coordinators to discuss the reporting tool format and questions during their bi-annual PROS meetings. These meetings also provided an opportunity to get buy-in from participants well before the data collection period began.

We used error reporting as a tool to help us obtain “contextual data” about errors. For the purposes of this study, errors were defined as “the failure of a planned action to be completed as intended, or the use of a wrong plan to achieve an aim; including problems in practice, products, procedures, and systems.” ¹⁴ The reporting tool was developed as an Internet-based, digitally encrypted tool.

Immediately following funding and Institutional Review Board (IRB) approval, PROS staff recruited five pediatricians from the network to participate in pilot-testing the Web-based reporting tool. Each of these pediatricians participated in a 15-minute training session via telephone. In addition, they received a detailed instruction manual that explained the study protocol and how to access and use the Web-based tool. Pediatricians were then asked to use the tool to report all errors that occurred in a 1-month period. Participants were asked to report each error that occurred during the study data collection period that he or she was either directly involved in or discovered while caring for patients in an ambulatory pediatric setting.

Furthermore, practitioners were instructed to not include any information that might identify the practitioner, the practice, office staff, another practitioner, or the patient. This was verified by a staff person at PROS who de-identified any data that might have been inadvertently included before the reports were passed to the principal investigators.

Data collection was pilot-tested in five pediatric practices in March 2003, using the secure, Web-based tool. During the 1-month pilot test, 108 errors were reported. Review of the reported errors revealed that numerous reports were made for “trivial” errors (e.g., inconsequential misspellings or immediately corrected clinical documentation that had no reasonably foreseeable potential to cause harm). The directions to practitioners about the reporting tool were revised to instruct practitioners not to the report minor, nonsubstantive errors that often occur. After revising the tool and the error-reporting instructions, an additional 10 sites were recruited (one practitioner was not able to participate in the second round) and a total of 14 sites collected data from June to September 2003. This 3-month data collection period resulted in 64 error reports.

The research team decided to combine the data from the 1-month pilot test (n = 108) and the 3-month reporting phase (n = 64) in order to have more reports (n = 172) to analyze. Data were cleaned to eliminate “trivial” reports; this eliminated 44 reports (42 of which were from the pilot data, prior to clarifying the definition of reportable errors). Error reports were then read to ensure that each report represented a mutually exclusive error. Reports that included more than one error were divided into separate error reports. Reading through the error reports and dividing them into mutually exclusive errors resulted in 19 additional error reports, for a total of 147 reports. For example, it was decided that this report included two separate errors:

Parent left message for refill of Adderall 15mg, 1QDay. Nurse wrote message for Adderall XR 15 mg, 1QDay. Mom discovered change at pharmacy and didn't want Adderall XR because of high cost. Upon further questioning, [clinicians] learned patient was actually taking ½ pill in morning and ½ at lunch.

The Critical Incident Analysis Technique ¹⁵ —a method first employed in aviation to examine aircraft training accidents and now frequently being applied to medicine—was the method used to analyze the error reports. ^16–18 It is a method for organizing reports of critical incidents into meaningful categories of types. In this investigation, the aim of the study was to identify the types of errors that occur in pediatric ambulatory care and the aspects of ambulatory medical care in which they occur. The physician reports provided the incidents for analysis.

The Constant Comparative Method ¹⁹ provided the analytic structure for the Critical Incident Analysis. This involves reviewing all the error reports, focusing on several reports to begin to formulate meaning, and then incrementally including more reports to begin to develop preliminary categories. Reports were coded into the preliminary categories to determine the need to add new categories, and related categories were grouped to develop overarching categories. The typology of categories was reviewed deductively to ensure coherence and parallelism. Finally, all of the data were coded using the developed categories. ¹⁹ Reliability of the categories was tested through a process of triangulation, which involved independent coding of the error reports by three independent reviewers (one pediatrician and two patient safety researchers).

This process of independent coding, triangulation, and reliability testing was performed on two different sets of categories: the aspect of ambulatory medical care in which the problems occurred (medical domains: Table 1) and the types of problems reported (problem types: Table 2). Coding discrepancies were reconciled by consensus of the research team. The intercoder reliability for medical domains was 87 percent and for problem types, the intercoder reliability was 72 percent.

Table 1

Medical domains used for coding error reports.

Table 2

Problem types used for coding error reports.

Results

Clinicians familiar with participating in research efforts of primary care networks successfully used a Web-based reporting system to document practice errors. In a brief post-study survey, the clinicians expressed a very high degree of satisfaction with the ease of using the online reporting tool.

Tables 3 and 4 provide the categories of medical error and the frequency of reports. Errors in the medical treatment domain were most frequent (38 percent) followed by administrative errors (22 percent). Reported errors occurred most commonly in the domains of preventive screening and immunization and of medical treatment (specifically, the medication process). Of the errors within the medical treatment domain (Table 4), 84 percent were medication errors. Communication problems were the most common cause of the reported errors, with problematic communication reported as contributing to 67 percent of the errors. Table 5 provides examples, from the data, of error reports coded by medical domain and problem type.

Table 3

Categories of medical error and frequency of harm.

Table 4

Types of errors within the medical treatment category and frequency of harm.

Table 5

Examples of error reports.

Physicians reported errors, yet various members of the care team (parents, nurses, pharmacists) often discovered the errors. As we expected, physicians discovered most of the errors (52 percent). Parents and nurses discovered 15 percent and 13 percent, respectively. Other errors were discovered by transcriptionists (7 percent), pharmacists (4 percent), lab technicians (3 percent), and other physicians (1 percent). Office receptionists, school personnel, a grandmother, and a patient found the remaining 3 percent of the errors. While we thought the person reporting the error (the pediatrician) would be the one who most often discovered the error, these results suggest the important role that the entire care team has in preventing errors from reaching the child.

The occurrence of an error did not imply harm or injury to the patient. When practitioners were asked to report whether an error resulted in harm, they reported that 6 percent (n = 9) of the errors resulted in harm, and five percent (n = 7) scored as “don't know.” The prospective nature of this study allowed us to detect errors that did not result in harm to the patient—these types of errors would be difficult to identify in a retrospective chart review.

Anecdotally, physicians described their personal learning about identifying errors and noted an improvement in the culture of learning in their practices. For example, one pediatrician said, “At first I was embarrassed to let other people in my office know that I was part of this study and that I made errors. But by the end [of the reporting period], I was going up to people, saying ‘let me tell you about an error I made—has this ever happened to you?’” This sort of change, and development of “a culture of learning,” is an important step toward developing a safer health care environment.

Participating clinicians also reported anecdotally an increased awareness of potential errors and harm in practice. One clinician mentioned the tendency to gloss over minor, inconsequential errors, especially during very busy times. She noted that study participation raised her consciousness about errors, and encouraged her to mention minor errors that would otherwise go unnoticed, to practice colleagues.

Another clinician reported that the busy nature of modern medicine, in itself, might contribute to medical errors and to the glossing over of minor medical errors, due to the clinician being too busy to notice or take action. This is another example of how the development of a “culture of learning” might lead to a safer health care environment. Several clinicians reported a fair degree of subjectivity over the definition of an error, the degree of its seriousness, and whether to report it. Another important facet of raising clinicians' awareness about errors in their practice—and of reporting medical error data in general—is a clearer definition about what constitutes an error, what leads to an error, and what can be done to prevent it.

Discussion

The LEAP study confirms that medical errors occur in ambulatory pediatric settings. LEAP demonstrates that data collection of medical error reports can be successfully accomplished in busy ambulatory practice settings via a Web-based tool. In addition, LEAP confirms what other studies have documented: medication errors are among the most common challenges in patient safety. Participating practitioners reported a high degree of satisfaction and a minimal number of problems with the Web-based tool. The success of this Web-based data collection tool points the way for future online data collection efforts.

Although error reporting was a component of this study, it is not the end-goal. Making care safer for children is our ultimate objective. LEAP suggests the utility of a simple mechanism for collecting and synthesizing errors. Anecdotal reports from participating clinicians indicate that this process facilitated important reflection about safety issues in outpatient pediatric health care settings. Identifying and learning from “near-misses” or errors that do not result in harm is the initial step in making health care safer. Training in error identification and error mechanisms, and in the systems, culture, conditions, and practices of patient safety can provide the necessary tools for clinicians and office-based practice staff to learn about key safety issues of communication, hand offs, and teamwork. Working collaboratively to test solutions may lead to the redesign of practice systems to improve the safety of the pediatric medical care delivered in their office.

Conclusion

Information obtained from this study will be instrumental in the design of interventions to reduce errors and improve pediatric patient safety in the outpatient setting. The LEAP study documents that errors occur in ambulatory settings, most frequently in the context of medical treatment. Pediatricians successfully used a Web-based reporting tool over a short period of time. Reported errors occurred most frequently in the domains of preventive screening and immunization, and of medical treatment (specifically the medication process). Communication problems were the most common cause of the reported errors. Our results suggest that all members of the health care team play a role in preventing errors from reaching the child. Further research will clarify categories of harm in ambulatory settings and explore venues for presenting errors and collaboratively designing and testing solutions. For future research, we will consider a larger study that will include a more representative sample of practices, as well as the participation by additional practice staff and possibly parents.

Improvement in ambulatory care safety will continue to be a challenge because pediatric ambulatory care clinics are busy, high-volume settings where hand offs are common. Pediatric protocols may often be individualized by clinicians, and children thus present unique opportunities for error. Furthermore, each clinic may be organized differently based on its location, practice type, and size. Additionally, while it is possible to learn from other practices, each office practice must adapt system changes meant to improve quality and patient safety to meet local needs. It isn't enough to know what works in another setting. Each practice must recognize how to assess its own culture and design, and conduct effective tests of change. The LEAP study is an important initial step in learning how to ensure that health care is safer for children and families.

Acknowledgments

Support for Learning from Errors in Ambulatory Pediatrics (LEAP) was provided by a grant from the University of North Carolina (UNC) Center for Education and Research on Therapeutics (CERTS) through a cooperative agreement with the Agency for Healthcare Research and Quality (grant number U18 HS10397). The PROS network also receives core funding from the Health Resources and Services Administration's Maternal and Child Health Bureau (U.S. Department of Health and Human Services).

The authors wish to thank the PROS practitioners for their dedication to the success of this project and for the continued efforts in making health care safer for children.

The authors also wish to thank David Kleckner at the University of North Carolina for his technical assistance with the Web-based reporting tool.

References

1.

Bates D, Cullen D, Laird N. et al. Incidence of adverse drug events and potential adverse drug events: implications for prevention. JAMA. 1995;274:29–34. [PubMed: 7791255]

2.

Classen D, Pestotnik S, Evans R. et al. Adverse drug events in hospitalized patients. JAMA. 1997;277:301–6. [PubMed: 9002492]

3.

Dovey S, Phillips R, Green L. Medical errors affecting vulnerable primary care patients in six countries: A report of the LINNAEUS Collaboration. Academy for Health Services Research and Health Policy (AHSRHP) Annual Research Meeting—Health services research: from knowledge to action. Washington, DC: AHSRHP, 2002.

4.

Kohn LT, Corrigan JM, Donaldson MS, editors. To err is human: building a safer health system. A report of the Committee on Quality of Health Care in America, Institute of Medicine. Washington, DC: National Academy Press; 2000.

5.

Kaushal R, Bates D, Landrigan C, et al. Medication errors and adverse drug events in hospitalized children. Panel presentation given at the Pediatric Academic Society; Boston, MA; 2000.

6.

Kaushal R, Jaggi T, Walsh K. et al. Pediatric medication errors: what do we know? What gaps remain? Ambul Pediatr. 2004;4:73–81. [PubMed: 14731088]

7.

Miller M, Provonot P, Burstin H. Pediatric patient safety in the ambulatory setting. Ambul Pediatr. 2004;4:47–54. [PubMed: 14731091]

8.

Johnson K, Davison C. Information technology: its importance to child health safety. Ambul Pediatr. 2004;4:64–42. [PubMed: 14731089]

9.

Wasserman R, Slora E, Bocian A, et al. Pediatric Research in Office Settings (PROS): a national practice-based research network to improve children's health care. Pediatrics 1998;102:1,350–7. [PubMed: 9832568]

10.

Wasserman R, Croft C, Brotherton S. Preschool vision screening in pediatric practice: a study from the Pediatric Research in Office Settings (PROS) network. Pediatrics. 1992;89:834–8. [PubMed: 1579390]

11.

Herman-Giddens M, Slora E, Wasserman R. et al. Secondary sexual characteristics and menses in young girls seen in office practice: a study from the Pediatric Research in Office Settings network. Pediatrics. 1997;99:505–12. [PubMed: 9093289]

12.

Darden PM, Taylor JA, Slora EJ, et al. Methodological issues in determining rates of childhood immunization in office practice. A study from Pediatric Research In Office Settings (PROS). Arch Pediatr Adolesc Med 1996;150(10):1,027–31. [PubMed: 8859133]

13.

Kelleher K, Childs G, Wasserman R, et al. Insurance status and recognition of psychosocial problems: A report from PROS and ASPN. Arch Pediatr Adolesc Med 1997;151:1,109–15.

14.

Reason J. Human error. Cambridge, MA: Cambridge University Press; 1990.

15.

Flanagan J. The Critical Incident Technique. Psychol Bull. 1954;51(4):327–58. [PubMed: 13177800]

16.

Dovey S, Meyers D, Phillips R. et al. A preliminary taxonomy of medical errors in family practice. Qual Saf Health Care. 2002;11:233–8. [PMC free article: PMC1743626] [PubMed: 12486987]

17.

Cote C, Notterman D, Karl H. et al. Adverse sedation events in pediatrics: a Critical Incident Analysis of contributing factors. Pediatrics. 2000;105:803–14. [PubMed: 10742324]

18.

Woods D, Holl J, Bhatia M, et al. Patient safety problems in children's medical care: a Critical Incident Analysis. Panel presentation given at the AcademyHealth Annual meeting, Nashville, TN; 2003.

19.

Creswell J. Research design: qualitative and quantitative approaches. Thousand Oaks, CA: Sage Publications; 1994.

The appendix cited in this report is provided electronically at http://www.aap.org/pros/ leapmain.htm.

Footnotes

*

The appendix cited in this report is provided electronically at http://www.aap.org/pros/ leapmain.htm.