Background: The testing process in primary care is complex, and it varies from one office to another. We sought to understand how family medicine offices create safety in this process. Methods: Using observations, interviews, and surveys, we collected data at four family medicine offices. We searched the interview and observation notes for stories of safety, error prevention, and recovery and coded them to a model of resilient engineering properties, work system components, and testing process steps. Results: We found only six examples of practices that were systematically creating safety in the testing process via organizational resilience. The most common resilience properties were top-level commitment and a learning culture applied to work system components of people and their tasks. Offices predominantly depended on individuals to double-check, remember, and work around ongoing problems. Conclusions: While family medicine offices overwhelming depend on individuals to work around testing process problems, important properties of office-wide safety practices included a top-level commitment and a learning culture.

Introduction

The doctor-patient relationship has long been considered the center of primary medical care. However, this relationship does not occur in a vacuum. Each office visit is supported by systems of individuals, procedures, technologies, regulations, and organizational structure. This larger system has a significant impact on patient care. Researchers have recently begun to take a more global perspective on primary care and to evaluate the impact of the larger system on the quality of patient care.^{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}

One of the most common and important processes in primary care is testing. Tests ordered in primary care include laboratory, imaging, and special tests (e.g., cardiac stress tests, electromyograms). The testing process can be defined as all the steps that occur from the time a physician decides to order a test until the appropriate followup action is discussed with the patient and follow-though has occurred.

Some low complexity tests are performed in physicians’ offices, but most tests are sent to outside facilities. Previous work^{11, 12, 13, 14, 15, 16, 17, 18, 19} has led to an understanding of the steps that make up the testing process in primary care and delineated the steps in which physicians and their staff members perceive the most errors occurring.^{11, 12, 14, 16, 17} Although some authors have broken these actions down into “pre-analytical, analytical and post-analytical” phases,^{15, 20} we have expanded the pre- and post-analytical office-based actions into a series of steps, which taken together define the testing process (Figure 1):

Figure 1

Steps in the testing process in primary care. Source: Adapted from Hickner JM, Fernald DH, Harris DM, et al. Issues and initiatives in the testing process in primary care physician offices. Jt Comm J Qual Patient Safety 2005; 31: 81–89.

• Ordering: A physician makes a decision to obtain a test and communicates that decision to the appropriate personnel.
• Implementation: The order is transmitted to those performing the test and/or obtaining the specimen(s); the patient is prepared for the test and/or the specimen(s) are obtained.
• Tracking: The test order is monitored internally (within the primary care practice) until the results are returned.
• Return of results: The results are sent back to the office (and to the physician) from testing facilities or locations.
• Response: The physician makes a decision as to the meaning of the results and creates an action plan.
• Documentation: Physician and/or staff note in the medical record that the result has been reviewed; that the physician has responded to the result; and that the patient has been notified.
• Notification: The patient is informed of his/her test result and the physician’s recommendations for action.
• Followup: The process whereby abnormal results and/or results requiring action are monitored until such action is taken or the patient refuses the action.

In a field as complex as medicine, there are multiple potential sources of ambiguity (e.g., patients with similar names) and small mistakes (e.g., incorrect filing of a test result) that can cascade into consequences disproportionate to their sources (e.g., allowing a critical condition to go untreated). Testing represents a common arena for these types of errors. Recent estimates show that the average family physician and general internist order laboratory tests in 29 percent and 38 percent of patient visits, respectively, and imaging studies in 10 percent and 12 percent, respectively.¹³ Therefore, it is not surprising that errors associated with these events are common; 15 to 54 percent of primary care medical errors reported by physicians and their staffs are related to the testing process.^{12, 13, 21, 22, 23, 24}

Errors have been reported in all office-based testing process steps, but those that occur in association with the implementation and return of results are the most frequently reported.^{12, 15, 21} Although these errors have rarely been associated with significant physical harm to patients, adverse consequences, including emotional distress, financial loss, and delay of diagnosis and treatment are common.¹²

The road to improved systems begins with an understanding of the testing process within the larger practice system.²⁵ The testing process can be described as a distributed cognitive system or a work system, where multiple people, tasks, technologies, and environmental and organizational factors interact to determine the outcome.^{2, 26, 27}

In order to move the focus from what is wrong with the testing process to what works well, we have framed our research in the context of resilient systems engineering.²⁸ In this context, a resilient testing process is viewed as a system process capable of adaptively learning to correct errors and to take advantage of new opportunities (e.g., information technology) to improve quality.^{28, 29, 30} Safety and resilience are not static properties of an organization but reflect a dynamic struggle to create safety. The properties necessary for resilient organizations have been described as follows:³⁰

• Top-level commitment: Top management recognizes performance concerns and addresses them with continuous and extensive follow-through.
• Just culture: Reporting of issues, problems, events, and errors throughout the organization is supported, but culpable behaviors are not tolerated.
• Learning culture: Issues, problems, events, and errors are handled with an eye toward repair and true reform, not denial.
• Opacity: Management is aware of how close they are to having serious problems and events due to weak safety defenses.
• Awareness: Management collects ongoing data to gather insight into quality of performance, problems, and the state of safety defenses.
• Preparedness: Management actively anticipates problems and prepares for them.
• Flexibility: New or complex problems are handled in a way that maximizes the ability to solve the problem without disrupting overall work.

To best understand how to increase safety in the testing process, we believe a model must describe both the complexities of the work place system and the existence of resilience properties in that practice (Figure 2). Resilience properties, such as those listed above, are exhibited through the work system: that is, the people, tasks, tools and technologies, environment, and organizational structure of the practice.

Figure 2

Creating safety: A model of possible components of officewide safety practices in family medicine offices. Note: For an organization to create safety, it must develop officewide safety practices that incorporate one or more properties of resilience. These (more...)

In the current study, we applied this model to describe how family medicine offices enhance safety. As part of a larger multimethod study of actual testing process performance in primary care, we analyzed observations and interviews in family medicine offices in order to describe how these offices are working to improve quality and decrease errors in the testing process. Specifically, we asked these general questions:

1. How do offices safely manage the multiple steps of the testing process?
2. What work system components and resilience properties do offices use to enhance safety in the testing process?

Methods

To better understand the testing process in primary care, we elected to intensively study four family medicine offices. Each office was visited for 2 to 4 days, with other data obtained before and after these visits by phone, e-mail, postal mail, and personal visits. The study was conducted between December 2006 and June 2007.

We used data collection methods that allowed us to gather the maximal amount of information while causing minimal interference to patient care and productivity. As data were collected at each site, we also conducted ongoing discussion and analyses. This approach allowed each day’s visit to build on the previously collected data.

This study received approval from the University of Cincinnati and Wright State University Institutional Review Boards.

Results

Testing Process Complexity

Prior to describing how these offices created safety, we will briefly describe the offices and the complexity of their testing processes. While the four medical offices we studied ranged in size, location, and patient characteristics (Table 1), they all performed a complicated series of tasks to move from a physician test order through patient notification and followup.

Table 1

Characteristics of participating family medicine offices.

All of the offices performed some of their own low-complexity laboratory testing, but they sent the majority of their laboratory work to hospital or reference laboratories. Only one site had its own radiology suite and staff for plain films; all the others used nearby hospitals and free standing radiology centers for imaging and special tests.

We found variation both between practices and within practices. For all practices, the type of test ordered (laboratory, imaging, or special test) and the site where it was conducted (office, reference laboratory, or hospital) affected the specific tasks performed. For example, at one office, the procedure was as follows: a physician ordered an imaging test at a local hospital via a written prescription, received the results by fax days later, and then waited for the patient to return for a followup office visit for patient notification. At this same office, physicians’ laboratory test orders were handwritten on the billing sheet; test samples were obtained by a medical assistant (MA) who entered the orders into an onsite laboratory computer terminal; results were returned via a dedicated printer the next day; and patients were notified of their results by mail.

Within offices, the procedure for ordering tests and managing results occasionally varied among individual providers and staff for identical tests performed at the same location. While each practice had preferred reference laboratories and radiology centers, a patient’s insurance status occasionally necessitated using different testing sites (often requiring different tasks.)

No single individual at any of the offices, including the office manager or medical director, could describe all the tasks involved in any of the testing processes. However, these administrators were more aware of the general process flow than were the MAs, physicians, or clerical staff, who rarely knew what happened to an order, sample, or result before it came to them or after it left them.

None of the offices had written protocols for all of their testing processes; two offices had no written protocols at all; and the other two offices had protocols for some, but not all of the testing process steps and tasks. There were physicians and staff at every office who, when asked to tell us “what happens next with this …” described tasks and processes that were totally incorrect. For example, at one office, a physician, when asked what happened to test results that came back when he was absent from the office, noted that the MA reviewed them all and sent all abnormal results to a partner for review and action. However, the MA remarked that this was not one of her duties, and that she had never performed those actions.

Organizational Safety

As noted above, we observed six instances that we felt fit within the “creating safety model” of officewide safety practices (Figure 2). One office had four safety practices; one had two; one had one; and one had none. The only demographic characteristic of the offices that separated them was the presence of a residency program at the office with four safety practices. These safety practices are described in Table 2, along with the major work system components and resilience properties involved.

Table 2

Creating safety in the testing process: Organizational safety practices identified in family medicine offices and their associated testing process steps, work system components, and resilience properties.

All of these safety practices involved the support and involvement of the entire office organization. How people performed their tasks was influenced by these organizational decisions. In three of the safety practices, specific tools were also used (e.g., computer printouts, date and signature stamps, and standardized patient correspondence forms). Although one safety practice influenced the overall safety culture of the office, the other practices all dealt with steps in the testing process: tracking and return of results (one safety practice) and response to results, documentation, patient notification and followup (four safety practices). We did not find any examples of officewide safety practices involving the ordering or implementation of tests.

The main resilience property found in these safety practices was the use of a learning culture. In almost all the cases described, the successful safety practice grew out of experiences with error, failure, or adverse events. The office organization then responded to the events with the development and implementation of the described safety practices. It was (and continues to be) a top-level commitment to these safety practices that is largely responsible for the success of these safety practices. In the two offices with one or no examples of organizational resilience, a top-level commitment to safety was noticeably absent.

Two other resilience properties were each found in one safety practice. There was awareness in the safety practice of abnormal Pap smear review. Data gathered about the management of abnormal Pap smears gave the clinic administration insight into what was going on regarding the quality of Pap smear care. We also found flexibility in those practices that used patient communication tools both to document physician response to a test result and to notify patients. When standardized throughout the office, these tools solved the problem of notifying patients about results without increasing work for the physician. We found no examples of opacity, preparedness, or a just culture within any of these organizational safety practices.

Discussion

Physicians order tests on their patients to screen for and diagnose disease, monitor treatment, and prevent complications. It is such a routine part of practice in primary care that physicians tend to give little thought to all the people, steps, and tasks that allow them to order a test today and receive a result tomorrow. But when an abnormal Pap smear is filed without physician review, or when a chest x-ray reveals a suspicious nodule but is never followed up, then physicians (and patients) ask, “What went wrong?”

In recent years, testing process research, especially in practice-based research networks, has focused on answering the “What went wrong?” question.^{11, 12, 13, 17, 18, 23, 24} However, it is equally important to answer the “What works well?” question.^{16, 31} “Creating safety” is one way to think about “What works well.” We found that creating safety was only rarely done via officewide safety practices. Instead, it depended on individual conscientiousness. Within our model of creating safety, we found that most officewide safety practices were characterized by a top-level commitment and a learning culture, and that these practices focused primarily on people and their tasks. Office practices without these organizational resilience properties depend almost exclusively on individuals working around dysfunctional systems to create safety.

Only a few officewide testing process safety practices were identified at these offices, but by examining these in the context of resilient systems, it is possible to better identify what family medicine offices need to do to create safety. The consistent resilience properties we found were a top-level commitment and a learning culture. Office improvement programs, such as the TransforMED^SM program of the American Academy of Family Physicians (AAFP)³² and the Clinical Microsystems initiatives,^33,34 note the importance of these factors in successfully implementing quality initiatives. While all of Wreathall’s organizational resilience properties³⁰ are important to the development of the highest quality office practices, using a learning culture to identify and remedy the immediate areas at risk, backed by a top-level commitment to address them with continuous and extensive follow-through, are natural starting places.

The organizational testing process safety practices also focused on the most elemental of work system components: the person and their tasks. While this is the logical and most appropriate starting place for safety practices, ultimately, incorporating technology, tools, and even environmental changes for safety will also be necessary.²

Since this was an exploratory study of only four primary care practice offices, we cannot determine whether demographic factors about the office were associated with the use of resilience factors. Although the residency office did have more safety practices, future research will be necessary to tell if that is true of training programs in general or just this specific office. In order to achieve an expansive view of testing process safety, we chose to study offices with a variety of demographic and geographic factors. Future research will also be necessary to study how incorporating safety practices will actually affect outcomes of patient care, including quality indicators, adverse events, harm, and patient satisfaction.

In offices without organizational resilience, safety is maintained almost exclusively by the diligence and conscientiousness of individual employees (and patients). While this diligence is important in even the most resilient organizations,^28,35 when it is applied mainly to working around dysfunctional systems rather than searching for, finding, ameliorating, and reporting the few errors that slip through, then the status quopersists, and little progress is made toward safety and quality.³⁶

In the participating offices, we found many examples of individuals who had developed their own way of implementing orders, notifying patients, or keeping track of results and followup actions, either because no officewide system existed or because they found the existing system ineffective or cumbersome. There was little if any discussion of these methods among office physicians and staff. Unfortunately, when these individuals were absent or found themselves working with others or in different locations in the office, their “work-arounds” tended to fall apart, and problems ensued. While allowing daily work to proceed, the overall quality of the office suffers when safety depends on individual diligence.

It is interesting to note that at the four offices we studied, no one could fully describe the testing processes that existed in their offices, and there were many misconceptions about what took place in their work setting. This is in no small part due to how complex these processes are. The role of complexity in health care, and in ambulatory care, is becoming more important, especially as a factor in safety and quality.^{37, 38, 39}

The fact that workers do not know what tasks their coworkers perform is consistent with our finding that individuals at these medical offices often perform their tasks in relative isolation. Yet, teamwork is one of the strongest components of safe and successful health care units.^{40, 41}; Instruments that measure safety culture include major sections on teamwork.⁴²

Teamwork is a necessary component to move beyond the attitude that maintains, “I don’t know how others do it, but this is what I do.” Some such processes may be excellent, but when performed in isolation, without organizational system support, and surrounded by several other methods for performing the same tasks and steps, inconsistency, confusion, and error are likely outcomes. This is why standardization is a common and well-accepted tenet of effective safety practices.^{43, 44}

Moving beyond the stage of individuals working independently is important in creating a safer system because it allows individuals to better coordinate their efforts. Through better coordination, the system will change from an open-loop process to a process that includes feedback, allowing for a system that can change in response to the ever-changing circumstances that face those working in ambulatory care.

Conclusion

The testing process, a common and important function in primary care offices, can be studied using numerous methods from human factors, cognitive systems engineering, and resilience engineering. There are dozens of tasks to be performed by multiple people to complete the testing process, from an initial order through patient notification and followup. The family medicine offices we studied depended on individuals to be diligent with their memory and to do double-checks and work arounds in order to provide safe testing care. We did identify a handful of officewide testing process safety practices. These offices incorporated a learning culture to identify and remedy areas at risk, backed by a top-level commitment to continuous and extensive follow-through. Further work is needed to study additional primary care offices to see if these findings are consistent and to find and implement best practices to assist offices in moving toward increased organizational resilience.

Acknowledgments

This study was funded by grant 1 K08 HS013914-01A2 from the Agency for Healthcare Quality and Research.

References

1.

Beasley JW, Escoto KH, Karsh BT. Human factors and ergonomics in primary care. In: Carayon P, editor. Handbook of human factors and ergonomics in health care and patient safety. Mahwah, NJ: Lawrence Erlbaum; 2007.

2.

Carayon P, Schoofs Hundt A, Karsh BT, et al. Work system design for patient safety: The SEIPS model. Qual Saf Health Care. 2006;15( Suppl 1):i50–i58. [PMC free article: PMC2464868] [PubMed: 17142610]

3.

Committee on Quality of Health Care in America. Crossing the quality chasm A new health system for the 21st century. Washington, DC: National Academies Press; 2001. [PubMed: 25057539]

4.

Donabedian A. The quality of care. How can it be assessed? JAMA. 1988;260:1743–1748. [PubMed: 3045356]

5.

Feifer C, Nemeth L, Nietert PJ, et al. Different paths to high-quality care: Three archetypes of top-performing practice sites. Ann Fam Med. 2007;5:233–241. [PMC free article: PMC1886483] [PubMed: 17548851]

6.

Kohn L, Corrigan J, Donaldson M. To err is human: Building a safer health system. Washington, DC: National Academies Press; 1999. [PubMed: 25077248]

7.

Reason J. Human error: Models and management. BMJ. 2000;320:768–770. [PMC free article: PMC1117770] [PubMed: 10720363]

8.

Scott J, Tallia A, Crosson JC, et al. Social network analysis as an analytic tool for interaction patterns in primary care practices. Ann Fam Med. 2005;3:443–448. [PMC free article: PMC1466914] [PubMed: 16189061]

9.

Shortell SM, Rundall TG, Hsu J. Improving patient care by linking evidence-based medicine and evidence-based management. JAMA. 2007;298:673–676. [PubMed: 17684190]

10.

Solberg LI, Hroscikoski MC, Sperl-Hillen JM, et al. Transforming medical care: Case study of an exemplary, small medical group. Ann Fam Med. 2006;4:109–116. [PMC free article: PMC1467006] [PubMed: 16569713]

11.

Elder N, Graham D, Brandt E, et al. The testing process in family medicine: Problems, solutions and barriers as seen by physicians and their staffs. A study of the American Academy of Family Physicians’ National Research Network. J Patient Safety. 2006;2:25–32.

12.

Hickner J, Graham D, Elder N, et al. Testing process errors and their harms and consequences reported from family medicine practices: A study of the AAFP National Research Network. Qual Saf Health Care. (in press) [PubMed: 18519626]

13.

Hickner JM, Fernald DH, Harris DM, et al. Issues and initiatives in the testing process in primary care physician offices. Jt Comm J Qual Patient Saf. 2005;31:81–89. [PubMed: 15791767]

14.

Boohaker EA, Ward RE, Uman JE, et al. Patient notification and follow-up of abnormal test results. A physician survey. Arch Intern Med. 1996;156:327–331. [PubMed: 8572844]

15.

Nutting PA, Main DS, Fischer PM, et al. Toward optimal laboratory use. Problems in laboratory testing in primary care. JAMA. 1996;275:635–639. [PubMed: 8594246]

16.

Mold JW, Cacy DS, Dalbir DK. Management of laboratory test results in family practice. An OKPRN study. J Fam Pract. 2000;49:709–715. [PubMed: 10947137]

17.

Poon EG, Gandhi TK, Sequist TD, et al. “I wish I had seen this test result earlier!” Dissatisfaction with test result management systems in primary care” Arch Intern Med. 2004;164:2223–2228. [PubMed: 15534158]

18.

Poon EG, Haas JS, Louise Puopolo A, et al. Communication factors in the follow-up of abnormal mammograms. J Gen Intern Med. 2004;19:316–323. [PMC free article: PMC1492194] [PubMed: 15061740]

19.

Poon EG, Wang SJ, Gandhi TK, et al. Design and implementation of a comprehensive outpatient Results Manager. J Biomed Inform. 2003;36:80–91. [PubMed: 14552849]

20.

Astion ML, Shojania KG, Hamill TR, et al. Classifying laboratory incident reports to identify problems that jeopardize patient safety. Am J Clin Pathol. 2003;120:18–26. [PubMed: 12866368]

21.

Dovey SM, Meyers DS, Phillips RL Jr, et al. A preliminary taxonomy of medical errors in family practice. Qual Saf Health Care. 2002;11:233–238. [PMC free article: PMC1743626] [PubMed: 12486987]

22.

Elder NC, Vonder Meulen MB, Cassedy A. The identification of medical errors by family physicians during outpatient visits. Ann Fam Med. 2004;2:125–129. [PMC free article: PMC1466644] [PubMed: 15083851]

23.

Fernald D, Pace WD, Harris DM, et al. Event reporting to a primary care patient safety reporting system: A report from the ASIPS Collaborative. Ann Fam Med. 2004;2:327–332. [PMC free article: PMC1466702] [PubMed: 15335131]

24.

Phillips R, Dovey S, Graham D, et al. Learning from different lenses: Reports of medical errors in primary care by clinicians, staff and patients. J Patient Safety. 2006;2:140–146.

25.

Auerbach AD, Landefeld CS, Shojania KG. The tension between needing to improve care and knowing how to do it. N Engl J Med. 2007;357:608–613. [PubMed: 17687138]

26.

Battles JB, Lilford RJ. Organizing patient safety research to identify risks and hazards. Qual Saf Health Care. 2003;12(Suppl 2):ii2–ii7. [PMC free article: PMC1765780] [PubMed: 14645888]

27.

Bogner M. Stretching the search for the “why” of error: The systems approach. J Clin Eng. 2002:110–115.

28.

Woods D. Essential characteristics of resilience. In: Hollnagel E, Woods D, Leveson N, editors. Resilience engineering: Concepts and precepts. Aldershot, UK: Ashgate; 2006.

29.

Weick K, Sutcliffe K. Managing the unexpected. San Francisco: Jossey-Bass; 2001.

30.

Wreathall J. Properties of resilient organizations: An initial view. In: Hollnagel E, Woods D, Leveson N, editors. Resilience engineering concepts and precepts. Burlington, VT: Ashgate; 2006.

31.

West D, Westfall JM, Araya-Guerra R, et al. Agency for Healthcare Research and Quality. Advances in patient safety: From research to implementation. Vol. 3. Rockville, MD: Agency for Healthcare Research and Quality; 2005. Using reported primary care errors to develop and implement patient safety interventions: A report from the ASIPS Collaborative. [PubMed: 21249992]

32.

TransforMED^SM. Empowering medical practice. [Accessed February 19, 2008]. Available at: www​.transformed.com/

33.

Godfrey M. Clinical microsystem action guide: Improving healthcare by improving your microsystem. Hanover, NH: Dartmouth-Hitchcock Medical Center; 2002.

34.

Mohr J, Batalden P, Barach P. Integrating patient safety into the clinical microsystem. Qual Saf Health Care. 2004;13(Suppl 2):ii34–ii38. [PMC free article: PMC1765806] [PubMed: 15576690]

35.

Parnes B, Fernald D, Quintela J, et al. Stopping the error cascade: A report on ameliorators from the ASIPS Collaborative. Qual Saf Health Care. 2007;16:12–16. [PMC free article: PMC2464918] [PubMed: 17301195]

36.

Tucker A, Edmondson A, Spear S. When problem solving prevents organizational learning. J Organ Change Manag. 2001;15:122–137.

37.

Plsek PE, Greenhalgh T. Complexity science: The challenge of complexity in health care. BMJ. 2001;323:625–628. [PMC free article: PMC1121189] [PubMed: 11557716]

38.

Stange KC, Zyzanski SJ, Jaen CR, et al. Illuminating the “black box.” A description of 4454 patient visits to 138 family physicians. J Fam Pract. 1998;46:377–389. [PubMed: 9597995]

39.

Stroebel CK, McDaniel RR Jr, Crabtree BF, et al. How complexity science can inform a reflective process for improvement in primary care practices. Jt Comm J Qual Patient Saf. 2005;31:438–446. [PubMed: 16156191]

40.

Leonard M, Graham S, Bonacum D. The human factor: The critical importance of effective teamwork and communication in providing safe care. Qual Saf Health Care. 2004;13( Suppl 1):i85–i90. [PMC free article: PMC1765783] [PubMed: 15465961]

41.

Baker D, Salas E, Barach P, et al. The relation between teamwork and patient safety. In: Carayon P, editor. Handbook of human factors and ergonomics in health care and patient safety. Mahwah, NJ: Lawrence Erlbaum; 2007.

42.

Modak I, Sexton JB, Lux TR, et al. Measuring safety culture in the ambulatory setting: The safety attitudes questionnaire-ambulatory version. J Gen Intern Med. 2007;22:1–5. [PMC free article: PMC2227589] [PubMed: 17351834]

43.

Leape LL. Error in medicine. JAMA. 1994;272:1851–1868. [PubMed: 7503827]

44.

Solberg LI. Improving medical practice: a conceptual framework. Ann Fam Med. 2007;5:251–256. [PMC free article: PMC1886486] [PubMed: 17548853]