Chapter 13Preoperative Checklists and Anesthesia Checklists

Treadwell JR, Lucas S.

Publication Details

How Important Is the Problem?

Surgical operations greatly benefit the public health; however, they can also be directly responsible for substantial morbidity and mortality. In industrialized countries, the rate of perioperative death directly due to inpatient surgery has been estimated at 0.4 percent to 0.8 percent, and the rate of major complications has been estimated at 3 percent to 17 percent.1,2 Sources of these complications are numerous, including wrong-patient/procedure/site surgery, anesthesia equipment problems, lack of availability of necessary equipment, unanticipated blood loss, non-sterile equipment, and surgical items (e.g., sponges) left inside patients. The complexity of most surgical procedures requires a well-coordinated team to prevent these events.

The medical community recognizes that anesthesia has reached a high level of safety; however, with increased awareness, it is believed that the risk, particularly morbidity risk, can be further reduced.3 As an example of increasing awareness, in June 2010, the European Board of Anesthesiology (EBA) and the European Society of Anesthesiology (ESA) jointly adopted the “Helsinki Declaration on Patient Safety in Anaesthesiology.”4 Also, the journal Health Devices listed, “Anesthesia hazards due to incomplete pre-use inspection” as one of the top ten technology hazards for 2012.5

What Is the Patient Safety Practice?

Preoperative checklists can help prevent errors and complications related to surgery. Checklists are often implemented within a multifactorial strategy of interventions; therefore they usually cannot be judged alone as a patient safety practice. The World Health Organization (WHO) Surgical Safety Checklist is a prominent example of a preoperative checklist intended to ensure safe surgery and minimize complications; it has been translated into at least six languages.6 Because of its prominence and importance, the majority of our review for this PSP details the WHO checklist: its development, pilot testing, context and implementation at different sites, and degree of adoption and diffusion around the world.

In addition to the WHO checklist, we also reviewed evidence on three other types of checklists:

  • The SURPASS checklist.7-10 The checklist encompasses not only the operation itself, but all events from admission to surgery to discharge
  • Checklists specifically intended to prevent wrong-site surgery. Two items on the WHO checklist address wrong-site surgery (“Has the patient confirmed his/her identity, site, procedure, and consent?” and “Is the site marked?”). In 2004, the Joint Commission created the Universal Protocol for Preventing Wrong Site, Wrong Procedure, Wrong Person Surgery.11 It comprises three sets of steps: pre-operative verification process, marking the operative site, and a “time out” immediately before the operation. A checklist can potentially be used to clarify the details of these three steps. The Universal Protocol is intended to prevent wrong surgery not just in the operating room but anywhere an invasive procedure is performed (e.g., interventional radiology unit).12
  • Checklists specifically intended to check anesthesia equipment. The WHO checklist also contains a specific item about preoperative anesthesia (“Is the anaesthesia machine and medication check complete?”). This single item could itself be addressed by a sub-checklist. In 2008, the American Society of Anesthesiologists provided general guidelines about items that should be checked before surgery, and institutions can implement the guidelines to tailor the checklist to their specific equipment and clinical settings.13

Checklists have also been developed, implemented, and assessed outside of the realm of surgery. The Michigan ICU checklist (also referred to as the Keystone project) has been shown to prevent central line-associated bloodstream infections (CLABSI).14,15 This program involved a multifactorial intervention at 108 Michigan ICUs. Data showed a reduction in CLABSI from 7.7 infections per 1,000 catheter-days before the program to only 1.4 infections per 1,000 catheter-days at 16 to 18 months after program initiation follow-up. A 2001 study by Dixon-Woods and colleagues15 proposed six reasons for this reduction, including “creating a densely networked community with strong horizontal links that exerted normative pressures on members” and “harnessing data on infection rates as a disciplinary force.” A recent systematic review of this program (and other PSPs to prevent hospital-associated infections) was conducted by the Blue Cross and Blue Shield Evidence-based Practice Center; please refer to that report for further information about the Keystone project.16

Background Information About Preoperative Checklists

In January 2007, the WHO Patient Safety group started work on the Second Global Patient Safety Challenge: Safe Surgery Saves Lives. This group of international experts identified four areas of potential improvement in surgical safety: surgical site infection prevention, safe anesthesia, safe surgical teams, and measurement of surgical services.17 Based on that work, in early 2008, the WHO published a guideline for safe surgery.18 This guideline was used as the basis for the WHO Surgical Safety Checklist, which was launched in June 2008.

The checklist, which was included as a Supplementary file in the original publication,6 contained 19 items in three phases with collaborative involvement of the surgeon, the anesthetist, and the nursing team:

  • Before induction of anesthesia (“Sign In”), covering areas such as patient identification, anesthesia equipment check, and a pulse oximetry check
  • Before skin incision (“Time Out”), covering areas such as team introductions, review of critical steps, and antibiotic prophylaxis
  • Before patient leaves operating room (“Sign Out”), covering areas such as checking counts of instruments, specimen labeling, and concerns for recovery

The SURPASS checklist (SURgical PAtient Safety System)7-10 is intended to address any events that occur between patient admission and discharge. Thus, it encompasses more potential areas of safety than the WHO checklist, which is focused only on the operating room. An estimated 53 percent to 70 percent of surgical errors occur outside the operating room.8,19,20 Within the operating room itself, the SURPASS checklist is less specific than the WHO checklist (for example, the SURPASS checklist does not specifically mention any of the following: pulse oximetry, difficult airway, risk of blood loss (although it asks whether blood products are available), team introductions, and anticipation of critical events).

In January 2004, the Joint Commission launched the first version of the Universal Protocol (UP) for Preventing Wrong Site, Wrong Procedure, Wrong Person Surgery.11,21 It comprises three sets of steps: pre-operative verification process, marking the operative site, and a “time out” immediately before the operation. The preoperative verifications (of person, procedure, and site) are supposed to occur not only in the operating room, but also (if applicable) when the procedure is scheduled, when the patient enters the health care facility, and anytime care is transferred between caregivers. Site marking should involve only the operative site and should be visible before the patient is draped. The “time out” is to occur before incision and involve the entire operating room team. The Universal Protocol is not a checklist,12 but it could be implemented using one or more checklists. Both steps 1 and 3 specifically mention the potential use of a checklist.

Anesthesia safety guidelines and standards are actively reviewed and modified globally through organizations such as the WHO and the World Federation of Societies of Anaesthesiologists (WFSA).22 The latest WFSA standard, which was developed as part of the “Safe Surgery Saves Lives” project, recommends that an appropriate “pre-list check” be performed prior to the start of each operating list and an appropriate “pre-patient check” be performed prior to each anesthetic. In addition, individual anesthesia societies are developing guidelines for pre-anesthesia checks, including the American Society of Anesthesiologists (ASA) and the Association of Anaesthetists of Great Britain and Ireland (AAGBI). The latest U.S. pre-use checkout guidelines, entitled “Recommendations for Pre-Anesthesia Checkout (PAC) Procedures,” were published in 2008 by the ASA.13 These guidelines were a result of a multi-year effort by an ASA task force consisting of members from the ASA, the American Association of Nurse Anesthetists (AANA), the American Society of Anesthesia Technicians and Technologists (ASATT), and major anesthesia system manufacturers. The latest AAGBI revision was published in 2004 and has been adopted by many institutions around the country.23 Similar to the WFSA guideline checklist, the full ASA and AAGBI checklists were designed to be used at the start of the day with a subset of the full checklists performed prior to each procedure. These societies' sample checklists were developed as a basis for institutions to develop their individual checklists.

Additional background information about preoperative checklists, including how they were developed and modified, and the overlap between different checklists, appears in Appendix A.

Why Should This Patient Safety Practice Work?

No formal “model” exists for why preoperative checklists should reduce surgical errors, but studies have cited several common reasons. These reasons include ensuring that all critical tasks are carried out, encouraging a non-hierarchical team-based approach; enhancing communication; catching near misses early, anticipating potential complications, and having technologies to manage anticipated and unanticipated complications. With regard to anesthesia checklists, Staender and Mahajan3 attribute the reduced anesthesia-related mortality rates to a combination of interventions, including incident reporting, simulations, and checklists.

What Are the Beneficial Effects of the Patient Safety Practice?

In this section, the primary issues surrounding checklists involve implementation, rather than whether they are effective. Consequently, we briefly summarize the primary results, and the bulk of our work appears later in detailed assessments of the implementation efforts.

World Health Organization Checklist

The 2008 WHO Surgical Safety Checklist was tested at eight sites (Prince Hamzah Hospital in Amman, Jordan; St. Stephen's Hospital in New Delhi, India; University of Washington Medical Center in Seattle, U.S.; St. Francis Designated District Hospital in Ifakara, Tanzania; Philippine General Hospital in Manila, Philippines; Toronto General Hospital in Toronto, Canada; St. Mary's Hospital in London, England; and Auckland City Hospital in Auckland, New Zealand).6 These settings varied greatly in the number of beds (range 371 to 1800), the number of operating rooms (range 3 to 39), and the income level of the country (four low, four high). Surgical safety policies prior to implementation of the WHO Checklist also differed regarding the use of routine intraoperative monitoring with pulse oximetry (six of eight sites), oral confirmation of patients' identity and surgical site in the operating room (only two of eight sites), and routine administration of prophylactic antibiotics in the operating room (five of eight sites). None of the eight sites had a “standard plan for intravenous access for cases of high blood loss,” or formal team briefings preoperatively or postoperatively.

Baseline data were obtained at each site for 3 months prior to checklist introduction, involving a total of 3,733 surgical procedures. In the subsequent 3- to 6-month period after checklist introduction, involving 3,955 procedures, data showed decreases in patient mortality (from 1.5% to 0.8%) and inpatient complications (from 11% to 7%). No single site was driving the findings, as evidenced by the persistence of findings after the removal of any single site in a sensitivity analysis. Authors found that the performance rates for six specific safety indicators (e.g., using a pulse oximeter) also increased after checklist introduction, suggesting that the safety indicators may have been responsible for the lower rates.

In discussing the results, authors acknowledged that the underlying reasons for the improvements were “most likely multifactorial” and included explanations such as the following:

  • The checklist itself
  • A Hawthorne effect (i.e., rates may have decreased because operating room personnel knew they were being measured). The authors argued against this possibility based on two aspects of their data: (1) that this knowledge was in place both before and after checklist introduction, and (2) the subset of procedures for which study personnel were present in the operating room had the same reductions in complications as procedures where study personnel were absent from the operating room.
  • The simple existence of a formal pause or preoperative briefing (which could be done without a “checklist”). Such a pause is a necessary component of the checklist.
  • Increased uptake of safety technologies (e.g., administering antibiotics in the operating room rather than in preoperative wards). This change could be considered a byproduct of checklist introduction (i.e., hospitals made more antibiotics directly available in the operating room because of the presence of an antibiotics-related item on the checklist)
  • A broad change in safety culture and teamwork at that site, an explanation supported by the finding that greater increases in safety attitudes at the pilot sites were associated with greater reductions in complications.24

In a subsequent, 2010, publication, Weiser and colleagues25 presented a subgroup analysis of the 2009 NEJM publication that was focused on urgent surgery (defined as surgery required to be performed within 24 hours of assessment in order to be beneficial). Complications dropped from 18 percent in the pre-intervention phase to 12 percent in the post-intervention phase, and death dropped from 3.7 percent to 1.4 percent. Also, a 2011 study by Haynes and colleagues24 reported data on the Safety Attitudes Questionnaire (SAQ) in the eight pilot sites before and after checklist introduction. The SAQ is scored on a 1 to 5 scale, where a 5 represents the most safety-conscious attitude. Scores on the SAQ were only slightly higher in the phase after checklist introduction than before introduction (4.01 vs. 3.91, representing an increase of only 2.5 percent of the scale range; this small difference was nevertheless statistically significant). However, the change in SAQ scores was associated with reduced complication rates (Pearson r=0.71), meaning that sites with greater improvements in safety attitude tended to have greater reductions in complications. The publication also reported that 80 percent of respondents considered the checklist easy to use, 20 percent believed it took too long, and when respondents were asked if they would want the checklist used if they were undergoing surgery, 93 percent said yes.

SURPASS Checklist

An empirical test of the 90-item SURPASS checklist was reported in a 2010 study by DeVries and colleagues.7 The design was a 6-month interrupted time series with concurrent controls; six hospitals using the checklist were matched with five other hospitals that did not, and researchers measured the rates of surgical complications in both groups. The 11 hospitals were distributed through the Netherlands and comprised six tertiary hospitals, three academic hospitals, and two regional hospitals; numbers of beds per hospital ranged from 380-1002. These hospitals had already been measured for their safety performance, so the potential Hawthorne effect is lower than it would have been in hospitals just starting to be measured. Regarding implementation, authors stated that the SURPASS checklist involved extensive time and effort. A random sample of cases generally revealed good compliance with the checklist (median 80%).

The 3-month period after the checklist was initiated (compared with the 3 months before) saw numerous improvements: decreases in the percentage of patients with complications (from 15% to 11%), in-hospital mortality (1.5% to 0.8%), patient temporary disability (9.4% to 6.6%), and reoperations (3.7% to 2.5%). No such improvements were found among the control hospitals. Interestingly, the extent of improvement was associated with greater compliance with the checklist: the 566 patients whose surgery involved greater checklist compliance had 7.1 complications per 100 patients, which was considerably lower than the 18.8 per 100 experienced by the 580 patients whose surgery involved less checklist compliance. This finding provided greater confidence that the checklist itself was the reason for the improvements. A subsequent retrospective review of 294 medical claims10 estimated that 40 percent of deaths and 29 percent of liability incidents might have been prevented if the SURPASS checklist had been used.

Wrong-Site Surgery Checklists

Wrong-site surgery is relative rare: Estimates for various procedures range from 1 in 13,000 procedures for wrong-site anesthesia block to 1 in 4,200 for wrong-side ureteral stents.26 A general systematic review estimated that the overall rate was 1 to 5 per 10,000 procedures.27 Given the rarity, demonstrating a statistical reduction would require an unfeasibly large study. A systematic review searched for literature and concluded there was “no literature to substantiate the effectiveness of the current JC [Joint Commission] Universal Protocol in decreasing the rate of wrong site, wrong level surgery.”27 Therefore, the preventive benefits of a checklist to prevent wrong-site surgery, are generally assumed based on clinical expertise.

Anesthesia Equipment Checklists

In evaluating beneficial effects, the same limitations apply to anesthesia checklists as apply to wrong-site surgery checklists. The rate of mortality associated with malfunction of anesthesia equipment is 1:100,000, and the rate of severe morbidity ranges between 1:170 and 1:500.3 Future research may be possible to evaluate the severe morbidity rate; however, addressing the benefits on mortality would require an unfeasibly large study.

A 2000 randomized cross-over study by Blike and Biddle28 compared the effectiveness of the 1994 hard-copy version of the FDA-approved AACR to a researcher-designed electronic checklist. Machine faults were purposely entered into an Ohmeda Modulus II Plus Anesthesia System. Participants using the electronic checklist were first given a researcher-drafted “philosophy of anesthesia apparatus checkout,” which outlined basic strategies to reduce anesthesia apparatus-related patient injury. They reported that the electronic checklist greatly improved the detection of prearranged anesthesia equipment faults. For 19 of the 20 faults studied, the electronic checklist was either equal or superior to the AACR. However, the electronic checklist missed 30 percent of the “difficult” faults (e.g., breathing circuit leak). While this percentage was better than when the AACR was used (60%), it is still substantial. Studies like these provided the basis for revising the AACR. For additional references to the effectiveness of the AACR, we refer the reader to the 2008 ASA guideline; most of this literature was published prior to 2000.

Ben-Menachem and colleagues29 performed a simulation study, published in 2011 that used the 2008 ASA guideline to measure the performance of anesthesia residents of Sheba Medical Center (Israel). The residents were instructed to complete the ASA checklists during simulation-based scenarios, which included two pre-arranged equipment failures. The study showed that 25 of 28 participants correctly performed 70 percent or more of the items on the checklist that is used before the first-morning case, and 27 of 30 participants correctly performed 70 percent or more of the items on the between-case checklist. Regarding the pre-arranged equipment failures, 30 of 31 participants identified O2 supply and pressure alarms and 30 of 30 participants recognized an abnormal capnograph waveform.

What Are the Harms of the Patient Safety Practice?

Direct harms of preoperative checklists have not been reported. In 2011, Sewell30 reported that after WHO implementation, the rate of lower respiratory tract infections actually increased from 2.1 percent to 2.5 percent. Whether this increase was caused by the checklist is unclear; however the authors attributed rate reductions to the checklist, so they could also have attributed rate increases to the checklist. In 2011, Kearns31 reported that 3 months after WHO checklist implementation, 30 percent believed it was an inconvenience in emergency cases; however, this percentage was lower than it had been prior to implementation of the checklist when staff were asked hypothetically whether they believed it would be an inconvenience in emergency cases (53 percent said it would be). In 2010, Thomassen and colleagues32 reported user experiences with their pre-induction anesthesia checklist. In this qualitative study, focus group interviews were conducted amongst the participating nurses and physicians. Users reported that checklist use could divert attention from the patient and that it interfered with doctor-nurse workflow, although the latter improved with increased use.

How Has the Patient Safety Practice Been Implemented, and in What Contexts?

World Health Organization Checklist

In 2011, one of the eight pilot sites that piloted the WHO Checklist reported checklist-related opinions of surgical team members 18 months after checklist introduction.33 Team members reported high levels of agreement with the questions “Do you think the use of the checklist has improved patient safety?” “Are you comfortable in reminding other members of the team to carry out the checklist?” “If you were to undergo surgery would you want the checklist to be used?” and “Do you think that use of the checklist generally has improved communication among members of the Operating Room team?” Team members generally estimated that it took about two minutes to complete the checklist.

We identified nine reports of the implementation of the WHO checklist at other sites; implementation details at each site appear in Table 1 in Appendix D in the section titled “Evidence Tables for Chapter 13.” Eight studies used the 2008 WHO checklist as a basis, and one did not say which version was used. Six studies modified the WHO checklist, according to either surgical specialty (three studies) or country (three studies). Of six studies that modified the checklist, five provided their modified checklist within the paper, and of these five, four included all of the WHO items and one did not (they had deleted some items).

Six studies were case series, and three were before-after studies. Regarding a theory or logic model, eight of nine provided some statements about why a checklist should work to reduce complications (e.g., “Checklists may be used to improve patient safety by ensuring that all elements of a practice are instituted for each new clinical event”).31 Six studies were conducted in the UK, two in the U.S., and one in Finland. Four studies involved surgical specialties (pediatrics, OB-GYN, orthopedics, and otorhinolaryngology), and the other five were general surgery.

Only two studies reported on the pre-existing safety infrastructure: one stated that a core group of patient safety experts was in place before checklist implementation, and another stated that a hospital quality infrastructure had been in place for five years prior to implementation. Two studies reported information on the pre-existing safety culture, and they both measured staff attitudes specifically about checklist-related items. In one, some safety aspects were fairly good (knowledge of OR-teams' names and roles, the rate of recording of postoperative follow-up instructions, and overall successful communication range from 61% to 93%); however the rate of discussing risks was only 24 percent. In the other study, most respondents (81% to 85%) believed that communication in the operating room could improve and that for elective surgery the checklist would be useful) and only 31 percent already felt familiar with other operating room team members. However, for emergency surgery, a slight majority (53%) believed that introducing the checklist would be inconvenient. All of these opinions were hypothetical as they were solicited before checklist introduction.

The results of the nine implementations appear in Table 2 in Appendix D in the section titled “Evidence Tables for Chapter 13.” Regarding checklist training, three sites mentioned educational sessions, three used posters in the operating room, two mentioned a hospitalwide publicity campaign, two mentioned that training was provided (however, no details were provided), and two either failed to mention training or stated that only limited training was provided. Four studies mentioned a pilot testing period; these pilot tests lasted 1 to 3 months and often involved minor modifications to the checklist. Three studies reported the degree of compliance with the checklist; one simply reported 97 percent compliance, and the other two reported improvement over time (from approximately 60% to approximately 80% in one study, and from 85% to 95% in another study). One study reported that it took about two minutes to complete the checklist, and that 20 percent of respondents believed it caused an unnecessary time delay.

Feedback from surgical teams was generally positive, but support tended to be greater from nurses and anesthetists than from surgeons. Two studies reported increases in certain attitudinal variables such as the degree to which people felt familiar with others in the operating room, the quality of communication, the anticipated safety of patients, and the usefulness of the checklist in either elective or emergency cases. Behaviorally, one study reported that after 3 months, team briefings were occurring in 77 percent of operations and time-outs in 86 percent. Another study reported improvements in anesthetists' knowledge about patients, their check of anesthesia equipment, and staff knowledge of patient identity/procedure/site.

Reasons cited for success included good training and staff understanding, a local champion, support from upper management, being able to modify the checklist, distribution of responsibility, the feeling of ownership by team members, and enhanced communication and teamwork. Barriers to implementation included general surgeon resistance to changing habits, the belief that they were already checking those things, awkwardness of self-introductions, steep interpersonal hierarchy, and a fear of legal responsibility if a complication occurred after they had signed a form.

One ongoing research project, funded by the Agency for Healthcare Research and Quality, is entitled “Factors Associated with Effective Implementation of a Surgical Safety Checklist.”34 This 2010-2013 project “will examine implementation processes in a large group of U.S. and international hospitals to identify factors supportive of effective implementation.” Further, the team will determine how teamwork helps explain the impact of the checklist.

The WHO Web site (www.who.int/patientsafety/safesurgery/en/) provides advice to hospitals for implementing the checklist.35 This advice includes statements such as “The key to successful implementation is to start small. Start with a single operating room on day 1 and see how it works. This will guide you to strategies for altering the checklist to fit your needs, as well as identify potential barriers to adaptation.”35 Other implementation advice from WHO is available in the Frequently Asked Questions section (www.who.int/patientsafety/safesurgery/faq_introduction/en/index.html), the 20-page “Starter Kit for Implementing the Surgical Safety Checklist” (www.who.int/patientsafety/safesurgery/testing/participate/starter_kit-sssl.pdf), and the “Checklist Adaptation Guide” (www.who.int/patientsafety/safesurgery/checklist_adaptation.pdf). Regarding checklist modification, the Web site states, “Do not hesitate to customize the checklist for your setting as necessary, but do not remove safety steps just because you are unable to accomplish them.” Also, regarding feasibility, the WHO states that “It should take no more than a minute to complete each section of the checklist” (i.e., three minutes in total). The pilot study reported that at various sites, introduction of the checklist took only 1 week to 1 month.6

The Web site www.safesurg.org also provides additional materials relevant to the WHO checklist. Those interested in implementing the checklist are encouraged to register with the Web site. The Web site provides a template for the 2009 checklist (www.safesurg.org/template-checklist.html) in Microsoft Word format (Microsoft Corporation, Redmond, Washington, U.S.), and an implementation manual is available (www.safesurg.org/implementation-manual.html). The Web site also provides a list of other institutions' modified checklists (www.safesurg.org/modified-checklists.html), where institutions can submit their modifications of the WHO checklist to be made publically available. On October 3, 2011, the publically-available list contained 79 checklists from 25 countries.

The site also provides several downloadable videos (www.safesurg.org/videos.html): one on how to use the WHO checklist; one on how not to use it; two from the National Patient Safety Agency in the United Kingdom; one from University Health Network Hospital in Toronto; one from the Surgical Care and Outcomes Assessment Program in Washington State; two from Great Ormond Street Hospital in the United Kingdom; one in French from Fattouma Bourguiba Hospital in Monastir, Tunisia; one in Spanish from la Agencia de Calidad de Andalucia; one Spanish translation of the WHO how-top video; and two from Auckland City Hospital in New Zealand.

SURPASS Checklist

We performed a citation search to determine if the SURPASS checklist has yet been attempted outside the Netherlands; however, no such attempts were identified. The SURPASS Web site (www.surpass-checklist.nl/home.jsf?lang=en) describes an electronic version of the checklist (called SURPASS Digital) that can be used by any web-connected computer. The electronic version allows one to modify the checklist, although the designers of SURPASS strongly encourage users to avoid modification (www.surpass-checklist.nl/content.jsf?pageId=FAQ&lang=en).

Wrong-Site Surgery Checklists

No implementation advice was found on the Joint Commission Web site or in other published documents. In August 2010, the Joint Commission conduced an online survey of over 2,100 people.36 The Web site did not report how many questions were asked or the wording of any given question. The Web site reports five findings from the survey: 1) 88 percent agreed that their organizations could fully implement the Universal Protocol; 2) 87 percent to 92 percent agreed that the three steps are appropriate; 3) More than 90 percent agreed that “there is benefit” in using it in the operating room, ambulatory surgery, and hospital units performing invasive procedures, but the rates of agreement of benefit were lower for ambulatory clinics and physician offices; 4) the need to modify policies and procedures varied greatly across respondents; and 5) no differences were found between different types of respondents (e.g., type of hospital, bed size).

We identified four sites describing pertinent checklists (see Table 3 in Appendix D in the section entitled “Evidence Tables for Chapter 13”). These sites were located in Switzerland, Sweden, the United Kingdom, and North Carolina:

The Swiss study37 was conducted in a large anesthesiology service and focused on verifying two key aspects: patient identity and surgical site. The protocol was developed by an interdisciplinary team and required patient participation in the verification of identity and surgical site (answering open-ended questions rather than closed-ended questions). Compared with the first 3 months of implementation, the next 3 months saw better compliance in checking patient identity (63% up to 81%) and proportion of surgical site checks performed (77% up to 93%). Compliance was stable in subsequent periods. Barriers to implementation included 1) surgeons saying they already knew the patients or the surgical site was obvious, and 2) the failure to include the input of all surgical services in developing the protocol.

The Swedish study38 involved two hospitals, each of which had a recent wrong-site surgery incident, and a root-cause analysis suggested that a time-out procedure might help. A time-out checklist was implemented, and one year later, a questionnaire was sent to all 704 team members. Of the 331 responders, 93 percent expressed the belief that the checklist contributes to increased patient safety (either “without a doubt,” or “probably”). When asked about eight specific components of the time-out checklists, the percentage of respondents who believed the component was “very important” varied widely, from a low of 14 percent for the introduction of team members to highs of over 80 percent for patient identity, correct procedure, and correct side.

The English study39 was conducted at a children's hospital in which staff had incorporated an eight-item correct-site surgery checklist into an existing preoperative checklist. Five people were required to sign the documentation: marking surgeon, operating surgeon, ward nurse, scrub nurse, and anesthetist. Comparing 2008 to 2006, correct completion was unimproved for four of the eight items (ward nurse signed, operating surgeon signed, scrub nurse, signed, and operating department practitioner signed) but was improved for the other four (mark site documented, no mark required documented, entries legible, and marking surgeon signed).

The North Carolina study37 implemented a checklist to prevent wrong-site surgery that was tailored to the hospital's preferences and procedures. Previously, the staff was using a cumbersome form to document their compliance with the Universal Protocol. Champions demonstrated the checklist during educational staff meetings, and new staff were given a primer. Staff commented favorably that they no longer had to remember everything.

The Association of PeriOperative Registered Nurses (AORN) Comprehensive Surgical Checklist (www.aorn.org/uploadedImages/Images/Images/comprehensive_surgical_checklist_RGB961.jpg) was a collaborative effort between AORN, the developers of the WHO checklist, and the Joint Commission. The Web site states that the checklist, created in April 2010, was “created to support a facility's need to use a single checklist that includes the safety checks outlined in the World Health Organization's (WHO) Surgical Safety Checklist, while also meeting the safety checks within The Joint Commission's Universal Protocol in order to meet accreditation requirements.” Our searches identified no empirical studies of this checklist.

Another combined checklist (called a “crosswalk”) combining the WHO checklist and the Universal Protocol was published in November 2011 by the Pennsylvania Patient Safety Reporting System.40 This document also addresses checking preparedness for surgical fires, as well as two intraoperative checks specifically for spinal surgery involving precise locations. Due to the recency, no studies exist yet on this crosswalk.

Anesthesia Equipment Checklists

The ASA guidelines identify 15 items: 7 to be performed only before the first procedure of the day, and 8 to be performed prior to each procedure. Similarly, the AAGBI guideline recommends that 11 items be checked prior to each operating session and that 3 of these items are to be checked again prior to each new patient procedure. These guidelines are to be implemented by individual hospitals and tailored to their departmental needs. As stated by the ASA on their Web site (ASA, 2011), “the updated recommendations are intended to serve as general guidelines for individual departments and practitioners to design pre-anesthesia checkout procedures specific for the delivery systems and the needs of the local practice.” Further, they state, “Guidelines are systematically developed recommendations that assist the practitioner and patient in making decisions about health care. These recommendations may be adopted, modified, or rejected according to clinical needs and constraints and are not intended to replace local institutional policies. In addition, practice guidelines are not intended as standards or absolute requirements, and their use cannot guarantee any specific outcome. Practice guidelines are subject to revision as warranted by the evolution of medical knowledge, technology, and practice. They provide basic recommendations that are supported by a synthesis and analysis of the current literature, expert opinion, open forum commentary, and clinical feasibility data.”41

The ASA encourages institutions to submit their version of the Pre-Anesthesia Checkout (PAC) for publication on the ASA Web site (www.asahq.org/For-Members/Clinical-Information/2008-ASA-Recommendations-for-PreAnesthesia-Checkout/Sample-Procedures.aspx). Currently, sample PACs are posted for the following anesthesia system models: (1) General Electric AESTIVA®, (2) Draeger Apollo, (3) Draeger Narkmoed GS, (4) Draeger 6000, (5) Draeger B/C/GS, and (6) Draeger Fabius GS. Eight U.S. hospitals are currently represented on the ASA collection of sample checkouts.

As an additional international example of implementing anesthesia checklist guidelines, the Columbian 2009 version of their “Minimum Safety Standards in Anaesthesia” states that anaesthesiologists and surgeons must collaborate in completing an overall check list, which is to include at least the items in the WHO checklist. In addition, before applying anaesthetic, the anaesthesiologist must complete a pre-anaesthetic checklist.42

In referencing earlier implementation strategies for aviation checklists, a 2000 article by Blike and Biddle28 propose the “three P's” for successful implementation of their anesthesia machine electronic checklist. They refer to the three P's as “a guiding philosophy, with procedures designed to achieve the goal of the philosophy using consistent policies for implementation.” They concluded that the earlier AACR was deficient in that the associated published checklist had no supporting philosophy.

Regarding staffing, the 2008 ASA guidelines identify particular aspects of the PAC that could be performed by a qualified anesthesia and/or biomedical technician. However, “regardless of the level of training and support by technicians, the anesthesia care provider is ultimately responsible for proper function of all equipment used to provide anesthesia care.”13

Are There Any Data About Costs?

Costs of implementing a checklist mostly involve checklist development (or checklist modification if the WHO checklist is used), formal staff notification that use of the checklist is expected, staff training, and additional operating room time. In 2010, Semel and colleagues43 performed a hypothetical decision analysis of checklist introduction in a U.S. hospital. The cost of implementing the checklist was estimated using the “opportunity cost of the work that would have otherwise been performed by the three department checklist champions and the implementation coordinator,” which was an estimated $12,635 in 2008 dollars; per-use cost was only $11. The cost of a major surgical complication was estimated at $13,372. In the base-case analysis, checklist introduction actually saved money. Regarding time, Sewell 201130 reported that 20 percent of staff thought the WHO checklist caused an unnecessary time delay. However, in 2011, Taylor and colleagues33 reported that the WHO checklist took only about two minutes on average.

With regard to operating room time, a 26-item anesthesia checklist developed in 2010 by Thomassen and colleagues44 was completed with a median time of 88.5 seconds (n=502 patients). Additionally, when cases were compared before and after implementation, checklist completion did not cause any significant difference in pre-induction time (25.1 vs. 24.3 minutes).

An additional potential cost benefit relates to reduced litigation claims. With regard to anesthesia, comparing the period prior to the1990s to the period from 1990 to 2003, the proportion of claims with substandard care decreased (from 39% to 22%), and payments were made less frequently (from 58% to 42% of the time).3

Are There Any Data About Adoption and Diffusion of This Patient Safety Practice?

On February 22, 2012, the WHO's Surgical Safety Web Map indicated that as of February 1, 2012, 4,120 hospitals had expressed interest in using the checklist and 1,790 of these hospitals have used the checklist in at least one operating theatre (Figure 1). On the map, red crosses represent those expressing interest, and yellow crosses represent previous/current users.

Figure 1 is a screenshot taken on February 22, 2012 of the WHO Surgical Safety Web Map. It depicts a global map on which red crosses represent hospitals that have expressed interest in using the WHO Surgical Safety Checklist (as of February 1, 2012); yellow crosses represent hospitals that have used the checklist in at least one operating theatre. A panel to the right of the map provides a key to additional symbols representing local endorsing organizations, international endorsing organizations, pilot sites, and countries with nationwide implementation. Permission for reusing the image was granted by the World Health Organization.

Figure 1, Chapter 13

Screenshot of adoption and diffusion of the WHO surgical safety checklist. Note: This figure is a screenshot taken on 2/22/2012 of the WHO Surgical Safety Web Map (http://maps.cga.harvard.edu:8080/Hospital/). Red crosses represent hospitals who have expressed (more...)

Our searches found that a number of professional organizations have recommended adoption of the WHO checklist (Table 1).

Table 1, Chapter 13. Governmental and nongovernmental organizations adopting or recommending adoption of the WHO checklist.

Table 1, Chapter 13

Governmental and nongovernmental organizations adopting or recommending adoption of the WHO checklist.

The webcast event “Check a Box, Save a Life” was launched on October 22, 2009 to promote the use of the WHO checklist.53 The event, run mostly by medical students, involved 182 hosting sites from 121 medical institutions and an estimated 1,400 online viewers. A Facebook page had enrolled 111 medical students who agreed to host the event at their institutions. At the Institute for Healthcare Improvement forum 6 weeks later, 15 case reports were presented that detailed checklist-related projects.

In January 2010 in the UK, just before the mandatory requirement to use the WHO checklist was instituted by the NPSA, Sivathasan and colleagues54 conducted telephone interviews with 238 hospitals in the UK (randomly selected from some 540 hospitals, therefore representing about 44% of UK hospitals). Almost all (99%) of the hospitals had heard of the checklist, and its use was already compulsory in 65 percent of them. In hospitals where it was not required, 81 percent used it voluntarily, and 75 percent had a plan to make it mandatory in the future. However, some operating rooms reported partial use of the checklist, i.e., intentionally skipping items or skipping the entire checklist because of time constraints.

In June 2009, the journal OR Manager received online data from 136 subscribers regarding use of the WHO checklist.55 About half (48.5%) said they had implemented the checklist, and 64 percent said the checklist has improved safety in the operating room. However, 11 percent of respondents stated that the checklist was not well accepted by surgeons, and another 63 percent said surgeons did accept it but “with reservations.” Nurses were believed to have a somewhat greater degree of acceptance, with only 2 percent “not well accepted” and 52 percent “accepted with reservations.”

A survey in October/November 2009 of 12 oral and maxillofacial consultants in Yorkshire England found that all were aware of the WHO checklist, but only 5/12 were actually using it.56 Ten of 12 expressed the belief that it would improve patient safety, but four of 12 said it would not improve team communication.

Regarding the Universal Protocol, accredited hospitals are required to comply. Therefore the “diffusion” of the Universal Protocol is large, by mandate. However, as stated earlier, the Universal Protocol is not a checklist. We found no published information on how many hospitals actually use a checklist in their efforts to comply.

Regarding anesthesia checklists, in 2009, Winters and colleagues57 at Johns Hopkins University republished the AAGBI checklist and discussed resistance of physicians to adopting anesthesia checklists in general. They cited the cases of some physicians who claimed to be insulted, whereas others expressed doubt that a checklist will prevent a medical mistake. They counter this argument by mentioning the complexity of modern medicine, which may inadvertently introduce devastating risks. In 2000, Thomassen and colleagues44 of Haukeland University Hospital in Norway developed an anesthesia checklist designed to identify “pre-induction deficiencies” (i.e., missing equipment or inadequate preparation). The checklist was improved over the course of 502 inductions. They reported that in 17 percent of the cases, missing items were identified, the most critical being lack of availability of a second laryngoscope, the introducer not having been fitted to the endotracheal tube, the endotracheal tube cuff not having been tested, and no separate ventilation bag available. Thomassen's 2010 study32 reported user experiences: Some of the senior physicians were skeptical of the usefulness of the checklist. They concluded that the success of implementation of the checklist depends on physician leaders having a positive attitude. The checklist itself improved confidence in unfamiliar contexts (see Table 4, Chapter 13 in Appendix D).

Conclusions and Comment

Several prominent authorities in the field of patient safety have proposed checklists in an attempt to prevent mistakes related to surgery. These checklists have been developed carefully by experts in the field, and have evolved over time to capture only the most essential considerations. Numerous implementation issues remain, including how to modify a given checklist to a specific hospital setting, or to a specific anesthesia system, or to a specific surgical staff. A recurrent theme in the literature on preoperative checklist is the explicit encouragement of a team-based approach. Further adoption and diffusion of these checklists will depend on the continued demonstration of effectiveness in preventing errors, checklist modifications to improve clarity and prevent misuse, proof that the benefits are worth the added time and cost, and flexibility to changes as needs arise. A summary table is located below (Table 2).

Table 2, Chapter 13. Summary table.

Table 2, Chapter 13

Summary table.

References

1.
Gawande AA, Thomas EJ, Zinner MJ, et al. The incidence and nature of surgical adverse events in Colorado and Utah in 1992. Surgery. 1999 Jul;126(1):66–75. [PubMed: 10418594]
2.
Kable AK, Gibberd RW, Spigelman AD. Adverse events in surgical patients in Australia. Int J Qual Health Care. 2002 Aug;14(4):269–76. [PubMed: 12201185]
3.
Staender SE, Mahajan RP. Anesthesia and patient safety: have we reached our limits. Curr Opin Anaesthesiol. 2011 Jun;24(3):349–53. [PubMed: 21330916]
4.
Helsinki declaration on patient safety in anesthesiology. Actual Anestesiol Reanim. 2010 Jul;20(3):136–9.
5.
ECRI Institute. Guidance article: top 10 technology hazards for 2012. Health Devices. 2011 Nov;40(11):358–73. [PubMed: 23444580]
6.
Haynes AB, Weiser TG, Berry WR, et al. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med. 2009;360(5):491–9. content​.nejm.org/cgi​/content/full/NEJMsa0810119. [PubMed: 19144931]
7.
de Vries EN, Prins HA, Crolla RM, et al. Effect of a comprehensive surgical safety system on patient outcomes. N Engl J Med. 2010 Nov 11;363(20):1928–37. [PubMed: 21067384]
8.
de Vries EN, Hollmann MW, Smorenburg SM, et al. Development and validation of the SURgical PAtient Safety System (SURPASS) checklist. Qual Saf Health Care. 2009 Apr;18(2):121–6. [PubMed: 19342526]
9.
de Vries EN, Dijkstra L, Smorenburg SM, et al. The SURgical PAtient Safety System (SURPASS) checklist optimizes timing of antibiotic prophylaxis. Patient Saf Surg. 2010 Apr 13;4(1):6. [PMC free article: PMC2867812] [PubMed: 20388204]
10.
de Vries EN, Eikens-Jansen MP, Hamersma AM, et al. Prevention of surgical malpractice claims by use of a surgical safety checklist. Ann Surg. 2011 Mar;253(3):624–8. [PubMed: 21209590]
11.
Facts about the universal protocol. [internet]. Oakbrook Terrace (IL): The Joint Commission; 2004. [2008 Jan 9]. [1 p]. www​.jointcommission.org​/assets/1/18/Universal​%20Protocol%201%204%20111.PDF.
12.
Using surgical checklists to comply with the universal protocol. Jt Comm Perspect Patient Saf. 2011 Jul;11(7):9–11.
13.
ASA Committee on Equipment and Facilities. 2008 recommendations for pre-anesthesia checkout procedures. Park Ridge (IL): American Society of Anesthesiologists; 2008.
14.
Pronovost P, Needham D, Berenholtz S, et al. An intervention to decrease catheter-related bloodstream infections in the ICU. N Engl J Med. 2006 Dec 28;355(26):2725–32. [PubMed: 17192537]
15.
Dixon-Woods M, Bosk CL, Aveling EL, et al. Explaining Michigan: developing an ex post theory of a quality improvement program. Milbank Q. 2011 Jun;89(2):167–205. [PMC free article: PMC3142336] [PubMed: 21676020]
16.
AHRQ Effective Healthcare Program. Closing the quality gap: prevention of health care-associated infections. [internet]. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); Aug 3, 2011. [2011 Nov 4]. [20]. www​.effectivehealthcare​.ahrq.gov/index.cfm​/search-for-guides-reviews-and-reports​/?productid=738&pageaction​=displayproduct#toc.
17.
Surgical safety checklist. Geneva: World Health Organization (WHO); Jan, 2009. 1 p. whqlibdoc​.who.int/publications​/2009/9789241598590​_eng_Checklist.pdf.
18.
World Alliance for Patient Safety. WHO guidelines for safe surgery. Geneva: World Health Organization (WHO); 2008.
19.
Greenberg CC, Regenbogen SE, Studdert DM, et al. Patterns of communication breakdowns resulting in injury to surgical patients. J Am Coll Surg. 2007 Apr;204(4):533–40. [PubMed: 17382211]
20.
Griffen FD, Stephens LS, Alexander JB, et al. The American College of Surgeons' closed claims study: new insights for improving care. J Am Coll Surg. 2007 Apr;204(4):561–9. [PubMed: 17382214]
21.
Mulloy DF, Hughes RG. Patient safety and quality: an evidence-based handbook for nurses. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); Apr, 2008. Chapter 36. Wrong-site surgery: a preventable medical error. pp. 1–15. [AHRQ Publication No. 08-0043] www​.ahrq.gov/qual/nurseshdbk​/docs/MulloyD_WSS.pdf. [PubMed: 21328777]
22.
Merry AF. Role of anesthesiologists in WHO safe surgery programs. Int Anesthesiol Clin. 2010 Spring;48(2):137–50. [PubMed: 20386233]
23.
Association of Anaesthetists. Checklist for anaesthetic equipment 2004. London (UK): Association of Anaesthetists of Great Britain & Ireland; 2004. 2 p. www​.aagbi.org/sites/default​/files/checklista404.pdf.
24.
Haynes AB, Weiser TG, Berry WR, et al. Changes in safety attitude and relationship to decreased postoperative morbidity and mortality following implementation of a checklist-based surgical safety intervention. BMJ Qual Saf. 2011 Jan;20(1):102–7. [PubMed: 21228082]
25.
Weiser TG, Haynes AB, Dziekan G, et al. Effect of a 19-item surgical safety checklist during urgent operations in a global patient population. Ann Surg. 2010 May;251(5):976–80. [PubMed: 20395848]
26.
Quarterly update on preventing wrong-site surgery [draft]. Plymouth Meeting (PA): ECRI Institute; 2012. 13 p.
27.
Devine J, Chutkan N, Norvell DC, et al. Avoiding wrong site surgery: a systematic review. Spine (Phila Pa 1976) 2010 Apr 20;35(9 Suppl):S28–36. [PubMed: 20407349]
28.
Blike G, Biddle C. Preanesthesia detection of equipment faults by anesthesia providers at an academic hospital: comparison of standard practice and a new electronic checklist. AANA J. 2000 Dec;68(6):497–505. [PubMed: 11272956]
29.
Ben-Menachem E, Ezri T, Ziv A, et al. Identifying and managing technical faults in the anesthesia machine: lessons learned from the Israeli Board of Anesthesiologists. Anesth Analg. 2011 Apr;112(4):864–6. [PubMed: 21288969]
30.
Sewell M, Adebibe M, Jayakumar P, et al. Use of the WHO surgical safety checklist in trauma and orthopaedic patients. Int Orthop. 2011 Jun;35(6):897–901. [PMC free article: PMC3103968] [PubMed: 20730425]
31.
Kearns RJ, Uppal V, Bonner J, et al. The introduction of a surgical safety checklist in a tertiary referral obstetric centre. BMJ Qual Saf. 2011 Sep;20(9):818–22. [PubMed: 21693466]
32.
Thomassen O, Brattebo G, Heltne JK, et al. Checklists in the operating room: help or hurdle? A qualitative study on health workers' experiences. BMC Health Serv Res. 2010;10:342. [PMC free article: PMC3009978] [PubMed: 21171967]
33.
Taylor B, Slater A, Reznick R. The surgical safety checklist effects are sustained, and team culture is strengthened. Surgeon. 2010 Feb;8(1):1–4. [PubMed: 20222395]
34.
Singer SJ. Researching implementation and change while improving quality: grant awards. Fact sheet. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); Oct, 2011. Grants awarded in FY 2010. Factors associated with effective implementation of a surgical safety checklist. p. 3. www​.ahrq.gov/qual/qualimp.pdf.
35.
Safe surgery saves lives [internet]. Geneva: World Health Organization; [2011 Sep 22]. www​.who.int/patientsafety​/safesurgery/en/
36.
Follow-up Survey on Universal Protocol. [internet]. Oakbrook Terrace (IL): The Joint Commission; Dec 14, 2010. [2011 Oct 18]. [2 p]. www​.jointcommission.org​/follow-up_survey_universal_protocol/
37.
Garnerin P, Ares M, Huchet A, et al. Verifying patient identity and site of surgery: improving compliance with protocol by audit and feedback. Qual Saf Health Care. 2008 Dec;17(6):454–8. [PubMed: 19064662]
38.
Nilsson L, Lindberget O, Gupta A, et al. Implementing a pre-operative checklist to increase patient safety: a 1-year follow-up of personnel attitudes. Acta Anaesthesiol Scand. 2010 Feb;54(2):176–82. [PubMed: 19719811]
39.
Owers C, Lees E, Roberts J. Correct site surgery - Are we up to standard? Cent Eur J Med. 2010 Aug;5(4):407–10.
40.
Possible integration of actions to satisfy Joint Commission universal protocol and WHO surgical safety checklist. [internet]. Harrisburg (PA): Pennsylvania Patient Safety Authority; 2011. [2011 Nov 21]. [1 p]. www​.patientsafetyauthority​.org/EducationalTools​/PatientSafetyTools​/PWSS/Documents/crosswalk.pdf.
41.
2008 ASA recommendations for preanesthesia checkout: sample procedures. [internet]. Park Ridge (IL): American Society of Anesthesiologists (ASA); [2011 Nov 11]. [2 p]. www​.asahq.org/For-Members​/Clinical-Information​/2008-ASA-Recommendations-for-PreAnesthesia-Checkout​/Sample-Procedures​.aspx.
42.
Ibarra P, Robledo B, Galindo M, et al. Minimum standards 2009 for practising anaesthesiology in Colombia: Safety Committee. Rev Colomb Anestesiol. 2009 Jul;37(3):235–53.
43.
Semel ME, Resch S, Haynes AB, et al. Adopting a surgical safety checklist could save money and improve the quality of care in U.S. hospitals. Health Aff (Millwood) 2010 Sep;29(9):1593–9. [PMC free article: PMC3069616] [PubMed: 20820013]
44.
Thomassen O, Brattebo G, Softeland E, et al. The effect of a simple checklist on frequent pre-induction deficiencies. Acta Anaesthesiol Scand. 2010 Nov;54(10):1179–84. [PubMed: 21069898]
45.
Mahajan RP. The WHO surgical checklist. Best Pract Res Clin Anaesthesiol. 2011 Jun;25(2):161–8. [PubMed: 21550541]
46.
The objectives and study aim of the Surgical Checklist Implementation Project. [internet]. London (UK): Imperial College; [2011 Oct 6]. [1 p]. www​.safesurgery.org.uk/
47.
Robbins J. Hospital checklists. Transforming evidence-based care and patient safety protocols into routine practice. Crit Care Nurs Q. 2011 Apr-Jun;34(2):142–9. [PubMed: 21407009]
48.
Warnock GL. Of surgeons and safety checklists. Can J Surg. 2010 Dec;53(6):364–6. [PMC free article: PMC2993035] [PubMed: 21092427]
49.
National push for surgical safety checklist under way. Hosp Peer Rev. 2009 May;34(5):53–7. [PubMed: 19453097]
50.
WHO's patient-safety checklist for surgery. Lancet. 2008 Jul 5;372(9632):1. [PubMed: 18603137]
51.
Grande L. [Improving safety in the operating room reduces hospital mortality]. Cir Esp. 2009 Dec;86(6):329–30. [PubMed: 19783243]
52.
Gough I. A surgical safety checklist for Australia and New Zealand. ANZ J Surg. 2010 Jan;80(1-2):1–2. [PubMed: 20575868]
53.
Henderson D, Carson-Stevens A, Bohnen J, et al. Check a box. save a life: How student leadership is shaking up health care and driving a revolution in patient safety. J Patient Saf. 2010 Mar;6(1):43–7. [PubMed: 22130303]
54.
Sivathasan N, Rakowski KR, Robertson BF, et al. The World Health Organization 's ‘Surgical Safety Checklist’: should evidence-based initiatives be enforced in hospital policy? JRSM Short Rep. 2010;1(5):40. [PMC free article: PMC2984370] [PubMed: 21103132]
55.
Patterson P. In survey, about half of ORs are using the WHO checklist. OR Manager. 2009 Sep;251(9):6–8. [PubMed: 19824166]
56.
Abdel-Galil K. The WHO surgical safety checklist: are we measuring up. Br J Oral Maxillofac Surg. 2010 Jul;48(5):397–8. [PubMed: 20153096]
57.
Winters BD, Gurses AP, Lehmann H, et al. Clinical review: checklists - translating evidence into practice. Crit Care. 2009;13(6):210. [PMC free article: PMC2811937] [PubMed: 20064195]