Improvement Development: The Context

Prior to the development of the MHS error-reduction system, it had been noted that JDCH oncology nurses were spending an inordinate amount of time verifying physician’s handwritten orders against complex COG protocols and roadmaps. In addition, the computer-generated MARs provided by the pharmacy system were failing to group patients’ treatment criteria in a chronologic format, making the MAR/original order reconciliation process extremely difficult and labor intensive and, consequently, increasing the risk of error. Despite adhering to strict double-check policies, near-miss errors were often being caught during the order and MAR reconciliation process (Figure 1).

Figure 1

Joe DiMaggio Children’s Hospital chemotherapy order process.

Oncology nurse-clinician verification of the discrepancies provided baseline measures of variability in the accuracy of the JDCH pediatric chemotherapy prescribing process. It was noted that from January through November 2005, 79 percent of chemotherapy orders were written correctly, but 58 problematic individual patient order sessions had chemotherapy errors. Flawed chemotherapy orders included incorrect protocols, faulty calculations, inaccurate scheduling and weight/body surface area calculations, and missing dates, as well as legibility issues. For a vulnerable population of patients, such near-miss prescribing errors had the potential to cause serious harm had they not been caught and remedied.

With the growth of the pediatric oncology practice at JDCH, protocols increased in complexity. Recognizing the safety issues involved, JDCH nurses and oncology clinicians saw the need to improve the efficiency and safety of this process. Networking with peers at other pediatric oncology centers revealed that many centers were struggling with similar issues and that no automated solution existed. Having identified the problem, MHS leadership empowered JDCH nurses to find a solution.

Standardization of the Pediatric Chemotherapy Process

With an awareness that nationwide centers were concerned about pediatric chemotherapy safety,^{3, 4, 8, 9, 10} that the pediatric chemotherapy system was excessively prone to errors, and that a systemwide initiative was needed to improve patient safety, a multidisciplinary team was formed to improve pediatric oncology patients’ outcomes. The team consisted of two pediatric oncology physicians (employed by MHS), a nursing director of pediatrics, the JDCH chief nursing officer, three JDCH oncology clinicians, a JDCH oncology pharmacist, and three clinical informatics pharmacists. Team members were chosen for their particular clinical or process expertise, their willingness to work toward a solution, and their ability to make and enforce administrative decisions.

The team’s intent was to streamline the pediatric chemotherapy process and implement standardized order sets and medication administration templates that would correlate with COG pediatric chemotherapy treatment protocols. The group’s hypotheses were that: one, use of COG protocol-specific standard order sets and multiday MARs would decrease the number of chemotherapy order errors; and two, decreasing the number of individual patient order sessions with problematic chemotherapy orders would improve patient safety.

Although significant obstacles were not encountered in designing the new system, the team struggled with a number of decisions, including creating an effective and efficient forms-review process; facilitating ongoing forms maintenance; and implementing an alternative process for access to forms (to ensure that clinicians always had the most current version). The forms access issue was resolved by the group’s decision to post the forms on the MHS physician portal site, which is accessible to any authorized MHS employee or medical staff member, either in-house or remotely.

The forms review/approval process was established as follows:

1. A protocol specialist develops order forms and MARs for an arm of the protocol.
2. Once the draft of these order forms is completed, a subset of the team members reviews that protocol arm.
3. The review process for each arm is assigned to a rotating group that includes two individuals from each discipline (i.e., physician, nurse, and pharmacist).
4. These individuals have 2 weeks to assess the protocol for accuracy and appropriateness, after which they share any feedback and/or necessary changes with the rest of the team at one of the scheduled monthly meetings. The approval process is usually completed within 2 weeks.
5. After changes are made, the team members perform their final review and then sign off using a sheet maintained by the protocol specialist.
6. Once finalized, the forms are converted to PDF format and attached to the intranet menu, where they can be accessed and printed as needed by pediatric oncology physicians or their designees.

The process for pharmacy orders was also changed. Previously, one pharmacy satellite entered orders for a patient’s non-chemotherapeutic medications, while another entered only chemotherapeutic agents. However, with the new protocol standardization and implementation, a single pharmacist would enter all protocol medications, regardless of whether they were considered chemotherapeutic or non-chemotherapeutic agents. Lastly, a decision was made to adopt the standardized order forms and MARs, whether the patient was being treated as an inpatient, in an outpatient clinic, or in a physician’s office.

Standardization of the Chemotherapy Process: Prescribing

Handwritten order sets are associated with significantly greater risk, compared with standardized, preprinted order sets.^{3, 12, 13} Incorrect, incomplete, or illegible chemotherapy orders require nurses and/or pharmacists to make assumptions that may be erroneous, thus putting the patient’s health at risk.¹ In contrast, standardized, complete order sets are an inexpensive and readily available method that substantially reduces the need for clarification and the number of changes required during order verification and processing.^{14, 15}

MHS made a commitment to create standardized preprinted order sets for all oncology protocols. Since more than 90 percent of pediatric oncology patients are treated based on open or closed protocols,¹⁶ the team agreed to begin with open protocol order-set creation for the most common pediatric malignancies. Figure 2 shows the AALL0331 protocol, the first such standardized order set we developed, for acute lymphoblastic leukemia (ALL), the most common malignancy of children.¹⁷

Figure 2

Standardized Pre-B ALL Standard Induction order form.

The AALL0331 protocol consists of 13 treatment arms (induction, consolidation, maintenance), for which 67 order sets/multiday MARs were drafted and validated by the multidisciplinary design group through the articulated iterative process. Once the first group of documents was completed for ALL, the group conducted a “parallel go-live” by using the forms in parallel with the existing process. The feedback from the clinicians for the new protocol was so favorable that the forms were immediately implemented. Two additional protocols were developed to study patients with pre-B type of leukemia. Pre-B type pediatric oncology patients may be either on the “low risk” (AALL0331) or “high risk” (AALL0232) protocol.

Once feedback from the first diagnosis-specific protocol was completed, the task force presented their findings to MHS executives and leaders for review. This resulted in the hiring within a year of a full-time protocol specialist to design, create, maintain, and manage the approval and ongoing education process for full implementation of all COG-specific order sets and MARs. To date, 238 order sets/MARs for four pediatric COG diagnoses have been implemented at JDCH. There are 10 new and eight existing patients on the AALL0331 and AALL0232 protocols, respectively, and two on the AREN0532 protocol. A fifth order set/MAR for Wilm’s tumor is in the final stages of approval. Our goal is an order set/MAR for 100 percent of all COG protocols.

Redundant, manual, multidisciplinary checks of chemotherapy orders are a proactive approach to error management.¹⁶ Thus, as an additional safety measure, two-physician verification and cosignature of all chemotherapy orders was implemented in December 2005. Oncology nurse-clinician assessment of both handwritten and template order sessions for chemotherapy order errors continues. These efforts appear to be paying off, as evidenced by a decline in chemotherapy order errors from 58 in January to November 2005 (prior to standardization and integration of the pediatric chemotherapy process components) to 6 such errors for the same time period in 2007. No chemotherapy order errors have so far occurred using the four standardized chemotherapy order sets/MARs. The significant decrease in chemotherapy order errors in handwritten order sessions is believed to be a function of increasing physician familiarity and utilization of the standardized order method and incorporation of the standardized structure and content into the handwritten order session process.

To promote ease of access and consistency of use, MHS placed the current standardized leukemia chemotherapy order sets on the MHS intranet page (Figure 3), making them accessible to any authorized MHS employee or medical staff member, either in-house or remotely. Clicking on the link opens the actual order document in PDF format, so that it can be printed, completed, verified, cosigned, and used to initiate chemotherapy administration. This MAR is then reconciled with another nurse against the protocol-driven roadmap and individual patient chemotherapy order.

Figure 3

Online standardized pediatric chemotherapy order sets.

Standardization of the Chemotherapy Process: Dispensing

Once the standardized chemotherapy form is completed, verified, and cosigned by two physicians, it is sent to the chemotherapy-dedicated pharmacy, where a pharmacy triple-check process is initiated (Figure 4). MHS’s pharmacy triple-check process was instituted to mitigate modeled probability dispensing error rates of 2 to 3 percent.¹⁸ (Note: Probability dispensing error rates are a quantifiable statistical relationship between a measure of workload [e.g., number of prescriptions dispensed by individual pharmacy staff during a single workday] and the risk of committing at least one dispensing error during that same workday period.)

Figure 4

Pharmacy pediatric chemotherapy order triple check.

The MHS pharmacy’s triple-check process ensures accuracy in interpreting and entering the chemotherapy order, as well as in compounding and dispensing the medication. Although the pharmacy triple-check process is an industry benchmark practice, the MHS triple-check profile is unique in that it corresponds with the COG-specific protocol. This protocol-specific triple-check profile provides the pharmacist with the entire standardized plan of care, thus decreasing the possibility of transcription errors and improving order process efficiency. Pharmacist review of the chemotherapy order, verification of the order with the chemotherapy protocol, and redundant checks of entering and dispensing chemotherapy orders are recommendations of the failure modes and effects analysis (FMEA) for safe order interpretation, compounding, and dispensing of chemotherapy medications to hospitalized children.³

Standardization of the Chemotherapy Process: Administration

Chemotherapy administration is the chemotherapy subprocess with one of the highest risks for errors.³ Every step in the standardized process—from prescribing to dispensing with redundant checks by physicians, pharmacists, and nurses against the protocol, roadmap, and individual order—could be performed, but if the nurse administers the wrong medication to the wrong patient, the entire process is a failure with potentially devastating consequences.

When caring for pediatric patients, health care organizations must have effective processes to ensure that their staff is competent in the use of devices, equipment, and drug administration.^{19, 20, 21} MHS’s culture of safety is proactively focused on safety promotion vs. reactive error management. All personnel who administer chemotherapy must have completed the Oncology Nursing Society’s chemotherapy provider course “Get Certified”²² and be able to demonstrate clinical competency with the clinical nurse specialist. All staff members are required to be competent in terms of drug therapy knowledge, ability to function safely in the medication administration system by adhering to policies and procedures, ability to foster communication and teamwork, and use of decision-support tools.

Upon receipt of the chemotherapy from the pharmacy, two certified chemotherapy provider nurses review the original chemotherapy order, the patient’s protocol-based roadmap for appropriate dosage, and the administration schedule with the dispensed, labeled chemotherapy bag. The patient’s absolute neutrophil count (ANC) is calculated from the most recent CBC, and relevant lab results, and specific tests are reviewed, verified, and documented. Inappropriate lab and test results are reported to the physician in a timely manner.

Immediately prior to chemotherapy administration, two certified chemotherapy provider nurses perform a critical check at the point of delivery. A critical check includes verification of the patient’s name and medical record number and of the chemotherapy agent, dose, route, volume, and infusion time. The critical check is documented on the patient’s MAR. To ensure safe administration of chemotherapy to hospitalized children, FMEA recommendations include redundant, two-RN verification of the chemotherapy order, the patient’s individualized protocol-based roadmap, and administration dose, times, and bedside patient identification.³

Technologic Advances

The Institute of Medicine’s (IOM) groundbreaking report, “To Err is Human,” documented as many as 98,000 deaths per year from avoidable medical errors. It also identified information technology as an important tool for decreasing adverse medical outcomes.²³ In the aftermath of the IOM report, hospitals have prioritized patient safety and investigated new methods for improving the delivery of health care. Motivated by an awareness that current health care delivery practices are not as efficient or effective as they need to be, MHS has embraced technology to assist physicians and staff in providing more efficient, effective, and safe patient care.

Standardization of pediatric chemotherapy order sets and a more efficient order verification process are foundational process improvement steps toward implementation of computerized physician order entry(CPOE) and bar code medication administration (BCMA) in this complex setting. CPOE is one technologic process of medical management that provides a well-organized, electronic strategy to enhance efficiency, improve the quality of patient care, and decrease adverse medical outcomes.²⁴

MHS has developed and is testing the CPOE order set for Protocol AALL0331-Standard Induction Arm (Figure 5). MHS is already in the initial stages of CPOE. The pediatric and adult emergency departments (EDs) at three of the six facilities are “live on CPOE” with plans for two additional ED facilities to “go-live” in 2008. In addition, nursing computer-based clinical documentation and BCMA have been implemented at several facilities. These initiatives offer exciting opportunities to fuse quality and best practices at the point of care.

Figure 5

CPOE test order set – Protocol AALL0331 Standard Induction Arm.