Chapter  138:  Cancer Care Quality Measures: Diagnosis and Treatment of Colorectal Cancer

Colon Imaging, Endoscopic Visualization, and Biopsy (Question 1a)

The best-developed measure identified was percentage of patients who underwent appropriate evaluation for a positive fecal occult blood test [FOBT] (I₅S₄T₄), which is linked to an outcome of interest (reduced mortality from CRC), based on an evidence-based recommendation, and fairly well tested. The leverage points are awareness of the positive FOBT and referral for colonoscopy.

Although the benefits of expeditious action are intuitive, the measure time from presentation to diagnosis (I₃S₁T₁) has not been explicitly linked to an outcome of interest, is not based on any evidence-based guidelines, and can therefore be considered to be relatively less well developed and untested. The same can be said for proportion of colonoscopies that were completed in a timely fashion (I₅S₄T_-).

Any measure that relates to miss rates (I₅S₃T₃) is unlikely to be useful to compare quality of care, since the methodological gold standard for determining a miss rate is tandem or closely timed colonoscopies, and such a gold standard procedure is not realistic within general clinical practice. Computed tomographic (CT) colonography, or “virtual colonoscopy,” appears to be a better alternative, but is not yet part of mainstream evaluation considering the flux in the technology utilized.

Also important were technical process measures of colonoscopy such as cecal intubation rate (I₅S₄T₄), intraprocedural complication rate (I₅S₄T_-), and postprocedural complication rate (I₅S₄T₄). These technical measures are endorsed by guidelines and are meaningful indicators of quality. These measures tend to be fairly well developed, but poorly tested.

Complication rate of colonoscopy (I₅S₄T₄) is an important technical measures; however, an accurate definition of the measure is required.

Pathologic Staging (Question 1b)

Our literature search retrieved two process measures that have been used to assess the quality of staging of CRC:

• Adequate lymph node retrieval and evaluation (I₅S₄T₅).

• Percentage of newly diagnosed CRC cases who were staged using the American Joint Committee on Cancer (AJCC) system (I₄S₃T₃).

Adequate lymph node retrieval and evaluation is linked to an outcome of interest (better staging, leading to appropriate management and better survival) and based on evidence (the 1999 consensus statement of the College of American Pathologists (CAP) recommends evaluating 10–15 lymph nodes in node-negative patients). It is well suited as a quality measure in a broader context, and is well developed and tested. Percentage of newly diagnosed CRC cases that were staged using the AJCC system is another measure that is well developed, has face validity, is related to an outcome of interest, and represents an important leverage point for improving the quality of CRC care. However, it does not distinguish between pathology staging and clinical staging.

Polypectomy for Malignant Polyps (Question 2a)

No quality measures were found addressing polypectomy for malignant polyps

Surgical Therapy (Question 2b)

The most basic measure of the quality of surgical management is whether surgical therapy was provided as appropriate to the location and stage of the cancer. Results could potentially be broken out by stage or presented as a single summary measure, surgical therapy appropriate to the location and stage of the cancer, for example, appropriate primary therapy for CRC as defined by the NCI guidelines (I₄S₄T₄). The former approach provides more information and is preferable. Regarding the denominator of this measure, patients for whom the therapy is inappropriate should be excluded. This point is particularly problematic for patients with stage IV colon or rectal cancer, for whom the intent of the surgery is palliation rather than cure. Ideally, the measure should also distinguish between patients who were offered “appropriate” surgery and refused, and those patients who were not offered such surgery. Making these distinctions increases the data collection demands.

Surgical complication rates provide another very natural measure. Most variations on this measure focus on mortality rates (e.g., 30-day mortality [I₅S₄T₅] or in-hospital mortality [I₅S₃T₃]), although a delineation of the actual complications would be helpful as well, this latter type of measure being relatively underdeveloped. Presumably, such a measure should be both stratified by stage and location, and risk adjusted to take into account other clinical characteristics of the patient.

For patients with rectal cancer, various measures have been proposed regarding the rate of sphincter-saving surgery (I₅S₄T₄). This is an outcome of importance to patients. However, quality measures should consider detailed information about the anatomical location and other characteristics of the tumor, which is not yet the case in practice.

Technical measures associated with surgery are underdeveloped.

Adjuvant Chemotherapy (Question 2c)

We considered measures for stage III colon cancer, stages II and III rectal cancer, and stage IV colorectal cancer. Roughly speaking, these measures were defined as the proportion of patients with stage III colon cancer receiving adjuvant chemotherapy (I₅S₄T₅), the proportion of patients with stages II or III rectal cancer receiving chemoradiotherapy (I₅S₃T₂), and the proportion of patients with stage IV colon cancer or stage IV rectal cancer receiving palliative chemotherapy (I₅S₃T₂).

Of these measures, perhaps the best-developed and best-tested measure applies to patients with stage III colon cancer. It is derived from an evidence-based recommendation, is an important leverage point, and is well tested.

Also evidence-based and well validated is the measure applying to patients with stages II or III rectal cancer. Several different versions of the measure have been created, but the ideal version of the measure is the one that most closely matches the most recent recommendations from guidelines, which specify that chemoradiotherapy is the preferred method of management for essentially all such patients.

The measure regarding palliative chemotherapy for stage IV colon cancer or stage IV rectal cancer is reasonable in principle, but suffers from the difficulty that the decision to provide this therapy should very much be made on a case-by-case basis, thus implying that prospective data collection is probably necessary.

For all the above measures, patient preference needs to be taken into account. However, this data element is particularly difficult to collect.

The measure on the proportion of patients referred to an oncologist (I₅S₃T₂) pertains to an important leverage point in practice, although it is possible that this is neither a necessary nor a sufficient condition for the provision of high-quality care.

Radiation Therapy (Question 2d)

We identified an important process measure: the percentage of patients with stage II or III rectal cancer receiving adjuvant chemoradiotherapy (I₅S₃T₂). A well-developed and tested measure, it needs to account for reasons for non-receipt of therapy (e.g., insurance coverage or patient preference).

We also identified two potential quality measures for radiation therapy related to the technical quality of its administration, with reference to two different guidelines: adherence to radiotherapy management treatment guidelines for patients with adenocarcinoma of the rectum or sigmoid colon (I₅S₄T₄) and rate of use of modern radiation therapy techniques and adherence to recommendations of NCI-sponsored randomized controlled trials in rectal cancer patients (I₃S₃T₂). However, neither of the two technical measures is appropriate for current use because they are based on outdated guidelines. These measures could be modified to make them relevant to current evidence and practice standards.

Colonoscopic Surveillance (Question 3)

Measures describing the performance of postoperative surveillance colonoscopies were fairly well developed and well motivated: their ratings ranged from I₅S₄T₅ for the measure percentage of patients with CRC receiving postoperative (surveillance) colonoscopy to I₄S₃T₂ for percentage of patients with local or regional CRC who had colonoscopy or flexible sigmoidoscopy with barium enema. The most problematic point was the lack of consensus regarding the optimal scheduling of this procedure. The numerator should include colonoscopies only, and not other tests such as barium enema or flexible sigmoidoscopy. The measure could also benefit by explicitly specifying whether patients without clearing colonoscopy should be included. “Clearing colonoscopy” (1) refers to a sufficiently comprehensive examination of the colon to exclude other significant neoplasia, such as adenomatous polyps or synchronous cancers; and (2) signifies that all polyps were removed. Technical criteria should be considered as well; for example, whether the colonoscopy was complete according to the standard of documented cecal intubation.

A measure describing the rate of perioperative colonoscopy (I₅S₄T₄) was similarly well motivated, but suffered from a lack of precision in distinguishing clearing colonoscopies from similar procedures pre- and postsurgery. Including this additional detail would be substantively helpful, although potentially increasing the data collection requirements, as not all databases (particularly administrative databases) will contain sufficient information to make this distinction.

Pathology, Operative, and Chemotherapy Reports (Question 4)

The quality measure proportion of CRC cases in which pathologic staging preceded chemotherapy and radiation treatment (I₅S₄T_-) is well developed, free from confounders, and is an important leverage point, but needs testing.

The quality measure pathology report in concordance with CAP guidelines (I₅S₄T_-) is well developed, free from confounders, and is an important leverage point, but needs testing. Not all items included in the CAP guidelines are related to outcomes of interest, and it may be useful to select those that address a leverage point, for example, local extent of tumor, regional lymph node metastases, or residual tumor at resected margin. This aspect is considered by the measure adequacy of pathology reports on CRC (I₅S₄T_-), which limits measurement to scientifically validated elements of the CAP guidelines, and is therefore a potentially ideal measure; however it needs field-testing.

The other measures (each being an item of the Association of Directors of Anatomic and Surgical Pathology [ADASP] guideline) take more of a micro-level perspective, addressing individual elements of the pathology report. These are well developed and address some important leverage points for improving quality of colon and rectum cancer pathology reporting standards. The most important ones are: local extent of tumor, regional lymph node metastases, and residual tumor at surgical resection margin (all I₅S₅T₄).

Technical measures relating to operative and chemotherapy reports were underdeveloped.

Patient Populations and Purposes; Age, Race/Ethnicity, and Socioeconomic Status (Questions 5a and 5b)

A number of studies have measured differences in process measures related to quality of care based on age, racial/ethnic, or socioeconomic differences; however, few studies have evaluated potential confounders that may explain the differences observed. More refined measures that evaluate a variety of covariates involved in clinical decisionmaking may be necessary to understand and explain these apparent quality-of-care differences.

Burden of Disease

The burden of illness from CRC is substantial. In 2003, it is estimated that there were 145,290 new cases and 56,290 deaths attributable to CRC in the U.S. CRC is the second leading cause of cancer death in the U.S.⁴

Conceptual Model

Our conceptual model of the process of care for colon and rectal cancer patients in the U.S. is depicted in Figures 1 and 2. The next sections discuss the various elements of care represented within these figures.

Diagnosis of CRC

The preferred method of CRC detection is through screening of asymptomatic patients. The clinical rationale for screening is that CRC is a cancer that is usually slow growing, and if the cancer is caught at an early stage (ideally, at the polyp stage), then it is more likely to be cured. However, in spite of demonstrated benefit and published clinical guidelines, CRC screening tests are underutilized.⁵ In part due to this underutilization, some patients will be diagnosed with CRC because of symptoms.

Quality measures related to CRC screening have been evaluated elsewhere. At the request of the NQF, we restricted our analysis to: (a) the additional pattern of tests used to diagnose CRC in patients for whom either a screening test result or the presence of symptoms suggests an increased risk of CRC; and (b) screening for recurrence of disease following surgery for CRC, this latter application being termed “surveillance.” The preferred test in both applications is a colonoscopy.

Colonoscopy is the most sensitive test for detection of colorectal neoplasia and the only method that allows both examination of the entire large bowel and the opportunity to biopsy or even remove suspected lesions.⁶ Various screening tests such as double contrast barium enema, flexible sigmoidoscopy, and fecal occult blood test may have preceded the colonoscopy. However, these modalities are no longer recommended as followup tests.⁶ Computed tomographic (CT) colonography (“virtual colonoscopy”) is an emerging technology with the potential to screen for CRC, but it is not currently endorsed for screening or diagnostic evaluation.⁶–¹⁰

Staging

Surgical resection remains the most effective therapy for colorectal carcinoma. Clinical staging involves evaluating the primary tumor and the presence of nodal and distant metastases; clinical staging begins with evaluation of signs and symptoms, radiographic, and laboratory tests, and includes findings from direct visualization via endoscopy or surgery. For rectal cancer, clinical staging also includes endoscopic ultrasound or Phased Array MRI preoperatively to determine the depth of penetration and the possibility of regional nodal disease, which cannot be accurately assessed by pelvic CT scanning.

Pathologic staging is performed after surgical resection of the primary tumor, and accurate staging is critical for appropriate patient management and also for comparability of data across research studies.¹¹

The prognosis of patients with colon cancer is clearly related to the degree of penetration of the tumor through the bowel wall, the presence or absence of nodal involvement, and the presence or absence of distant metastases. These three characteristics form the basis for all staging systems developed for this disease. Bowel obstruction and bowel perforation are additional indicators of poor prognosis.¹²

Although the microscopic appearance of adenocarcinoma of the colon and rectum tends to be straightforward, a detailed pathology review is essential for diagnosis, characterization, and staging. Pathologic stage combines the clinical staging information with surgical findings, incorporating data from the pathologic examination of resected primary and regional lymph nodes.¹³ The pathology report is a critical step in the process of pathologic staging. Information on pathology reports should be sufficient to stage the patient using the Tumor, Node, Metastasis Staging System (TNM).¹⁴ Other key factors include the grade of the tumor and the surgical margin status.

The assessment of nodal metastases at the time of surgical treatment is particularly crucial for patients with apparently localized carcinoma of the colon and rectum, since the presence of nodal metastases is currently the most important factor in determining whether an individual is a candidate for adjuvant therapy.¹⁵ Moreover, stage of disease is the single most important factor in predicting survival.¹⁶–¹⁸ The small size of many mesenteric nodes, including some with micrometastatic carcinoma, increases the risk of missing metastases. Recent studies have demonstrated that 10 to 15 lymph nodes are needed for reliable staging.¹⁹–²¹

Management: Surgery

Standard treatment for patients with stages I–III colon cancer has been open surgical resection of the primary tumor and regional lymph nodes for localized disease. Polypectomy for malignant polyps is adequate for polyps that have not invaded the submucosa, because these tumors do not metastasize. For polyps that reveal submucosal, lymphovascular, or neural invasion, a bowel resection should be performed. Although surgery is usually reserved for patients with non-metastatic disease, palliative surgery or diverting colostomy is sometimes performed in patients with metastatic disease and bowel obstruction. Some patients who have developed metastases in the liver are also candidates for surgery. Improved surgical techniques and advances in preoperative imaging have allowed for better patient selection for resection.

Localized rectal cancer is treated by surgical resection of the primary tumor and regional lymph nodes. Although previously patients underwent surgery and then postoperative chemoradiotherapy, currently patients with clinical stage T1,2 disease proceed to surgical resection, while most patients with clinical stage T3 or T4 or N+ tumors now receive preoperative chemotherapy and radiotherapy. Most rectal cancers that are located 5 cm or more from the anal verge can be removed while preserving the anal sphincter. For small rectal cancers with favorable pathologic features, local full-thickness excisions may be adequate. The resective technique of total mesorectal excision has led to reduction in local recurrences and improvement in overall survival.²²

Management: Chemotherapy

Because their cancers have spread beyond the colon and rectum, patients with stage III colon cancer without medical or psychosocial contraindications to treatment are recommended to have systemic adjuvant chemotherapy following surgical resection.¹⁵ Patients with stage II and III rectal cancer also benefit from chemotherapy.^15,23 Palliative multiagent chemotherapy also increases survival for those with stage IV disease, although it is not curative.^24,25

The steps in the process of receiving appropriate adjuvant chemotherapy are that patients with CRC must be staged appropriately, be referred to an oncologist, receive an adequate performance status evaluation, and be ascertained to be free of major postoperative complications. Chemotherapy must then be offered and administered at an appropriate dosage for an appropriate length of time.²⁶ Finally, adequate reporting must be provided, this requirement applying equally to surgical therapy, chemotherapy, and radiation therapy.

Management: Radiation Therapy

Because the pattern of recurrence following surgical excision is more often local in patients with rectal cancer than in those with colon cancer, the potential impact of radiation therapy is greater in patients with rectal cancer than in patients with colon cancer.²⁷ Both preoperative and postoperative radiation therapy decrease local failure.²⁸–³¹ Patients with stage II or III rectal cancer are at high risk for local and systemic relapse and are thus the best candidates for adjuvant therapy including chemotherapy and/or radiation therapy. However, it is important to follow precise procedures for radiation therapy in order to achieve optimum results.

The role of adjuvant radiation therapy for patients with colon cancer (above the peritoneal reflection) is not well defined and has no current standard role in the management of patients with colon cancer following curative resection.³²

Surveillance

Surveillance refers to the ongoing followup of CRC patients after treatment. Colonoscopic surveillance after curative resection of CRC is recommended, but the optimal timing of the surveillance coloscopies after a perioperative “clearing” colonoscopy is yet to be determined. If a preoperative colonoscopy is not possible (e.g., because of an obstructing lesion), it should be performed 6 months after surgery.

Non-endoscopic or combined modality surveillance, such as following tumor markers (carcinoembryonic antigen [CEA]) or abdominal imaging (CT), is controversial. More generally, while there is an overall survival benefit for intensifying the followup of patients after curative surgery for CRC, the optimal combination of clinic visits, laboratory tests and procedures is not currently known.³³–³⁵

Sources

The primary sources of literature were MEDLINE^® (1966-January 2005) and the Cochrane Database of Systematic Reviews. Searches of these databases were supplemented by reviews of reference lists contained in seed articles, all included articles, relevant review articles, and meta-analyses. We made special attempts to retrieve grey literature, loosely defined as institutional or government reports, professional society documents, research papers, fact sheets, and other literature, in print or electronic format, that is not controlled by commercial publishing interests. Sources for grey literature included professional organization websites and the Internet.

Search Strategies

The basic search strategy used the National Library of Medicine's Medical Subject Headings (MeSH) key word nomenclature developed for MEDLINE^® and was adapted for use in the other databases. The searches were limited to the English language. The text of the major search strategy is given in Appendix B^*, as well as a history of search terms considered.

The final search, conducted in January 2005, yielded 5450 citations. When limited to publication date since 1990 and English language, this totaled 3,771 citations. These records were maintained in a ProCite (Thompson ISI ResearchSoft, Berkeley, CA) database.

Abstract and Full-Text Screening

Paired content experts from the Duke research team independently reviewed a set of abstracts and classified each as “include” or “exclude.” An abstract was retained at this stage if at least one of the paired reviewers recommended that it be included, and abstractors were instructed to apply the inclusion criteria liberally. Inter-rater reliability for include/exclude decisions was tested by having five pairs of readers review a subset of abstracts (n = 765). Agreement ranged from a kappa of 0.23 to 0.46. Samples of abstracts for which abstractors disagreed were reviewed, and it was noted that most disagreements occurred for marginal articles and reflected the abstractors' attempts to be as inclusive as possible during this preliminary screening stage. A total of 874 abstracts were included for the further “full-text review” stage.

At the full-text review stage, the paired researchers independently reviewed a set of the articles, and indicated a decision to “include” or “exclude” the article for the data abstraction stage. Detailed inclusion and exclusion screening criteria were developed for this purpose (see “Full-text Screening Criteria,” below). When a pair of reviewers arrived at a different opinion about whether to include an article, they were asked to reconcile the difference. Detailed inclusion and exclusion screening criteria were developed and are listed below.

Full-Text Screening Criteria

The full-text screening criteria were as follows:

Include if:

• The study population pertains to adults undergoing diagnostic evaluation for, or in treatment for, colon or rectal cancer, and at least one of the following apply:

  • Asymptomatic patients who have been screened AND have a positive finding suspicious for colon or rectal cancer (e.g., polyps).

  • Patients being diagnosed through the use of colon imaging, endoscopic visualization, or biopsy (Question 1a).

  • Diagnosed patients undergoing pathology staging (Question 1b).

  • Patients undergoing polypectomy for malignant colonic polyps (Question 2a).

  • Patients treated with surgical therapy for either rectal cancer or colon cancer (Question 2b).

  • Patients with metastatic, but potentially curable (hepatic/pulmonary-resectable) disease, when a majority of the study population has CRC (Question 2c).

  • Patients treated with adjuvant chemotherapy or radiation therapy (Questions 2c and 2d).

  • Patients treated with palliative chemotherapy or radiotherapy (Questions 2c and 2d).

  • Patients undergoing colonoscopic surveillance (no other form of surveillance) for CRC (Question 3).

• AND

• The study refers to at least one of the following:

  • An explicit quality measure (process, structure, or outcome).

  • A measure that assesses the adequacy and completeness of documentation of pathology, operative, or chemotherapy reports (operative procedures will include reports for colonoscopy and radiation therapy) (Question 4).

  • The testing of a quality measure's validity or reliability.

  • A data source (e.g., registry) used for measuring quality, but may not specify the quality measure itself.

• Exclude if one or more of the following apply:

• The study uses non-U.S. data.

• The study was published prior to 1990.

• The report is in a non-English language.

• The study population involves any of the following:

  • The screening of asymptomatic patients.

  • Patients with familial polyposis syndrome.

  • Patients undergoing polypectomy for hereditary polyposis.

  • Patients with hereditary non-polyposis colorectal cancer.

  • Patients with multiple metastases, BUT the majority of the patients ARE NOT CRC patients.

Summaries of the results of the abstract screening and full-text review are provided in Tables 1 and 2. A list of excluded articles by reason for exclusion is found in Appendix C^*

Background

The preferred method of colorectal cancer (CRC) detection is through screening of asymptomatic patients. However, in spite of demonstrated benefit and published clinical guidelines, CRC screening tests are underutilized.⁵ In part due to underutilized screening, some patients will be diagnosed with CRC because of symptoms. The American Society for Gastrointestinal Endoscopy (ASGE) strongly endorses colonoscopy in the diagnosis of CRC and further states that “multiple biopsy specimens should be obtained from all suspicious lesions, and polypoid lesions should be removed.”³⁸ In addition, patients who have a positive CRC screening test by any modality other than colonoscopy (i.e., fecal occult blood test [FOBT], flexible sigmoidoscopy, or double contrast barium enema [DCBE]) require a full colonoscopy for further evaluation.⁶ DCBE was previously considered acceptable for the followup of a positive FOBT, but the latest guidelines emphasize that colonoscopy is the preferred followup test based on the sensitivity of colonoscopy.⁶ Virtual colonoscopy is another promising and emerging imaging technique that has some advantages over traditional colonoscopy.⁶–¹⁰

Studies of colonoscopy miss rates for adenomas underscore that even this “gold standard” is not perfect.³⁹ In 2002 a Multi-Society Task Force published its recommendations for assessment of quality in the technical performance of colonoscopy.⁴⁰ Quality targets included: use of recommended screening and surveillance (e.g., postcancer resection) guidelines, cecal intubation rates of ≥ 90 percent overall and ≥ 95 percent of screening cases, documentation of cecal intubation by landmarks, mean colonic examination time (generally on withdrawal of colonoscope) of at least 6 to 10 minutes, incidence of perforation < 1/1,000 (all) and < 1/2,000 (screening exams), and incidence of postpolypectomy bleeding (< 1/100). Recommendations for the colonoscopy procedure report included documentation of procedure time; quality of bowel preparation; cecal landmarks; and polyps identified, removed (technique), and recovered.

Results

Our literature search revealed several quality measures for outcomes of interest related to this question:

• Percentage of patients with positive FOBT who underwent an appropriate evaluation.⁴¹–⁴⁵

• Time from patient presentation with symptoms to cancer diagnosis.⁴⁶

• Proportion of colonoscopies that were completed in a timely fashion.⁴⁷

• Percentage of patients with colon or rectal cancer undergoing colonoscopy as part of their evaluation.⁴⁸

• Colonoscopic miss rate for significant colonic neoplasia.^49,50

• Complication rate of colonoscopy.⁵¹

• Serious postendoscopic procedure complication rates.^47,52

• Rate of unplanned reversal of sedation medication.⁴⁷

• Intraprocedure colonoscopy complication rate.⁵³

• Colonoscopy completion rate.⁵⁴

• Cecal intubation rate.^47,52

• Adenoma removal rate for patients over 50 years old.⁴⁷

• Percentage of patients with adequate bowel preparation prior to colonoscopy.^47,52,55

• Proportion of colonoscopies performed by physicians with specialized training.⁴⁷

• Proportion of patients who adequately understood the colonoscopy procedure.⁵³

General Process Measures. The first, and most widely used, measure we consider is the percentage of patients who underwent appropriate evaluation for a positive FOBT. This is a general process measure that depends critically on the specification of what constitutes an “appropriate” evaluation. The measure is based on evidence-based guidelines that have evolved over time as additional data on what constitutes an appropriate evaluation have become available. The older American Cancer Society (ACS) guidelines⁵⁶ recommended the combination of barium enema and flexible sigmoidoscopy if a colonoscopy was not performed. The 1997 guidelines⁵⁷ allowed for either colonoscopy or barium enema for followup of a positive FOBT. The updated 2003 guidelines⁶ emphasized that colonoscopy was the preferred method of followup for a positive FOBT because of newer data supporting the increased sensitivity of colonoscopy to detect adenomatous polyps. Therefore, depending on the guideline cited, age of the data, and the timing of the study publication, “adequate evaluation” could be defensibly defined with some variation. Ideally, “appropriate evaluation” should also include consideration of the maximum acceptable time lag between the positive FOBT and a more definitive diagnostic workup, although the above guidelines do not explicitly specify such a time lag.

In applying this measure, Shields et al.⁴¹ used data from 1986-1988 and defined “adequate evaluation” as colonoscopy or DCBE with flexible sigmoidoscopy per the cited 1980 ACS guidelines.⁵⁸ The measure was applied in a community-based mass screening program, and the source included patient self-report up to 3 years after the evaluation. Such an extensive delay might call the reliability of self-report data into question.

Levin et al.⁴² used data from 1993 and defined “adequate evaluation” as colonoscopy or DCBE per the 1992 ACS guidelines.⁵⁶ The data sources were patient and physician self-report, the accuracy of which were not tested. Although formal validation was not performed, it was found that consultation of a gastroenterologist significantly increased the chance of adequate evaluation, thus supporting the discriminant validity of the measure.

Baig et al.⁴³ defined “appropriate evaluation” to include colonoscopy or DCBE (without flexible sigmoidoscopy), which is consistent with the 1997 guidelines,⁵⁷ although these guidelines were not explicitly cited. The followup period was restricted to 60 days, which may limit the usability and adaptability of this measure in systems where 2-month followup may not be possible (e.g., the Veterans Administration [VA] system). The data sources were provider survey and health maintenance organization (HMO) claims data.

Myers et al.⁴⁴ defined “adequate evaluation” as colonoscopy or DCBE with flexible sigmoidoscopy within 180 days, which may be a more realistic followup period. This study was in the setting of an intervention to improve followup rates for positive FOBT. Data sources were physician surveys and HMO administration claims data.

Mandel et al.⁴⁵ is another variation on this same measure and defined “appropriate evaluation” as followup colonoscopy or DCBE with flexible sigmoidoscopy. What is most noteworthy about this version of the measure is the study population: an 18-year followup from one of the original clinical trials that demonstrated the efficacy of FOBT. These data may be difficult to generalize, since patients that are enrolled in a clinical trial could be expected to have higher rates of appropriate followup than general practice.

Two measures focus on timeliness. One is the time from patient presentation with symptoms to physician diagnosis. Marble et al.⁴⁶ applied this measure to tumor registry data from a single medical center. Details of how time to diagnosis was determined are lacking. Moreover, there was no explicit demonstration that delays in diagnosis are related to survival (or any other outcome of interest). This measure is not well developed.

A related measure, proposed by Minnesota Gastroenterology PA as part of their internal quality improvement initiative,⁴⁷ is the proportion of colonoscopies that were completed in a timely fashion, where timeliness is operationally defined as “acceptable to patient and referring physician according to internal practice standards.” Ideally, this internal practice standard should reflect the indication for the procedure. While potentially important, this measure is not clearly linked to outcomes of interest or validated.

Another general process measure was applied by Beart et al.:⁴⁸ percentage of patients with colon or rectal cancer undergoing colonoscopy as part of their evaluation. The major flaw in this measure is the failure to distinguish between those colonoscopies used for the purpose of diagnosis and those used for preoperative clearance. This measure is not considered further.

Yet another general process measure is the colonoscopy miss rate (i.e., the miss rate for detection of significant colonic neoplasia). Haseman et al.⁵⁰ defined this measure as “percentage of colon cancers not detected by colonoscopy in symptomatic patients.” He performed a retrospective review of colon cancer cases that were not detected during a colonoscopy performed up to 3 years prior to cancer diagnosis. The numerator and denominator were well defined; however, the assumption that lesions should have been detected up to 3 years prior is not validated. Further limitations to the study were that the hospitals had volunteered to participate and that this was a retrospective review.

Shehadeh et al.⁴⁹ estimated the miss rate for “big” (≥ 10 mm) polyps by retrospectively reviewing the endoscopy database and pathology records of patients with an initial colonoscopy including polypectomy and at least one subsequent colonoscopy at a single, academically affiliated VA medical center. “Big” polyps seen on the subsequent colonoscopy were considered “missed” by the first colonoscopy. While the rate of missed advanced adenomatous polyps is a potentially important quality indicator, the method of measuring this value is more appropriately obtained with a prospective design and consideration of tandem colonic examinations.

Technical Process Measures. Some of the most fundamental technical measures concern procedural complication rates. Various specific measures have been proposed. Ure et al.⁵¹ use the measure complication rate of colonoscopy. The numerator included the following complications: bowel perforation, intraprocedural complications (hypoxia [O₂ saturation < 90 percent], hypotension, bradycardia, hemorrhage), postpolypectomy bleeding, cardiopulmonary arrest, and death. The presence or absence of any complications was noted and the rates of individual complications were separately described, as was the total number of complications. Some of the complications (i.e., the numerator) were more precisely described than others. Overall this measure could be better developed and needs to be risk-adjusted. Furthermore, the importance of the individual complications is not uniform. For example, in terms of patient outcomes, perforation, postpolypectomy bleeding and death have established importance, whereas complications such as intraprocedural hypoxia treated with supplemental oxygen may not be clinically important.

Minnesota Gastroenterology PA⁴⁷ defined the complication rate measure as serious postendoscopy complication rate, where the serious complications considered included death within 30 days, perforation, bleeding with transfusion, cardiopulmonary arrest, and unplanned surgery. Focusing on these severe and well-described complications adds to the development, importance, and face validity of this measure. Similarly, a large VA study⁵² was conducted in order to study major complication rates following screening colonoscopy. The complications were defined accurately and evaluated rigorously for up to 30 days after the procedure.

Potentially less serious, but clinically important, complications of sedation are captured by a second complication measure, proposed by the same group, the rate of unplanned reversal of sedation medication.⁴⁷

The National Quality Measures Clearinghouse (NQMC) publishes a similar measure, intraprocedure colonoscopy complication rate (proposed by the Accreditation Association for Ambulatory Health Care [AAHC] Institute for Quality Improvement), which includes arrhythmia, bleeding requiring treatment, extended recovery, hospital transfer, hypotension, hypoxia, perforations, and respiratory arrest.⁵³ The specific definitions of these complications are lacking, although guidelines by the Accreditation Association for Ambulatory Health Care (AAAHC) Institute for Quality Improvement are cited.⁵⁹

Another technical measure is the colonoscopy completion rate, i.e., whether the colonoscopy was adequately completed. The Colon Cancer Workgroup⁵⁴ applied the definition “colonoscopy to the ileocecal valve performed prior to surgical resection of CRC.” This measure is well developed and is based on National Comprehensive Cancer Network (NCCN) guidelines.²⁴ Cases of obstructing or perforated carcinoma should be excluded from the denominator of this measure. Ure et al.⁵¹ similarly defined colonoscopy completion rate as “visualization of the cecum or ileocolic anastomosis.” The rationale is similar to the above.

Cecal intubation rates and adenoma removal rates for patients over 50 years old were quality measures proposed by Minnesota Gastroenterology PA.⁴⁷ Both are well defined, linked to outcomes of interest, and based on guidelines.⁴⁰ Cecal intubation was defined as examination of the entire colon to the cecum or to a surgical anastomosis (when applicable). Adenoma removal rates were assessed in patients over 50 years of age. The rationale for this latter measure appears to be that if the adenoma detection rate is less than “population norms,” then polyps are probably being missed. “Population norms” were well defined.

Another technical measure pertains to the quality of bowel preparation preceding the colonoscopy. Harewood et al.⁵⁵ examined the relationship between adequate bowel preparation and the detection of colonic polyps or cancer. The data source was a national gastrointestinal endoscopic database (Clinical Outcomes Research Initiative [CORI]), and the findings supported significantly greater detection of colonic neoplasia (although not necessarily cancer) in patients with adequate bowel preparation compared to without adequate preparation. This measure is important, but a more appropriate quality measure for health care might be the documentation of the bowel preparation quality at the time of the procedure, as this may affect the interval of the next examination and the level of certainty that neoplasia or malignancy has been ruled out. Weaknesses of the study itself included the lack of validation of the CORI data, lack of standardized definitions for the quality of bowel preparation, and inconsistent histology data for colonic lesions.

Minnesota Gastroenterology PA⁴⁷ also utilizes a measure of the “proportion of colonoscopies with adequate preparation.” This version of the measure is more specific: “no fecal material obscured a lesion 5 mm or more in diameter and a high quality examination takes place.”

All measures proposed by Minnesota Gastroenterology PA have been implemented within their system and will require further testing for broader application.

Other Measures. Minnesota Gastroenterology PA⁴⁷ propose a structural measure of the proportion of colonoscopies performed by physicians with specialized training (gastroenterology, colorectal surgery, general surgery, pediatric surgery). This follows ASGE guidelines.⁶⁰ The measure is well defined, but needs validation and testing.

The NQMC⁵³ publishes the measure proportion of patients who respond “yes” to whether they have an adequate understanding of their procedure as asked in a post procedure telephone interview (proposed by the AAHC Institute for Quality Improvement). This is guideline-based,⁵⁹ but not validated (e.g., patients may want to give socially acceptable answers in a telephone interview), and measures only perception of understanding and not true understanding.

A final quality measure is an indirect one: incidence of metastatic diagnosis among health plan members aged 50 or older.⁶¹ Reduction of metastatic disease is supported by NCCN guidelines²⁴ as a measure of the long-term effectiveness of cancer screening programs, one component of which is quality of diagnosis. Because the primary determinant of this outcome is likely to be the proportion of eligible patients undergoing screening, this measure is considered to be outside the scope of interest here.

Conclusions

We retrieved several general, technical, and structural measures that have been utilized in the United States for assessing the quality of care related to diagnosis of CRC. Ratings for these measures are given in Table 4; further details are provided in Evidence Tables 1 and 2 in Appendix E^*. Perhaps the most important, well-developed, and widely tested⁴¹–⁴⁴ measure identified was percentage of patients who underwent appropriate evaluation for a positive FOBT (I₅S₄T₄), which is linked to an outcome of interest (reduced mortality from CRC) and is based on an evidence-based recommendation. The leverage points are awareness of the positive FOBT and referral for colonoscopy.

Although the benefits of expeditious action are intuitive, time from presentation to diagnosis has not been explicitly linked to an outcome of interest, is not based on any evidence-based guidelines, and can therefore be considered to be relatively less well developed and untested (I₃S₁T₁). The same can be said for the technical measure proportion of colonoscopies that were completed in a timely fashion (I₅S₄T_-).

Any measure that relates to miss rates (I₅S₃T₃) is open to debate because no matter how the miss rate is estimated it is unlikely to be useful to compare quality of care, since the methodological gold standard for determining a miss rate is tandem or closely timed colonoscopies, and such a gold standard procedure is not realistic within general clinical practice. In other words, it is not realistically possible, in practice, to determine whether a case has in fact been “missed.”

Also important is the technical process measure of colonoscopy: cecal intubation rate (I₅S₄T₄). This technical measure is endorsed by guidelines, but is inconsistently linked to outcomes of interest; it tends to be fairly well developed, but poorly tested. While cecal intubation rate is an important measure, a precise definition of intubation is required for appropriate evaluation. Toward this end, colonoscopy completion rate (I₅S₄T_-), specified in the studies we examined as visualization “to the ileocecal valve” or “of the entire cecum or ileocolic anastomosis,” may be a better measure.

Intraprocedural complication rate (I₅S₄T_-), postprocedural complication rate (I₅S₄T₄), and complication rate of colonoscopy (I₅S₄T₄) are other important technical measures; however, an accurate definition of the complication rate is required. Complications should be divided into those of a severe nature with links to outcomes of interest (e.g., bleeding requiring transfusion or hospitalization) and those of a more minor nature (e.g., intraprocedural transient hypoxia with oxygen saturation < 90 percent reversed with oxygen supplementation via nasal canula and resolution during the recovery period). For purposes of data collection (particularly since colonoscopy is primarily an outpatient procedure) it may also be useful to divide complications into intraprocedural complications, immediately postprocedural complications (e.g., within 2 hours), and 30-day postprocedural complications. This final category would require not only examination of medical records, but potentially patient survey in order to determine whether the patient was treated at another facility.

Patient understanding of the colonoscopy procedure was fairly well developed, but its relationship to outcomes has not been definitively demonstrated (I₅S₄T_-).

The Multi-Society Task Force guidelines⁴⁰ suggest an additional potentially useful quality measure, colonoscopic withdrawal time (which has been linked to polyp detection rates), but we found no data on development or validation of this potential measure.

The recommendations made herein are based on a review of currently available data. Obviously, future data or clinical realities (e.g., costs, changes in societal goals) or the development of new technologies may alter the recommendations in the future. Nonetheless, for the major recommendations, we do not expect their basic core importance to change.

Future Research

Since several potentially useful quality measures for diagnosis of CRC were retrieved, future work should focus on correcting any deficiencies of these measures. For the measure percentage of patients who underwent appropriate evaluation for a positive FOBT, the numerator should include only those patients who underwent colonoscopy, as this reflects the most recent guidelines.⁶ The denominator should include all patients who underwent FOBT by the appropriate technique (stool collection at home, any applicable dietary or medication restriction [depends on the specific FOBT used], results evaluated by a certified laboratory) and had a positive test (defined as any positive window). The following patients should be excluded from the numerator and denominator: those with positive occult blood test based on a digital rectal exam only; individuals for whom screening is inappropriate either because of life-limiting comorbidity, because they are younger than screening guidelines recommend or because they are not yet due for screening (e.g., had a colonoscopy within the past 5 to 10 years); and individuals who were offered colonoscopy, but refused. The first and second exclusions are based on the premise that one cannot measure the quality of screening followup if the screening itself was incorrectly performed or inappropriately applied. The third exclusion reflects that patient preference, if based on an informed decision, should not penalize the provider or system being evaluated.

One additional point to resolve in the definition of “adequate evaluation” is the maximum acceptable time lag between the positive FOBT and the subsequent colonoscopy. At present, this specification cannot rely on recourse to a guideline, but might instead be derived from basic first principles and the structural characteristics of the health systems being studied.

Finally, it should be noted that this quality measure needs to undergo additional testing. The link between followup of positive FOBT and CRC-specific survival is well supported. The reliability and accuracy of various data sources, however, require additional investigation. Data sources could include chart review, administrative claims data, patient survey, and provider survey.

Other technical process measures of colonoscopy also have potential for improvement to become useful quality measures. These measures should address the accuracy and safety of the procedure. Complication rates require precise descriptions including the specific complication (e.g., bleeding), the definition (e.g., bleeding that results in hospitalization or transfusion), and the time frame (e.g., within 30 days of the procedure). It would be best to separately collect the data for individual complications within prespecified time frames (e.g., intraprocedural, within 2 hours of the procedure, within 30 days) even if the complications might be reasonably reported in combination. Potential data sources are procedural flow sheets, chart review, and patient survey. These sources require testing for reliability and accuracy. Cecal intubation rate requires a precise definition and should include those patients postcecal resection by the inclusion of “to the surgical anastomosis.” The numerator should include all patients who meet this definition. The denominator should include all colonoscopies regardless of test indication. Patients with obstructing lesions should be excluded from the numerator and denominator. Patients with other reasons for failed complete colonoscopy, such as poorly tolerated examination, poor bowel preparation, or intraprocedural complications, should still be included in the denominator. Success and accuracy of this quality measure depends on accurate documentation of cecal landmarks or surgical anastomoses in the colonoscopy report. The first step is to have documentation of the extent of the colonoscopy as a required part of the colonoscopy report template. The next and more difficult step is validation of the report itself and of collective endoscopic databases, such as CORI. Corroboration by another health provider or inclusion of a still picture or video documentation are possible methods to compare the report to another source of information. Results may be biased if those performing the colonoscopy know that they are being observed.

Background

Although the microscopic appearance of adenocarcinoma of the colon and rectum tends to be straightforward, a detailed pathology review is essential for diagnosis, characterization, and staging. Stage is the most accurate predictor of survival for patients with CRC, and accurate staging is critical for appropriate patient management and meaningful clinical research.⁶² Pathologic stage combines the clinical staging information with surgical findings, incorporating data from the pathologic examination of resected primary and regional lymph nodes.¹³

Historically, numerous different staging systems for CRC have been used, but a single internationally recognized system is required to ensure a common language for cancer that is understood by clinicians in all specialties. The Tumor, Node, Metastasis (TNM) Staging System of the American Joint Committee on Cancer (AJCC) and the International Union Against Cancer (UICC) fulfills this mandate and has the added advantages of being data-driven, updated on a regular basis, multidisciplinary in design, and applicable to all current techniques of stage evaluation.¹⁴ Uniquely, the TNM Staging System is governed by rules of application, both general and site-specific.⁶³

The single most important determinant of prognosis in patients with apparently localized carcinoma of the colon and rectum is the presence or absence of nodal metastases at the time of surgical treatment. Patients with stage I or II cancer of the colon or rectum have an anticipated 5-year survival rate in excess of 75 percent. In contrast, individuals with N1 disease have a 5-year survival rate of only 45 to 60 percent.⁶⁴ The presence of nodal metastases is currently the most important factor in determining whether an individual is a candidate for adjuvant therapy¹⁵ and is arguably the single most important factor in predicting survival.¹⁶–¹⁸

Nodal staging accuracy for colorectal carcinoma is currently under scrutiny. Several factors contribute to variation in nodal staging accuracy, including the amount of mesentery resected, fixative, diligence of search for nodes, and number of histologic levels examined. The small size of many mesenteric nodes, including some with micrometastatic carcinoma, increases the risk of missing metastases. Recent studies demonstrated that 10 to 15 lymph nodes are needed for reliable staging. Although the exact number of lymph nodes required for adequate lymph node evaluation in patients has been debated, the 1999 consensus statement by the College of American Pathologists (CAP) recommended evaluating 12 to 15 nodes in node-negative patients.¹⁹

There is evidence to indicate that the quantity of lymph node harvest has a direct effect on staging.^20,21,65 The principles of surgical resection of colon and rectal cancer dictate en bloc removal of the cancer and sufficient proximal and distal bowel to encompass potential submucosal lymphatic tumor spread and include the regional mesenteric draining lymphatics. Standard resections of the colon and rectum are founded on anatomic structures that provide not only for adequate radial and longitudinal margins but for adequate regional lymph node clearance as well. Despite these well-recognized guidelines, there may be substantial variability in the type of resection performed for CRC, which could lead to variability in the number of nodes removed. Additionally, the number of nodes in a surgical specimen may vary from one patient to another and may perhaps depend on the extent and diligence of the pathologic examination,⁶⁶ since many pathologists are unfamiliar with these rules or unsure how to interpret them in specific situations.⁶⁷

Results

Our literature search retrieved two process measures that have been utilized to assess the quality of staging of CRC:

• Adequate lymph node retrieval and evaluation.^{13,54,61,68,69}

• Percentage of newly diagnosed CRC cases who were staged using the AJCC system.^70,71

Applications have ranged from clinical epidemiology to quality improvement.

Baxter et al.⁶⁸ applied the measure adequate lymph node retrieval and evaluation. Adequate sampling was defined as an examination of at least 12 lymph nodes, based on the UICC and the AJCC definitions. Population-based data from the National Cancer Institute's (NCI's) Surveillance, Epidemiology, and End Results (SEER) program was used for this purpose. Exclusion criteria were clearly mentioned (in situ or metastatic disease, patients with prior malignancies, non-adenocarcinomas, patients with appendiceal malignancies who underwent radical surgery and did not receive neoadjuvant radiation). The study evaluated the mean and the median number of lymph nodes examined. More traditional ratio-based measures of the proportion of patients who had no lymph nodes examined and the proportion of patients who had at least 12 nodes examined were also calculated. Trends over time were evaluated and tested for any association with patient age, race/ethnicity, sex, or geographic location; year of diagnosis; or tumor stage, grade, and anatomic site. The study was limited by its use of population-based data with partial information on patient factors, tumor factors, and specimen adequacy. The measure does not account for the fact that lymph node evaluation depends on the surgical quality, which could lead to variability in the number of nodes removed. Additionally, the number of nodes in a surgical specimen may vary from one patient to another.

An Institute of Medicine (IOM) report¹³ proposed a similar measure to assess the quality of cancer care in Georgia. In this case, appropriate histological assessment of CRC was defined as an assessment of 12 or more lymph nodes; this measure is thus essentially equivalent to one of the variations of the measure described by Baxter et al.,⁶⁸ above.

Two other measures used an essentially identical structure, but prescribed retrieval of different numbers of lymph nodes for their thresholds. The National Committee for Quality Assurance (NCQA)⁶¹ proposed 14 lymph nodes, based on NCCN guidelines.²⁴ The Colon Cancer Workgroup,⁵⁴ on the other hand, proposed eight lymph nodes and based this decision on the SEER median of eight.

Galvis et al.⁶⁹ utilized a similar measure to compare the pathology specimen gross examination performance of pathology residents and pathologists' assistants. Investigators retrospectively reviewed surgical pathology reports for 176 colorectal specimens (submitted from 1997-1999) from one anatomic pathology laboratory serving a teaching center. For all specimens, the total number of lymph nodes retrieved, the number of positive nodes retrieved, and the length of colorectal specimens were recorded. The measure is not explicitly validated; however it is known that the number of lymph nodes sampled is an important predictor of CRC outcomes.^20,21 The measure was applied at a single center and needs further testing.

It should be noted that this measure (adequate lymph node retrieval and evaluation) requires documentation of the examination on operation notes and pathology gross specimen examination reports. The choice of the number of lymph nodes could be an institutional decision. A trigger that was utilized by the Colon Cancer Workgroup⁵⁴ may be insightful: this institution evaluates any situation where a hospital or a surgeon-pathologist combination regularly obtains fewer than eight lymph nodes. The group also recognizes that rectal cancer resection specimens may have fewer lymph nodes.

Steele⁷¹ analyzed the National Cancer Data Base (NCDB) to examine the trends in stage at diagnosis. Although not explicitly defined as a quality measure, the percentage of newly diagnosed CRC cases that were staged using the AJCC system was reported and used as a marker of appropriate cancer diagnosis and treatment over the time period studied. The study evaluated trends over time, census region, and hospital type. This measure can be generalized, but is limited by its inability to differentiate between those cases that were staged using pathology criteria versus those that utilized clinical criteria for staging.

Chiaverini et al.⁷⁰ utilized essentially the same measure to assess the progress in the control of CRC from 1987-2000 in Rhode Island using data from the Rhode Island Cancer Registry.

Conclusions

We retrieved two major quality measures that have been utilized in the United States for assessing the quality of pathology staging of patients with CRC. Ratings for these measures are given below in Table 5; further details are provided in Evidence Tables 3 and 4 in Appendix E^*

Adequate lymph node retrieval and evaluation is linked to an outcome of interest (better staging, leading to appropriate management and better survival) and based on evidence.^20,21,65,72–⁷⁷ It is also well developed and tested (I₅S₄T₅) and could be utilized for internal quality assessment of personnel.

It may be important to note that the number of lymph nodes examined reflects not only the quality of pathology care, but also reflects an interaction between tumor factors (right-sided colon resections may often contain a larger mesentry and more lymph nodes) and the quality of the surgical procedure (removal of an adequate number of lymph nodes, an adequate length of specimen, and surgical clearance).⁷⁸

Percentage of newly diagnosed CRC cases that were staged using the AJCC system is another measure that is well developed, has face validity, is related to an outcome of interest, and represents an important leverage point for improving the quality of CRC care.⁷¹ However, it does not distinguish between pathology and clinical staging (I₄S₃T₃).

The recommendations made herein are based on a review of currently available data. Obviously, future data or clinical realities (e.g., costs, changes in societal goals) or the development of new technologies may alter the recommendations in the future. Nonetheless, for the major recommendations, we do not expect their basic core importance to change.

Future Research

For newly diagnosed CRC cases to be staged according to the AJCC (or another) system is a necessary, but not sufficient, condition for accurate pathologic staging. Recognizing this, future research should focus on clarifying precisely what constitutes an adequate lymph node evaluation. In particular, first principles suggest that not all patients should have exactly the same number of lymph nodes evaluated, and expanding the evidence base on this subject would be of help. Also, in order to better establish the points of leverage, components of these measures that are attributable to the surgeon should be separated from those that are attributable to the pathologist.

Background

Adenomas of the gastrointestinal tract may undergo malignant transformation in the adenoma-carcinoma sequence and are regarded as precursors of most colorectal carcinomas. Colonoscopy is considered a safe and effective method of screening for colorectal adenomas, and endoscopic polypectomy has been demonstrated to prevent progression to adenocarcinoma. The term “malignant polyp” refers to a macroscopically benign-appearing adenoma in which invasive carcinoma is detected histologically in the resected specimen. With tumor infiltration through the lamina propria and possible invasion of venous and lymphatic vessels, malignant polyps have the potential for lymph node metastases, and endoscopic polypectomy may not be curative. The decision on subsequent management of malignant polyps may be difficult because of controversial studies in the literature. Several studies show adequacy of endoscopic polypectomy alone for selected cases, but some authors advocate colectomy for any invasive carcinoma arising in an adenoma. Many endoscopists tend to favor subsequent surgery as the strategy of choice for all these patients.⁷⁹ Some elements of this decision include level of invasion (probably the most important factor), the type of polyp, the presence of lymphatic or blood vessel invasion, poor differentiation of the tumor, and positive margins of resection in the polypectomy section.

Even though subsequent management decisions are difficult and must be individualized to the circumstances of individual patients, one point of general agreement is that a crucial piece of information is provided by the pathology report, the successful generation of which requires the surgeon to collect the specimen according to sound clinical principles and the pathologist to provide a comprehensive report. In some cases, the result of this report will suggest a clear plan of action: for example, for a level 0 invasion of a pedunculated adenoma a polypectomy should suffice, while a level 4 invasion of a sessile adenoma into subserosal connective tissue or microscopic cancer at the resection margin⁸⁰ suggests surgery. However, there are many intermediate cases between these two extremes for which guidelines are either not present or not helpful.

Results

No quality measures on this topic were found.

Conclusions

In the absence of any current quality measures, our recommendations are derived from basic first principles and are provided immediately below, under “Future Research.”

Future Research

The clinical decision for the treatment of a malignant polyp (e.g., endoscopic polypectomy alone, endoscopic polypectomy followed by segmental colectomy, colonic resection alone, primary colotomy and polypectomy) is not straightforward, and the evidence base is relatively modest (e.g., including the lack of guidelines). Accordingly, potential measures of the form percentage of patients with malignant polyps receiving a certain form of treatment do not seem to be a fruitful approach at present, although as the evidence base improves this kind of measure might be specified for those subgroups of patients for which the clinical decision is clear. On the other hand, a measure such as percentage of patients with malignant polyps receiving a sufficiently comprehensive evaluation (however defined), and treatment selected from a menu of appropriate alternatives (this menu perhaps specified using expert consensus) might be helpful.

Recognizing the importance of the collection and analysis of the pathologic specimen, various technical measures could be generated. The portion of Question 4 that pertains to pathology reports illustrates how such reports might be formatted.

In summary, for future research we recommend extending the pathology reports described in Question 4 to the analysis of polyps, developing more general measures addressing the overall adequacy of the evaluation of possibly malignant polyps, and developing measures that specify whether a patient received one out of a set of clinically reasonable treatments. Notably, the CAP has a checklist available for polypectomy specimens.

Background

In responding to this question, we extended the patient population to include patients with colon cancer, as well as patients with rectal cancer.

Surgery is part of the standard management of patients with colon and rectal cancer stages I, II, and III, and for selected patients with stage IV disease. In addition, chemotherapy is the standard adjuvant therapy for stage III colon cancer, and chemoradiotherapy is the standard adjuvant therapy for stage II and III rectal cancer.⁸¹ However, for stage IV colon and rectal cancer, chemotherapy alone may be appropriate,⁸¹ with surgery and/or radiation reserved for palliation of symptoms due to bulky primary tumors.

Although adjuvant therapies have improved local recurrence rates and survival, it remains clear that proper surgical resection is critical in determining the overall outcome of a patient with CRC. In regards to colon cancer, it is important to remove the tumor with adequate margins and provide the pathologist proper mesenteric tissue to allow for proper lymph node sampling. Precise surgical technique is even more critical for rectal cancer.

One of the most significant advances in the treatment of rectal cancer has been the concept of performing rectal cancer resection according to the principle of total mesorectal excision (TME). Previously, blunt dissection irrespective of the anatomic planes of the rectum was widely utilized and taught as the proper mechanism to excise rectal tumors. However, 15 to 50 percent local recurrence rates were reported using these blunt techniques. In 1982, Heald et al.⁸² recognized that most local recurrences after rectal cancer resection were a result of inadequate and imprecise surgical excision of the rectal lymphovascular pedicle that may contain micrometastases. Moreover, they recognized that by using meticulous, sharp dissection under direct vision and not violating the anatomical planes of the visceral and parietal pelvic fascia down to the level of the levator ani muscles, the rectum and its mesentery could be removed as an intact unit. The quality of the excision can be examined by careful macroscopic as well as microscopic assessment of the resection specimen.⁸³ These techniques dramatically reduced the incidence of positive lateral margins and led to astonishingly low local failure rates while still performing a sphincter-preserving operation. Recently, several studies have concluded that using combined modalities and TME, local recurrence remains under 10 percent, with cancer-specific survival of over 70 percent²² compared with conventional blunt surgical techniques, where local failure rates range from 15 to 45 percent.^84,85 In these studies, precise anatomic dissection also prevented injury to the sympathetic and parasympathetic nerves in the pelvis, which reduced the previously high incidence of bladder and sexual dysfunction following rectal cancer surgery. TME has rapidly been adopted throughout the world.

The appropriate approach to surgery for rectal cancer depends to a large extent on tumor characteristics as measured with such tools as endoscopic ultrasound (EUS), computed tomography (CT) scans, magnetic resonance imaging (MRI), and history and physical examination. NCI guidelines currently recommend rigid proctoscopy, EUS, or MRI and an abdomino-pelvic CT to stage the cancer preoperatively. By implication, measures that focus on sphincter preservation rates should take the characteristics of the tumor into consideration.

Regardless of the location of the tumor, one of the critical tasks of the surgeon is to generate a high-quality specimen for the pathologist to analyze. Classically, this has been perceived as removing the major vasculature to the section of colon being removed or a total mesorectal excision for rectal cancers. This technique assures adequate lymph node harvesting for the pathologist, although the precise number of nodes cannot be described by the surgeon. These measures are difficult to ascertain and are again a function of the surgeon's technical abilities. These should be considered valid measures of the quality of surgery since they determine the subsequent pathology staging of the tumor and management decisions based on the stage. Survival of CRC increases with the number of lymph nodes recovered, regardless of their positivity.⁶⁵

An important point to note in any outcome measure related to surgery is the need to examine all management processes beyond surgical resection (as appropriate to stage and location of tumor): was adjuvant therapy offered subsequent to surgery?; did the patient refuse it?; was radiotherapy offered?; was surveillance performed?

Results

We identified a number of measures that have been utilized to assess the quality of surgery in colon and rectal cancer. These are either general measures that assess whether surgery was performed, or technical measures that assess the quality of the surgical technique employed. Each measure takes the form of a numerator describing the number of eligible patients receiving some form of surgery for colon cancer, rectal cancer, or for both types of cancers, divided by a denominator describing the number of eligible patients.

Regarding the denominator, there is considerable variation: some measures focus on colon cancer only, some focus on rectal cancer only, and some focus on both. The best approach to quality measures is to focus on a single location and stage (e.g., stages I and II colon cancer, or stage III colon cancer) or to present the data disaggregated by location and stage.

Newcomb et al.⁸⁶ and Demissie et al.⁸⁷ have studied rates of refusal in patients and have determined that a significant percentage of patients refuse to undergo surgery (from 9 to 33 percent, with refusal rates being higher among African-Americans) and chemotherapy (from 16 to 33 percent, with refusal rates being higher among the elderly). With these observations in mind, all measures should incorporate patient preference within them, or else adopt a measure that assesses “intent-to-treat rates” instead of rates of “treatment received.”

Measures That do not Account for Stage. Several general process measures have been utilized to assess the surgery-related quality of care in all available colon and rectal cancer patients, without accounting for stage and/or location. For example, such a strategy would be necessitated when available administrative databases do not contain this information. A common and very serious problem with all these measures is that not performing surgery may be appropriate for some stages of colon and rectal cancer (e.g., stage IV colon and rectal cancer). Accordingly, these measures are simply listed here and are not considered further.

The aforementioned measures include:

• Rate of surgery for CRC.^88,89

• Percentage of patients with CRC who received surgery.^86,90,91

• Number of CRC patients who received surgery as part of primary treatment.⁹²

Measures That Assess Appropriate Receipt (or Non-Receipt) of Surgery for Colon or Rectal Cancer for Patients Having Cancers of Various Location and Stage. Demissie et al.⁸⁷ analyzed the SEER database (which contains specific information regarding physician recommendations regarding surgical treatment, the actual receipt of the treatment, and reasons for not receiving treatment) to determine non-receipt of surgery. The numerator and denominator were well defined. Results were presented by stage of tumor and histologic type. The study considered the intention of the surgeon to treat as an important factor in categorizing patients as “having received standard surgical treatment;” a sensitivity analysis was performed using the actual treatment received to observe if the results were significantly different.

Cooper et al.⁹³ utilized Medicare provider analysis and review files from 1984 to 1987 to study whether race was associated with differences in curative surgical resection rates. The numerator and denominator were moderately well defined. The results were presented by race, by differentiating black and white patients in the population studied. Differences in race-related patient preferences may have accounted for the observed difference in receipt of surgical resection among black and white patients.

Merrill et al.⁹¹ also analyzed the rate of surgical resection by using two tumor registries, part of the SEER databases, Medicare inpatient claims data, two HMO databases, and 1990 U.S. census data. An appropriate definition was used to specify the numerator and denominator, for all stages of tumor 0–IV. The aim of the study was to compare treatment utilization in HMO and fee-for-service (FFS) settings for Medicare colorectal cancer cases.

Steele⁷¹ used the NCDB to assess trends in colon and rectal cancer diagnosis and management in order to provide benchmarks for clinicians and policymakers. An appropriate rate was used, and the numerator and denominator were specified: percentage of stage 0–III colon cancer patients who underwent surgery, and stage III patients who received chemotherapy in addition to surgery. AJCC staging was used, and the measure was based on AJCC guidelines. The NCDB may overstate performance because of its voluntary nature.

Roetzheim et al.⁹⁴ studied the impact of health insurance and race on colorectal cancer surgery using the measure percentage of patients with CRC who underwent cancer-directed surgery. Cancer-directed surgery rates were identified from the Florida Cancer Database System, supplemented with information from discharge abstracts using Current Procedural Terminology (CPT) codes. The measure was precisely defined, and confounders examined were age, sex, marital status, household income, and educational level; site of cancer; stage at diagnosis (information was not detailed); residence; and comorbid scores. The study evaluated colon and rectal cancer patients separately and concluded that treatments received by patients varied considerably according to insurance payer: those with HMO insurance were more likely to receive definitive surgical treatment, and this difference increased with advancing cancer stage. Among non-Medicare patients, those having Medicaid and the uninsured were less likely to receive surgical treatment than private FFS patients. There were no differences between races in receipt of definitive surgical treatment.

Temple et al.⁹⁵ assessed two measures for stage IV rectal cancer: cancer-directed surgery rates and metastatectomy rates. The general approach regarding the measure was well structured, numerators and denominators were defined, and the SEER-Medicare registry was used as the data source. The main difficulty with this measure is that the evidence for a health benefit from removing the primary cancer in patients with stage IV cancer is uncertain and likely to be highly patient-dependent. In many cases, such patients may derive greater benefit from more aggressive systemic chemotherapy than from surgical intervention.

O'Connell et al.⁹⁶ compared cancer-directed surgery rates by age using data from the 1991-1999 SEER database. The measure was precisely specified, AJCC staging was used for defining the tumor stage, and results were appropriately broken out by stage. Patient preference was not accounted for. There was no validation of the receipt of surgery beyond the usual policies of the SEER registry, which are quite stringent: accuracy is ensured by checking the accuracy of sample cases by re-abstracting data from medical records every year. The completeness has been recorded as 98 percent.⁹⁷ The study is limited by the age and geographic constraints of the linked SEER-Medicare files.

O'Connell et al.⁹⁸ later assessed cancer-directed surgery rates in order to compare rectal cancer patient outcomes in younger and older patients using the SEER registry.

Hyman et al.⁹⁹ also measured the curative resection rate by utilizing a voluntary registry created by the Vermont chapter of ACS with the Vermont Program for Quality Health Care. The study did not specifically aim to compare quality of care among participating surgeons, but rather to compare quality of care with national benchmarks.

A better defined measure was percentage of patients with colon or rectal cancer who had curative resection.¹⁰⁰ Curative surgery in this 10-year retrospective study using a tumor registry was defined as surgery performed in the absence of documented spread. The well-defined measure was utilized to compare surgery rates and outcomes in black and white patients with CRC with some risk-adjustment. There were no significant differences in treatments received between black and white patients; but significantly worse survival was noted among black patients in spite of similar rates of chemotherapy and radiation treatments received by patients of the two races.

Tropman et al.¹⁰¹ assessed the rates of appropriate primary therapy (wide surgical resection and anastomosis) for CRC as defined by the NCI guidelines. The denominator included patients with stages I–III colon cancer, all of whom should receive surgery in the absence of contraindications or refusal. Medical records and physician office records were used to collect data. The only addition that may be suggested is the documentation of patient preference for or against surgical treatment.

Govindrajan et al.¹⁰² studied the impact of race on rate of surgery in stage II and III colon cancer patients in a small group of CRC patients over the period 1984-1994 in a single institution through a retrospective analysis of a hospital tumor registry. The numerator and the denominator were appropriately defined, and results were presented by location of the tumor and stage of the disease. The degree to which these results can be generalized is unknown, as is the quality of the data obtained from this registry.

Sphincter-Saving Surgery for Rectal Cancer. Schrag et al.¹⁰³ used the SEER-Medicare database to study ostomy rates after adjusting for various patient characteristics. The measure is well defined, risk-adjusted, and relevant. However, details regarding the tumor (e.g., size of tumor, distance from the anal verge) were not considered.

Hodgson et al.¹⁰⁴ utilized a similar measure, the rate of colostomy for patients with rectal and rectosigmoid cancers. The adequately defined measure was studied by examining the California Cancer Registry. The validity and risk-adjustment for this study was fair.

Morris et al.¹⁰⁵ used the SEER database to examine the rate of sphincter-preserving operations. However, this study did not allow the authors to examine the anatomical location of the tumor, which would influence the type of operation performed. The measure used precise specifications for including patients, and the data sources were fairly reliable. Risk-adjustment was not adequately performed.

A similar measure was used by Simons et al.,¹⁰⁶ who analyzed a regional cancer surveillance program registry (the quality of the registry is unknown) to study the sphincter-saving procedure rate for patients with different stages of rectal cancer. (The main predictor of interest was hospital volume, and the data set was matched by stage.) The numerator and the denominator were well defined.

In a recent study, Purves et al.¹⁰⁷ compared rates of sphincter-saving procedures and abdomino-pelvic resections among surgeons with varying caseload volumes. They utilized the 1997 Nationwide Inpatient Sample (NIS) and the Hospital Cost and Utilization Project (HCUP) databases; the measure was well defined, but the data sources had no information regarding tumor staging.

The NCQA⁶¹ proposed the quality measure percentage of rectal cancer cases that received a sphincter preservation procedure at time of surgery. It was based on the NCCN guideline,²⁵ was well defined, and suggested that the data sources could be cancer registries and claims data.

Schrag et al.¹⁰⁸ followed 2,815 patients to assess the abdominal perineal resection (APR) rates. The denominator was all patients identified from the SEER-Medicare linked registry for 1992-1996, aged 65 years and older, with stage I–IV rectal cancer undergoing surgery and receiving FFS care. APR rates were compared by providers with different case volumes. A deficiency of this measure is that while APR is not the ideal procedure for most patients, it is clearly indicated for some. As currently constructed, the measure does not differentiate between those patients for whom APR is indicated.

Measures That Assess the Quality of Surgery for Colon and/or Rectal Cancer. Controlling for tumor stage, Read et al.¹⁰⁹ evaluated the impact of surgeon specialty on the extent to which patients experienced local tumor recurrence, or local control rate. Patients undergoing curative treatment for primary rectal adenocarcinoma with neoadjuvant radiotherapy followed by proctectomy were followed for 5 years using medical records, cancer registry, or physician/patient data. The measure was well specified and adjusted for tumor size and location.

One of the measures most often utilized to measure the technical quality of CRC surgery is whether the surgical specimen provided was sufficient to accurately stage the cancer. Such measures^13,54,61 are discussed under Question 1b (pathologic staging). This section will focus on measures that assess the complications and outcomes of surgery.

Measures That Assess the Complication Rates of Colon and/or Rectal Cancer Surgery. Patients greater than 80 years old with colon or rectal cancer during 1961-1987 from a single institution were studied by Coburn et al.,¹¹⁰ who calculated the postsurgical complication rate (i.e., any complication), including anastomotic leak rates. Inpatient, outpatient, and office records were utilized as data sources. The measures were not clearly specified (e.g., the period within which postsurgical complications or leak rate would be assessed was not defined), and the ability to generalize to other data sources is uncertain.

Measures That Assess Intermediate Outcomes of Surgery for Colon and/or Rectal Cancer. Thirty-day mortality and in-hospital mortality are two intermediate outcomes that can assess the quality of surgery. However, important considerations for these measures are the comorbidity score and the stage of cancer. Unless these two factors are accounted for, these measures are unlikely to be valid. Another important consideration is the nature of the lesions: whether the lesions operated upon were obstructed or perforated is important to consider because these factors are associated with a higher risk of postoperative mortality.

Rabeneck et al.¹¹¹ assessed 30-day operative mortality rates using the national VA patient treatment file. The measure is well-defined and compares mortality in elderly versus younger patients following surgical resection for colorectal cancer.

Coburn et al.¹¹⁰ also assesses 30-day mortality for patients who received operative treatment for colon and rectal cancer between 1961-1987. This measure is well defined using inpatient, outpatient, and office records as data sources. The study adjusted for obstructed and perforated lesions, but not for comorbidities.

Cooper et al.⁹³ performed a similar analysis using Medicare provider analysis and review files. Comobidities and nature of lesions were not considered.

Schrag et al.^103,108,112 compared 30-day mortality rates for low- and high-volume hospitals. In several studies, they assessed the rates for colon cancer, rectal cancer, or both cancers. They adjusted for most confounders: stage and hospital volume, comorbid illnesses, age, sex, race, socioeconomic status, and acuity of hospitalization.

Hodgson et al.¹⁰⁴also examined the impact of hospital volume on postoperative (30-day) mortality rates of patients diagnosed with rectal cancer stages I–III using state registries.

Temple et al.⁹⁵ restricted the measure to 30-day mortality for stage IV colon cancer. Using the SEER database and Medicare claims data, they defined the measure precisely and adjusted for comorbidities. Whittle et al.¹¹³ also used perioperative mortality rate for colon cancer patients enrolled in Medicare. The measure was not adjusted for by disease stage, comorbid factors, or emergency presentation, all which can affect peri- and postoperative survival rates.

A well-defined measure was used by Agarwal et al.¹¹⁴ to assess 30-day operative mortality rate, by analyzing a tumor registry from 1975 to 1980. Five-year survival distributions, as well as analyses by stage, were done to determine what factors influence operative mortality.

Dimick et al.,¹¹⁵ Harmon et al.,¹¹⁶ and Ko et al.¹¹⁷ assessed in-hospital mortality rates. All of them defined the numerator and denominator appropriately. All of them adjusted for comorbid conditions, nature of admission (urgent versus emergent), age, and sex of patients. Dimick and colleagues¹¹⁵ used the NIS for colon cancer resections to compare mortality by hospital volume. Harmon et al.¹¹⁶ utilized Health Services Cost Review Commission (HSCRC) data, and Ko and colleagues¹¹⁷ used the NIS to examine both hospital and surgeon volume impact on mortality rates.

Conclusions

Ratings for the measures described here are given in Table 6; further details are provided in Evidence Tables 5 and 6 in Appendix E^*. The most basic measure of the quality of surgical management is whether surgical therapy is provided as appropriate to the location and stage of the cancer. Results could potentially be broken out by stage or presented as a single summary measure, rates of appropriate primary therapy for CRC as defined by the NCI guidelines (I₄S₄T₄). Regarding the denominator of this measure, patients for whom the therapy is inappropriate should be excluded. This point is particularly problematic for patients with stage IV colon or rectal cancer, for whom the intent of the surgery is palliation rather than cure. Ideally, the measure should also distinguish between patients who were offered “appropriate” surgery and refused, and those patients who were not offered such surgery. Making these distinctions increases the data collection demands (e.g., although sometimes included within registries, such information is not included in the typical administrative database).

Surgical complication rates are another very natural measure. Most variations on this measure focus on mortality rates (e.g., 30-day mortality [I₅S₄T₅], in-hospital mortality [I₅S₃T₃]), although a delineation of the actual complications would be helpful as well, this latter type of measure being relatively underdeveloped. Presumably, such a measure should be both stratified by stage and location, and also risk-adjusted to take into account other clinical characteristics of the patient.

For patients with rectal cancer, various measures have been proposed regarding the rate of sphincter-saving surgery (I₅S₄T₄). This is an outcome of importance to patients. However, quality measures should consider detailed information about the anatomical location and other characteristics of the tumor, which is not yet the case in practice.

Technical measures associated with surgery are underdeveloped.

The recommendations made herein are based on a review of currently available data. Obviously, future data or clinical realities (e.g., costs, changes in societal goals) or the development of new technologies may alter the recommendations in the future. Nonetheless, for the major recommendations, we do not expect their basic core importance to change. For example, no matter how surgery changes technically, it will not be able to cure all stage III patients.

Future Research

Staging Workup. Considering the relevance of an appropriate staging workup and the availability of NCI guidelines defining it, quality measures that assess this workup would be appropriate.

Appropriate Primary Therapy for Stage IV CRC. Although the NCI standards have been used effectively for stage I–III colorectal cancers, there does not seem to be a consensus on the effectiveness, timing, or type of surgery that should be performed on a patient with metastatic disease. The reason for this may be multi-factorial, including physicians' historic pessimism, the heterogeneous nature of stage IV disease (e.g., ranging from a single liver metastasis to carcinomatosis), the risk of obstruction, and the use of new highly potent chemotherapy agents. At present, this heterogeneity greatly complicates the use of quality measures based on anything but data sources that describe in detail both the patient's clinical condition and preferences. The question of metastasectomy for patients with stage IV CRC remains controversial. Future research should focus on determining appropriate management modalities for stage IV CRC. Quality measures can only be developed subsequent to this research.

Improvement in Sphincter-Preserving Data. Following complete oncologic resection, sphincter-preservation remains an important aspect of rectal surgery for the patient. However, the current data comparing sphincter-preserving operations to APRs reveal very high APR rates. However, these data often lack critical details, including the precise preoperative measurement of the tumor from the anal verge, the assessment of the tumor's involvement of the anal sphincter complex, the “fixed” nature of the tumor, preoperative sphincter function of the patient, the body habitus of the patient, and finally the patient's preference for kind of surgery performed. Ideal measures should account for all these aspects that lead to decisions regarding APR.

Anastomotic Leak Rates. Quality measures need to be developed to determine what factors influence this potentially life-threatening risk following colon or rectal excision. In regards to rectal cancer, numerous factors may affect the leak rates, including comorbid illnesses, obesity, and use of preoperative chemoradiation. Finally, surgical experience and potential volume/outcome measures could be examined for relationship to this complication rate.

Recurrence Rates Following Resection of CRC. For colon cancer, recurrence rates may be influenced by the number of lymph nodes harvested during the colectomy. With rectal cancers, the use of TME has clearly decreased the local recurrence rates; however, the margin that should be considered adequate is controversial. Twenty years ago, the surgical dictum was to obtain a 5 cm margin; however, within the last 10 years, this has been reduced to a 2 cm margin, and now even this distance is being questioned. There clearly is a need for a well-constructed trial to examine this critical surgical issue, the results of which could lead to specifying the optimal surgical margins as part of a quality measure.

Improvement in Quality of Rectal Surgery. Quality measures need to be developed to ascertain that the operation was performed according to the principles of TME and the specimen retrieved is ideal.

Complications of Rectal Cancer Surgery Due to the Use of Neoadjuvant Therapy. Although several recent studies demonstrate no apparent increase in surgical complications with the use of preoperative chemoradiation, newer and potentially more toxic approaches may further complicate this already difficult procedure. If so, this should be accounted for by quality measures. Bladder and sexual dysfunction also need to be assessed, although recent evidence indicates that these complications may be multifactorial, with radiation playing a much larger role.118–122

Appropriate Treatment for T1 Rectal Cancer. This is very controversial. Recommendations vary from transanal excision with or without chemoradiation to proctectomy. Unless there is adequate research leading to consensus regarding treatment, no quality measure can be developed in this area.

Background

In responding to this question, we extended the original patient population to include patients with metastatic but potentially curable (hepatic/pulmonary-resectable) disease. We also extended the definition of adjuvant therapy to include chemotherapy, radiation therapy, and their combination.

We identified a number of measures that have been utilized to assess the appropriate use of adjuvant chemotherapy. Each measure takes the form of a numerator describing the number of eligible patients receiving some form of adjuvant therapy (e.g., adjuvant chemotherapy, adjuvant chemotherapy plus radiation therapy) divided by a denominator describing the number of patients in whom the treatment is indicated. The 1990 National Institutes of Health (NIH) Consensus Guidelines state that patients with stage II and III rectal cancer should receive adjuvant chemotherapy plus radiation therapy, whereas patients with stage III colon cancer should receive chemotherapy alone.¹⁵ Although most of the quality measures regarding appropriate use of adjuvant chemotherapy and radiotherapy for patients with colorectal cancer used the 1990 NIH consensus guidelines as a benchmark, it is important to note that these guidelines may no longer represent state-of-the-art treatment for all patients. For example, for carefully selected patients with T1-2N1 or T3N0 rectal cancer, some authorities have suggested that omission of adjuvant radiation therapy be evaluated in clinical trials.¹²³ More recent recommendations from the American Society of Clinical Oncology say that adjuvant chemotherapy might also be clinically reasonable, on a case-by-case basis, for patients with stage II colon cancer.²³ The NCCN guidelines state that both chemotherapy and radiation therapy might be considered for palliation of patients with stage IV rectal cancer,²⁵ and chemotherapy plus monoclonal antibodies (cetuximab and bevacizumab) as palliation for patients with stage IV colon cancer,²⁴ at least among those patients that can tolerate these therapies. Regarding resected stage IV colorectal cancer, the NCCN guidelines also state that patients who have completely resected liver metastases should be offered 6 months of adjuvant chemotherapy.

Regarding the denominator, there is considerable variation in stage of cancer: this includes reporting the results of stages I, II, III, and IV separately; limiting consideration to a single stage (e.g., stage III); grouping stages II and III; and grouping stages into local (stages I and II), regional (stage III), and distant (stage IV). Some measures focus on colon cancer only, some focus on rectal cancer only, and some focus on colorectal cancer without differentiating between the two. The best approaches are to focus on a single location and stage (e.g., stage III colon cancer) or to present the data disaggregated by location and stage (e.g., stage I rectal cancer, stage I colon cancer, and stage IV colon cancer). It is also reasonable to combine those circumstances where the guideline recommendation is the same (e.g., stage II rectal cancer and stage III rectal cancer). The approach to be avoided is the grouping of circumstances for which the guideline recommendation differs (e.g., the recommendation for stage III rectal cancer is chemotherapy plus radiation therapy, while the recommendation for stage III colon cancer is chemotherapy only, so the grouping of stage III CRC should be avoided).

It should be noted that there is substantial overlap in the measures considered under Questions 2c and 2d. As an organizing principle, all general measures that described chemotherapy and/or radiation therapy are included under Question 2c. The primary focus under Question 2d is technical measures pertaining to radiation therapy (regardless of whether this radiation therapy is given with chemotherapy).

Results

Measures That do not Specify Stage and/or Location. A few versions of the basic measures do not specify stage of disease. Because the clinical issues are so different among patients with different stages of disease, measures that do not explicitly include stage cannot be recommended. Examples include Dominitz et al.,⁸⁸ which used an administrative database that did not include information on stage, the National Patient Treatment File from the Department of Veterans Affairs. This was also attempted by Wudel et al.¹⁰⁰ in a circumstance where the sample size was too small to subdivide patients by stage. Coia et al.¹²⁴ used the Patterns of Care Study database, a nationally based sample of patients receiving radiation therapy during 1988, to inquire how many patients receiving radiation for cancer of the colon or the rectum also received surgery. Beginning with the subset of patients receiving radiation, rather than all patients with disease of a certain stage, makes the measure impossible to map back to an inception cohort of all patients with a certain stage of disease, and thus difficult to interpret. Coburn et al.¹¹⁰ analyzed the effect of age on the provision of adjuvant therapy, but did not break out the analysis by stage (or provide a precise description of the database used).

Measures That Use Suboptimal Combinations of Stage and/or Location. The measure by Newcomb and Carbone⁸⁶ makes an inadequate differentiation between types of cancer, using the categories of local, regional, and distant colorectal cancers, with colon and rectal cancers grouped (i.e., the measure does not recognize that the guideline recommendations within the category “regional” differ according to location and stage). Stage was based on a state tumor registry, and adjuvant therapy was based on patient report (which is unvalidated). This version of the basic adjuvant therapy measure is not recommended.

The measure by Govindarajan et al.¹⁰² also makes an inadequate differentiation between types of cancers. Medical records of patients with stage II or III colon or rectal cancer treated at a hospital in Arkansas between 1984 and 1997 were used to report the numbers receiving surgery, radiation, and chemotherapy. Not only were the data for stages and locations with different guideline recommendations combined, it was not possible to tell how many patients had, for example, both surgery and radiation (e.g., the number of patients with surgery, regardless of radiation status, is presented). This approach is not recommended.

Rogers et al.⁹⁰ identified patients with CRC from Tennessee Medicare and Medicaid files over the period 1984-1994. The primary intention was the elucidation of racial differences in processes and outcomes of care, including the use of chemotherapy, in stage III colorectal cancer. Moreover, there was no validation of the very low use of adjuvant therapy against medical records, suggesting that these administrative data files might not have been appropriate for this purpose. Combining stage III colon cancer and stage III rectal cancer is suboptimal, and this measure is not recommended.

NCQA⁶¹ described the measure percent of late stage (stage ≥ III) colon cancer patients that received one or more courses of adjuvant chemotherapy within 1 year of initial cancer surgery and specified the window for receiving adjuvant therapy (to allow inclusion of therapy up to 1 year from surgery), the stages of colon cancer when adjuvant therapy is administered (stage III and greater), and the number of courses of chemotherapy. A potential problem with this measure is that adjuvant therapy for resected stage IV patients is not a standard recommendation because there is not universal agreement on its benefits, so application of this measure could lead to the impression that too few patients are receiving adjuvant therapy.

Measures for Stage IV Colorectal Cancer. Temple et al.⁹⁵ followed patients identified from the SEER-Medicare linked registry for 1991-1996, aged 65 years and older, with stage IV colon or rectal cancer, enrolled in a FFS health plan. Patients were cross-classified according to the use of primary-cancer-directed surgery, chemotherapy, and radiation therapy. The use of adjuvant chemotherapy, with or without radiation therapy, might be considered to be a potential quality measure, although it is not supported by a guideline that suggests that this strategy is likely to be beneficial for all patients. In particular, this is a decision that should be made on a case-by-case basis in consideration of the characteristics of the tumor, comorbidities (e.g., the poor performance status of many patients would preclude the use of chemotherapy), and patient preferences. The most reasonable application of such a measure would be in a database that includes a record of those cases where a physician considered the potential application of adjuvant therapy, but in consultation with the patient decided against it. In the absence of such information, this is not a version of the basic quality measure that can be strongly recommended.

Measures for Stage III Colon Cancer. Potosky et al.¹²⁵ followed patients in the SEER registry (1990-1991 and 1995), aged 21 and above, who had completed surgery. For approximately 81 percent of sampled cases, the offer or administration of adjuvant therapy was verified by physician contacts. The authors note that “doctor verification significantly improves the completeness of chemotherapy ascertainment because adjuvant chemotherapy is frequently given outside the hospital setting, and a note of any planned chemotherapy is typically not reported in the hospital record.” The numerator was the number of patients “offered, recommended or administered adjuvant 5-Fluorouracil (5-FU) plus levamisole or leucovorin.” This version of the measure reflects physician intention rather than actual receipt of therapy, since patients that are offered but refuse the therapy are nevertheless included in its numerator. Adjuvant chemotherapy use was broken out by various sociodemographic characteristics. The study does not explicitly validate the quality measure.

Hyman et al.⁹⁹ reported data from a surgeon-initiated registry in Vermont during 1999-2001. The measure is the number of stage III colon cancers that were offered referral for adjuvant therapy. Because this is a prospectively collected registry, this information is likely to be collected with particularly high accuracy. The degree to which this database can be generalized is uncertain.

Mahoney et al.¹²⁶ followed 69 patients identified from a New Jersey tumor registry for 1989-1996 with stage III colon cancer and focused on a chart-based examination of the reason that adjuvant chemotherapy might not be provided. Of 35 patients not receiving chemotherapy therapy, 12 were not offered such therapy, 11 refused, 7 were too old, and 5 had significant concomitant disease. Although a small study, this provides support for the notion that administrative databases will understate provider compliance with guidelines by omitting information that might document that this therapy was either offered but declined, or else might not have been clinically appropriate.

Ayanian et al.¹²⁷ followed patients identified from three regional registries within the California Cancer Registry during 1996-1997 who were 18 or older, with stage III adenocarcinoma of the colon, surviving at least 30 days postsurgery. Information about adjuvant therapy was obtained from physician surveys and office records. The numerator of the quality measure was the number of patients receiving adjuvant chemotherapy. The study does not validate the quality measure, but does include some validation of the data sources. The sensitivity of routinely collected registry on receipt of chemotherapy (i.e., using medical record review as the gold standard) was 87 percent. Various sociodemographic characteristics were considered. Results from a single state registry might not reflect national patterns of care.

Schrag et al.^103,112,128 followed patients identified from the SEER-Medicare linked registry for 1991-1996, aged 65 years and older, with stage III colon cancer, enrolled in a FFS health plan and surviving at least 3 months. The numerator was the number of these patients who received chemotherapy within 3 months of surgery. Among others, the impacts of age, race, and hospital volume were considered. There was no validation of the receipt of chemotherapy beyond the usual policies of the SEER registry, although an unpublished reference asserting that the SEER data has 90 percent sensitivity for identifying the use of chemotherapy was cited. The study is limited by the age and geographic constraints of the linked SEER-Medicare files.

Sundararajan et al.¹²⁹ followed patients identified from the SEER-Medicare linked registry for 1992-1996, aged 65 years and older, with stage III colon cancer, enrolled in a FFS health plan and surviving at least 4 months. The numerator was the number of patients receiving 5-FU within 4 months. Strengths and weaknesses of the measure are essentially identical to the discussion of Schrag et al. above.

Keating et al.¹³⁰ utilized SEER-Medicare linked data for 1993-1999 and followed patients aged 66 years and above, with stage III colon cancer, who were enrolled in FFS Medicare Parts A and B. The numerator was the number of patients who received adjuvant chemotherapy. County-specific utilization rates were calculated and compared according to market share of FFS plans versus HMOs. The study does not validate the quality measure and is limited by the age and geographic constraints of the SEER-Medicare linked files. Use of adjuvant chemotherapy is not validated against medical records (with the exception of the usual SEER Registry procedures) and might be underestimated.

Beart et al.⁴⁸ presented a similar analysis, subdivided by both location and stage, for patients treated in 1983 and 1988 in the national survey of the Commission on Cancer, a nationally based survey including up to 25 cases per hospital per year. The numerator was the number of patients receiving chemotherapy, of all stage III colon cancer patients.

Tropman et al.¹⁰¹ followed patients identified from tumor registries in North and South Carolina during 1991 and 1996 and supplemented by review of medical records. The numerator was the number of patients that were treated with the consensus-based guidelines at the time of the cancer. This application was otherwise unremarkable.¹³¹

The Colon Cancer Workgroup⁵⁴ attempted to refine the numerator of the basic quality measure further by specifying “lymph node positive colon cancer” instead of using the terminology “stage III.” Specifically, the measure is defined by the declarative statement “adjuvant chemotherapy is administered to patients with lymph node positive colon cancer.” It is noted that older patients and those patients with many comorbidities might be poor candidates for chemotherapy. This refinement of the basic measure was not explicitly validated beyond the argument for its face validity.

The IOM, in Assessing the Quality of Cancer Care,¹³ described the measure adjuvant chemotherapy after surgery for stage III colon cancer. It is well defined: “number of patients with stage III colon cancer who receive a full course of adjuvant chemotherapy after surgery, divided by the number of node-positive stage III patients who undergo surgery.” The rationale is as before, the primary distinction being the basis of the measure in the more recent 2005 guidelines from the NCCN.²⁴

Measures for Stages II and III Rectal Cancer. Ayanian et al.¹²⁷ followed patients identified from three regional registries of the California Cancer Registry during 1996-1997 who were 18 or older, with stage III adenocarcinoma of the colon, surviving at least 30 days postsurgery. Information about adjuvant chemotherapy and radiation therapy was obtained from physician surveys and office records. The numerator of the measure was the number of patients receiving both adjuvant chemotherapy and radiation therapy. The study does not validate the quality measure, but validates the data sources. The sensitivity of routinely collected registry data for whether patients actually received chemotherapy was 87 percent, while the sensitivity for radiation therapy was 93 percent. Various sociodemographic characteristics were considered. Results from a single state registry might not reflect national patterns of care.

Keating et al.¹³⁰ utilized SEER-Medicare linked data for 1993-1999 and followed patients aged 66 years and above, with stage II or III rectal cancer, who were enrolled in FFS Medicare Parts A and B. The numerator was whether patients received adjuvant chemotherapy and radiation therapy. County-specific utilization rates were calculated and compared according to market share of FFS versus HMOs. The study does not validate the quality measure and is limited to the age and geographic constraints of the SEER-Medicare linked files. Neither adjuvant chemotherapy nor radiation therapy was validated against medical records (with the exception of the usual SEER Registry procedures), and might be underestimated.

Schrag et al.¹³² followed patients identified from the SEER-Medicare linked registry for 1992-1996, aged 65 years and older, with stage II or stage III rectal cancer, enrolled in a FFS health plan and surviving at least 4 months. Two versions of the numerator were given: the number of these patients who received adjuvant chemotherapy (with 5-FU) and radiation therapy within 4 months of surgery, and the number of these patients that received radiation alone; the number of patients receiving adjuvant chemotherapy alone was also noted. Among others, the impacts of age and race were considered. There was no validation of the receipt of chemotherapy and radiation therapy beyond the usual policies of the SEER registry, so it is possible that these are underreported. Results are also presented separately by stage. The analysis recognizes that radiation might be given preoperatively, thus making the distinction between preoperative and postoperative use of radiation therapy. The study is limited by the age and geographic constraints of the linked SEER-Medicare files.

Hyman et al.,⁹⁹ mentioned above, reported data from a surgeon-initiated registry in Vermont during 1999-2001. The measure is the number of stage II or stage III rectal cancers that were offered referral for adjuvant therapy. Presumably, this would be the stage at which radiation therapy would be considered as well, although it was not precisely specified whether this adjuvant therapy included chemotherapy, radiation therapy, or both. Because this is a prospectively collected registry, this information is likely to be collected accurately. The degree to which this database can be generalized is uncertain.

Potosky et al.¹²⁵ followed patients in the SEER registry (1990-1991 and 1995), aged 21 and above, who had completed surgery. The numerator was the number of patients “offered, recommended or administered adjuvant 5-FU plus levamisole or leucovorin.” For approximately 81 percent of sampled cases, the offer or administration of adjuvant therapy was verified by physician contacts. While the data were presented separately according to whether the numerator was the number of patients receiving chemotherapy only, the number of patients receiving radiation therapy alone, or the number of patients receiving both, the authors considered the primary quality measure to be the receipt of chemotherapy (i.e., regardless of radiation). Adjuvant chemotherapy use was broken out by various sociodemographic characteristics. The study does not explicitly validate the quality measure.

Neugut et al.¹³³ followed patients identified from the SEER-Medicare linked registry for 1992 and 1996, with stage II or stage III rectal cancer, enrolled in a FFS health plan and surviving at least 4 months. The numerators of the potential measures were the number of patients receiving chemotherapy only, the number of patients receiving radiation therapy alone, and the number of patients receiving both. Among others, the impacts of age, sex, and race were considered. There was no validation of the receipt of chemotherapy and radiation therapy beyond the usual policies of the SEER registry, so it is possible that these are underreported. Results are also presented separately by stage. The study is limited by the age and geographic constraints of the linked SEER-Medicare files.

Reports That are Disaggregated by Stage and Location (or Stage Only). Any report that disaggregates results by location and stage has the potential to create a quality measure for each available subgroup. For example, Steele et al.¹³¹ uses data on colon cancer cases from the NCDB during 1985, 1988, and 1990; breaks this out into stage 0, stage I, stage II, stage III, and stage IV; then classifies treatment as surgery only, surgery and chemotherapy, surgery radiation and chemotherapy, other, and none. An example of a possible quality measure is the number of patients with stage III cancer receiving surgery and chemotherapy. Steele⁷¹ presents a similar analysis using both colon and rectal cancer patients (i.e., subdivided by both location and stage) from 1991, as does Steele¹³⁴ using patients from 1992. Jessup et al.¹³⁵ presents a similar analysis of this data base for the years 1986, 1987, 1988, 1990, 1991, 1992, and 1993. Participation in the NCDB is voluntary and thus has the potential to skew the results in the direction of overstating performance.

Beart et al.⁴⁸ presents a similar analysis, subdivided by both location and stage, for patients treated in 1983 and 1988, in the national survey of the Commission on Cancer, a nationally based survey including up to 25 cases per hospital per year.

Roetzheim et al.⁹⁴ presents a similar analysis, subdivided by location and stage (but not both), for patients treated in 1994, from Florida's statewide cancer registry. Because the guidelines recommendations for stage III colon cancer differ from those of stage III rectal cancer, it would have been preferable to disaggregate the analysis by both location and stage.

Retchin and Brown¹³⁶ collected data using medical record review by nurse abstractors (for a comparison of HMO versus FFS, after first identifying potential cases from administrative files), and disaggregated the data by stage but not location and stage. Inter-rater reliability of the abstraction was assessed, as was a comparison versus a sham chart. Because the guideline recommendations for stage III colon cancer differ from those of stage III rectal cancer, it would have been preferable to disaggregate the analysis by both location and stage.

Other: Intermediate Process Measures. A quality measure developed by Oliveria and colleagues¹³⁷ is referral to a medical oncologist for consideration of adjuvant chemotherapy. This intermediate process measure is indirectly based on the 1990 NIH Consensus Statement on chemotherapy utilization in stage III disease.¹⁵ Since most chemotherapy is administered by medical oncologists, “referral to a medical oncologist” is a first step in assuring compliance with this guideline. The denominator consists of members of a Massachusetts HMO newly diagnosed with CRC from 1997 to 1999. For purposes of assessing chemotherapy utilization, the analysis was performed by stage. The numerator is the subset of these patients referred to a medical oncologist within 4 months of surgery. Data were obtained from the computerized database of the HMO. The study does not specifically validate the quality measure, but reliability of the data sources was ascertained by reviewing medical records to confirm data in the computerized database. The study is limited by the use of data from a single HMO.

Conclusions

With the exception of the intermediate process measure described by Oliveria et al.,¹³⁷ all the measures we identified took the form of number of patients receiving one or more forms of adjuvant therapy divided by the number of patients with some form of CRC. Not all of these measures were applied in a formal quality improvement context; for example, for the purpose of documenting national patterns of care, Steele^71,131 cross-classifies the universe of CRC patients into all possible combinations of location and stage, then for each of these possible combinations reports the number of patients receiving adjuvant chemotherapy, adjuvant radiation therapy, both chemotherapy and radiation therapy, and so forth. These potential quality measures more closely approximate actual quality measures when an additional analytical step is taken; that is, to select (e.g., based on guidelines) for its numerator the most appropriate item from the list of possible adjuvant therapies. (For example, the appropriate numerator for patients with stage III colon cancer is the number of patients receiving adjuvant chemotherapy).

Our presentation is organized according to the denominator of the quality measure (e.g., measures for patients with stage III colon cancer). Measures having certain denominators will not be formally rated. In particular, when an investigator uses as the denominator every possible combination of location and stage then (so long as the numerator is also reported at a high level of detail) the reader has complete freedom to define whatever quality measure is desired. We also do not formally rate those measures that fail to specify stage and/or location, or else specify suboptimal combinations of stage and/or location (here, the former is an example of the latter). The most typical such combination of a stage and location involves the grouping of circumstances for which the guideline recommendation differs (e.g., the recommendation for stage III rectal cancer is chemotherapy plus radiation therapy, while the recommendation for stage III colon cancer is chemotherapy only, so the grouping of stage III colorectal cancer should be avoided).

With this in mind, we considered measures for stage III colon cancer, stages II and III rectal cancer, and stage IV colorectal cancer. Roughly speaking, these measures were defined as the proportion of patients with stage III colon cancer receiving adjuvant chemotherapy (I₅S₄T₅), the proportion of patients with stages II or III rectal cancer receiving chemoradiotherapy (I₅S₃T₂), and the proportion of patients with stage IV colon cancer or stage IV rectal cancer receiving palliative chemotherapy (I₅S₃T₂).

Of these measures, perhaps the best developed and best-tested measure applies to patients with stage III colon cancer. It is derived from an evidence-based recommendation.^15,138–¹⁴⁰ Various investigators have attempted to fine-tune the measure; for example, by further defining stage III as lymph node positive, by specifying the maximum time between surgery and the receipt of adjuvant therapy, and so forth. Such enhancements seem clinically plausible and follow the principle that quality measures should be specified as precisely as possible, but they do not change the basic nature of the measure. In particular, applications that use slightly different approaches to specifying the above details should be sufficiently comparable for most practical purposes.

Perhaps more salient are other features of the measure and/or the databases from which this measure can be estimated. In particular, because chemotherapy is often administered on an outpatient basis, administrative inpatient databases tend to underestimate the number of patients that receive chemotherapy. Some investigators have supplemented the use of administrative databases by contacting the treating physician directly, although it must be acknowledged that such an approach is costly and time-consuming. At the very least, users of administrative databases should either find or generate some estimate of the likely number of patients for whom the administration of chemotherapy will be missed.

Another important distinction applies to the difference between the numbers of patients receiving adjuvant chemotherapy, the greater number of patients who are offered this therapy, and the (perhaps) still greater number of patients who should be offered this therapy. In some quality measurement applications the appropriate numerator is the number of patients for whom the recommended therapy is offered, for others it is the number of patients for whom this therapy is actually received. A significant number of patients refuse chemotherapy when offered. Moreover, failure to consider the number of patients who should be offered this therapy can potentially bias the comparison between groups; for example, one explanation for the lower rate of utilization of adjuvant chemotherapy among older patients is that poorer functional status is more likely to preclude its use. Comorbidities, particularly including the presence of post-operative complications, should also be considered. Investigators that work with administrative databases can probably do no better than to use standard techniques of risk-adjustment, while at the same time recognizing that these techniques cannot capture all the nuances of clinical decisionmaking. When data are collected prospectively, the ideal is to document each component of the decision: whether and why each patient was considered by the physician to be eligible or ineligible for adjuvant chemotherapy, whether this therapy was offered, whether this therapy was accepted, and whether this therapy was completed.

With the above background in mind, it can be noted that the quality measure proportion of patients with stage III colon cancer receiving adjuvant chemotherapy (I₅S₄T₅) is important, represents a significant leverage point in the care of patients that is strongly associated with outcomes, and has been used in various applications such as comparing quality of care provided over years of diagnosis, clinical variables of the condition, sex, age, race, marital status, median income of patients, geographic regions of the country, and across health care delivery systems.

Also evidence-based and well validated is the measure assessing the percentage of patients with stages II and III rectal cancer receiving chemoradiotherapy (I₅S₃T₂). Most considerations are similar to those for stage III colon cancer and are not repeated here. For stages II and III rectal cancer, somewhat different versions of the measure have been created. As discussed under Question 2d, the ideal version of the measure is the one that most closely matches the most recent recommendations from guidelines, which specify that chemoradiotherapy is the preferred method of management for essentially all such patients.

The measure proportion of patients with stage IV colon cancer or stage IV rectal cancer receiving palliative chemotherapy (I₅S₃T₂) is reasonable in principle, but suffers from the difficulty that the decision to provide this therapy should very much be made on a case-by-case basis, thus implying that prospective data collection is probably necessary.

The measure on the proportion of patients referred to an oncologist (I₅S₃T₂) pertains to an important leverage point in practice, although it must be admitted that this is neither a necessary nor a sufficient condition for the provision of high-quality care. Thus, the relationship between the quality measure and outcomes, while plausible, seems less direct that is the case for the other measures in this section.

Final ratings for these measures are summarized below in Table 7; further details are provided in Evidence Tables 7 and 8 in Appendix E^*

The recommendations made herein are based on a review of currently available data. Obviously, future data or clinical realities (e.g., costs, changes in societal goals) or the development of new technologies may alter the recommendations in the future. Nonetheless, for the major recommendations, we do not expect their basic core importance to change. For example, the need for adjuvant therapy for stage III colon cancer is unlikely to change.

Future Research

Our recommendations for future research involve better definitions of the measures discussed above and development of new measures.

Regarding the quality measure adjuvant chemotherapy administered to patients with lymph node positive colon cancer patients, it is important to determine if the same number of cases will be found for the numerator as would be found if “stage III” or “regional disease” is used to define these patients.

Regarding the quality measure use of adjuvant chemotherapy for stage III colon cancer, the ideal denominator would be patients with stage III colon cancer, who have had a complete surgical resection, and who have no contraindications to chemotherapy or severe comorbidities. Two possible ideal numerators arise. The first is the number of individuals offered chemotherapy, and the second is the number of individuals who received chemotherapy. More research would need to focus on determinants of actually receiving chemotherapy among patients offered chemotherapy. Also, research should center on which of the two numerators yields a better indicator of quality care. Furthermore, for the measure to be useful, adequate documentation must be available. Most databases discussed above do not take into account the fact that some patients will decline the offer of chemotherapy. More research is necessary on how to best derive data for whether chemotherapy was offered and received.

The receipt of chemotherapy is only a surrogate marker for whether patients received the correct agents, dose, and schedule. Data in breast cancer and lymphoma therapy demonstrate that inadequate dose intensity leads to inferior outcomes. While these data are not available for CRC, it is possible that similar observations would be made. Research on the ideal measure of “proper chemotherapy administration” is necessary. Further study is necessary on whether a global measure of chemotherapy adequacy (dose intensity) or individual measures (adequate dose, adequate frequency, adequate duration) is preferred.

The toxicities of chemotherapy can be substantial, and mortality is reported. Measures of the quality of chemotherapy could include the rate of toxicities, but more study on whether this is a true indication of quality is necessary.

Regarding the quality measure referral to medical oncologist for consideration of adjuvant chemotherapy, it is necessary to determine how this measure correlates with a measure of whether patients were offered chemotherapy. Also, this measure is entirely untested.

Although perhaps outside the current scope of interest, as a point of context it might be noted that the quality of the facility in which chemotherapy is administered is also quite important. Such issues as ensuring sterility, correct dose and concentration, correct diluent, correct vessel and tubing for holding the chemotherapy, and care in how the drug is actually administered to the patient are crucial not only for colon cancer, but for chemotherapy in general.

Background

In patients with locally advanced rectal cancer, multiple randomized controlled trials have demonstrated that adjuvant chemoradiotherapy decreases local failure and improves overall survival.^141,142 Based on these results, the NIH issued a Consensus Statement in 1990 recommending that all patients with TNM stage II (i.e., a tumor penetrating the rectal wall, without regional lymph node involvement) or stage III (i.e., any tumor with regional lymph node involvement) rectal cancer receive postoperative adjuvant chemoradiotherapy.¹⁵ Since 1990, a number of important developments have occurred which have led to significant changes in the management of patients with rectal cancer. Preoperative chemoradiotherapy has been compared to postoperative chemoradiotherapy, and has been found to improve local control and reduce toxicity while being equally effective with regard to survival.¹⁴³ The use of TME has reduced rates of locoregional recurrence compared with older surgical techniques. (Local recurrence may further reduced with the use of adjuvant radiotherapy in combination with TME, although the absolute benefit of radiotherapy is lower in patients undergoing TME as compared to older, less comprehensive resections.¹⁴⁴) Finally, refinements in risk stratification for the heterogeneous group of patients with stage II and III rectal cancer have enabled tailoring of treatment approaches based on the estimated risks of locoregional and distant failure associated with each Tumor-Node stage combination.¹²³

Given the importance of adjuvant chemoradiotherapy for rectal cancer, delivered either preoperatively or postoperatively, efforts to ensure that it is offered to patients who can tolerate it (likely the vast majority of patients who are medically fit for surgery), and that it is delivered in accordance with accepted standards are crucial components of quality medical care for rectal cancer patients. The outcomes of interest regarding adjuvant chemoradiotherapy are locoregional recurrence, overall survival, sphincter preservation, toxicity, and quality of life. The major leverage points are as follows:

• 1

  ) Adequate preoperative staging.

• 2

  ) Referral to medical and radiation oncology if tumor stage T3 or T4, or if nodal involvement suspected.

• 3

  ) Delivery of chemotherapy and radiotherapy in accordance with accepted professional standards.

The first leverage point is addressed under Question 1b. Our literature search did not reveal any measures that addressed the second leverage point. The available measures with regard to the third leverage point are discussed below.

It should be noted that there is substantial overlap in the measures considered under Questions 2c and 2d. As an organizing principle, all general measures that described chemotherapy and/or radiation therapy are included under Question 2c. The primary focus under Question 2d is technical measures pertaining to radiation therapy (regardless of whether this radiation therapy is given with chemotherapy).

Results

We identified two types of quality measures. The first type, a process measure, addresses the question of whether patients with rectal cancer receive radiation therapy as appropriate to their stage of disease. The second type of measure, a technical measure, addresses the question of whether the technique used to deliver radiation therapy conformed to accepted standards of quality.

Measures of the first type are discussed as part of the response to Question 2c, above. In particular, we first eliminated from further discussion those measures that did not specify stage and/or location,^{88,100,110,124} and also those measures that used suboptimal combinations of stage and/or location.^86,90,102 Then we noted that investigators who report the data according to the finest possible disaggregation of stage and location, and use as their numerators a cross-classification of chemotherapy versus radiation therapy, implicitly allow the users to create a guideline-based measure for each stage of disease;^48,71,94,134–¹³⁶ these measures are not discussed further. In other words, those investigators who used the most extensive possible treatment of their data by stage and location could, by implication, group the data in whatever way seems appropriate; in particular, the group could map to the guidelines when discussing grouping by stage and location. This report does not consider these further and instead only considers those studies that grouped the data before presenting results. Finally, Temple et al.⁹⁵ measures the proportion of patients with stage IV colon or rectal cancer receiving palliative chemotherapy, radiation therapy, or both. This measure has been included, although there are no guidelines or evidence regarding the use of palliative radiotherapy.

The main focus of the potential process measures was patients with stages II or III rectal cancer. Measures by Ayanian et al.,¹²⁷ Keating et al.,¹³⁰ Schrag et al.,¹³² Hyman et al.,⁹⁹ Potosky et al.,¹²⁵ O'Connell et al.,⁹⁸ Baxter et al.,¹⁴⁵ and Neugut et al.¹³³ were discussed; additional and similar applications by Morris et al.¹⁰⁵ and Demissie et al.⁸⁷ were identified as part of the literature review for Question 2d. This set of measures primarily differs according to whether their potential numerators were broken out according to the presence of chemotherapy or radiation therapy, or instead were broken out according to the presence of chemoradiotherapy (i.e., adjuvant chemotherapy plus radiation therapy). Implicitly, the former set of measures is agnostic about the most appropriate treatment for this class of patients and simply strives to describe utilization among the two clinically plausible approaches of chemotherapy and radiation therapy. On the other hand, the latter set of measures, which are based on the NCI and NCCN guidelines, explicitly assume that chemoradiotherapy is the approach of choice. The issues will be discussed more fully in the “Conclusions” section of this Chapter.

Since the above process measures have already been described under Question 2c, this section is limited to a description of the various technical measures that have been applied to radiation therapy for patients with rectal cancer.

The report by Kline et al.¹⁴⁶ serves as the basis for the potential technical quality measure adherence to radiotherapy management treatment guidelines for patients with adenocarcinoma of the rectum or sigmoid colon. This measure is based on consensus guidelines from the Patterns of Care Study (PCS) Treatment Planning Committee. It is well developed. The numerator and denominator are well specified. The data source was the 1989-1990 PCS survey. The study compares quality indicators across three different types of radiotherapy facilities including academic centers, hospital-based practices, and freestanding facilities. The following indicators of quality of radiation therapy were assessed: placement of clips by surgeon when tumor adhered to other pelvic structures; isodose distribution generated for dose prescription; beam energy used should be ≥ 4 MeV; use of at least three radiation therapy fields; fields should be shaped with custom blocks; wedges or compensators should be used as needed; port films should be taken; patients should be simulated with contrast in the rectum; a small bowel series should be done if the total dose will exceed 50 Gy; and prone setup should be used. The PCS Treatment Planning Committee guidelines also suggested that information from a pelvis CT should be used for radiation therapy planning. The fact that a random sampling of health care facilities in the U.S. was used suggests that the results are a valid representation of radiotherapy practice in the U.S. However, the study is limited by the fact that the PCS survey was conducted during the 1989-90 time period, and radiotherapy practices have changed considerably since then.

The report by Minsky et al.¹⁴⁷ formed the basis for the potential technical quality measure rate of use of modern radiation therapy techniques and adherence to recommendations of NCI sponsored randomized controlled trials in rectal cancer patients. This measure is based on recommendations by a consensus of experts (the PCS Rectal Cancer Committee) and on adherence to recommendations from randomized control trials. It is well developed. The numerator and denominator were clearly specified. The data source was the PCS national survey in radiation therapy for the years 1992-1994.¹⁴⁸ Many factors considered indicative of high-quality radiation therapy were measured in the survey, including the field arrangement; the prescription point; the beam energy; the patient position; treatment of all fields each day; treatment with a full bladder; use of a belly board; attempt to exclude the small bowel in patients treated postoperatively; use of small bowel contrast; placement of surgical clips; and inclusion of the scar in all treatment fields in patients treated with APR. Information on the radiotherapy dose to the whole pelvis and to the boost field, the radiation fraction size, and the total treatment time were obtained by the survey, although the authors do not make any assessment of what they view as the appropriate radiation dose, fraction size, or treatment duration. Therefore, adherence to guidelines with regard to those radiotherapy factors cannot be assessed from this report. The data source was not validated. The fact that a random sampling of health care facilities in the U.S. was used suggests that the results are a valid representation of radiotherapy practice in the U.S. However, the study is limited by the fact that the PCS survey was conducted during the 1992-1994 time period, and radiotherapy practices have changed considerably since then.

Conclusions

Perhaps the most important process measure for appropriate use of radiation therapy for rectal cancer identified by our literature search is the percentage of patients with stage II or III rectal cancer who received adjuvant chemoradiotherapy (I₅S₃T₂). In particular, this measure is linked to an outcome of interest (locoregional control and survival) and is based on an evidence-based recommendation.^15,25 It is well developed, fairly well tested, and a significant leverage point. An ideal version of this measure would explicitly consider possible contraindications to therapy (e.g., serious medical comorbidity, previous pelvic radiotherapy, inflammatory bowel disease, or connective tissue disorder), the distinction between being offered and receiving therapy, and the reasons for non-receipt of adjuvant therapy. Indeed, while this measure can be used for comparing the quality of care for patients of different demographic characteristics, treated in different health care settings, and with different insurance coverage, unless the reasons for non-receipt of therapy are analyzed, then the potential remains that the application of this measure could be biased (e.g., older patients are more likely to have contraindications to chemoradiotherapy than younger patients).

The other version of this process measure that might reasonably be considered is to break out the numerator into patients receiving chemotherapy alone, radiation therapy alone, both chemotherapy and radiation therapy, and neither chemotherapy nor radiation therapy. Although concurrent administration of chemotherapy and radiotherapy is the preferred approach to treatment, there may be circumstances when clinicians recommend one adjuvant modality but not the other. For example, it might be reasonable to offer adjuvant chemotherapy but omit adjuvant radiotherapy in a patient with a significant connective tissue disease (e.g., systemic lupus erythematosus or scleroderma), or in a patient who has received prior pelvic irradiation for prostate cancer or cervical cancer.

While it is vitally important to know whether a patient received chemoradiotherapy, it is also critical to assess the technical quality of its administration. Obtaining information on the details of chemotherapy and radiotherapy obviously presents a significantly greater challenge than determining simply whether chemotherapy or radiotherapy was used at all. The two potential quality measures for the technical administration of radiotherapy identified in this report demonstrate that this type of measure is feasible. Final ratings for these measures are described in Table 8; further details are provided in Evidence Table 9 in Appendix E^*. In addition to demonstrating feasibility, these studies also illustrate some of the hazards of attempting to ascertain this type of data. In particular, a significant challenge is obtaining data in a timely fashion. If there is excessive delay between data collection and reporting of the results, there is a significant risk that the results will not be applicable to modern practice. Because of this time lag, neither of the two technical radiotherapy quality measures reviewed above is ready for immediate use.

The recommendations made herein are based on a review of currently available data. Obviously, future data or clinical realities (e.g., costs, changes in societal goals) or the development of new technologies may alter the recommendations in the future. Nonetheless, for the major recommendations, we do not expect their basic core importance to change. For example, the need for adjuvant therapy for stage III colon cancer is unlikely to change.

Future Research

The limitations of the process measures for the stage-appropriate administration of radiation therapy for colorectal cancer and the avenues for future research on these process measures are discussed above in the “Future Research” section of Question 2c. Herein we provide suggestions for future research on technical measures of the quality of radiation therapy for patients with colorectal cancer.

One major drawback to both of the technical radiotherapy measures described above (adherence to radiotherapy management treatment guidelines for patients with adenocarcinoma of the rectum or sigmoid colon and rate of use of modern RT techniques and adherence to recommendations of NCI-sponsored randomized controlled trials in rectal cancer patients) is obsolescence due to the time lag between the collection of the data and the reporting of the results.

The feasibility and cost-effectiveness of real-time technical measures of the quality of radiation therapy for CRC need to be investigated. With modern telecommunications, the delay between collection of data on radiotherapy treatment parameters and reporting could potentially be greatly reduced and meaningful, timely quality measures obtained. A panel of radiotherapy experts could determine the components of high quality radiotherapy in modern day practice and a survey could potentially be conducted via telephone or the internet so that the data could be rapidly obtained and used to ensure uniform quality across radiotherapy facilities.

The principle of radiotherapy quality assurance in “real time” has been demonstrated by several ongoing radiotherapy trials administered by the Radiation Therapy Oncology Group (RTOG). The RTOG has established mechanisms for (1) credentialing of facilities participating in RTOG trials, and (2) rapid review and approval of radiotherapy plans for specific patients enrolled on these trials. The costs of reviewing every colorectal cancer patient's radiotherapy treatment plan would likely be prohibitively large. However, facilities might be encouraged/required to undergo random sampling of cases as part of a quality assurance mechanism.

The major shortcoming of both of the technical measures of the quality of radiation therapy for CRC that are reviewed in this report is that neither have been linked to outcomes such as reduced locoregional recurrence rates, improved disease-free or overall survival, or reduced complication rates. An implicit assumption has been made that the technical quality of radiation therapy is linked to at least some of these outcomes, but there is very little if any data demonstrating that such a link exists. Establishing links between specific technical measures of the quality of radiotherapy and these outcomes should be a priority of future research on this type of quality measure.

Finally, in creating measures of the technical quality of radiation therapy, one significant issue for future research is whether the quality of radiation therapy should be reported with regard to specific technical parameters or whether it can be summarized by one global measure. In other words, should the results of a quality measure's individual items should be combined into a total score (in effect, creating a single measure), or should they be assessed as individual items (in effect, creating a large number of measures)? If one argues that any deficiency in the provision of radiation therapy can potentially lead to a worsening of patient outcomes, and furthermore that the points of leverage are each individual element of radiation therapy practice, then developers of quality measures should perform at least some (although not necessarily all) analyses at the level of the individual element. On the other hand, if one argues that there will inevitably be some heterogeneity in radiotherapy practices, even among high quality radiation therapy centers, then developers should employ a global measure of the quality of radiation therapy. Research is needed to determine which of these types of measures is more important, usable, and scientifically acceptable.

Background

Surveillance refers to the ongoing followup of those at increased risk for disease after treatment. Surveillance leads to decreased incidence of CRC and improved survival;^33,149,150 however, heterogeneity of followup strategies has precluded assessment of the optimal combination of clinic visits, laboratory tests, and procedures.³³ The effectiveness of non-endoscopic or combined modality surveillance, such as following tumor markers (carcinoembryonic antigen [CEA]) or abdominal CT imaging, is controversial.

Several professional societies have endorsed guidelines for colonoscopic surveillance after curative resection of CRC (Table 9). These guidelines are similar, but differ on the timing of the first colonoscopy after a perioperative “clearing” colonoscopy. “Clearing the colon” (1) refers to a sufficiently comprehensive examination of the colon to exclude other significant neoplasia, such as adenomatous polyps or synchronous cancers, and (2) signifies that all polyps were removed.

All of the most recent guidelines specifically endorse colonoscopy for bowel surveillance. Several older articles described measures that included sigmoidoscopy with or without barium enema as part of a more generic “bowel surveillance.” Since these other two modalities are no longer endorsed for surveillance, we have excluded articles where the colonoscopy rate was not reported separately.

Sigmoidoscopy and barium enema are not currently recommended for surveillance in CRC patients and are beyond the scope of the present question. Several of the studies we considered combined colonoscopy with other bowel imaging in the numerator,¹⁵⁵–¹⁵⁷ which made these measures less useable and important. In cases where receipt of colonoscopy could not be separated from other bowel imaging modalities, we excluded the study.^155,156

Results

Postoperative (Surveillance) Colonoscopy. Several studies, using slightly different definitions, examined the measure percentage of patients receiving postoperative surveillance colonoscopy. Cooper and colleagues published two studies evaluating the same measure in the same study population.^158,159 These studies defined the quality measure the “percentage of patients with non-metastatic CRC who had colonoscopy up to 3 years after diagnosis.” This process (general) measure was based on earlier versions of current guidelines,^57,160,161 which recommended postoperative surveillance, but the timing of the surveillance (relative to the cancer resection) was not specifically included in the guideline. The numerator and denominator for the measure were ill defined. The two stages of CRC were “local” and “regional.” The data source was a merged SEER and Medicare administrative database. Neither the measure nor the data source was validated in this study, although the measure has face validity, being guideline-based. The results are limited due to the inclusion of patients aged 65 and older, restriction to the nine SEER regions, and the lack of clinical details from an administrative database to determine if these were truly “surveillance” colonoscopies or diagnostic procedures to evaluate symptoms. The data are relatively dated (diagnosis in 1991 with followup through 1994), and although subjects had to have survived at least 6 months for inclusion, other attempts to risk-adjust this measure (e.g., it is inappropriate to perform surveillance colonoscopy in patients with poor anticipated survival) were not attempted.

Lafata et al.¹⁶² used administrative data and tumor registry data and defined the measure as the “percentage of patients with CRC who received a followup colonoscopy.” “Followup” meant 2 months to 8 years postresection with curative intent. The measure is fairly well developed. The timing of the colonoscopy was not included in the measure. Cited guidelines included ASCRS 1992,¹⁶¹ ASCO 1999,¹⁶³ NCCN 1996,¹⁶⁴ and AGA 1997.⁵⁷ The setting was a large multispecialty group and affiliated HMO in Michigan. The restricted geographic location and exclusive HMO setting are limitations.

Keating et al.¹³⁰ used the SEER-Medicare database to measure the “percentage of patients who underwent colonoscopy.” They specifically examined whether or not the colonoscopy was performed 7 to 18 months after diagnosis. A comorbidity score was measured and used in an adjusted analysis that was primarily aimed at examining the relationship between managed care market share and the quality measure, but the measure did not explicitly exclude any patients with high comorbidity burden who may not have been surveillance candidates.

Retchin and Brown¹³⁶ used HMO clinical data and Medicare claims data to examine the “proportion of patients with CRC who had undergone surgery who had had a colonoscopy by 6 months postresection.” This definition of the measure was based on a physician advisory panel and not on published guidelines. It is possible that some of these colonoscopies were in fact “clearing colonoscopies” and not the first surveillance examination. This measure was well described, but not well developed or well tested.

An IOM report¹³ described a similar measure with the suggested use of the Medicare-SEER merged database or databases derived from medical records. In this case, the measure was refined as number of stage I to stage III CRC cases with a colonoscopy within 1 year of surgery, which was based on the NCCN guideline. This was a better developed measure because of the explicit inclusion of surveillance colonoscopy timing congruent with at least one published guideline. The usability of the specific timing, however, is unclear since conflicting guidelines exist. This measure, as described in this report,¹³ was not risk-adjusted although it was noted that “local practice patterns and patient-related factors affect the use of endoscopic procedures.”

Rulyak et al.¹⁵⁷ used HMO claims data, HMO pathology databases, and SEER data from Seattle to measure the percentage of patients with local or regional CRC who had colonoscopy or flexible sigmoidoscopy with barium enema. They examined two time points: 18 months and 5 years after diagnosis. The weaknesses were the inclusion of a small number of cases of non-colonoscopic surveillance and the use of a 5-year time point (which is not related to any published guidelines). In this regard, the 18-month time frame is reasonable and arguably close to the 1-year recommendation of some guidelines (NCCN, ACS). The measure was not explicitly risk-adjusted, but patients who did not live at least 6 months after diagnosis were excluded.

NCQA⁶¹ proposed the measure percentage of colon cancer cases who receive followup colonoscopy within 36 months of surgical treatment. This version of the quality measure is well developed in that it includes only colonoscopy and specifies a guideline-endorsed surveillance interval. Its weakness is a lack of adjustment for patients too sick to benefit from surveillance either because of advanced cancer (i.e., in the absence of specifying disease stage or curative intent, patients with palliative operations could have been included in the denominator), or from comorbid illnesses.

NCQA⁶¹ also proposed another measure: percentage of rectal cancer cases that received endoscopic examination within 12 months per NCCN guidelines. The caveats listed above apply to the importance, validity, and usability of this measure as well.

Perioperative (Clearing) Colonoscopy. Keating et al.¹³⁰ used the SEER-Medicare database to measure the percentage of patients who underwent colonoscopy pre- or postoperatively (defined as in the period of 45 days prior to diagnosis through 30 days after surgery). While the measure is referred to as “complete colonoscopy,” this can be difficult to determine from procedure codes alone, as any endoscopy past the splenic flexure may be coded as a “colonoscopy” even if it was not “complete” by the criteria of documented cecal intubation. Furthermore the guidelines (see Table 9, above) allow for up to 6 months postresection to perform the “clearing” colonoscopy in cases where preoperative complete colonoscopy was not possible due to obstruction by the tumor.

Conclusions

Ratings for the quality measures discussed in this section are given below in Table 10; further details are provided in Evidence Tables 10 and 11 in Appendix E^*

The measures describing the performance of postoperative surveillance colonoscopies were fairly well developed and well motivated: their ratings ranged from I₅S₄T₅ for the measure percentage of patients with CRC receiving postoperative (surveillance) colonoscopy to I₄S₃T₂ for percentage of patients with local or regional CRC who had colonoscopy or flexible sigmoidoscopy with barium enema. The most problematic point was the lack of consensus regarding the optimal scheduling of this procedure. (Even in the absence of such consensus, an ideal measure would be based on at least one guideline and roughly concordant with the others.) The numerator should include colonoscopies only, and not other tests such as barium enema or flexible sigmoidoscopy. The measure could also benefit by explicitly specifying whether patients without clearing colonoscopy should be included. Technical criteria should be considered as well; for example, whether the colonoscopy was complete according to the standard of documented cecal intubation.

The measure describing the performance of perioperative colonoscopy was similarly well motivated, but suffered from a lack of precision in distinguishing clearing colonoscopies from similar procedures pre- and postsurgery (I₅S₄T₄). Including this additional detail would be substantively helpful, although potentially increasing the data collection requirements as not all databases (particularly administrative databases) will contain sufficient information to make this distinction.

An additional methodological issue involves the linkage between clearing and surveillance colonoscopies. One approach is to measure the rates of clearing colonoscopy and surveillance colonoscopy, separately evaluating both the performance and timing of a complete perioperative colonoscopy and complete surveillance colonoscopy. A second approach is to define the colonoscopic surveillance measure to include only those patients who underwent a clearing colonoscopy. The limitation of the latter approach is that a patient who failed to undergo perioperative clearing colonoscopy, but then had a first postoperative colonoscopy at 2 years, would be excluded from any measure of colonoscopic surveillance quality of care. As a result, an opportunity to improve CRC care will be missed. A limitation of only evaluating presurgical colonoscopy (rather than perioperative colonoscopy) is that some patients will be unable to have a complete colonoscopy at that time due to obstruction or need for emergent surgery (e.g., perforation).

Various technical measures examining critical details of the colonoscopy are examined under Question 1a.

The recommendations made herein are based on a review of currently available data. Obviously, future data or clinical realities (e.g., costs, changes in societal goals) or the development of new technologies may alter the recommendations in the future. Nonetheless, for the major recommendations, we do not expect their basic core importance to change.

Future Research

Several potential quality measures related to postoperative colonoscopy were retrieved. They relate to two important clinical issues that should be the focus of future research: (1) performance of postoperative surveillance colonoscopy, and (2) the performance of perioperative clearing colonoscopy. The actual surveillance colonoscopy should be defined as the first followup colonoscopy after both resection of the cancer and after “clearing of the colon” for other significant neoplasia (adenomatous polyps or synchronous cancers). Therefore, the clearing colonoscopy is a leverage point for the surveillance colonoscopy and is arguably worth measuring separately. It should be noted at this point that patients with unresectable metastases should be excluded from the denominator for measures regarding surveillance, since they do not benefit from it.

While several professional societies endorse guidelines for colonoscopic surveillance after curative resection of CRC (Table 9), the evidence upon which these guidelines are based is somewhat indirect. The mortality benefit of colonoscopic surveillance has not been tested in randomized controlled trials. The general logic is that individuals with a personal history of cancer are at increased risk for colon or rectal cancer and therefore should have at least as much and likely more followup as those individuals at average risk for CRC. The first step to improve the quality measure is to strengthen the link of the quality measure to outcomes of interest. The ideal outcome would be survival, but intermediate outcomes such as changes in patient management, or detection of metachronous or recurrent cancers at a treatable stage, may also be of interest. Without outcomes data it is not surprising that the guidelines differ on the timing of the first surveillance colonoscopy. The lack of consensus regarding the timing is one of the limiting factors for developing an appropriate measure. Outcomes data would encourage consistency across the guidelines, but even without the data consensus could be achieved. Once a standard interval for surveillance colonoscopy is established, the quality measure should reflect this interval (e.g., percentage of patients who receive the first surveillance colonoscopy at 3 years).

Any measure of colonoscopic surveillance should exclude all patients with greater than stage III disease at diagnosis, patients who died before the first surveillance colonoscopy was due, and patients who did not undergo resection of the cancer (with curative intent). It is reasonable to assume that patients in sufficient health to undergo and survive the cancer resection would be in sufficient health to undergo at least the first surveillance colonoscopy. This is particularly true if the ACS and NCCN recommendations of a 1-year interval are followed. On the other hand, it is conceivable that within a 3-year time frame the health status of an individual could change considerably and therefore need to reevaluated for the appropriateness (and safety) of followup colonoscopy.

The clearing colonoscopy, while not itself postoperative surveillance, also requires evaluation to make the measure of the colonoscopic surveillance precise and guideline-concordant. The clearing colonoscopy, which is an important opportunity to improve CRC care, should be defined as “complete” (to cecal landmarks or surgical anastomosis), with “adequate bowel preparation” (“no fecal material obscured a lesion 5 mm or more in diameter and a high quality examination takes place”), and with all polyps removed. The denominator is “the number of patients diagnosed with colon or rectal cancer.” The numerator is “the number of patients who undergo the clearing colonoscopy either prior to surgical resection or within 6 months of resection.”

Finally, the data sources need evaluation for reliability and accuracy. While administrative or claims data can provide cancer diagnosis and documentation of procedures (surgical resection, colonoscopy) they will need supplementation with other data for accurate staging (e.g. registry data, chart review) and for details of the colonoscopy itself. The latter will most likely require examination of colonoscopy reports as preparation quality, extent of procedure, and documentation of polyps or masses seen and removed are not documented in other data sources. A standard colonoscopy report template, entered into an electronic database, would facilitate use of the quality measurement. In addition, chart review may be necessary to determine changes in the health status of the patient if the first surveillance interval follows the 3-year rather than the 1-year recommendation.

Background

Our literature search did not reveal any measures that related to the documentation of surgical and chemotherapy reports. Accordingly, this section will focus on pathology reports.

The pathology report is the product of several steps, which include surgical dissection and histologic block selection, communication between surgeon and pathologist, pathologic dissection with macroscopic and microscopic analysis, and finally dictation and transcription of the completed report. It is important for making decisions regarding adequacy of surgical resection, need for adjuvant therapy, and surveillance. Therefore, a poor quality pathology report can adversely affect clinical outcomes.¹⁶⁵ Additional value of pathology reports includes their ability to provide information for clinical audits, for assessing accuracy of new diagnostic and preoperative staging techniques, and for comparing patient groups in clinical trials.

In recent years, the Association of Directors of Anatomic and Surgical Pathology (ADASP) and CAP have both undertaken to publish guidelines for the reporting of common cancers.^166,167 While the formats of the two guidelines are somewhat different, their contents are essentially the same.

The American College of Surgery Commission on Cancer (ACS-COC) accredits cancer centers in the United States. Recently, the ACS-COC decided to require elements deemed as essential by the CAP to be described in all pathology reports in their accredited cancer centers as of January 2004. However, they do not require that the specific CAP protocols or synoptic reports be utilized. ADASP has also updated all of its protocols and checklists to comply with the ACS-COC requirements. The different elements in the revised ADASP Diagnostic Checklists have been divided into the categories Required and Optional. The term Required in this context signifies only compliance with the ACS-COC guidelines. ADASP realizes that specimens and practices vary, and it will not be possible to report these elements in every case.

CAP has developed and updated guidelines for reporting cancers of the colon and rectum and offers protocols to assist pathologists in providing clinically useful and relevant information in the reporting of the results of their examination of surgical specimens.¹⁶⁸ CAP regards the reporting elements in the Surgical Pathology Cancer Case Summary portion of its protocols as essential elements of the pathology report. However, the manner in which these elements are reported is at the discretion of each pathologist - taking into account clinician preferences, institutional policies, and individual practice. CAP also recognizes that the ACS-COC mandated the use of the checklist elements of the protocols as part of the Cancer Program Standards for Approved Cancer Programs. However, not all these elements stipulated in the CAP checklist are required for accreditation (for approved cancer programs) purposes, since not all of them are validated for use in patient management (e.g., venous invasion).

Results

The IOM report Assessing the Quality of Cancer Care¹³ proposed the quality measure proportion of CRC cases in which pathologic staging preceded chemotherapy and radiation treatment. The numerator is the number of new CRC cases with a medical chart documentation of pathologic stage before chemotherapy or radiation is initiated; the denominator is the number of new CRC cases with chemotherapy or radiation treatment. The measure recommends that medical records be used as the data source.

The Colon Cancer Workgroup⁵⁴ recommended the measure pathology report in concordance with CAP guidelines. They further define the measure as pathology reporting that includes lymph nodes resected, number of nodes positive, tumor characteristics (grade, depth of invasion), and mucosal and radial margins. They stipulate the data required for application of the measure (pathology and surgical reports); it is based on the CAP and NCCN guidelines, clinically relevant, is amenable to change by providers and the system, and is free from confounders. The measure was utilized for assessing and improving quality of care in a single health system. The measure is well developed, and is linked to an outcome of interest. No information was available regarding testing.

The IOM¹³ recommended the measure adequacy of pathology reports on CRC to assess the quality of cancer care in Georgia. Adequacy is defined as those reports that meet the CAP data elements as required by the ACS-COC. The measure is important, usable, precisely defined, valid, recommends fairly reliable data sources, and the conditions for use of the measure are stated. Risk-adjustment need not be applied in this context.

Wei et al.¹⁶⁵ evaluated variations in colon carcinoma reporting by laboratory type and hospital volume, as related to the ADASP recommendations, in order to identify areas for improvement. Data were collected from pathology reports from patients participating in a population-based cancer study (North Carolina Cancer Study [NCCS] from 1997 to 2000). The denominator consisted of patients who had surgically resected T2-4 pathologically confirmed invasive adenocarcinoma and for whom surgical pathology reports were available and who resided in the 33-county area used for NCCS. Patients with T0s and T1 were excluded because several items of the pathology report were not applicable to localized malignancies. The following items were included in the evaluation: percentage of reports that mentioned (a) how specimen was received; (b) how it was identified; (c) part of intestine included; (d) the tumor site; (e) proximity of tumor to the nearest margin; (f) macroscopic subtype; (g) tumor dimensions; (h) macroscopic depth of penetration; (i) appearance of serosa adjacent to the tumor; (j) appearance of residual bowel; (k) histological features including histologic type and grade; (l) depth of infiltration; (m) lymph node metastases; and (n) involvement of margins. Overall percentage of each reported characteristic was noted. Pathology laboratories were categorized as contract, teaching hospital, or community hospital laboratories in one analysis, and categorized by hospital case volume in another. The study did not assess the accuracy of the reported findings, and did not assess the impact of pathology reports on patient outcome. Each of these items can be considered to be a separate quality measure. A global score was not calculated.

We did not identify any manuscripts dealing with the adequacy of chemotherapy reports. Although currently the most important measure is whether chemotherapy is administered to stage III colon cancer patients, there may be future interest in determining whether an adequate dose intensity was given. Giving an adequate amount of chemotherapy is important for the outcome of patients with breast cancer, and if similar data are reported for colon cancer in the future, then it will be important to be able to measure whether adequate chemotherapy has been given. In this case, the documentation contained in chemotherapy reports will need to be of high quality (e.g., reporting the agents given and their dosage).

Conclusions

Pathology reports play a very important role in providing clinically useful and relevant information from the examination of surgically resected specimens.

Ratings for the quality measures discussed in this section are given below in Table 11; further details are provided in Evidence Tables 12 and 13 in Appendix E^*. The quality measure proportion of CRC cases in which pathologic staging preceded chemotherapy and radiation treatment,¹³ is well developed, free from confounders, and is an important leverage point, but needs testing (I₅S₄T_-).

The quality measure pathology report in concordance with CAP guidelines⁵⁴ is well developed, free from confounders, and is an important leverage point, but needs testing (I₅S₄T_-). Not all items included in the CAP guidelines are related to outcomes of interest, and it may be useful to select those that address a leverage point. This aspect is considered by the measure adequacy of pathology reports on CRC, which limits measurement to scientifically validated elements of the CAP guidelines, and is therefore a potentially ideal measure; however, it needs field-testing (I₅S₄T_-).¹³

The other measures (each being an item of the ADASP guideline) take more of a micro-level perspective, addressing individual elements of the pathology report. These are well developed, and address some important leverage points for improving quality of colon and rectum cancer pathology reporting standards.¹⁶⁵ The most important ones are: local extent of tumor, regional lymph node metastases, and residual tumor at surgical resection margin. If those elements are selected, the process measures will address a leverage point, and be related to an outcome of interest (I₅S₅T₄).

The advantage of using individual items as measures is that they allow a comparison between laboratories that utilize different guidelines (e.g., ADASP guidelines instead of CAP recommendations), or permit a comparison of only those items that have a proven link to survival and therefore are leverage points, or allow one to perform a comparison over time, as the continued availability of evidence may alter the relevance of some elements of the checklists/guidelines.

When utilizing any of these measures to assess and/or compare quality of pathology reporting the following considerations are relevant: (a) the measures evaluate the presence or absence of certain items in a report, not the accuracy of those items; and (b) some omissions in a pathology report may not represent poor quality of care: some microscopic reporting (e.g. depth of penetration) may obviate the need for reporting on macroscopic features (e.g. appearance of tumor); and (c) the measures may not distinguish between quality of the surgical procedure and quality of pathology services.

The recommendations made herein are based on a review of currently available data. Obviously, future data or clinical realities (e.g., costs, changes in societal goals) or the development of new technologies may alter the recommendations in the future. Nonetheless, for the major recommendations, we do not expect their basic core importance to change.

Future Research

The basic and sound premise behind the currently available measures of the quality of pathology reports is that standardization of reports is a crucial step toward the goal of ensuring their quality. Additional and related areas for development include the development of summary scoring systems (i.e., for those measures that include multiple items), assessing the relative importance of various items, and extending the evidence base that links elements of the pathology report to patient outcomes.

This same premise can be applied with equal force to operative and chemotherapy reports, and generating analogous measures should be a high priority for future research. With special reference to rectal cancers, TME of rectal tumors has clearly demonstrated a reduction in local recurrence and an improvement in long-term survival. However, it is difficult to discern if in fact a TME has been performed, since operative notes tend to be extemporaneous. Perhaps the inclusion of specific anatomic landmarks in the operative note, as well as the pathologist comment on the gross integrity of the rectal facial planes, would help confirm the fact that a TME of the rectum was performed. Then surgical results of rectal cancer procedures could be directly compared without the confounding factor of surgical technique.

Standardized chemotherapy notes could also be developed to include information on patient preferences regarding chemotherapy. In addition, since “receipt” of chemotherapy is only a surrogate marker for whether patients receive the correct agents, dose, and schedule, reports should include the latter sort of information. In order for chemotherapy reports to be of use, more research will be needed on whether information that would be contained in such reports (agents used, dose used, over what period of time) will need to be established for colon cancer. Subsequently, a more global measure, such as chemotherapy dose intensity, will need to be evaluated for whether it is related to outcome.

As illustrated in the development of quality measures for pathology reports, the role of the professional society can be key.

Background

We identified several types of reports that were relevant to the question of the usefulness of quality measures to assess differences in the quality of care across various patient-level characteristics. A relatively small number of studies were specifically intended to assess the impact of patient, provider, or health system characteristics on measures of either process or outcome. There were a far greater number of studies that evaluated and reported, to a limited extent, data on the association of such patient, provider, or system characteristics with process or outcome measures.

This section of the report is intended to provide a comprehensive list and description of those studies that were designed to measure and examine the impact of patient characteristics on processes and outcomes. The report identifies a representative, but not necessarily complete, list of studies for which evaluation of patient level characteristics was not a stated goal. Reasons for making this distinction include: (1) studies designed to assess patient factors will be more reliable in their measurement and analysis; (2) studies for which patient characteristics were not the major focus of the analysis are more difficult to identify (and, thus, were more subject to biased ascertainment); and (3) possible reporting bias (studies that mention age, race, or socioeconomic status associations in passing are more likely to do so if a statistically significant result is found).

Results

Tables 12-15 summarize the use of various CRC quality-of-care measures by various patient characteristics including age, sex, race/ethnicity, and socioeconomic status. The investigators reported 29 instances in which age was related to either outcomes (n = 6) or process measures (n = 23; Table 12). Many studies described lower rates of use of chemotherapy or radiation therapy for older compared with younger patients using age cutoffs ranging from 65 to 85; however, few studies controlled for relevant covariates (e.g., performance status or comorbid illnesses) that might explain this difference.

Similarly, in five cases (Table 13) sex was related to either outcomes (n = 1) or process measures (n = 4). Four of the five studies described lower rates of use of adjuvant chemotherapy or radiation therapy for women compared with men; however, none controlled for relevant covariates (e.g., performance status or comorbid illnesses) that might explain this difference.

Race was related to process measures, most commonly receipt of certain types of treatment in 30 instances (Table 14). With the exception of several measures in the VA system, which did not show racial differences,⁸⁸ each study found that whites were more likely than blacks to receive certain types of treatment including surgery, chemotherapy, and radiotherapy.

Finally, in 16 instances (Table 15) socioeconomic status was related to process measures, most commonly receipt of certain types of treatment. Socioeconomic status was most commonly measured as income or health insurance status. More than half of the studies found that patients of higher socioeconomic status were more likely than patients of lower socioeconomic status to receive certain types of treatment; however, several studies showed no statistically significant differences in quality measures across socioeconomic status.

Conclusions

The question of whether the evidence supports the use of any of the available quality measures ultimately devolves into asking whether the databases in question exhibit differential bias according to the patient subgroups being studied. For example, suppose that patients with stage III colon cancer are followed, and the process measure of interest is the percentage of patients receiving adjuvant chemotherapy. As discussed elsewhere, most but not all of such patients are good candidates for adjuvant chemotherapy, and not all patients that are offered such therapy will accept. If the data source is prospectively collected and designed to be sensitive to such issues (e.g., a form in the patient's medical record that documents whether or not the patient was a good candidate for chemotherapy and, if so, whether the therapy was accepted), then sufficient information is present to make an accurate comparison of, for example, rates of adjuvant chemotherapy use by age. On the other hand, if such information is absent, then the possibility exists that older patients may have more contraindications and/or a greater propensity to refuse therapy, thus biasing the conclusions. The answers to the above question about differential bias depend entirely on the quality measure, the groups to be compared, and the available database, but clearly this is the question that should be of fundamental importance to the analyst.