NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Patwardhan MB, Samsa GP, McCrory DC, et al. Cancer Care Quality Measures: Diagnosis and Treatment of Colorectal Cancer. Rockville (MD): Agency for Healthcare Research and Quality (US); 2006 May. (Evidence Reports/Technology Assessments, No. 138.)

  • This publication is provided for historical reference only and the information may be out of date.

This publication is provided for historical reference only and the information may be out of date.

Cover of Cancer Care Quality Measures

Cancer Care Quality Measures: Diagnosis and Treatment of Colorectal Cancer.

Show details

Executive Summary


The burden of illness from colorectal cancer (CRC), the second leading cause of cancer death in the U.S., is substantial. One way to reduce the burden of illness is to improve the quality of care for patients with CRC. Measuring quality of care can help identify specific areas in need of improvement. Sponsored by the Agency for Healthcare Research and Quality (AHRQ), the National Cancer Institute (NCI), the Centers for Medicare & Medicaid Services (CMS), and the Centers for Disease Control and Prevention (CDC), the Duke Evidence-base Practice Center undertook this evidence report to identify measures that are currently available to assess the quality of care provided to patients with CRC, and to assess the extent to which these measures have been developed and tested. With this basis, we also report on areas for future research with an explicit aim of developing a set of measures that can effectively evaluate the process of care provided to CRC patients. Information from this evidence report will be used by technical panels convened by the National Quality Forum (NQF) to consider cancer care quality measures.

Quality measures may be classified as structure, process, and outcome measures. While all three types are considered here, the primary focus is on process measures, whether or not such measures have been formally applied within the context of quality improvement.


Five key questions are addressed by this report:

Question 1: What quality-of-care measures are available and what evidence is available for these measures to assess the quality of diagnosis of colorectal cancer, including: (a) appropriate use of colon imaging, endoscopic visualization, and biopsy; and (b) availability and accuracy of pathologic staging?

Question 2: As appropriate to specific stages of colorectal cancer, what quality-of-care measures are available and what evidence is available for measures of quality of care of treatment of colorectal cancer, including: (a) polypectomy for malignant polyps, including evaluation of surgical margins; (b) surgical therapy for colon and rectal cancers; (c) appropriate use of adjuvant chemotherapy and adjuvant radiation therapy, including for patients with metastatic but potentially curable (hepatic/pulmonary-resectable) disease; and (d) appropriate use of radiation therapy for either curative or palliative therapy, specifically for rectal cancers?

Question 3: What quality-of-care measures are available and what evidence is available for measures of colonoscopic surveillance for colorectal cancer?

Question 4: What measures are available and what evidence is available for measures to assess the adequacy and completeness of documentation of pathology, operative, and chemotherapy reports?

Question 5: For Questions 1–4 above: (a) in what patient populations and for what purposes have these quality of care measures been used; and (b) does evidence support the use of any of these measures to assess differences in quality of care across patients' age, race/ethnicity, and/or socioeconomic status?

After finalizing the key questions in consultation with study sponsors, NQF, and a panel of national experts, we identified measures through a computerized search of English-language citations in MEDLINE® (1966-January 2005), the Cochrane Database of Systematic Reviews, and the National Guideline Clearinghouse. We also reviewed reference lists from all relevant articles and made special attempts to retrieve grey literature, loosely defined as institutional or government reports, professional society documents, research papers, fact sheets, and other literature, in print or electronic format, that is not controlled by commercial publishing interests. The basic MEDLINE search strategy was adapted for use in the other databases. The final search yielded 3,771 citations published since 1990.

Measures described in individual studies were selected by reviewers according to standardized criteria relating to each question, and were then rated according to their importance and usability (I), scientific acceptability (S), and extent of testing (T); each domain was scored from 1 (poor) to 5 (ideal). A final “I-S-T” rating was assigned to each quality measure across studies.


Colon Imaging, Endoscopic Visualization, and Biopsy (Question 1a)

The best-developed measure identified was percentage of patients who underwent appropriate evaluation for a positive fecal occult blood test [FOBT] (I5S4T4), which is linked to an outcome of interest (reduced mortality from CRC), based on an evidence-based recommendation, and fairly well tested. The leverage points are awareness of the positive FOBT and referral for colonoscopy.

Although the benefits of expeditious action are intuitive, the measure time from presentation to diagnosis (I3S1T1) has not been explicitly linked to an outcome of interest, is not based on any evidence-based guidelines, and can therefore be considered to be relatively less well developed and untested. The same can be said for proportion of colonoscopies that were completed in a timely fashion (I5S4T-).

Any measure that relates to miss rates (I5S3T3) is unlikely to be useful to compare quality of care, since the methodological gold standard for determining a miss rate is tandem or closely timed colonoscopies, and such a gold standard procedure is not realistic within general clinical practice. Computed tomographic (CT) colonography, or “virtual colonoscopy,” appears to be a better alternative, but is not yet part of mainstream evaluation considering the flux in the technology utilized.

Also important were technical process measures of colonoscopy such as cecal intubation rate (I5S4T4), intraprocedural complication rate (I5S4T-), and postprocedural complication rate (I5S4T4). These technical measures are endorsed by guidelines and are meaningful indicators of quality. These measures tend to be fairly well developed, but poorly tested.

Complication rate of colonoscopy (I5S4T4) is an important technical measures; however, an accurate definition of the measure is required.

Pathologic Staging (Question 1b)

Our literature search retrieved two process measures that have been used to assess the quality of staging of CRC:

  • Adequate lymph node retrieval and evaluation (I5S4T5).
  • Percentage of newly diagnosed CRC cases who were staged using the American Joint Committee on Cancer (AJCC) system (I4S3T3).

Adequate lymph node retrieval and evaluation is linked to an outcome of interest (better staging, leading to appropriate management and better survival) and based on evidence (the 1999 consensus statement of the College of American Pathologists (CAP) recommends evaluating 10–15 lymph nodes in node-negative patients). It is well suited as a quality measure in a broader context, and is well developed and tested. Percentage of newly diagnosed CRC cases that were staged using the AJCC system is another measure that is well developed, has face validity, is related to an outcome of interest, and represents an important leverage point for improving the quality of CRC care. However, it does not distinguish between pathology staging and clinical staging.

Polypectomy for Malignant Polyps (Question 2a)

No quality measures were found addressing polypectomy for malignant polyps

Surgical Therapy (Question 2b)

The most basic measure of the quality of surgical management is whether surgical therapy was provided as appropriate to the location and stage of the cancer. Results could potentially be broken out by stage or presented as a single summary measure, surgical therapy appropriate to the location and stage of the cancer, for example, appropriate primary therapy for CRC as defined by the NCI guidelines (I4S4T4). The former approach provides more information and is preferable. Regarding the denominator of this measure, patients for whom the therapy is inappropriate should be excluded. This point is particularly problematic for patients with stage IV colon or rectal cancer, for whom the intent of the surgery is palliation rather than cure. Ideally, the measure should also distinguish between patients who were offered “appropriate” surgery and refused, and those patients who were not offered such surgery. Making these distinctions increases the data collection demands.

Surgical complication rates provide another very natural measure. Most variations on this measure focus on mortality rates (e.g., 30-day mortality [I5S4T5] or in-hospital mortality [I5S3T3]), although a delineation of the actual complications would be helpful as well, this latter type of measure being relatively underdeveloped. Presumably, such a measure should be both stratified by stage and location, and risk adjusted to take into account other clinical characteristics of the patient.

For patients with rectal cancer, various measures have been proposed regarding the rate of sphincter-saving surgery (I5S4T4). This is an outcome of importance to patients. However, quality measures should consider detailed information about the anatomical location and other characteristics of the tumor, which is not yet the case in practice.

Technical measures associated with surgery are underdeveloped.

Adjuvant Chemotherapy (Question 2c)

We considered measures for stage III colon cancer, stages II and III rectal cancer, and stage IV colorectal cancer. Roughly speaking, these measures were defined as the proportion of patients with stage III colon cancer receiving adjuvant chemotherapy (I5S4T5), the proportion of patients with stages II or III rectal cancer receiving chemoradiotherapy (I5S3T2), and the proportion of patients with stage IV colon cancer or stage IV rectal cancer receiving palliative chemotherapy (I5S3T2).

Of these measures, perhaps the best-developed and best-tested measure applies to patients with stage III colon cancer. It is derived from an evidence-based recommendation, is an important leverage point, and is well tested.

Also evidence-based and well validated is the measure applying to patients with stages II or III rectal cancer. Several different versions of the measure have been created, but the ideal version of the measure is the one that most closely matches the most recent recommendations from guidelines, which specify that chemoradiotherapy is the preferred method of management for essentially all such patients.

The measure regarding palliative chemotherapy for stage IV colon cancer or stage IV rectal cancer is reasonable in principle, but suffers from the difficulty that the decision to provide this therapy should very much be made on a case-by-case basis, thus implying that prospective data collection is probably necessary.

For all the above measures, patient preference needs to be taken into account. However, this data element is particularly difficult to collect.

The measure on the proportion of patients referred to an oncologist (I5S3T2) pertains to an important leverage point in practice, although it is possible that this is neither a necessary nor a sufficient condition for the provision of high-quality care.

Radiation Therapy (Question 2d)

We identified an important process measure: the percentage of patients with stage II or III rectal cancer receiving adjuvant chemoradiotherapy (I5S3T2). A well-developed and tested measure, it needs to account for reasons for non-receipt of therapy (e.g., insurance coverage or patient preference).

We also identified two potential quality measures for radiation therapy related to the technical quality of its administration, with reference to two different guidelines: adherence to radiotherapy management treatment guidelines for patients with adenocarcinoma of the rectum or sigmoid colon (I5S4T4) and rate of use of modern radiation therapy techniques and adherence to recommendations of NCI-sponsored randomized controlled trials in rectal cancer patients (I3S3T2). However, neither of the two technical measures is appropriate for current use because they are based on outdated guidelines. These measures could be modified to make them relevant to current evidence and practice standards.

Colonoscopic Surveillance (Question 3)

Measures describing the performance of postoperative surveillance colonoscopies were fairly well developed and well motivated: their ratings ranged from I5S4T5 for the measure percentage of patients with CRC receiving postoperative (surveillance) colonoscopy to I4S3T2 for percentage of patients with local or regional CRC who had colonoscopy or flexible sigmoidoscopy with barium enema. The most problematic point was the lack of consensus regarding the optimal scheduling of this procedure. The numerator should include colonoscopies only, and not other tests such as barium enema or flexible sigmoidoscopy. The measure could also benefit by explicitly specifying whether patients without clearing colonoscopy should be included. “Clearing colonoscopy” (1) refers to a sufficiently comprehensive examination of the colon to exclude other significant neoplasia, such as adenomatous polyps or synchronous cancers; and (2) signifies that all polyps were removed. Technical criteria should be considered as well; for example, whether the colonoscopy was complete according to the standard of documented cecal intubation.

A measure describing the rate of perioperative colonoscopy (I5S4T4) was similarly well motivated, but suffered from a lack of precision in distinguishing clearing colonoscopies from similar procedures pre- and postsurgery. Including this additional detail would be substantively helpful, although potentially increasing the data collection requirements, as not all databases (particularly administrative databases) will contain sufficient information to make this distinction.

Pathology, Operative, and Chemotherapy Reports (Question 4)

The quality measure proportion of CRC cases in which pathologic staging preceded chemotherapy and radiation treatment (I5S4T-) is well developed, free from confounders, and is an important leverage point, but needs testing.

The quality measure pathology report in concordance with CAP guidelines (I5S4T-) is well developed, free from confounders, and is an important leverage point, but needs testing. Not all items included in the CAP guidelines are related to outcomes of interest, and it may be useful to select those that address a leverage point, for example, local extent of tumor, regional lymph node metastases, or residual tumor at resected margin. This aspect is considered by the measure adequacy of pathology reports on CRC (I5S4T-), which limits measurement to scientifically validated elements of the CAP guidelines, and is therefore a potentially ideal measure; however it needs field-testing.

The other measures (each being an item of the Association of Directors of Anatomic and Surgical Pathology [ADASP] guideline) take more of a micro-level perspective, addressing individual elements of the pathology report. These are well developed and address some important leverage points for improving quality of colon and rectum cancer pathology reporting standards. The most important ones are: local extent of tumor, regional lymph node metastases, and residual tumor at surgical resection margin (all I5S5T4).

Technical measures relating to operative and chemotherapy reports were underdeveloped.

Patient Populations and Purposes; Age, Race/Ethnicity, and Socioeconomic Status (Questions 5a and 5b)

A number of studies have measured differences in process measures related to quality of care based on age, racial/ethnic, or socioeconomic differences; however, few studies have evaluated potential confounders that may explain the differences observed. More refined measures that evaluate a variety of covariates involved in clinical decisionmaking may be necessary to understand and explain these apparent quality-of-care differences.


Although we also considered measures of structure and outcome, the primary focus of this report was on measures of CRC-related processes of care. Here, the main distinction is between those measures that are general (dealing with processes of care on a broader scale) versus technical (pertaining to specific details of a procedure).

The formula for creating a general process measure is to take the sequence of steps describing the optimal pattern of care for patients with CRC, identify each of these steps as both a leverage point and a potential quality measure, and then define such a measure. For example, one of the earliest steps in the process involves the proper diagnosis of patients with a suspicion of CRC. A well-developed and well-tested measure associated with this step is percentage of patients who underwent appropriate evaluation for a positive FOBT (I5S4T4). As with all ratio-based measures, it is critical to carefully define both the numerator and denominator of the measure. Here, the numerator can be based on a guideline and involves colonoscopy with complete visualization of the colon, although the literature did discuss various nuances, such as the precise definition of what complete visualization entails, how soon the colonoscopy should be performed after the positive FOBT, and so forth. The numerators of some general process measures will have a stronger evidence base than others, with those that are based on guidelines (especially those guidelines that are consistent across professional societies, when multiple such guidelines have been proffered) having the strongest evidence base, and those that are based on basic first principles supported by some research findings being relatively weaker, but often quite sufficient for the task at hand.

Regarding the denominator, “patients with positive FOBT” comprise only a subset of those patients of ultimate interest, namely, “all patients with sufficiently high clinical suspicion of CRC to require followup testing.” These patients would include those with positive tests according to other modalities (e.g., barium enema, flexible sigmoidoscopy), and also those who come to the provider's attention because of symptoms such as bleeding. Often, a clinically precise statement of the denominator involved deleting various patients; for example, those that would be ineligible because of comorbid conditions or other clinical issues, patients that refuse a form of therapy such as chemotherapy, and so forth.

A consistent source of tension is the distinction between the clinically derived fine-tuning of the definition of the denominator of a quality measure and the limitations of available data sources (which often do not contain sufficient information to act on such distinctions). Our approach here has been to encourage the elicitation of denominators that are as precise as possible, even if the available data sources might not meet all their requirements. In practice, it is crucially important to assess both the strengths and weaknesses of available data sources. Formal validation efforts are encouraged, but are not always strictly necessary. The assessment of data sources should include the differential impact of data quality on patient subgroups. For example, older patients tend to have more comorbidity and might thus be less likely to tolerate adjuvant chemoradiation therapy; an analysis that does not take this tendency into account could incorrectly lead to the conclusion that an age-related bias exists in the provision of such therapy. A similar caveat applies to analyses comparing patients by race and ethnicity, hospitals according to type, and so forth. Another example of bias is the inability of data sources to account for patient preferences.

Overall, the coverage of general process measures (i.e., across the various leverage points in the sequence of care) is extensive. At least some process measures are available for the steps involving diagnostic imaging, staging, surgical therapy, adjuvant chemotherapy, adjuvant radiation therapy, and colonoscopic surveillance. Additional general process measures might be developed for polyp assessment and removal, although it must be recognized that the proper clinical course of action when presented with a malignant polyp is not always clear. While the above formula could be followed to generate even more general process measures within the above categories, perhaps the task of greater importance is to continue to refine those measures that already exist. The continued expansion of the evidence base, and the continued updating guidelines, should assist this process. Moreover, the basic structure of general process measures facilitates such updating. For example, if future research suggests that the optimal interval for colonoscopic surveillance should be changed, then the structure of the existing quality measures related to surveillance need not be changed; all that would require change is the replacement of one recommended interval with another.

In contrast to general process measures, as might be anticipated, technical measures tended to be used in a formal quality improvement context. Although some excellent technical measures were found, the overall development of technical measures seems somewhat less advanced than that of the general process measures. In particular, for some elements of the care pathway, such as operative reports and chemotherapy reports, no technical measures were found. Various technical measures for quality of colonoscopy were found (e.g., cecal intubation rate). These were guideline-based, well developed, but less well tested, and the linkage between the technical measures and patient outcomes, although intuitive, was not always explicitly provided. To the extent that complication rate of colonoscopy (I5S4T4) can be considered a technical measure, this area of inquiry is relatively well developed, but needs further testing. Adequate lymph node retrieval and evaluation (I5S4T5) is an excellent technical measure related to disease staging, although residual areas requiring clarification remain (e.g., optimal number of lymph nodes to examine, consideration that the optimal number of lymph node depends to some extent on patient and tumor characteristics). Technical measures associated with surgery are underdeveloped. Chemoradiation therapy has a number of potential and well-justified technical measures; here, the challenge is to identify and focus on those measures having the most impact on patient outcomes. Technical measures for pathology reporting are well developed, reflecting among other things the areas of emphasis among the relevant professional societies. As noted above, no such technical process measures were identified for operative reports or chemotherapy reports, although it can quite reasonably presumed that with sufficient attention from professional societies and other stakeholders such measures could be developed.

Future Research

Perhaps the most important areas for future development of technical measures are: (a) developing such measures in those areas for which they are lacking; and (b) in areas where such measures exist, identifying and focusing on those measures that have the greatest impact on outcome. Clarifying and otherwise fine-tuning the measures is a process that should be ongoing, both for technical and general measures.


  • PubReader
  • Print View
  • Cite this Page

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...