Chapter  137:  Cancer Care Quality Measures: Symptoms and End-of-Life Care

Reporting the Evidence

This report addressed the following questions:

• 1. What quality-of-care measures are available and what evidence supports those measures to assess pain, depression, dyspnea, and advance care planning for patients with cancer, including:

  • a. Patient assessment,

  • b. Intervention and appropriate management, and

  • c. Timeliness and effectiveness of intervention.

• 2. What gaps in knowledge about quality measurement are evident from the currently available literature, including absence of measures or measures lacking evidence of their scientific soundness, whether for the population of cancer patients as a whole or for specific subpopulations?

Literature Search and Review

We searched MEDLINE^®, CINAHL^©, and PsycINFO^© for English language literature (January 1995 - February 2005) using terms for pain, dyspnea, depression, and ACP combined with terms for cancer, quality assessment or improvement, and palliative care. Because our previous experience identified limitations in the indexing of quality measures in scientific databases, we used pre-specified terms to search the Internet for evidence-based standards or quality measures that had been developed by healthcare organizations or specialty societies. (February to March 2005). We directly contacted developers of all measures or measure sets we identified and sought additional information using a ‘contact form.’

Study Selection and Abstraction

Six reviewers, with clinical backgrounds in internal medicine and nursing, and research careers in palliative care, organized into four teams by domain, each reviewed documents independently, resolving disagreement by consensus. We applied the following inclusion and exclusion criteria at both the title/abstract and article screening phases. Study reports or other documents were accepted only if they contained either indicators or measures. An indicator was defined as “a descriptive statement with a normative relationship to quality that includes a numerator and denominator and that is expressed as a measurable standard.” A measure is an indicator fully specified for measurement, including data elements, the data collection approach, data sources, analysis, and presentation.

Documents were excluded if they:

• Were non-Western (i.e., not United States, Canada, Australia, New Zealand, Great Britain, or European),

• Were published in a non-English language,

• Focused exclusively on a disease other than cancer UNLESS they were on the topic of pain or depression, or

• Focused on cancer but considered only a topic other than ACP, depression, dyspnea, or pain,

• Addressed only specific types of cancer and/or processes of care that were not generalizable across the most common cancer conditions (e.g., particular pain syndromes for tumor subtypes),

• Guidelines were excluded if they did not contain any description of their methods or how they were developed.

For the domains of pain and depression, if a measure or indicator was developed for populations (e.g., hospice) that generally include cancer patients, but the development or field testing for the indicator did not explicitly include cancer patients, we did not exclude the document or citation.

Articles accepted at the screening stage were subjected to full abstraction using standardized forms. Some studies or documents described only indicator or measure development, whereas others described testing or use in an actual population. Detailed information on the attributes of these tools was abstracted.

The Report

In general, the review was organized around fundamental domains of symptom management—assessment, treatment, and follow-up. Assessment includes all activities related to evaluating the presence of a symptom. Treatment includes all activities subsequent to identification related to mitigating the symptom. Follow-up includes re-assessment and/or re-treatment of the symptom. For consistency, ACP was also organized around these domains since they are reasonably analogous, although for ACP, the task of ‘assessment’ is to evaluate goals and preferences, ‘treatment’ is the application of advance care plans, and ‘follow-up’ refers to actual implementation.

Peer Review

We received comments from eight peer reviewers and eleven TEP members who reviewed our draft report. We compiled the comments and made appropriate changes to the report.

Literature Review

The literature search identified 4,580 titles. We identified an additional 389 articles from the database we had assembled for a systematic review of research on End-of-Life Care and Outcomes. Our web search identified 105 guidelines. An additional 109 articles were suggested from by the project members. Four additional articles were suggested by our TEP upon review of the draft version of this evidence report. In total, the RAND reviewers examined 5,187 titles.

Of the 5,187 possibly relevant titles, 4,599 were excluded at abstract review, leaving 588. Repeat review by one principal investigator (KL) excluded an additional 51 titles, leaving 537 reports and guidelines.

Of the 537 articles ordered, we retrieved 536 prior to the cut-off date (June 10, 2005). Screening resulted in the exclusion of 485 articles: 347 included no indicator; 113 had no domain of interest; 8 considered a disease other than cancer; 5 were guidelines with no methods; 6 originated in a non-Western location; 2 presented duplicate data; 2 were duplicate articles; and 2 were useful for background only.

Of the remaining 51 articles, 25 that contained measures and/or indicators and described their methods were reviewed in detail. Potential indicators were identified by each team based on their working knowledge of research literature and prioritized based on each team's opinion of the most important gaps in measurement. The remaining 26 articles contained only potential indicators. As one article can report on multiple topics and may include multiple measures and/or indicators/potential indicators the numbers in the domain boxes in Figure 1 represent the numbers of measures, indicators or potential indicators that were abstracted into the Evidence Tables and not the actual number of articles.

Pain - Measures and Indicators

This category addressed the general conceptual areas of pain assessment (five measures, two indicators) with numeric or nonspecific scales; treatment (two measures, two indicators), including timely intervention; side effect/constipation prophylaxis; radiotherapy; and follow-up (four measures, one indicator) including timely relief, satisfaction, and degree of relief (Table 2). Assessment measures and indicators specifically focused on cancer addressed routine pain assessment using numeric or non-specific scales in inpatient, intensive care, outpatient, and unspecified settings.

Dyspnea - Measures and Indicators

Dyspnea measures and indicators addressed the general conceptual areas of routine assessment (one measure) with a numeric scale; treatment (three indicators), which included any treatment, terminal treatment, and addressing hypoxia; and follow-up (one measure), addressing timely relief.

Depression - Measures and Indicators

Depression measures and indicators addressed the general conceptual areas of assessment (one measure, one indicator), including psychosocial, spiritual assessment, treatment (two measures) including regular assessment or treatment.

ACP - Measures and Indicators

ACP measures and indicators addressed the general conceptual areas of assessment (1 measures, 4 indicators), including surrogate identification, preference evaluation, patient/family participation, and timely planning; ACP application (2 measures), addressing documentation across venues and evaluation of specific life sustaining preferences; and ACP follow-up (10 measures, 1 indicators), which addressed preference-treatment consistency, hospice admission, late life utilization of hospital or intensive care settings, emergency care, and chemotherapy.

Field Testing in Relevant Populations

Many of the measures may be useful now for local quality assessment and improvement, but most measures lack even basic published evidence of reliability and validity. Before these measures are deployed at higher levels of the health care system, information about their ability to discriminate quality performance and responsiveness to change will be required. Furthermore, health care systems must consider the cost and feasibility of data collection and must be assured that measures are at low risk for perverse incentives and that they can provide timely feedback.

The Need for Measures To Address Impaired Self-Report

None of the indicators address how to evaluate symptoms among individuals with impaired self-report, which can be due to either temporary (e.g., delirium) or durable factors (e.g., dementia, brain metastasis). There is no doubt that delirium is common among cancer patients, although its incidence and prevalence are not well studied. Some recent reports noted a prevalence of 18–50% among patients with cancer in general, and observed rates are even higher in certain treatment settings such as the intensive care unit or the nursing home, where almost 60% of the population is also living with dementia.

The Lack of Pediatric Measures

We noted the complete absence of pediatric measures or indicators. Cancer represents the second leading cause of death among children ages 5 to 15 in the United States, and serious deficiencies in the care of pediatric cancer cases have been noted. It is insufficient to propose that adult measures or indicators could simply be adopted for pediatric care, as the kinds of disease, challenges of symptom reporting and intervention, and basic approach to care differ so significantly between pediatric and adult oncology care.

Defining the “End-of-Life”

The lack of consensus on definitions of “end-of-life” presents what has been called the “denominator” problem. Many of the indicators define end-of-life retrospectively (e.g., last six months of life). This definition does not provide guidance on important clinical service, and retrospective analyses may introduce a number of biases, since the timing of death is often not predictable. Prospective indicators would account for patient preferences but will require methodological work to define which cancer patients (those with certain types that are usually fatal, those at advanced stage) should be in the denominator.

The Publication Gap for Quality Measurement

An important finding of our report was that much of the evidence on the measures that we identified was unpublished. Additional information that might be available to inform the evidence base for these measures was likely collected by non-academic organizations and is probably unpublished. Much of the actual experience with measures and measurement has probably been reported in an anecdotal rather than an archival form. Those who conduct future systematic reviews should be aware of this important issue and make an explicit effort to capture this unpublished information. Methods for systematically compiling and analyzing this type of data might help to improve the evidence base for quality measurement.

Strengthening Quality Measurement of Depression in Cancer

Given the recognized prevalence of depression among cancer patients, surprisingly few indicators have been evaluated depression in cancer. Indicators developed for other settings may be appropriate but have not been tested in this population. For late-stage disease, more fundamental clinical research is needed, for example to assess the optimal approach to screening and the usefulness of short term treatment.

Building an Evidence Base To Compare Population Subgroups

We found insufficient evidence to comment on population differences. An important priority for future research is evaluating measures in important sub-populations (e.g., those that differ by race/ethnicity, language, or gender). It is also important to define subgroups of cancer by stage of illness and to understand the performance of indicators among major subgroups, including disease type and settings of care. Fewer indicators are available for outpatient than for inpatient care evaluation, even though the majority of patient care over time occurs in the outpatient setting.

Literature Searches

Sources for our review included MEDLINE^®, the Cumulative Index to Nursing & Allied Health Literature (CINAHL^©) database, and PsycINFO.^© The RAND Library staff performed the searches. Members of the project team worked closely with the TEP and the librarians to refine the search strategy. We limited the searches to articles published in the English language, appearing in journals between the years 1995 and 2005, involving human subjects, and excluding individual case reports. We limited our search to the past decade because our prior experience demonstrated that most of the progress in the science of supportive or palliative care was limited to that period.

The main search strategy included an extensive list of terms intended to identify all research publications associated with all of the following:

• Each of the domains of interest (pain, dyspnea, depression, and advance care planning)

• Quality assessment or improvement

• Quality measurement

As described above, searches for pain and depression were limited with terms related to cancer. Searches for dyspnea and ACP were not limited with cancer-specific terms. We searched both the adult and pediatric cancer literature. We also searched “related articles” (e.g., using similar articles linked by MEDLINE^®) for citations that described using measures within the Assessing Care of the Vulnerable Elders (ACOVE) measure set that were likely to meet our criteria. All of these searches were conducted during February 2005. The search strategies can be found in Appendix C ^*.

Grey Literature - Internet Search

Because of limitations in the indexing of quality measures in traditional scientific databases, we employed a number of additional search strategies. We sought to identify guidelines that would point to recommended evidence-based standards of practice or specific quality measures that had been developed by health care organizations or specialty societies. Most of these searches involved the Internet. First, we used the National Guideline Clearinghouse (www.guidelines.gov) and the National Quality Measures Clearinghouse (http://www. qualitymeasures.ahrq.gov/) to identify guidelines and indicators that fit our criteria. In addition, we accessed the websites of specific professional societies, disease societies, medical organizations, medical associations, and patient advocacy groups to search for any relevant guidelines or measures, as follows:

• ASCO (American Society of Clinical Oncology) - www.asco.org

• European Society for Medical Oncolgy - www.esmo.org

• Canadian Cancer Society - http://www.cancer.ca/ccs/internet/frontdoor/ 0,,3172___langId-en,00.html

• Canadian Association of Psychosocial Oncology - http://www.capo.ca/eng/index.asp

• ONS (Oncology Nursing Society) - www.ons.org/evidence

• Canadian Association of Nurses in Oncology - http://www.cos.ca/cano/web/en/ index.html

• American Cancer Society - http://www.cancer.org/docroot/home/index.asp

• AAHPM (American Academy of Hospice and Palliative Medicine) - www.aahpm.org

• NHPCO (National Hospice and Palliative Care Organization) - www.nhpco.org

• EAPC (European Association of Palliative Care) - www.eapcnet.org

• PCA (Palliative Care Australia) - www.pallcare.org.au/

• Canadian Palliative Care Association - http://www.chpca.net/

• Growthhouse - www.growthhouse.org

• Center to Advance Palliative Care (CAPC) - www.capc.org

• Center for Palliative Care Studies - www.medicaring.org

• Americans for Better Care of the Dying - www.abcd-caring.org

• Innovations in End of Life Care (online journal) - http://www2.edc.org/lastacts/

• IHI (Institute for Healthcare Improvement) - www.ihi.org

• Promoting Excellence in End of Life Care - www.promotingexcellence.org

• Supportive Care of the Dying - www.careofdying.org

• MacColl Institute for Healthcare Innovation - http://www.improvingchroniccare.org/

• American Alliance of Cancer Pain Initiatives - http://www.aacpi.wisc.edu/

• American Pain Society - http://www.ampainsoc.org/

• Pain and Policy Studies Group - http://www.medsch.wisc.edu/painpolicy/

• American Thoracic Society - www.thoracic.org

• The National Comprehensive Cancer Network - www.nccn.org.

We also conducted Internet searches using Google^™ and the following search terms: guideline, clinical practice guideline, indicator, measure, quality measure, performance measure, cancer, metatstatic cancer, prostate cancer, breast cancer, lung cancer, colorectal cancer, depression, depressive symptoms, emotional depression, antidepressive agents, tricyclic antidepressants, selective serotonin re-uptake inhibitors, pain, morphine, opioids, nonsteroidal anti-inflammatory drugs, treatment of bone metastasis, radiation therapy, dyspnea, shortness of breath, anemia, erythropoeitin therapy, advance care planning, advance directives, goals of care, DNR, DNI, withholding or withdrawal of treatment.

Finally, we searched the following published sources for relevant guidelines or indicators: Directory of Clinical Practice Guidelines, Guide to Clinical and Preventive Services, Health Plan Employer Data and Information Set (HEDIS^©) 2005, The Medical Outcomes and Guidelines Sourcebook, Clinical Process and Outcomes Measurement Directory, and Behavioral Outcomes and Guidelines Sourcebook. ⁴⁴–⁴⁹ Whenever possible, we downloaded publications from the Internet; otherwise, we ordered them from the UCLA Biomedical Library or directly from the publishing organizations. All of these additional searches were conducted from January 2005 through March 2005.

Grey Literature - Measure Developers

We sought supplemental information on measures that we identified by directly contacting developers of those measures or measure sets. In our communications, we specified which additional types of information might be helpful for this report and used a “contact form” as a guide to solicit this additional information (Appendix D ^*).

End-of-Life Care and Outcomes Literature Database

We searched the reference database for the End-of-Life Care and Outcomes Project, a systematic review recently completed by this report's authors, for articles coded as “cancer.”²¹ This search yielded 389 articles that met the criteria and were screened for relevance to this project.

Introduction

For the review of pain measures, we reviewed all relevant clinical guidelines, indicator and measure sets, potential indicators, systematic reviews, and articles related to quality improvement initiatives and clinical trials, including one review of quality improvement initiatives for cancer pain. Please refer to Evidence Table Appendix F1 ^*. Pain was among the topics most commonly addressed within existing measure sets (6/8 of the sets identified). We designated most of the items we identified within these measure sets as quality measures since all of them have been operationalized and used for at least quality assessment. The exception is a handful of indicators that are either under development (e.g., Cancer Care Nova Scotia) or for which we were unclear whether or not they had previously been used for actual measurement or we were unsuccessful in obtaining information about that experience (e.g., Georgia Cancer Coalition).

The report of IOM's Committee on Assessing Improvements in Cancer Care in Georgia (National Cancer Policy Board)⁵⁵ (Chapter 6, Treating Cancer) has two indicators related to pain, one each for assessment and follow-up.

The performance measurement framework for Cancer Care Ontario⁵⁶ has a measure relevant to pain management, defined as patients' self-reported pain and perception of pain management by providers; this measure was operationalized as an indicator as “satisfaction with pain management.”⁵⁷

ACOVE generally excluded cancer patients, so the ACOVE set of measures for chronic pain⁵⁸ were not included in our review, although some of these measures may be appropriate for patients with cancer. However, ACOVE measures related to the end -of life did not exclude cancer; thus, that set includes one measure relevant to pain assessment.⁵⁹ This measure has also been adapted for nursing homes.⁶⁰

The VHA Transforming the Intensive Care Unit (TICU) initiative includes two measures for pain assessment and follow-up that are currently being used in 30 ICUs and have evidence for efficacy in a cancer ICU.^{61, 62}

The University Health Consortium (UHC) included two measures for assessment, one for treatment, and one for follow-up. QA Tools included three items relevant to pain, although only two were accepted by the expert panel. However, the indicator not accepted by the QA Tools panel has been proposed as part of Cancer Care Nova Scotia.

The NHPCO set includes one outcome measure for pain.⁶³

A broader cancer quality measurement set, QOPI (The Quality Oncology Practice Initiative), includes one measure for assessment of pain at the end of life. We also identified a treatment indicator in a guideline for radiation therapy.⁶⁴

Measures and Indicators

Assessment. We identified five measures and two indicators relevant to cancer pain assessment.

Measure - Regular Assessment of Pain. Quality Assurance Tools (QA Tools), proposed in 2000,⁶⁴ includes one measure relevant to pain assessment. The numerator of this measure is “patients with the presence or absence of pain noted every six months,” and the denominator is “all patients with cancer metastatic to the bone.” The measure was limited to patients with bone metastases because this group was felt to be well defined and to have the highest prevalence of cancer pain. The prevalence of every six months was chosen as a minimum requirement, based on older versions of clinical practice guidelines from the early 1990s. Although this measure was tested as part of the QA Tools evaluation in a VA- and community-based population,^{65, 66} the numbers of patients eligible for this indicator was too small to provide any useful validity, reliability, or comparability information. This measure is intended for use in patients with any type of cancer metastatic to the bone. The measure is not in current use.

Measure - Routine Inpatient Pain Assessment. The University Health Consortium (UHC) Palliative Care Benchmarking Project⁶⁷ included two measures relevant to pain assessment. The numerator for the first measure is “persons with pain assessment within 48 hours of admission.” The denominator is “adults ≥18 years of age, with admission for CHF (DRG 127), Cancer (DRG 82, 203, 172, 274, 346, 10) HIV (DRG 489) OR respiratory (DRG 475, 483) AND length of stay > 4 days AND 2 prior admissions for any cause in preceding 12 months.” This denominator was developed by an expert panel to represent a potential palliative care population in hospitals. A mean of 96% (median 98.5%, range 83–100%) of charts in the UHC cohort satisfied this indicator, including 98% of cancer patients, and little variability was seen among participating sites. No reliability or validity information is available; however, the benchmarking project found that using a combination of all of the UHC measures in a “palliative care bundle” was associated with lower patient length of stay. In addition, several elements of the bundle could be used to identify hospitals with “best practices” for palliative care. Scores from the bundle were instrumental in achieving advances in palliative care at several hospitals, including justifying at least one new palliative care program. The measure is intended for use as a tool for comparing performance in all hospitals. However, it is very broad, does not include an element of timeliness (within 48 hours of admission is late for pain assessment to occur), and is not in current use.

The numerator for the second UHC pain assessment indicator (routine inpatient pain assessment with a numeric scale) is “persons assessed with a numeric pain scale.” The denominator is “adults ≥ 18 years of age, with admission for CHF (DRG 127), Cancer (DRG 82, 203, 172, 274, 346, 10) HIV (DRG 489) OR respiratory (DRG 475, 483) AND length of stay > 4 days AND 2 prior admissions for any cause in preceding 12 months in the hospital, AND reported pain within the first 48 hours of admission.” A mean of only 76% (median 85%, range 13–100%) of the charts reviewed in the UHC cohort met this criterion, including 77% of those of cancer patients. Some variability was seen among sites; at ten low-performing sites, only 15 to 70% of charts included this information. Information is the same as for the first UHC indicator; no additional reliability or validity information is available.

Measure - Regular ICU Pain Assessment. The VHA TICU project, initially proposed in 2002,⁶¹ includes one pain assessment measure, which was modified for inclusion in the VHA TICU palliative care bundle in 2005 (Quality Indicator #5).⁶⁸ The numerator is “the total number of 4-hour intervals (within the first 24 hours following ICU admission) for which pain was assessed and documented, using an appropriate rating scale.” The denominator is “the total number of 4-hour intervals (within the first 24 hours following ICU admission) for patients with an ICU length of stay ≥ 24 hours.” Documentation of pain assessment may include (1) a numeric value of 1 to 10, or (2) pain was assessed but the patient was unresponsive (e.g., heavily sedated, comatose). A four-hour interval is defined as a four-hour interval of patient care (patient-nursing interval). The four-hour intervals are set at 8 am; 12pm; 4 pm; 8 pm; 12 am; and 4 am. An appropriate rating scale is defined as either a visual analogue scale (VAS with numeric translation of 1–10) or another numerical rating scale of 1–10. Exclusion criteria include intervals for which a patient was not physically available in the ICU (e.g. patient expired; not yet admitted to the ICU; was discharged/transferred out of the ICU (for part of the day); went into the operating room or off the unit for a procedure); and intervals for which pain was assessed, but without the use of an appropriate rating scale as defined above. This measure is supported by the JCAHO Hospital Standards RI.1.2.9, PE.1.4 and the National Consensus Project for Quality Palliative Care Clinical Practice Guideline, 2.1.⁶⁹ Evidence of its validity includes a rapid-cycle quality-improvement intervention in two ICUs, including a surgical cancer ICU, demonstrating that its use as part of a rapid-cycle quality improvement intervention, in which the percent of nursing intervals where pain scores were measured using a VAS scale increased from 42 to 94%, contributed to a decrease in pain scores ≥3 from 41% to 6%.⁶² In addition, a similar rating scale was used in an RCT of interrupted sedation in the ICU.⁷⁰ No reliability information is available.

This measure is intended for use in all ICU populations. As with other pain outcome measures, variability in the quality of pain assessment may limit its use for comparing performance. In the ICU population, as at the end of life or in nursing homes, many patients may have difficulty or may be unable to verbalize a numerical pain score; depending on local standards of assessment, this outcome variable may therefore apply to very different proportions of the population in different ICUs or settings. The measure also does not specify whether this score should represent minimum, maximum, or average pain. Also, the measure applies only to the first 24 hours in the ICU. This measure is currently being used as part of the VHA TICU initiative and is part of the VHA palliative care bundle, which will be initiated in September 2005.

Measure - Assessing Pain in Patients Close to Death. QOPI (the Quality Oncology Practice Initiative) included one end-of-life pain assessment measure in their practice-based system of quality assessment. The numerator of this measure is “medical records where there is an explicit practitioner's notation quantifying their physical pain or lack thereof on their last visit to the office prior to death,” and the denominator is “recently deceased patients from oncology practices.”⁷¹ QOPI was developed and sponsored by individual oncologists and ACSO (the American Society of Clinical Oncology). QOPI selected and developed quality measures that were easily identifiable through outpatient medical record review. The measures also had to relate to important issues, as defined by (1) consensus among QOPI participants, (2) evidence-based standards, or (3) requirements from organizations such as JCAHO. This measure was evaluated through review of 10 medical records of patients who had died in 41 oncology practices. The initial round of assessment showed that 56% of medical records met this measure, 69% met the measure in a second round 6 months later, and there was substantial variation among practices (range, 30–90%). No validity or reliability information is available.

This measure is intended for use in self-assessment in outpatient oncology practices. The retrospective nature of the measure is a drawback, since death is often not predictable, and improvements might be due to changes in documentation that do not necessarily reflect improvement in practice. This measure will be part of a new voluntary quality improvement initiative for oncology practices sponsored by ASCO, NCI, and the National Health Council, to begin in 2005. End-of-life will be the first initiative and will include this measure as well as whether the pain was quantified by either a numerical scale or another descriptor.

Indicator - Routine Pain Assessment in Expected Dying. The Assessing Care of the Vulnerable Elderly (ACOVE) end-of-life measure set⁵⁹ includes one measure relevant to assessment for pain. Its numerator is “the number of vulnerable elders with documentation of pain or lack of pain during the last three days of life.” The denominator is “vulnerable elders who are conscious during the last three days of life and who died an expected death.” Although the eight measures from the ACOVE end-of-life set were evaluated for reliability and validity⁷² as part of the ACOVE project, this measure was not included in the evaluation. The denominator may be difficult to define, since “expected death” requires explicit documentation of that expectation in the medical record more than one day before death. This measure is intended for use in all vulnerable elders. An expert panel accepted a modified version of this indicator as part of a revised set for nursing homes.⁶⁰ The nursing home indicator requires a denominator of all nursing home residents who died an expected death and were conscious at all during the last seven days of life.

Indicator - Routine Assessment of Pain. The IOM's Committee on Assessing Improvements in Cancer Care in Georgia (National Cancer Policy Board) proposed an indicator set in 2005 with two indicators related to pain, one of which targeted regular assessment.⁵⁵ The numerator for this indicator is “the number of cancer patient encounters where the patient was assessed for pain,” and the denominator is “the number of cancer patient encounters.” Available information does not specify the appropriate frequency of cancer pain assessment (i.e., every encounter) or how pain should be measured. The indicator is intended for use for all cancer patients, across the continuum of care and all health care settings. This indicator is newly proposed and information on actual use is not available.

Treatment. We identified three measures and one indicator relevant to cancer pain treatment.

Measure - Responsive Pain Treatment. QA Tools includes one measure relevant to treatment of pain, proposed in 2000.⁶⁴ The numerator is “patients offered a change in pain management within 24 hours of the pain complaint,” and the denominator is “all cancer patients whose pain is uncontrolled.” As with the other QA tools indicators, although this measure was tested as part of the QA Tools evaluation in VA and community-based populations,^{65 66} the number of eligible patients was too small in these studies to determine validity or reliability. This measure is intended for use in patients with any type of cancer across the continuum of care. The measure is not in current use.

Measure - Regular Prophylaxis of Opioid-Induced Constipation. The UHC Palliative Care Benchmarking Project also included one measure relevant to pain treatment that addresses safety or avoiding side effects that can be effectively prevented. The numerator is “patients with a bowel regimen initiated within 24 hours of an opioid, and in whom a bowel regimen is not contraindicated.” The denominator is “adults ≥ 18 years of age, with admission for CHF (DRG 127), Cancer (DRG 82, 203, 172, 274, 346, 10) HIV (DRG 489) OR respiratory (DRG 475, 483) AND length of stay > 4 days AND 2 prior admissions for any cause in preceding 12 months in the hospital, AND treated with narcotics.” The treatment of constipation as a side effect of opioid administration was addressed in a recent systematic review.⁷³ The review found that 25–50% of cancer patients have constipation and that it is the most frequently occurring adverse effect of opioid use in advanced cancer; however, the review of 17 studies found none that directly compared patients treated with an opioid regimen with those not treated. A similar potential measure has also undergone some field testing in a population that included cancer patients as part of the JCAHO-AMA-NCQA pain management performance measure development project.⁷⁴ A mean of only 59% (median 59%, range 20–93%) of the charts reviewed in the UHC cohort met this criterion, and the distribution of the outcome appears broadly normal. Eighty-three percent of cancer patients met the indicator. No validity or reliability information is available for this indicator; more details on the testing of the measurement set are described in the UHC pain assessment measures. This measure is intended for use across all types of cancer. The measure set is not in current use.

Indicator - Minimizing Radiotherapy Burden. Cancer Care Ontario's Program in Evidence-Based Care reported a clinical practice guideline in 2003, Radiotherapy Fractionation for the Palliation of Uncomplicated Painful Bone Metastases, Practice Guideline Report #13-2. This report includes an indicator relevant to the choice of radiotherapy regimen in patients with advanced cancer.⁷⁵ The numerator is “patients receiving single dose therapy as a single 8Gy treatment, prescribed to the appropriate target volume.” The denominator is “all adult patients with single or multiple radiographically confirmed bone metastases of any histology corresponding to painful areas in previously nonirradiated areas without pathologic fractures or spinal cord/cauda equina compression.” It does not apply to the management of malignant primary bone tumor and is intended for patients receiving radiotherapy where the objective or intent is pain relief. The report, which was based on a systematic review, also includes a number of qualifying statements. The evidence was reviewed by a four-member group and then approved by a multidisciplinary group. External review was obtained through a mailed survey of Ontario practitioners. The indicator is based on 2 systematic reviews and 16 RCTs. A meta-analysis of eight RCTs, conducted as part of the guideline process, found no significant difference in response rate between single-fraction and fractionated radiotherapy. There were no differences in quality of life (for the few studies where this was assessed) or side effects. Observed re-irradiation rates were higher with single-fraction than with multiple-fraction therapy, but details were not described. A separate systematic review and meta-analysis⁷⁶ derived the same conclusions about pain, but concluded that the pathologic fracture rate was significantly higher for single-fraction than for multiple-fraction treatment. The indicator is intended for use in a mixed cancer population, for palliative treatment. This indicator is relevant to the efficiency of care. The indicator is not known to be in current use.

Indicator - Effective Treatment for Painful Bony Metastasis. Cancer Care Nova Scotia⁷⁷ has proposed a set of quality indicators for palliative cancer care that includes one pain treatment indicator, adapted from a proposed QA tools indicator that was not accepted by the QA Tools expert panel and was not part of the final set.⁶⁴ The numerator of this indicator is “patients who are offered one of the following within one week of the notation of pain: radiation therapy to the sites of pain, or radioactive strontium therapy.” The denominator of this indicator is “patients with cancer metastatic to the bone and pain uncontrolled by opioids.” Although there is excellent meta-analysis evidence of the efficacy of radiation therapy for bone pain,⁷⁶ another meta-analysis found that the evidence supporting the efficacy of radioisotopes such as strontium was inconclusive,⁷⁸ and we did not identify any evidence that supports the use of radiation therapy only for pain uncontrolled by opioids or that supports the timing of radiation therapy. Although this indicator was proposed, based on feedback from CCNS expert panels and available datasources, the indicator will not be field tested at the current time because “uncontrolled pain” could not be operationalized. The indicator is intended for use in patients with metastatic breast cancer and addresses both effectiveness and timeliness. The indicator may be limited by the lack of definition of “pain uncontrolled by opioids.” The indicator set is proposed and is not in current use.

Follow-up. We identified four measures and one indicator relevant to the follow-up of cancer pain treatment.

Measure - Timely Treatment of Pain in Hospice. The NHPCO indicator set for hospice,^{54, 63, 79} includes one outcome measure for pain. The numerator is “the proportion of patients between 48 and 120 hours after admission who report ‘yes’ when asked, ‘Was your pain brought to an acceptable level within 48 hours of your admission to the hospice program?’” The denominator is “patients admitted to hospice who answer ‘yes’ when asked, 'Are you uncomfortable because of pain?” The measure does not apply to patients in nursing facilities, children, non-English-speaking patients, or others who cannot self-report, because of concerns about measurement. It assumes that all patients are assessed for pain on admission to hospice, and detailed information on how to administer the measure is available.⁶³ The NHPCO chose to focus only on outcome measures, and considered and rejected alternative data collection methods for pain outcomes, including chart audits and retrospective surveys. They also chose not to use proxy reports because of issues with reliability and validity.⁵⁴ This concern, as well as the difficulty in assessing pain in the last days of life, led to the timing for the indicator at hospice admission. A non-numerical dichotomous rating was chosen because of concerns about the validity of numerical ratings. For construct validity, in pilot testing, more people said that they were “uncomfortable” because of their pain than said their pain was “unacceptable,” so the word “uncomfortable” was chosen as more sensitive for the denominator.⁵⁴ Results from the use of this measure as part of the measurement set have also been published.⁷⁹ For the 78 agencies that submitted data in 2002, the mean was 87% (median, 90%; 25^th & 75^th percentiles, 79% & 100%). These data suggest that there may be sufficient variation among hospices for this measure to be useful for comparing performance or quality improvement. No other reliability or validity information is available. A previous review of the palliative care literature found that general measures of satisfaction or subjective ratings of health care were subject to a strong ceiling effect, and items or measures that addressed specific processes tended to exhibit better variability.²¹ This measure is intended for use in hospice care. It is limited by the inability of many hospice patients to report their pain. In addition, as an outcome measure, it would require evaluation for whether there needs to be risk adjustment for variation among organizations. Population characteristics such as the percentage of patients who report pain on admission, patients who cannot self-report, and patients with a history of drug abuse may significantly influence results. It is currently being used by some hospice programs for internal quality assurance purposes and voluntary, confidential reporting to NHPCO.

Measure - Effective Treatment of Pain in the ICU. This measure, initially evaluated in 2002,⁶¹ was modified for inclusion in the VHA TICU palliative care bundle in 2005 as an outcome measure for pain (Quality Indicator #6).⁶⁸ The numerator is “the total number of 4-hour intervals (within the first 24 hours following ICU admission) for which the documented pain score was <3.” The denominator is “the total number of 4-hour intervals (within the first 24 hours following ICU admission) with numerical pain values of 1 to 10, for patients with an ICU length of stay ≥ 24 hours.” Exclusion criteria include intervals for which a patient was not physically available in the ICU (e.g. patient expired; was not yet admitted to the ICU; was discharged/transferred out of the ICU [for part of the day]; went into the operating room or off the unit for a procedure); intervals for which a pain score was not documented; intervals for which pain was assessed, but the patient was unresponsive (e.g., heavily sedated; comatose); and intervals for which pain was assessed, but without the use of an appropriate rating scale as defined above. This measure is based upon a rapid-cycle quality-improvement intervention in two ICUs, including a surgical cancer ICU, demonstrating its responsiveness to changes in care processes (including the other VHA measure listed under assessment as well as whether house staff documented the pain score). It is also based on the JCAHO Hospital Standards RI.1.2.9., PE.1.4; and the National Consensus Project for Quality Palliative Care Clinical Practice Guideline, 2.1. No additional reliability or validity information is available.

This measure is intended for use in all ICU populations. As with other pain outcome measures, variability in the quality of pain assessment may limit its use for comparing performance. In the ICU population, as at the end of life or in nursing homes, many patients find it difficult or may be unable to verbalize a numerical pain score; depending on local standards of assessment, this measure may therefore be applicable to very different proportions of the population in different ICUs. The measure also does not specify whether this figure should be a minimum, maximum, or average pain score. It is currently being used as part of the VHA TICU initiative and is part of the VHA palliative care bundle, which will be initiated in September 2005.

Measure - Satisfaction With Pain Treatment. This measure is part of the indicator set proposed by Cancer Care Ontario in 2005.⁵⁷ The numerator of this measure is reported in three categories: “Patients where the response to the question “Do you think staff did everything they could to control your pain or discomfort?” was (1) yes, completely; (2) yes, somewhat; or (3) no.” The denominator is “outpatient cancer patients reporting mild to severe pain.” The source of this indicator is patient surveys. Information on reliability/validity as an indicator is not available. In a survey of more than 5,000 cancer outpatients in 2004, 70% of patients answered “yes, completely,” 25% answered “yes, somewhat,” and only 5% answered “no.” Variability by center was not reported, and all four provinces where the survey was done apparently had similar results.⁵⁷ The low number of “no” answers is consistent with previous studies of satisfaction for pain, which have found that overall satisfaction reports may not reliably correlate with actual pain scores and may be difficult to use because of ceiling effects.²¹ The indicator is intended for use in ambulatory/outpatient care. It may be affected by recall bias (used for care in the previous six months), low response rate among palliative care patients, and a requirement for a minimum denominator of 300 patients per participating center. It is currently being used in Ontario and three other Canadian provinces.

Measure - Timely Treatment of Inpatient Pain. The UHC Palliative Care Benchmarking Project,⁶⁷ introduced in 2004, includes one outcome measure for pain. The numerator is “persons with pain relief or reduction within 48 hours of admission to ≤ 3 on a 0–10 scale.” The denominator is “adults ≥18 years of age, with admission for CHF (DRG 127), Cancer (DRG 82, 203, 172, 274, 346, 10) HIV (DRG 489) OR respiratory (DRG 475, 483) AND length of stay > 4 days, 2 prior admissions for any cause in preceding 12 months in the hospital, AND reported pain within the 1st 48 hours of admission.” A mean of 76% (median 78%, range 46–92%) of the UHC cohort met this indicator. Only 70% of cancer patients met the indicator. There was moderate variability in this measure; although the range of performance was somewhat restricted, the distribution of results appeared to be relatively normal. As with the other UHC measures, no individual reliability or validity information is available. It is not known to be in current use.

Indicator - Effective Treatment of Pain. The IOM Committee on Assessing Improvements in Cancer Care in Georgia's (National Cancer Policy Board) 2005 indicator set proposed an indicator relevant to the follow-up of pain treatment.⁵⁵ The numerator is “the number of cancer patients who report being in more than minor pain.” The denominator is “the number of cancer patients who are not comatose.” This indicator is based on the 2004 cancer pain guideline from the National Comprehensive Cancer Network (NCCN),⁸⁰ the 1996 cancer pain guideline from the World Health Organization,⁸¹ the cancer pain position statement from the Oncology Nursing Society (ONS),⁸² and the American Pain Society Quality Improvement Guidelines for the treatment of acute pain and cancer pain.⁸³ Evidence is based on the AHRQ evidence report on treatment of cancer pain⁷ and NIH Consensus State of the Science Statement on symptom management in cancer.⁸⁴

The indicator is intended for use for all cancer patients, across the continuum of care and all health care settings. A validated pain scale that defines “minor pain” must be used in each health care setting, and the threshold should be reported along with the indicator. Poor medical record documentation may lead to low prevalence estimates. Potential benchmark sources are baseline patient surveys and medical records. Evidence suggests that members of ethnic minority groups, women, and elderly patients may be less likely to be adequately treated for pain (Chapter 6, p. 23).⁵⁵ This indicator is newly proposed and information on use is not available.

Potential Indicators and Measurement Gaps

We identified a large number of potential indicators in the substantial guideline, quality improvement, and research literature that we reviewed. Many of these guideline statements and potential indicators address specific management issues, such as the treatment of neuropathic pain, or more details on the indicators we have identified, such as more-comprehensive assessment for pain or pain outcomes.⁷⁴ We highlight here eight potential indicators with relatively strong supporting evidence and/or expert consensus, highlighting critical issues not addressed by the indicators and measures listed above, that may have potential for demonstrating variability in quality and improving patient outcomes, and applicable to cancer patients across the continuum and to diverse populations.

Assessment

Physician Documentation of Pain Assessment. The numerator is “patients where a pain assessment is documented in the physician note,” and the denominator is “all patients with cancer.” This potential indicator was used as a part of the pain continuous quality improvement study that led to the VHA pain measures ⁶¹ and is supported by evidence that pain assessment by nurses or aides often does not lead to documentation of pain by the physician. Improvements in pain assessment by non-physician staff may not lead to improved outcomes because the physician may not attend to the report of pain. ⁸⁵ Another consideration with pain assessment indicators is the extent to which they might address aspects of assessment including specific symptom attributes (e.g., timing, duration, and functional impairment) and specific etiologies.

Disparities in Pain Assessment. The numerator is “non-white patients with cancer who are assessed for pain,” and the denominator is “all patients with cancer who are assessed for pain.” Ensuring that pain assessment is equitable across ethnic groups is important because large studies have demonstrated that members of minority groups with cancer have higher rates of inadequate pain assessment and treatment than others.86,87 A recent nonsystematic review of this topic found that this disparity persists for both assessment and treatment, across multiple settings and types of pain.88 Whenever possible, assessment of disparity by ethnicity should also include assessment of other contributing factors, such as socioeconomic status and access to care, since these are often the primary cause of perceived disparities by race.

Assessment in Patients Who Have Difficulty Verbalizing Their Pain. The numerator of this potential indicator is “patients who have difficulty verbalizing their pain but who are assessed for pain,” and the denominator is “cancer patients who have difficulty verbalizing their pain.” This potential indicator is important because many of the indicators and measures we identified exclude patients who are unable to verbalize their pain. Many cancer patients at high risk for pain would therefore be excluded from many of these measures, including infants and small children, many seriously ill or perioperative cancer patients, cancer patients with severe dementia, and many patients in the last days of life, where a decreased level of consciousness is very common. In addition, patients with some degree of cognitive impairment or sedation may also be excluded from the denominator even though many of these patients are able to report their pain.⁴⁰ Variable exclusion of these patients limits the usefulness of pain assessment measures for comparisons between facilities or over time. Recent systematic reviews have summarized the variety of tools available for assessing pain in older adults with cognitive impairment⁸⁹ and in infants.⁹⁰ In addition, some evidence exists that cognitively impaired adults receive less pain medication than those who are cognitively intact and that untreated pain may lead to higher rates of adverse outcomes, such as delirium.⁹¹

Assessment for Depression in Patients With Pain. The numerator is “patients assessed for depression,” and the denominator is “all cancer patients with pain.” This potential indicator is supported by evidence that cancer patients with pain are more likely to be depressed than cancer patients without pain.92 In addition, there is substantial descriptive evidence that the symptoms of depression and pain often occur together.¹⁰

Treatment

Educating Patients About Pain Management. The numerator is “patients who are educated about pain management,” and the denominator is “cancer patients with pain.” This guidance is included in many cancer pain guidelines, some of which also include education of caregivers. A related potential outcome measure, that patients should be adequately informed and knowledgeable about pain management, is included in the 2005 American Pain Society Recommendations for Improving the Quality of Acute and Cancer Pain Management,⁷⁴ based on an expert review of the measurement tools used in 20 quality improvement studies in pain management.⁹³ A systematic review of educational interventions for patients and family caregivers found eight studies, two of which were randomized, controlled trials; both of these and all of the nonrandomized studies where pain management education was measured found a significant impact of education on pain.⁹⁴

Use of Opioids for Severe, Persistent Pain. The numerator of this potential indicator is “patients who are treated with opioids,” and the denominator is “cancer patients with severe, persistent pain.” This potential indicator is supported by a number of clinical practice guidelines for pain, including the World Health Organization⁸¹ and NCCN.⁸⁰ One of the pilot measures that was field-tested as part of the JCAHO-AMA-NCQA initiative also relates to this issue,⁷⁴ but it needs further development before use as a measure. A randomized trial in advanced cancer patients comparing the use of strong opioids to the WHO stepped-care approach where patients are started on non-opioids, then given weak opioid medications, found that patients treated with opioids had better pain outcomes.⁹⁵

Use of Long-Acting Pain Medications. The numerator is “all adults who are prescribed long-acting opioids,” and the denominator is “all adults with cancer who have persistent pain and are prescribed opioids.” This expert guidance is in numerous pain guidelines, including the Agency for Health Care Policy and Research cancer pain guideline⁹⁶ and the American Geriatrics Society cancer pain guideline for older adults. ⁹⁷ A systematic review of the cancer pain literature found only two relevant trials, which showed no difference between long-acting and around-the-clock short-acting formulations.⁷ However, the expert consultants for the Cancer Symptom Management State-of-the-Science Statement⁸⁴ also included this recommendation because improved adherence with long-acting drugs in clinical practice may improve outcomes.

Procedure-Related Pain in Children. The numerator is “all children who receive appropriate preparation and/or anesthesia” and the denominator is “all children with cancer receiving painful procedures.” We chose to highlight this potential indicator because it is emphasized in guidelines for cancer pain in children but is typically not mentioned in guidelines for adults. The World Health Organization³⁴ describes this area as important because the pain from procedures is often worse than that from the cancer and because inadequate pain control may lead to anxiety, reduced adherence, and poorer relationships with health care providers.

In addition, many pharmacological and nonpharmacological approaches, including the presence of parents whenever possible, local anesthetics, and opioids when needed, can reduce procedural pain. An assessment tool for postoperative pain management in children has been developed and psychometrically tested in children as young as eight years old.⁹⁸ Many randomized, controlled trials have demonstrated that use of pharmacologic measures such as topical anesthetics not only reduces pain, but reduces procedure time and improves procedure success rates.⁹⁹ Such an indicator might also be reasonable to consider for adults.

Introduction

We failed to identify any quality measures that were developed specifically for dyspnea in cancer. Please refer to the Evidence Table in Appendix F2 ^*. Two measures were developed and tested as part of the UHC Project on palliative care for dyspnea assessment and dyspnea reduction/relief within 48 hours of hospital admission.⁶⁷ The UHC indicators were used in an adult population with cancers that included lung, gastrointestinal, breast, male reproductive, and nervous system neoplasms - approximately 25% of the UHC cohort had cancer.

We identified two indicators for dyspnea applicable to the cancer population derived from the Assessing Care of Vulnerable Elders (ACOVE) project⁵⁹ and one indicator developed for COPD from the RAND QA Tools project that is applicable to patients with cancer and hypoxia.⁶⁵

Measures and Indicators

Assessment. We identified one measure and zero indicators relevant to dyspnea assessment.

Measure - Regular Assessment of Dyspnea. The University Health System Consortium (UHC) Palliative Care Benchmarking Project used a performance measure across an alliance of academic health centers in the United States in 35 participating hospitals for dyspnea assessment within 48 hours of admission.⁶⁷ The numerator is “persons with dyspnea assessment within 48 hours of admission.” The specific criteria for how dyspnea assessment was operationalized were not described in available reports (other than the use of a 10-point scale). The denominator is “adults 18 years of age and older, with admission for congestive heart failure (DRG 127), cancer (DRG 82, 203, 172, 274, 346, 10), HIV (DRG 489), or respiratory illness (DRG 475, 483) and a length of stay more than 4 days and 2 prior hospital admissions for any cause in the preceding 12 months.” This denominator was developed by an expert panel to represent a potential palliative care population in hospitals. No reliability or validity information is available; however, the benchmarking project found that a combination of use of all of the UHC measures in a “palliative care bundle” was associated with shorter patient length of stay. In addition, several elements of the bundle could be used to identify hospitals with “best practices” for palliative care. Scores from the bundle were instrumental in achieving advances in palliative care at several hospitals, including justifying at least one new palliative care program.

The dyspnea assessment measure is intended for use as a tool for comparing performance in all hospitals. The data for this measure come from billing, administrative, and clinical databases combined in the UHC data set. The UHC report indicates that a mean of 91% (median 95%, range 53–100%) of patients was assessed for dyspnea within 48 hours with the average rate for cancer being 89.9%. The measure criterion appears broad and does not include an element of timeliness (within 48 hours of admission may be too long for dyspnea assessment to occur). Further details about operationalization and details on psychometric testing are needed. The measure is not known to be in use.

Treatment. We identified zero measures and three indicators relevant to dyspnea treatment.

Indicator - Effective Dyspnea Treatment in Expected Dying When Forgoing Ventilator. ACOVE describes a set of quality indicators for adults age 65 years and older in 2001.⁵⁹ The numerator is “whether or not the patient received or had an order for an opioid, benzodiazepine, or barbiturate infusion to reduce dyspnea and the chart documented whether the patient has dyspnea.” The denominator is “all non-comatose vulnerable elders who are not expected to survive or those for whom a mechanical ventilator is withdrawn or intubation is withheld.” The indicator is designed to be evaluated by retrospective chart review and is most applicable to hospitalized patients; however, the indicator could be applied to any setting where ventilator withdrawal or withholding could occur. The withdrawal and withholding for mechanical ventilation indicator is supported by documentation of variation and inadequate management of dyspnea,^{100, 101} best-practice recommendations,^{102, 103} and statements from professional societies.^{59, 104, 105}

Indicator - Regular Treatment and Follow-up of Dyspnea in Expected Dying. A second ACOVE indicator addresses the follow-up of dyspnea treatment at the end of life.⁵⁹ The numerator is “whether or not the medical record documents how dyspnea was treated and followed up.” The denominator is “all vulnerable elders who had dyspnea in the last seven days of life and died expected deaths.” The indicator is designed to be evaluated by retrospective chart review and is applicable to any setting of care. This indicator was based on the premise that symptoms such as dyspnea should be a focus of end-of-life care and consensus calling for improvement in the care and management of dyspnea. In supporting documentation, the developers stated that dyspnea can be effectively treated with oxygen and pharmacologic agents.⁵⁹ Like the withholding and withdrawal of ventilation indicator, this indicator was not evaluable in the community-based study of ACOVE measures.⁷²

Indicator - Treatment of Dyspnea Caused by Hypoxia. QA Tools did not include indicators for dyspnea within its multiple cancer sets.⁶⁴–^{66, 106} However, within the category of chronic obstructive pulmonary disease (COPD), one indicator for hypoxia is applicable to cancer patients with dyspnea caused by hypoxia.^{65, 66, 107} The numerator is “whether home oxygen therapy was used.” The denominator is “all patients with baseline room air oxygen saturation < 88% at rest.” The original indicator in the QA Tools project for patients with COPD read as follows: “COPD patients should receive home oxygen if their baseline room air oxygen saturation is <88% at rest (not during an exacerbation).”⁶⁵ This indicator was based on multiple studies demonstrating benefit to both survival in hypoxic patients and subjective improvement of dyspnea symptoms through oxygen therapy.¹⁰⁷ More recent data supporting use of oxygen for symptomatic improvement in hypoxic patients exists, including a recent review by a working group of the scientific committee of the Association of Palliative Medicine.¹⁰⁸ This review included primarily patients with COPD and demonstrated more consistent symptom improvement with oxygen for those with hypoxia.

The indicator is designed to be evaluated by retrospective chart review. This indicator was used in the COPD population for outpatient care during routine clinical interactions that did not occur during an exacerbation, however it may be pertinent to other settings such as physician offices, home health care, and home hospice. The indicator was operationalized by looking at the lowest oxygen saturation at least two weeks before the close of the study and scored as meeting the criterion if an order, prescription, or discharge order for home oxygen was identified within two weeks of a saturation value less than 88% or a note that the patient was already on home oxygen that was not discontinued. Reliability was based on a 4% duplicate abstraction from the medical record and demonstrated a kappa of 0.83 for agreement on the condition (COPD), a kappa of 0.76 for agreement on eligibility for the indicator, and a kappa of 0.80 for scoring. To become a measure for cancer care, the numerator and denominator of this indicator would need to be explicitly operationalized and tested in cancer populations.

Follow-up. We found one measure and zero indicators related to follow-up for dyspnea.

Measure - Timely Treatment of Inpatient Dyspnea. The UHC Palliative Care Benchmarking Project used a second performance measure regarding dyspnea relief or reduction of dyspnea.⁶⁷ The numerator is “persons with dyspnea relief or reduction within 48 hours of admission.” Significant reduction was operationalized as a 3-or-more-point reduction in dyspnea on a 10-point scale. The denominator is “adults 18 years of age and older, with admission for congestive heart failure (DRG 127), cancer (DRG 82, 203, 172, 274, 346, 10), HIV (DRG 489), or respiratory illness (DRG 475, 483) and a length of stay more than 4 days and 2 prior admissions for any cause in the preceding 12 months and reported dyspnea within the first 48 hours of admission.” No reliability or validity information is available. Information is the same as for the other UHC dyspnea indicator. The UHC report indicates that 62% of patients assessed had dyspnea within 48 hours after admission and that of these, 77% overall and 77% of cancer patients had relief or reduction of these symptoms. There was substantial variation in results with a range of 37.5% to 96.6% criteria met across the 35 hospitals participating in the benchmarking project. Further details about operationalization and details on psychometric testing are lacking. The indicator set is not known to be in use.

Potential Indicators and Measurement Gaps

Assessment. We identified a large number of potential indicators in the substantial guideline, quality improvement, and research literature that we reviewed. We highlight here six potential indicators with relatively strong supporting evidence and/or expert consensus, highlighting critical issues not addressed by the indicators and measures listed above. These indicators have potential for demonstrating variability in quality and improving patient outcomes, and are applicable to cancer patients across the continuum and to diverse populations.

Although guidelines and evidence support assessment of dyspnea,^{15, 18, 21, 109} future research is needed to understand the most appropriate approach and instruments for assessment (i.e., provider observation, single-item symptom measures, or multi-item self-report effects of dyspnea) in particular settings and diseasesⁱ as well as operating characteristics as quality indicators.^{15, 16, 109} Subsequent research is indicated for indicators in dyspnea assessment. In the interim, it would be reasonable to develop and use a general indicator that at least some form of dyspnea symptom assessment was performed for all cancer patients.

Assessment for Treatable Causes of Dyspnea. The numerator is “all patients who have been assessed for treatable causes of dyspnea.” The denominator is “all cancer patients with dyspnea.” We identified four reports on the assessment of underlying causes of dyspnea.^{14, 109}–¹¹¹ The American College of Chest Physicians (ACCP) and American Thoracic Society (ATS) have recommended specific assessment for potentially correctable causes of dyspnea in this setting.^{109, 110} A prospective observational study demonstrated potentially correctable causes for dyspnea such as hypoxia (40%), anemia (20%), and bronchospasm (52%).¹⁴ Another report supports the use of spirometry to detect any airway obstruction in patients with dyspnea.¹¹¹ A reasonable potential indicator would be to use medical record review to document that correctable conditions were sought in evaluation of dyspnea in cancer patients. Some consideration of time windows for recent evaluation would be important in operationalizing this as a standard.

Treatment. We identified eight potential indicators related to the treatment of dyspnea, from seven reports.^{110, 108, 109, 112}–¹¹⁵ As with the general research gap in assessment of dyspnea, the operationalization of how much and when to assess for response to therapy requires future research and refinement of quality indicators.

Treatment for Malignant Pleural Effusions. The numerator is “patients who have had thoracentesis and appropriate subsequent therapy (pleurodesis or chemotherapy) as indicated.” The denominator is “cancer patients with dyspnea and malignant pleural effusions.” Evidence-based guidance from the ACCP Lung Cancer Guidelines, published by the American College of Chest Physicians includes (1) Patients with malignant pleural effusions that cause dyspnea initially should be drained by thoracentesis; (2) Patients with non-small cell lung cancer and better performance status and recurrent malignant pleural effusions, and whose lungs re-expand with initial thoracentesis or thoracoscopy, should be followed up by pleurodesis;¹¹⁰ and (3) Patients with small cell lung cancer receive systemic chemotherapy when malignant effusions are present to relieve the symptoms of dyspnea.¹¹⁰ Potential indicators for these specific conditions could be developed and tested. This guidance is clinically applicable to managing dyspnea in other cancers that cause malignant pleural effusions (breast, gastrointestinal, prostate, melanoma, others).

Bronchoscopic Therapy. The numerator is “patients who are offered, or who have, bronchoscopic therapy.” The denominator is “cancer patients with central airways obstruction.” The ACCP¹¹⁰ also present evidence based on a review of the literature that bronchoscopic therapy can reduce dyspnea in the presence of central airway obstruction, recommending that patients with central airway obstruction should have bronchoscopy and that in patients with central airway obstruction, rapid relief of dyspnea can be accomplished by removal of intraluminal tumor and/or by inserting a stent. A potential indicator may be that if a patient has cancer and central airway obstruction, then bronchoscopy and bronchoscopic therapy should be considered. This potential indicator is most applicable to lung cancer.

Patient Education and Non-Pharmacologic Interventions. The numerator is “patients who have received education about their dyspnea.” The denominator is “all cancer patients with dyspnea.” Kvale and colleagues¹¹⁰ recommended that all lung cancer patients with dyspnea receive non-pharmacologic, non-interventional treatments including patient education to reduce the dyspnea symptoms. Patient education can encompass breathing control education, relaxation techniques, and psychosocial support. The evidence for this potential indicator was poor based on their review of the literature, and these interventions were felt to have poor supporting evidence but moderate net benefit due to the lack of side effects. This indicator could be broadly formulated based on clinical practice to all forms of malignancy with dyspnea.

Palliative Radiation and Chemotherapy. ^{113, 114} The numerator is “patients who are offered or have received appropriate palliative chemotherapy or palliative radiation therapy.” The denominator is “all patients with particular malignancies.” The Lung Cancer Guidelines published by the American College of Chest Physicians¹¹⁴ present evidence that chemotherapy can reduce dyspnea symptoms among patients with Stage IV non-small cell lung cancer. Socinski et al., identified seven studies that explored the palliation of symptoms by chemotherapy; four of the seven reduced dyspnea symptoms in over half of the patients enrolled.¹¹⁴ Palliative chemotherapy should be considered for patients who are deemed suitable for treatment, including other malignancies.

The American Society of Clinical Oncology published a guideline for the treatment of unresectable non-small cell lung cancer¹¹³ in 2003 in which they cite research that radiotherapy can provide relief from dyspnea symptoms. However, the evidence cited in this report included only nonrandomized comparisons to conventional therapy. This indicator may likely be operationalized only in retrospective medical record review with fulfillment of criteria as consideration for therapy.

General Approaches. The numerator is "patients who have had oxygen, bronchodilators, corticosteroids, opioids, or antibiotics attempted. The denominator is “patients with cancer and dyspnea (and possibly co-morbid conditions such as COPD).”¹¹⁰ The evidence for the effectiveness of particular interventions is modest, but the net benefit may be moderate.¹¹⁰ The ATS endorsed oxygen therapy, opioids, anxiolytics, and bronchodilator trials to diminish dyspnea symptoms among terminally ill patients with lung disease.¹⁰⁹ Several systematic reviews support the efficacy of opioids and oxygen in patients with advanced illness including cancer.^{21, 112, 116} A reasonable criterion may be to consider whether therapy was considered, offered, or tried. The operationalization of this indicator would require further specification based on underlying disease, the physiologic mechanism of dyspnea, and particular therapeutic interventions (for example, specifying concomitant obstructive lung disease when evaluating use of bronchodilators).

Introduction

For the condition of depression, we searched for measures and indicators restricted to the condition of cancer. We identified one measure and three indicators that were relevant. Please refer to the Evidence Table in Appendix F3 ^*.

The UHC Palliative Benchmarking Project has evaluated one indicator related to formal psychosocial assessment in mixed hospitalized populations that include cancer patients.⁶⁷

ACOVE's end-of-life indicators addressed spiritual assessment prior to death, which we considered broadly applicable.⁵⁹ ACOVE also included measures and indicators for depression, as did QA Tools.^{65, 72} Some of these may be regarded as relevant, although we included only two additional ACOVE indicators, because one explicitly addressed depression in the elderly in the context of a new cancer diagnosis, and the other addressed somatic symptoms in vulnerable elders that overlap with presenting symptoms of cancer. QA Tools relied on AHCPR depression guidelines that emphasized cancer as an associated risk factor, although none of its indicators address this situation directly.

Measures and Indicators

Assessment. We identified one measure and one indicator relevant to cancer and depression assessment.

Measure - Regular Assessment for Psychosocial Well-Being. The UHC Palliative Care Benchmarking Project⁶⁷ included one measure relevant to depression assessment. The numerator is “persons with a formal psychosocial assessment (usually an assessment by a social worker) up to one year prior to admission during a previous admission OR within four days of the index admission.” The denominator is “adults ≥18 years of age, with admission for CHF (DRG 127), Cancer (DRG 82, 203, 172, 274, 346, 10) HIV (DRG 489) OR respiratory (DRG 475, 483) AND length of stay > 4 days AND 2 prior admissions for any cause in preceding 12 months.” This denominator was developed by an expert panel to represent a potential palliative care population in hospitals. Only a mean of 25% (median 17%, range 0–95%) of the UHC sample met the criteria for this indicator. This indicator identified four high performing facilities where more than 60% of patients had been assessed in the prior year. No reliability or validity information is available; however, the benchmarking project found that a combination of use of all of the UHC measures in a “palliative care bundle” was associated with lower patient length of stay. In addition, several elements of the bundle could be used to identify hospitals with “best practices” for palliative care. Scores from the bundle were instrumental in achieving advances in palliative care at several hospitals, including justifying at least one new palliative care program. Finally, results for the “bundle” varied significantly among the 35 hospitals participating in the project. It is intended for use as a tool for comparing performance in all hospitals. The measure is not known to be in current use.

Indicator - Regular Spiritual Assessment in Expected Dying. ACOVE identified one indicator relevant to depression assessment. The numerator is “vulnerable elders with a documentation of a spiritual assessment in the medical record.” The denominator is “vulnerable elders who were conscious during the last three days of life and who died an expected death.” Although eight items from the ACOVE end-of-life set were evaluated for reliability and validity⁷² as part of the ACOVE project, this indicator was not included in the evaluation. The denominator may be difficult to define, since “expected death” requires explicit medical record documentation more than one day before death that the patient is expected to die. This measure is intended for use in all vulnerable elders. An expert panel accepted a modified version of this indicator as part of a revised set for nursing homes. ⁶⁰ The nursing home indicator requires a denominator of “nursing home residents who died expected deaths and were conscious at all during the last seven days of life.” The numerator for the adapted measure requires that pain, spirituality, and emotional distress all have been addressed. Neither indicator is in current use.

Treatment. We identified two measures relevant to depression treatment among cancer patients.

Measure - Regular Assessment or Treatment of Depression in Newly Diagnosed Cancer. ACOVE identified two indicators relevant to both depression assessment and treatment.^{117, 60} For the first indicator, the numerator is “patients asked about or treated for depression, or referred to a mental health professional within two months of the diagnosis of a condition.” The denominator includes “all persons diagnosed with stroke, myocardial infarction, dementia, malignancy (excluding skin cancer), chronic pain, alcohol or substance abuse or dependence, anxiety disorder, or personality disorder.” These conditions were accepted for the indicator on the basis of their association with incident or prevalent depression. Although 13 indicators from the ACOVE depression set were evaluated for reliability and validity in a community population,⁷² this indicator was not included, although it was operationalized for cancer patients. No information on reliability and validity is available for this indicator. The indicator is not in use.

A second ACOVE measure (routine assessment or treatment of depression in symptomatic patients) relevant to depression assessment and treatment addressed both affective and somatic symptoms among vulnerable elders.^{117, 60} The numerator is “patients asked about or treated for depression, or referred to a mental health professional within two weeks of presentation.” The denominator is “vulnerable elders with new onset of one of the following symptoms-sad mood, feeling down, insomnia or difficulties with sleep, apathy or loss of interest in pleasurable activities, reports of memory loss, unexplained weight loss of more than 5% of body weight in the past month or 10% over one year, or unexplained fatigue or low energy.” This indicator was included in the 13 indicators from the ACOVE depression set, although ACOVE field evaluation excluded patients undergoing active cancer treatment. Only 26 of the 34 quality indicators triggered were passed.⁷² No information on reliability or validity is available. This indicator is not in use.

Follow-up. We identified zero measures and zero indicators relevant to the follow-up of depression treatment in cancer patients.

Potential Indicators and Measurement Gaps

Within the literature that we identified, only the ACOVE set provided measures and indicators for depression among a population that did not include cancer patients, per se or was not limited to cancer patients; thus we considered these as potential indicators. Quite a number of other measurement sets address depression care in general. ACOVE provides only a general example of that class of non-cancer depression indicators that might be considered. However, many of these indicators may not be appropriate for hospice or similar settings, since some of them address issues such as follow-up within months of depression diagnosis, and life expectancy in such settings may be very short. Nevertheless, within ACOVE, we identified the following additional potential indicators for cancer care.

Assessment. We identified three potential indicators relating to initial depression assessment.

Appropriate Diagnosis. Two ACOVE indicators are based on the assumption that depression diagnosis should document the presence or absence of major affective symptoms and suicidality. Both of these indicators use as a denominator population “all vulnerable elders with a new diagnosis of depression.” The first addresses the need to distinguish between major depression and other affective syndromes and uses as a numerator “patients where the medical record documents at least three of nine DSM-IV target symptoms within the first month of diagnosis.” The second indicator addresses the need to evaluate suicidality and uses as a numerator “patients where the medical record on day of diagnosis documents the presence or absence of suicidal ideation and psychosis.” These indicators are based on the findings that only patients with major depression are known to respond to medication¹¹⁷ and that evidence of psychosis and suicidality should be considered in the approach to treatment.¹¹⁷ The documentation of depressive symptoms may differ somewhat for cancer patients, especially late stage patients where fatigue and other manifestations of illness overlap with the “vegetative symptoms.”

Evaluating Suicidality. One ACOVE indicator addresses the need to evaluate and treat suicidality appropriately when it is present in a patient with newly diagnosed depression. The numerator for this indicator is “patients where the medical record documents that the person has no immediate plans for suicide or that the patient was referred for evaluation for psychiatric hospitalization.” Expert opinion supports this approach even though ethical considerations make studies of interventions difficult. Suicide is a disproportionate concern among the elderly in general, and cancer patients in particular.¹¹⁷

Treatment. We identified 2 potential indicators that were related to initial treatment.

Use of the Most Appropriate Anti-Depressants. Two ACOVE indicators address the appropriate use of medication in patients newly treated for depression. The denominator for both of these indicators is “vulnerable elders being treated for depression with antidepressants.” The numerator for the first potential indicator is “patients where the following medications are not used as first or second-line therapy: tertiary amine tricyclics, monoamine oxidase inhibitors (unless atypical depression is present), benzodiazepines, or stimulants (except methylphenidate).” This indicator is based on the fact that for the most part, antidepressants have equal efficacy, although side-effect profiles differ and are generally worse for older antidepressants highlighted by the indicator.¹¹⁷

Dose Titration. The denominator for this indicator is “vulnerable elders being treated for depression with antidepressants.” The numerator for the second potential indicator is “patients with antidepressants started at appropriate doses, and with an appropriate titration schedule to a therapeutic dose, therapeutic blood level, or remission of symptoms by 12 weeks.” Many studies demonstrate that older patients do not receive appropriate follow-up including therapeutic doses of antidepressant medications.¹¹⁷

Follow-up. We identified four potential indicators that were related to follow-up of initial treatment.

Addressing Non-Responders to Initial Treatment. The numerator for this potential indicator is “patients with antidepressant treatment, psychotherapy, or electroconvulsive therapy (ECT) offered within two weeks after depression diagnosis unless there is documentation within that period that the patient has improved, or unless the patient has substance abuse or dependence, in which case treatment may wait until eight weeks after the patient is in a drug or alcohol free state.” The denominator is “vulnerable elders with a new diagnosis of depression.” This indicator addresses the fact that psychotherapy may be the initial treatment attempted in depression, and that although the time-course of response is unclear, other treatments should be started within weeks of diagnosis if it is ineffective.¹¹⁷

Addressing Partial Responders to Treatment. Two ACOVE indicators addressed the appropriate care for patients who were only partial responders to initial pharmacotherapy at 6 and 12 weeks. The denominators for those potential indicators are “vulnerable elders who have only had a partial response to therapy (at each time period).” For the first indicator that addresses week-6 nonresponders, the numerator is “patients with one of the following treatment options initiated by the eighth week of care: medication dose should be optimized or the patient should be referred to a psychiatrist (if initial treatment was medication); or medication should be initiated or referral to a psychiatrist should be offered (if initial treatment was psychotherapy alone).” The second indicator addresses partial responses at 12 weeks. The numerator for this indicator is “patients with one of the following options instituted by the 16^th week of treatment: switch to a different medication class or add a second medication to the first (if initial treatment includes medication); add psychotherapy (if the initial treatment was medication); try medication (if initial treatment was psychotherapy without medication); consider ECT; or refer to a psychiatrist.”

Maintaining Effective Anti-Depressant Care. This potential indicator addresses the need for maintenance therapy. The denominator is “vulnerable elders who have responded successfully to antidepressant medication treatment.” The numerator is “patients continued on the drug at the same dose for at least six months, and making at least one clinician contact (office visit or phone) during that period.” This potential indicator acknowledges uncertainty in the exact amount of time that therapy should be continued in responders but upholds a lower limit of six months.¹¹⁷

Introduction

The review of literature in the area of advance care planning was not limited only to cancer. We identified a total of 30 indicators concerning the identification and documentation in the patient's medical record of advance directives, designation of a proxy decisionmaker, preferences for care in cognitively impaired elderly, referral to hospice, and utilization-based indicators of high intensity care. Please refer to the Evidence Table in Appendix F4 ^*. We included utilization indicators because much of the literature on advance care planning has focused on avoiding (e.g., resuscitation) or facilitating (e.g., hospice referral) particular treatment or utilization outcomes. However, we acknowledge that such indicators are limited as patient-centered metrics and may inadequately account for patient preferences. Furthermore, measures of particular types of utilization (e.g., hospitalization) may also reflect structural factors or the adequacy of alternative services. For that reason, other investigators might have made a reasonable decision to exclude such outcomes from consideration, although we included them for completeness. The indicators tended to overlap significantly and, where there was overlap, the primary differences were in care setting (e.g., ICU vs. nursing home, nursing home vs. hospital)⁶⁸ or populations (e.g., dementia vs. incapacitated patients).¹¹⁸ In addition, the literature on advance directive/advance care planning overlaps with the literature on good end-of-life/palliative care standards. Where appropriate, these overlapping indicators will be discussed together. Some items that did not clearly address “assessment,” “application,” or “follow-up” but crossed all domains are addressed separately. Finally, indicators may identify potential standards for hospital, system,⁶⁷ or regional^{55, 119, 120} performance.

Measures and Indicators

Assessment. We identified eight measures and four indicators relevant to advance care planning assessment.

Measure - Regular Identification of a Surrogate in Outpatient Setting. ACOVE's end-of-life indicators included one measure addressing the identification of a surrogate decisionmaker among outpatients.^{59,72 72} The numerator is “the number of vulnerable elderly patients with medical record documentation of their surrogate decisionmaker's name and contact information, or of a discussion with the patient of who would be surrogate, or documentation of a search for a surrogate, or an indication that there is no identified surrogate.” The denominator is “all vulnerable elderly outpatients.” No clinical trials or observational studies have examined this issue, yet documentation is an important tool for clinician access to an incapacitated patient's decisionmaker,^{21, 121} and many consensus statements promote designation of a surrogate decisionmaker (e.g., AGS Ethics Committee, 1995). One prospective study of physicians discussing advance directives with patients revealed that the physicians recognized the correct surrogate decisionmaker 89% of the time after such a discussion.¹²² Only 4% of 370 eligible charts in the 420-person ACOVE community-based sample passed this indicator.⁷² No reliability or validity information is available for the individual measure. The measure is not in use.

Measure - Regular Identification of a Surrogate Among Hospital Admissions With Impaired Cognitionⁱⁱ. One ACOVE measure addresses surrogacy among patients admitted to the hospital.^{59, 72} The numerator is “the number of vulnerable elders (among those with the denominator conditions) who have chart documentation within 48 hours of either an advance directive indicating the patient's surrogate decisionmaker, a discussion about who would be a surrogate decision maker or a discussion about a search for a surrogate, or an indication that there is no identified surrogate.” The denominator is “all vulnerable elders with dementia, coma, or altered mental status admitted to a hospital who survive 48 hours.” This indicator has implications for cancer patients who may have temporary (e.g., delirium) or durable cognitive impairment. Research has shown that the majority of patients admitted to the hospital have neither named a surrogate decision maker nor expressed their treatment preferences in a written document.¹²³ However, observational data suggest that physician understanding of patient resuscitation preference (a component of advance directives) is associated with a better match between those preferences and resuscitation attempts and is associated with less end-of-life resource use.¹²⁴–¹²⁶ Only 25% of 20 eligible charts in ACOVE's 420-person community-based sample passed this indicator.⁷² No reliability or validity data are available on this individual item. The measure is not in use.

Measure - Regular Assessment of Preferences Among Inpatients With Dementia. One ACOVE measure addresses documentation of preferences among inpatients with severe dementia.^{59, 72} The numerator is “the number of severely demented elders with documentation in their medical record that the patient's prior preferences for care have been considered or that these preferences could not be elicited or are unknown.” The denominator is “all vulnerable elderly patients with severe dementia admitted to the hospital who survive 48 hours.” This indicator has implications for cancer patients who may develop cognitive impairment associated with their cancer (e.g., delirium). Aggressive medical interventions and inadequate pain management are common in hospitalized dementia patients.¹²⁷ This measure could be evaluated among only 2 of 420 cases in ACOVE's community-based cohort.⁷² No reliability or validity information is available for the specific measure. The measure is not in use.

Measure - Regular Assessment of Preferences in an ICU. This ACOVE measure addresses documentation of preferences in an ICU.^{59, 72} The numerator for this indicator is “the number of vulnerable elders admitted directly to the intensive care unit from an outpatient or ER setting and surviving 48 hours with documentation in the medical record that the patient's preferences for care have been considered or that these preferences could not be elicited or are unknown within 48 hours of admission.” The denominator for this indicator is “all vulnerable elderly patients admitted directly to the ICU from the outpatient setting or emergency department and who survive 48 hours.” Only 6 of 420 patients in ACOVE's community sample were eligible for this measure, and 17% passed.⁷² No reliability or validity data are available for this specific measure. The measure is not in use.

Measure - Regular Assessment of Preferences in Hospice. NHPCO⁵⁴ measure addresses common decisions about activities of daily living, life closure, and care received by patients and family members facing a terminal illness. The numerator for this indicator is “the number of hospice patients who received care (either hospitalization or resuscitation) consistent with their expressed preferences as assessed at the time of admission.” The denominator is “all patients admitted to hospice.” Experts identified a number of global categories of decisionmaking, such as determination of the degree of symptom management desired, setting goals in relation to sentinel events, expressing or not expressing religious and spiritual needs, determination of one's wishes for the site of death, hospitalization, and CPR.⁶³ The intent of this indicator is to assist hospice patients in identifying care preferences and then meeting these needs by conducting a comprehensive assessment, including physical, functional, emotional, and spiritual and preferences, and allowing full autonomy to the patient and family to make decisions about how the remainder of the patient's life is to be spent upon admission. Hospices report these data voluntarily to NHPCO. In 2002, 120 agencies reported this measure, and the mean and median values were 97%. No reliability or validity data are available. This measure is in current use as a voluntary reporting standard.

Measure - Regular Patient Participation in Decisions To Limit Treatment. This ACOVE measure focuses on ensuring participation in decisions for the withdrawal or withholding of life-sustaining treatments (LST), which is consistent with ethical and professional guidelines.^{59, 72} The numerator is “the number of vulnerable elderly patients with documentation in the medical record of participation in the decision or why the patient did not participate in a decision regarding LST.” The denominator is “all vulnerable elderly patients in the hospital or nursing home with written orders to withdraw or withhold a LST.” A small study comparing chart documentation to patient recall about life-sustaining treatment decisions showed that medical record documentation reflected patient understanding of these decisions.¹²⁸ Ten of the 420 cases in ACOVE's community-based sample met criteria for the measure, and 70% passed.⁷² No reliability or validity information is available on this specific measure. The measure is not in use.

Measure - Regular Family Meetings Among Hospitalized Patients. The numerator for this UHC measure is “the number of adults admitted to the hospital with documentation in the medical record that the health care team conducted a patient/family meeting within one week of admission that included discussion of the patient's treatment preferences and/or a plan for discharge disposition.”⁶⁷ The denominator is adults ≥18 years of age, with admission for CHF (DRG 127), cancer (DRG 82, 203, 172, 274, 346, 10) HIV (DRG 489) OR respiratory (DRG 475, 483) AND length of stay > 4 days AND 2 prior admissions for any cause in preceding 12 months." This indicator is similar to the ACOVE measures^{59, 72} that specify documentation of patient participation in decisionmaking (implying communication of some sort between the patient and provider team) and the VHA indicators⁶⁸ that specify conducting an interdisciplinary team meeting with the family within 120 hours of ICU admission. A mean of 39% (median 41%, range 0–93%) of the UHC cohort met this indicator. Only 55% of cancer patients met the indicator. The distribution of results appeared to be broadly normal. As with the other UHC measures, no individual reliability or validity information is available. It is not known to be in current use.

Measure - Timely and Effective Discharge Planning. The numerator for this UHC measure is the number of adults admitted to the hospital with documentation in the medical record of a plan for discharge disposition within four days of admission."⁶⁷ The denominator is “adults ≥18 years of age, with admission for CHF (DRG 127), cancer (DRG 82, 203, 172, 274, 346, 10) HIV (DRG 489) OR respiratory (DRG 475, 483) AND length of stay > 4 days AND 2 prior admissions for any cause in preceding 12 months.” This measure implies that all patients who enter the hospital and stay at least one week should receive a plan for discharge disposition within four days of admission. Planning for discharge is usually not explicitly included in discussions of advance directives, although many critically ill patients frequently move in and out of the hospital and across a variety of health care settings as they progress on a downward trajectory toward death. This explicit recognition of the importance of planning for care after leaving the hospital is a contribution to the advance directives discussion. A mean of 53% (median 53%, range 18–94%) of the UHC cohort met this indicator. Only 62% of cancer patients met the indicator. The distribution of results appeared to be broadly normal. As with the other UHC measures, no individual reliability or validity information is available. It is not known to be in current use.

Indicator - Regular Identification of a Surrogate in the ICU. This VHA indicator focuses on the identification of a medical decisionmaker (family member or other appropriate surrogate) and is similar to the ACOVE indicator^{68, 72, 129} identifying a surrogate/proxy for community-dwelling or hospitalized vulnerable elders. Other than setting, the major difference with the ACOVE indicators is the timeframe identified for performance of the indicator (24 vs. 48 hours): The denominator is “the total number of ICU patients with a stay of ≥ 24 hours.” The numerator is “the number of patients who have documentation on the status of identification of a health care proxy (or other appropriate surrogate decisionmaker) within 24 hours.” Support for this indicator is derived from JCAHO Hospital Standard RI.1.2.3, literature focusing on the withdrawal of life-support in the ICU,¹³⁰ and practice guidelines.¹⁰⁰ This measure is currently being used as part of the VHA TICU initiative and is part of the VHA palliative care bundle, which will be initiated in September 2005.

Indicator - Regular Assessment of Advance Directives for ICU Patients. This VHA indicator focuses on the specification and documentation of the patient's advance directive if admitted to the ICU within 24 hours of admission.⁶⁸ The numerator for this indicator is “the number of patients admitted to the ICU with documentation of the patient's advance directive (written or oral instructions from the patient specifying the type of medical treatment that is desired if the patient becomes incapacitated, including a living will, durable power of attorney (note State-specific status) or any document that State law recognizes as an “advance directive” entered in the medical record within 24 hours following admission.” The denominator is “the total number of patients with an ICU length of stay greater than 24 hours.” Support for this indicator is derived from JCAHO Hospital Standard RI.1.2.5; the National Consensus Project for Quality Palliative Care Clinical Practice Guideline 1.2, 8.1^{69, 131} and literature on decision making and medical outcomes.^{132, 133} This measure is currently being used as part of the VHA TICU initiative and is part of the VHA palliative care bundle, which will be initiated in September 2005.

Indicator - Regular Assessment of Specific Resuscitation Preferences in the ICU. This VHA indicator focuses on the percent of patients with documentation in the medical record of resuscitation status within 24 hours following admission to the ICU.⁶⁸ The numerator for this indicator is “the number of patients admitted to the ICU with documentation of the patient's advance directive (written or oral instructions from the patient specifying the type of medical treatment that is desired if the patient becomes incapacitated, including a living will, durable power of attorney (note State-specific status) or any document that State law recognizes as an ‘advance directive’) entered in the medical record within 24 hours following admission.” The denominator is “the total number of patients with an ICU length of stay greater than 24 hours.” Support for this indicator is derived from the literature on decisionmaking and medical outcomes.¹³² This measure is currently being used as part of the VHA TICU initiative and is part of the VHA palliative care bundle, which will be initiated in September 2005.

Indicator - Regular Clinician-Patient-Family Communication in the ICU. This VHA indicator addresses inadequate communication among patients, family members, and health care providers in ICUs.^{68, 134} The numerator for this indicator is “the number of patients admitted to the ICU who survive more than 5 days (120 hours) with documentation in the medical record that an interdisciplinary team meeting (with at least the attending physician (primary care or ICU attending) and a nurse) and the patient and/or family was conducted within 72 hours of ICU admission and involved a discussion addressing each of the following topics: the patient's condition (diagnosis and prognosis), goals of treatment, the patient's and family's needs and preferences, and the patient's and family's understanding of the patient's condition and goals of treatment at the conclusion of the meeting.” The denominator is “total number of patients with an ICU length of stay more than 5 days (120 hours).” Support for this indicator is derived from JCAHO Standards RI.1.2.2.-RI.1.2.8, RI.1.3.6; NCP_CPG 1.3.^{135, 136} A joint meeting with the surrogate and family members and the health care team, labeled a “family conference,” should be used to answer any questions family members might have regarding the condition of the patients and to explore goals of care. The literature identifies various elements of good communication as well as barriers to advance directive discussions.¹³⁷–¹⁴¹ This measure is currently being used as part of the VHA TICU initiative and is part of the VHA palliative care bundle, which will be initiated in September 2005.

Application. We identified two measures and zero indicators addressing the application of existing care plans.

Measure - Documentation of Care Preferences Across Venues. Several ACOVE measures address advance directive continuity.^{59, 72} The goal of advance care planning is to ensure that patient's preferences for care are identified and followed. The denominator is “all vulnerable elderly with an advance directive in the outpatient, inpatient, or nursing home medical record or who report the existence of an advance directive in an interview, and are receiving care in another venue.” The numerator is “those cases with documentation in their medical record of the existing advance directive at the second venue or documentation acknowledging the existing advance directive, it contents, and the reason that it is not included in the medical record.” Five observational studies have documented that even when advance directives are completed, patients' physicians often do not know that they exist,^{126, 142, 143} and the directives are not always documented in the patients' charts¹⁴⁴ or transferred from outpatient to inpatient settings.¹⁴⁵ This measure passed in 25% of the 8 cases among 420 community-dwelling elderly in the ACOVE study.⁷² No reliability or validity information is available on the specific measure. The measure is not in use.

Measure - Documentation of Specific Life Sustaining Preferences. This ACOVE measure addresses the need to document decisions a vulnerable elder makes concerning future health states.^{59, 72} The numerator is “patients with one of the following documented in the medical record: a discussion of life-sustaining treatment preferences, an advance directive, or that the patient discussed this topic with the physician or does not wish to discuss this topic.” The denominator is “all vulnerable elderly patients who indicate (during an interview) that they would rather die than live permanently comatose, ventilated, or tube fed.” Observational studies show that large proportions of patients would prefer to die than live permanently comatose, mechanically ventilated, or tube fed and that physicians and surrogate decision makers often do not know patients' preferences about life-sustaining treatments.⁵⁹ Consensus statements promote the documentation of these preferences (e.g., The American Geriatric Society Ethics Committee, 1995). Of the 238 of 420 eligible patients in ACOVE's community-based sample, only 12% passed the measure.⁷² No reliability or validity information is available on this specific measure. The measure is not in use.

Follow-up. We identified 14 separate measures (covering 10 general categories) and 7 indicators (covering 8 general categories) addressing follow-up. Many of these administrative measures and indicators are closely related. For example, Cancer Care Ontario¹¹⁹ and CCNS¹⁴⁶ are both using indicators and measures very similar to those developed by investigators at the Dana-Farber Cancer Institute.¹⁴⁶ For that reason, these similar measures and indicators are discussed together whenever possible.

Measure - Consistency of Preferences With Use of Ventilator Support. This ACOVE measure focuses on consistency between use of ventilator support and documented preferences.^{59, 72} The numerator is “the number of records with documentation of the goals of care and the patient's decision for mechanical ventilation or why this information is unavailable.” The denominator is “all hospitalized vulnerable elders requiring mechanical ventilation (except short-term and post operative mechanical ventilation).” This indicator is intended for use with all vulnerable elderly patients requiring mechanical ventilation, including cancer patients. Practices concerning withdrawal of treatment for patients receiving mechanical ventilation vary widely.^{123, 135, 143, 147, 148} Practices regarding withholding of specific treatments among surveyed physicians also varied greatly.¹⁰⁰ The SUPPORT study found that a primary predictor of whether a ventilator was withdrawn was the existence of communication about care preferences.¹⁴⁹ Only 2 of 420 persons in ACOVE's community-based study met criteria for the measure. No reliability or validity information is available on this specific measure. The measure is not in use.

Measure - Late Life Hospital Use. ^{77, 150} Several measures or indicators address late-life hospital use. One measure was developed by Dana-Farber Cancer Institute¹⁴⁶ investigators, and a closely related measure is in use by Cancer Care Ontario.¹¹⁹ A similar indicator is in the process of being operationalized by CCNS.¹⁴⁶ The numerator for the Dana-Farber measure is "the proportion of cancer decedents with more than one hospitalization or ER visit in the last month of life. The denominator is “all cancer decedents.” Support for its validity comes from literature reviews; focus groups of patients, caregivers, and providers; expert panel review; and quantitative analysis of the indicator in Medicare claims data. This measure was evaluated in a Medicare-SEER database, and using the highest performing decile of this measure implied that in high performing health care service areas (HCSAs), fewer than 4% of cancer decedents would have more than one hospitalization or emergency room visit in the last month of life. When a sample of 150 charts at Dana-Farber were abstracted as the gold standard against which the sensitivity and specificity of the measure would be determined, this measure was determined to have a sensitivity of 0.96, specificity of 1.00, and accuracy of 0.097. As a measure of variability, investigators determined the 5^th/95^th percentile ratio (2.38) (range: 1.85–3.16). The Cancer Care Ontario measure is “hospital use in days by stage of cancer.” The populations for the “stage of cancer” consist of those in initial care (within six months of diagnosis), continuing care, and terminal care (within six months of death). A separate category, investigative care, is defined as patients with a benign outcome. Provincial resource use in 2002 indicated that as a proportion of total cost, the terminal phase of care used 29 to 42% of resources across Ontario. The initial phase of care used 31 to 38% of total resources.¹²⁰ A closely related indicator, hospital days near the end of life, is being operationalized by CCNS.

Measure - Late Life ICU Use. ^{77, 146, 150} Several measures or indicators address late-life ICU use. One measure was developed by Dana-Farber Cancer Institute. A similar indicator is currently being operationalized by CCNS. The numerator for the Dana-Farber measure is “the proportion of cancer decedents with admission to the ICU in the last month of life.” The denominator is “all cancer decedents.” Support for its validity comes from literature reviews; focus groups of patients, caregivers, and providers; expert panel review; and quantitative analysis of the indicator in Medicare claims data. This measure was evaluated in a Medicare-SEER database and, using the highest performing decile of this measure, the data implied that in high performing HCSAs fewer than 4% of cancer decedents would have more than one ICU admission in the last month of life. When a sample of 150 charts at Dana-Farber were abstracted as the gold standard against which the sensitivity and specificity of the measure was determine, this measure was determined to have a sensitivity of 0.87, specificity of 0.97, and accuracy of 0.095. As a measure of variability, investigators determined the 5^th/95^th percentile ratio (3.28) (range: 2.38–4.67). A closely related indicator, ICU days near the end of life is being operationalized by CCNS. Neither the measure nor the indicator are currently in use.

Measure - Late Life Rate of Emergency Care. This measure was developed by Dana-Farber Cancer Institute^{77, 146, 150} investigators, and a closely related measure is in use by Cancer Care Ontario.¹¹⁹ A similar indicator is currently being operationalized by CCNS.¹⁴⁶ The numerator for the Dana-Farber measure is “the proportion of cancer decedents with more than one emergency room visit in the last month of life.” The denominator is “all cancer decedents.” Support for its validity comes from literature reviews; focus groups of patients, caregivers, and providers; expert panel review; and quantitative analysis of the indicator in Medicare claims data. This measure was evaluated in a Medicare-SEER database, and, using the highest performing decile of this measure, the data suggested that in high performing HCSAs, fewer than 4% of cancer decedents would have more than one emergency room visit in the last month of life. When a sample of 150 charts at Dana-Farber were abstracted as the gold standard against which the sensitivity and specificity of the measure was determined, this measure was determined to have a sensitivity of 0.82, specificity of 0.96, and accuracy of 0.89. As a measure of variability, investigators determined the 5^th/95^th percentile ratio (2.78) (range: 2.04–3.88). The Cancer Care Ontario measure is “the rate of cancer patients utilizing an emergency room in the last two weeks of life.” The denominator is “all cancer patients dying in a hospital.” Provincial rates of this measure varied between about 20% and 25% in 2002. A closely related indicator, the frequency of emergency visits, is being operationalized by CCNS.

Measure - New Chemotherapy Regimen in Last 30 Days of Life. ^{77, 150} This measure was developed at the Dana-Farber Cancer Institute¹⁴⁶ and a closely related indicator is being operationalized in ongoing work by CCNS.⁷⁷ The numerator is “the number of dying cancer patients started on a new chemotherapy regimen in the last 30 days of life.” The denominator is “all deceased cancer patients.” A high proportion (more than 2%) of cancer patients receiving a new chemotherapy regimen in the last 30 days of life indicates poor quality care.¹²⁰ Support comes from literature reviews; focus groups of patients, caregivers, and providers; expert panel review; and quantitative analysis of the indicator in Medicare claims data. This measure was normed in a Medicare-SEER database and, using the highest performing decile of this measure, the data implied that in high performing HCSAs, fewer than 2% of patient would initiate new chemotherapy regimens in the last month of life. A sample of 150 charts at Dana-Farber were abstracted as the gold standard against which the sensitivity and specificity of the measure was determined. This measure was determined to have a sensitivity of 0.83, specificity of 0.94, and accuracy of 0.85. As a measure of variability, investigators determined the 5^th/95^th percentile ratio (3.19) (range: 2.03–5.41). CCNS is operationalizing a closely related indicator - the interval between new chemotherapy and death. This measure is not in use.

Measure - Chemotherapy in Last 14 Days of Life. ^{77, 150} This measure is derived from the Dana-Farber Cancer Institute¹⁴⁶ measure set and a closely related indicator is being operationalized in ongoing work by CCNS.^{77, 120} The numerator is “the number of dying cancer patients receiving chemotherapy in the last 14 days of life.” The denominator is “all deceased cancer patients.” A high proportion (more than 10 %) of cancer patients receiving chemotherapy in the last 14 days of life indicates poor quality care. Support for this indicator comes from literature reviews; focus groups of patients, caregivers, and providers; expert panel review; and quantitative analysis of the indicator in Medicare claims data. This indicator was normed in a Medicare-SEER database and, using the highest performing decile of this measure, the data suggest that in high performing HCSAs, fewer than 10% of patients would receive chemotherapy in the last 14 days of life. When a sample of 150 charts at Dana-Farber were abstracted as the gold standard against which the sensitivity and specificity of the measure was determined, this measure was determined to have a sensitivity of 0.92, specificity of 0.94, and accuracy of 0.92. As a measure of variability, investigators determined the 5^th/95^th percentile ratio (2.24) (range: 1.74–2.97). CCNS is evaluating a closely related indicator, using a “short” interval between the last chemotherapy and death. This measure is not in use.

Measure - Admission to Hospice. ¹⁵⁰ This measure was developed by investigators at the Dana-Farber Cancer Institute¹⁴⁶ and is currently being operationalized as an indicator by CCNS^{77, 120} and Cancer Care Ontario.⁷⁷ A closely related measure was also selected for use by the Georgia Cancer Coalition.⁵⁵ The numerator is “the number of cancer patients not admitted to hospice.” The denominator is “all deceased cancer patients.” Numerous observational studies as well as several recent methodologically rigorous systematic reviews qualitatively support the effectiveness of hospice and palliative care in addressing the full range of quality-of-care domains.^{4, 21} Support for the validity of this item comes from literature reviews; focus groups of patients, caregivers, and providers; expert panel review; and quantitative analysis of the indicator in Medicare claims data. This measure was normed in a Medicare-SEER database and, using the highest performing decile of this measure the data suggested that in high performing HCSAs, fewer than 45% of patients would die without being admitted to hospice. When a sample of 150 charts at Dana-Farber were abstracted as the gold standard against which the sensitivity and specificity of the measure was determined, this measure was determined to have a sensitivity of 0.24, specificity of 0.96, and accuracy of 0.88. As a measure of variability, investigators determined the 5^th/95^th percentile ratio (5.00) (range: 3.76–6.89). Several closely related indicators are being evaluated and operationalized by CCNS, including enrollment in hospice, access to palliative care or palliative physician assessment, and periodic palliative care. The Georgia Cancer Coalition has operationalized this item as “the rate of cancer deaths in hospice.” Using a SEER-Medicare dataset, the proposed numerator is “the number of adults with cancer discharged due to death” and the denominator is “the number of adults with cancer (ICD-10 codes C00–C97, ICD-9 140–208).”⁵⁵ No reliability or validity information is available other than from the Dana-Farber experience. This item is not known to be in current use as a measure.

Measure - Late Referral to Hospice. ^{77 150} This measure was developed by investigators at the Dana-Farber Cancer¹⁴⁶ and is very similar to a measure developed by CCNS¹⁴⁶ and the Georgia Cancer Coalition.⁵⁵ The numerator is “the number of dying cancer patients referred to hospice or palliative care less than three days before death.” The denominator is “all deceased cancer patients.” A high proportion (more than 8 %) of dying cancer patient being referred to hospice or palliative care less than three days before death indicates poor quality care.¹⁴⁶ Support for the validity of this measure comes from literature reviews; focus groups of patients, caregivers, and providers; expert panel review; and quantitative analysis of the indicator in Medicare claims data. This measure was normed in a Medicare-SEER database, and using the highest performing decile of this measure, the data implied that in high performing HCSAs, fewer than 8% of dying patients would be admitted to hospice in the last three days of life. When a sample of 150 charts at Dana-Farber were abstracted as the gold standard against which the sensitivity and specificity of the measure was determined, this measure was determined to have a sensitivity of 0.97, specificity of 1.00, and accuracy of 0.97. As a measure of variability, investigators determined the 5^th/95^th percentile ratio (2.39) (range: 1.99–2.95). The Georgia Cancer Coalition has operationalized a closely related measure as cancer patients who receive hospice care for at least seven days. They propose to report this measure separately for inpatient and outpatient settings and to operationalize it using SEER Medicare data; the denominator is “the number of adults with cancer (ICD-10 codes C00–C97, ICD-9 140–208).”⁵⁵ With the exception of the data from the Dana-Farber experience, no reliability or validity information is available. This item is not known to be in current use as a measure.

Measure - Site of Death. ¹⁵⁰ This measure was developed by investigators at the Dana-Farber Cancer Institute¹⁴⁶ and is also being used as an indicator by CCNS⁷⁷ and as a measure by Cancer Care Ontario.^{119, 120} In the case of the Dana-Farber measure, the numerator is “number of cancer patients who die in the hospital.” The denominator is “all deceased cancer patients.” This item focuses on high rates of hospital cancer deaths (greater than 17%)¹⁵⁰ as indicating poor quality care. Support for this item comes from literature reviews; focus groups of patients, caregivers, and providers; expert panel review; and quantitative analysis of the indicator in Medicare claims data. The British Gold Standards Framework of dying has embraced a similar metric. This measure was normed in a Medicare-SEER database, and using the highest performing decile of this measure, the data implied that high performing HCSAs would have fewer than 17% inpatient deaths among cancer patients. When a sample of 150 charts at Dana-Farber were abstracted as the gold standard against which the sensitivity and specificity of the measure was determined, this measure was determined to have a sensitivity of 0.95, specificity of 1.00, and accuracy of 0.97. As a measure of variability, investigators determined the 5^th/95^th percentile ratio (2.49) (range: 2.05–3.12). Cancer Care Ontario is currently using this measure to evaluate provincial cancer care. About 56% of patients died in the hospital in Ontario, and rates varied between approximately 45% and 60% province-wide. Patients who died in the hospital were less likely to receive home care in the last six months, palliative care assessments, or physician house calls in the last two weeks of life.¹¹⁹ The same item is currently being operationalized as a set of comprehensive administrative data-based indicators for CCNS.

Measure - Safe Dying in Hospice. This measure was developed by investigators at the NHPCO.⁶³ The numerator is "the number of deceased hospice patient's caregivers with documentation in the patient's medical record who were contacted and surveyed as to the quality of care provided by the hospice using the following question: “If you cared for the patient at home, did hospice increase your confidence to safely care for your loved one as death approached?” The denominator is “all caregivers of hospice patients.” This indicator is based on the assumption that caregivers who lack confidence or unsafe environments will aggravate dying or may hasten death. The “high risk, high volume, problem prone” home care environments are those in which there may be variable, uncertain, or absent caregiver competencies. Family caregivers, who may be uneducated in health care delivery, stressed by the anticipation of a loved one's anticipated death, and fatigued by the labor associated with care of a physically dependent person, are given significant responsibility for the care of the patient. For the 116 agencies that voluntarily reported this information in 2002, the mean and median values were 98% for caregivers reporting safe dying experience.^{54, 79}

Indicator - Care Consistency With Documented Care Preferences. The numerator for this ACOVE measure is "the number of vulnerable elders whose specific treatment preferences were followed. The denominator is “all vulnerable elderly patients in any health care setting with specific treatment preferences documented in their medical record.”⁵⁹ The assumption is that the patient's preferences and designated surrogate will be elucidated and documented within the patient's medical record, usually within a specified timeframe that depends on the site within the hospital, or there will be documentation as to why surrogate and preferences were not able to be obtained. Thus, the documentation in patients' records of their preferences would provide the standard by which care consistency would be evaluated. Both patients and health care providers believe that end-of-life discussions are important and should be held while the patient is still able to meaningfully and actively participate in decision making. However, studies have shown that these conversations are not common.¹⁴¹ Moreover, research assessing the effectiveness of written advance directives in the care of seriously ill, hospitalized patients shows that even if the patient has completed an advance directive, and it is accessible in the patient's chart, physician-patient communication or decision making about resuscitation was not substantially enhanced. Current practice patterns indicate that increasing the frequency of advance directives is unlikely to be a substantial element in improving the care of seriously ill patients.¹⁵¹ Recent efforts to improve end-of-life care have focused on the application of documented preferences, as there is a strong theoretical basis and expert consensus that patient preferences should drive care. Danis and colleagues (1991)¹²⁴ showed a high correlation between documented care preferences and life-sustaining treatments. However, similar work has not been performed in the outpatient or hospital settings. No trials have studied whether documentation improves the concordance of care with values and preferences. This indicator was not tested in ACOVE's community-based sample and is not in current use.

Potential Indicators and Measurement Gaps

We reviewed a number of sets of proposed potential indicators, clinical practice guidelines, and quality improvement studies that did not meet the standards set for inclusion of relevant indicators. Overall, guidelines did not rise to the level of indicators because of the lack of specification of various elements required for this review: patient population, denominator, conceptual clarity, or data source information. We identified fourteen potential indicators. These potential indicators overlapped with those identified in the indicator table and also addressed a number of gaps in the indicator literature, including the special population of children with cancer.¹⁵²–¹⁵⁵

Although many of the issues in advance care planning in adults are similar to those in children, indicators for children may need to differ in some respects (Potential Indicators #1; #7, #8, #10, #11, #12). Potential indicators identified for children are derived from the Initiative for Pediatric Palliative Care (IPPC) Quality Domains of Children Living with Life-Threatening Conditions¹⁵² (Potential Indicator #1, #11, #12, #13), the Society of Industrial and Organization Psychology (SIOP) Working Committee on Psychosocial Issues in Pediatric Oncology¹⁵⁴ (Potential Indicator #7), the American Academy of Pediatrics Committee on Bioethics and Committee on Hospital Care¹⁵⁵ (Potential Indicator #8), and a literature review on key components of quality pediatric end-of-life care¹⁵³ (Potential Indicator #11). These potential indicators overlap with those identified for adults but address children's needs in light of their developmental level, and include assessment of needs, documentation of advance directives, support of the child and family, and continuity-of-care plans across provider sites.

Another potential indicator focused on adults in the ICU¹³⁵ (Potential Indicator #2). This indicator was derived from the American College of Physicians intervention literature and specified the components of the clinical identification of end-of-life ICU patients and the components of end-of-life care planning.

A number of potential indicators were identified from palliative care guidelines (Indicator #3, #4, #5, #14)^{69, 131, 156} and focused on end-of-life palliative care for hospitalized patients. These indicators were derived from the National Consensus Project for Quality Palliative Care Clinical Practice Guidelines^{69, 131} and the Palliative Care Australia Standards for Palliative Care Provision.¹⁵⁶ Continuity of patient information across settings was addressed in Potential Indicator #6¹⁴⁸ and was derived from the intervention literature.

The use and withdrawal of feeding tubes in institutionalized patients (hospital, nursing home) is addressed in Potential Indicator #9.¹⁵⁷

A final potential indicator was identified from the National Consensus Project^{69, 131} (Indicator #14) and focused on the treatment of the body in a culturally sensitive and respectful manner after a patient has died.

Cancer Care Ontario¹¹⁹ is using several measures related to waiting times for chemotherapy and radiotherapy that could be adapted for supportive or palliative care with appropriate denominator adjustments.

Measure That Crosses All Domains

Follow-up. We identified one composite measure that crosses all domains.

Measure - Family Evaluation of Hospice Care (FEHC). The FEHC is a 61-item survey administered after the death of a hospice patient to assess family members' perceptions of care provided by the hospice. The survey is based on the After-Death Bereaved Family Interview, which was developed based on a systematic review of the measurement literature relative to end-of-life care, consensus conferences, and focus groups.¹⁵⁸ Please refer to the Evidence Table in Appendix F5 ^*. The numerator is “the percentage of respondents reporting opportunity for improvement in attention to family needs for support, attending to family needs for information, provision of desired physical comfort and emotional support, and mean overall satisfaction.” The denominator is “family members of hospice decedents.” Although there are individual questions that relate to pain, depression, dyspnea, advance care planning, and other domains, and might be used for independent performance measurement, the FEHC is intended to be administered as a complete instrument or as modules and is therefore described here as a composite measure. The measure is intended for a mixed-hospice population. Hospices submit data voluntarily to NHPCO. In the first half of 2004, 352 hospices submitted data on 29,292 patients: 51% had cancer, 93% were white, 3% were African-American, and 3% were Hispanic.¹⁵⁸ For pain, 6% of family members reported unmet need (25th and 75th percentiles among hospices with more than 30 observations, 4% and 8%); for dyspnea, 5% (3% and 7%); and for emotional support, 9% (5% and 13%). Some questions related to advance care planning as well, such as satisfaction questions about respecting patient wishes and dying on one's own terms, but these values are reported only as a composite with other questions. Validity and reliability information are not available for this measure, although it is derived from other measures that have undergone extensive psychometric testing. Finally, little variation was found among hospices on this survey. However, these results are from early adopters; thus, it is possible that more variation would have been found if the survey had been administered among all hospices. It continues to be in voluntary use by hospices and NHPCO.

Field Testing in Relevant Populations

The lack of reported experience with these tools may not be surprising, as the measures or indicators we identified were largely developed in the last five years, reflecting the newness of the field of supportive cancer care.²¹ In addition, the science of quality improvement in particular has suffered from confusion about optimal methodological approaches.¹⁶¹ It is encouraging that some of the measures or indicators that we identified have been recently implemented or will be implemented in the near future for monitoring and improving population care (e.g., in Georgia, Ontario, Nova Scotia, and in the VHA system), which should offer opportunities for nurturing a fundamental knowledge base to foster improving the delivery of supportive or palliative cancer care. An especially important endeavor to foster progress in measurement would be to apply the better- developed general measure and indicator sets (e.g., ACOVE) to supportive or palliative care populations with cancer.

An important objective of future testing is to obtain more robust information about the performance characteristics of the strongest measures. Even given limitations, many of them may be useful now for institutional-level quality assessment and improvement activities. However, most of these measures lack even basic published information about reliability and validity. Before they are deployed at higher levels of the health care system, information about their ability to discriminate quality performance and responsiveness to change will be required. In addition, testing in a variety of settings and institutions may also be needed. Differences in instrumentation, data source, and data collection protocols may significantly affect the performance and feasibility of measures, and research on these methodological issues and how they affect palliative care evaluation is needed. Furthermore, health care systems must consider the cost and feasibility of data collection and must be assured that measures are low risk for perverse incentives and that they can provide timely feedback on performance so that measurement can contribute to operational decisionmaking.

The Need for Measures To Address Impaired Self-Report

None of the indicators address how to evaluate symptoms among individuals with impaired self-report, which can be due to either temporary (e.g., delirium) or durable factors (e.g., dementia, brain metastasis). There is no doubt that delirium is common among cancer patients, although its incidence and prevalence are not well studied. Some recent reports noted a prevalence of 18–50% among patients with cancer in general, ¹⁶²–¹⁶⁴ and observed rates are even higher in certain treatment settings such as the intensive care unit or in the nursing home, where almost 60% of the population is also living with dementia. The clinical needs of these populations are undoubtedly profound; a recent description of the clinical profile of cancer patients in US nursing homes reveals that, of the 190,976 cases, 51% of cases surviving more than one assessment had persistent pain, 25% of all cases were using oxygen, only 45% had DNR orders, and only 29% of those who were defined as “terminally ill” received hospice services.¹⁶⁵ A major effort to expand basic research in this area and ultimately address it through quality measurement would benefit many people at the end of life.^{166, 167}

Considering the example of pain, health professionals' estimates of patient's pain correlate poorly with patient self-reports, and discrepancies between patients and physicians in perceptions of pain severity are predictive of inadequate management.⁸ Without regular screening for pain, many patients with significant pain do not have pain documented in the medical record and do not receive analgesia.⁸⁵ Patient vulnerability makes data collection challenging even among patients capable of self-report, yet many patients who need palliative care are incapable of self-report at least part of the time, often at times when they are also at great risk of pain. Research that addresses methodological challenges of obtaining patient reports and methods that allow monitoring of pain without self-report are an urgent need. All of the pain quality improvement interventions identified in this review that succeeded in reducing population pain intensity in different settings also included increasing the frequency of pain assessment as a key component of the intervention.^{62, 159, 160}

The Lack of Pediatric Measures

A related and obvious shortcoming we noted is the complete absence of pediatric measures or indicators. As in the case of many adults, care assessment in young children is challenged by their inability to self report. Cancer represents the second leading cause of death among children ages 5–15 in the United States.¹⁶⁸ However, profound challenges and serious deficiencies in the care of pediatric cancer cases have been noted.^{169, 170} It is insufficient to propose that adult measures or indicators could simply be adopted for pediatric care, as the kinds of disease, challenges of symptom reporting and intervention, and basic approach to care differ so significantly between pediatric and adult oncology care. Many but not all of the adult measures have potentially useful analogues in the pediatric population,¹⁷¹ but substantial basic research will be needed to translate such efforts into useful pediatric tools. A comprehensive review in 2003 found only 22 research studies and 6 guideline statements in the area,¹⁵³ and most of this limited literature addresses pain, to the exclusion of other symptoms or experiences.¹⁵² In addition, a systematic review of measure and indicator sets relevant to pediatrics found no measures or indicators for end-of-life care.^{34, 172}–¹⁷⁵

Defining the “End-of-Life”

The lack of consensus on definitions of “end-of-life” presents what has been called the “denominator” problem. Of central importance is the continuing challenge of establishing a consistent and accepted definition of end-of-life that is not confined to that of the “imminently” dying. The correct definition of the end-of-life may well depend upon what use is to be made of the definition. For the measures reviewed, many of the indicators explicitly define end-of-life retrospectively (e.g., last six months of life). While appropriate for evaluating “aggressive treatment” in the last days before death, this definition does not provide guidance on important clinical services such as pain prevention or the timing of advance care planning. In addition, retrospective analyses may introduce a number of biases, since the timing of death is often not predictable. Prospective indicators, such as the quality of the discussion about whether the patient wants chemotherapy, would account for patient preferences. The ACOVE indicators come closer to a prospective method by identifying “vulnerable community-dwelling elders” for which the identification and documentation of life-sustaining treatment decisions may be most appropriate. Prospective analyses for cancer will require methodological work to define which cancer patients (certain types that are usually fatal, those at advanced stage) should be in the denominator for measures of the aggressiveness of care.

The Publication Gap for Quality Measurement

An important finding of our report was that much of the evidence on the measures that we identified was unpublished. Additional information that might inform the evidence base for these measures, (i.e., from local quality assessment and improvement efforts) probably exists but is likely unpublished and compiled by non-academic organizations. Further, much of the actual experience with measures and measurement probably exists in an anecdotal rather than archival form. We found that a strategy that consisted of an Internet search coupled with networking widely with organizations and individuals in the fields of oncology and supportive or palliative care was relatively effective for identifying such information. Future systematic reviews should be aware of this important issue and make an explicit effort to capture this unpublished information. Methods for systematically compiling and analyzing this type of data might help to improve the evidence base for quality measurement. Methods or other strategies to better promote the use of candidate measures in quality improvement work or to otherwise link measurement development work to quality improvement projects in cancer, such as the Veterans Administration Collaborative,¹⁶⁰ might help to provide the information on measure responsiveness that is needed.

Strengthening Quality Measurement of Depression in Cancer

Given the strength of the literature on measurement in mental health in general and the recognized importance of depression in the care of cancer patients, surprisingly few indicators have been evaluated for depression in advanced cancer. This lack of indicators is striking because indicators for mental health are well represented within available measure sets in general. Indicators developed for other settings may be appropriate but have not been tested in this population. Furthermore, depression is widely recognized to be an important influence on the experience of cancer. Certainly, many of the drugs used to treat depression are relevant within the scope of cancer care, especially for earlier stage disease. More fundamental clinical research is needed to address late stage disease - such as the optimal approach to screening and the usefulness of short-term treatment, which has not, for the most part, been addressed by this literature.²¹ Further research on similarities between measurements in cancer populations and those in non-cancer populations might allow elements of this work to be used as supporting evidence for particular measures. This effort may be particularly important for areas where there is significant non-cancer-related evidence, such as in patients with cognitive impairment.

Building an Evidence Base To Compare Population Subgroups

We found insufficient evidence to comment on important differences by population. This finding is not surprising, as the tools for examining these differences are limited by the lack of basic knowledge.²¹ However, an important priority for future research is evaluating measures in important sub-populations (i.e., by race/ethnicity, age, language, and gender). With regard to pain, members of minority groups with cancer have higher rates of inadequate pain assessment and treatment than others.^{86, 87} A recent nonsystematic review of the topic of cancer pain found that differences in pain management persist for both assessment and treatment, across multiple settings and types of pain.⁸⁸ Older cancer patients⁸ and cancer patients in nursing homes⁸⁷ are also less likely to receive adequate pain treatment. This issue was addressed in some of the guidelines we reviewed (e.g., the Oncology Nursing Society Position in Cancer Pain Management)⁸² but has not been operationalized into indicators. It is important to define subgroups of cancer by stage of illness and to understand the performance of indicators among major subgroups.

Durable improvement in symptom assessment will require measures that are capable of being implemented with other routine metrics of performance in important care settings. For example, improvements in pain management often decay over time. In one study to increase daily pain assessment, performance on the indicator was 82% after an educational intervention but decreased to 59% over a seven-month period.⁹ Evaluations of the use of pain assessment in clinical settings have shown good feasibility with minimal burden for staff.⁹ However, in one study with a high prevalence of unreported pain, training of home health aides to document pain scores resulted in a large increase in screening; however, physician documentation increased by only 5- to 8%.⁸⁵ In addition, although most of the indicators or measures that we identified were focused on the inpatient setting, much routine cancer care occurs in the ambulatory setting, and we found fewer indicators or measures to address outpatient performance.