NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Dy SM, Aslakson R, Wilson RF, et al. Closing the Quality Gap: Revisiting the State of the Science (Vol. 8: Improving Health Care and Palliative Care for Advanced and Serious Illness). Rockville (MD): Agency for Healthcare Research and Quality (US); 2012 Oct. (Evidence Reports/Technology Assessments, No. 208.8.)

Cover of Closing the Quality Gap: Revisiting the State of the Science (Vol. 8: Improving Health Care and Palliative Care for Advanced and Serious Illness)

Closing the Quality Gap: Revisiting the State of the Science (Vol. 8: Improving Health Care and Palliative Care for Advanced and Serious Illness).

Show details



Advanced and Serious Illness

For the purposes of this review, we defined the relevant population as “seriously ill patients and those with advanced disease (such as people living with advanced cancer or intensive care unit patients at high risk of dying), who are unlikely to be cured, recover, or stabilize.” We adapted this definition from the National Consensus Project41 definition of palliative care, which covers a broader population, to keep the scope of the review relatively focused on patients with advanced and serious illness. We covered the entire lifespan, including populations from pediatrics to geriatrics. We also included outcomes related to families and caregivers of these patients (e.g., depressive symptoms of family members of intensive care unit (ICU) patients). We included patients with all conditions meeting the population definition (e.g., critically ill patients and those with heart failure, end-stage lung disease, dementia, and advanced or metastatic cancer).

Many studies included diverse populations (e.g., cancer survivors, patients with early-stage disease, and patients with advanced disease). In these situations, we included studies where more than 50 percent of the included population would fit the above definition, or where results were reported separately for the advanced disease population. Where results were reported separately, we included the results only for the relevant portion of the population. For studies where the “seriously ill” portion of the definition of end-of-life care was most relevant (e.g., ICU populations), we included studies where the expected one-year mortality of the included population would be at least 50 percent.

Interventions To Improve Care and Quality Improvement

In determining how to define and categorize health care and palliative care interventions to improve care in this field, including quality improvement, we considered several ways that care for patients with advanced and serious illness may differ from care in prevention or chronic illness. Palliative care is a diverse field that often addresses complex and diverse diseases using a multidisciplinary approach. This type of care can have many different types of targets and outcomes, and can therefore involve many different types of interventions. Palliative care interventions can include palliative care consultation or case management programs; ordersets addressing multiple targets or programs targeting transitions between settings; and interventions targeting nonmedical aspects of care, such as psychosocial stresses or caregiver burden. These types of palliative care interventions will likely require different types of quality improvement, as well as different targets, and outcomes, than would be needed in more medical, unidimensional illnesses (such as asthma or hypertension). In addition, a key informant study of palliative care and hospice providers42 identified potential external barriers to quality improvement, such as the lack of resources and models for quality improvement, lack of evidence for best practices, and concerns that traditional models of quality improvement may not work well for hospice. Finally, improvement efforts in this area often involve interventions such as counseling for psychosocial and/or spiritual distress, which are more challenging to discern from quality improvement because of the inclusion of interpersonal interactions than, for example, administration of a medication.

We reviewed definitions from systematic reviews of quality improvement, (Table 1) including the Closing the Quality Gap series,7,43 as well as the reviews themselves to better understand how the definition was applied to different clinical areas like care coordination, asthma, and hypertension. We reviewed resources developed by the Cochrane Effective Practice and Organization of Care Group (, which focuses on systematic reviews of interventions designed to improve the delivery, practice, and organization of health care services. We examined another review on quality improvement studies that evaluated the reliability of a definition for identifying quality improvement interventions, and a review on quality improvement theory.43,44 Because psychological distress and support were targets of our review, we also contacted experts and searched for definitions of quality improvement in mental health care used in reviews of mental health quality improvement interventions, but did not locate any that were relevant. We used these definitions to develop a clear definition for quality improvement in patients with advanced and serious illness that met the needs of this review.

Table 1. Quality improvement definitions relevant to palliative care.

Table 1

Quality improvement definitions relevant to palliative care.

Based on this review, we developed the following definition of quality improvement as applied to patients with advanced or serious illness and palliative care:

Interventions targeting improvement for patients with advanced or serious illness that are conducted within or linked with the health care system and that have at least some element of system change, or intended change in how the patient/family/caregiver would interact with the system, and address one of the palliative care targets.

Taxonomy of Quality Improvement

We based our categories on the taxonomy from the Closing the Quality Gap Series,43 but adapted them slightly to meet the literature in this area, and added one category on provider training/support on quality improvement skills, as well as a category for “other” for any interventions that did not fit these categories. Interventions could include multiple types of quality improvement interventions (Table 2).

Table 2. Types of quality improvement in hospice and palliative care (adapted from CQG43).

Table 2

Types of quality improvement in hospice and palliative care (adapted from CQG43).

A consensus report published by the Improving Palliative Care in the ICU group in September 2010 noted that there are “two main models for ICU-palliative care integration: (1) the ‘consultative model,’ which focuses on increasing the involvement and effectiveness of palliative care consultants in the care of ICU patients and their families… and (2) the ‘integrative model,’ which seeks to embed palliative care principles and interventions into daily practice by the ICU team for all patients and families facing critical illness.” 47 The consensus report listed multiple examples of consultative versus integrative palliative care initiatives for ICU patients, but noted that it is unclear which “structure of a palliative care initiative... can best meet the needs of ICU patients, their loved ones, clinicians, and the hospital.” A complete systematic review of these initiatives with initiative classification as primarily “integrative” or “consultative” could determine which model is more efficacious, and would be applicable to most settings outside the ICU as well where palliative and other medical services could work together. From a comparative effectiveness perspective, understanding in what circumstances each model is best supported by the evidence can help practitioners choose which model to adopt in particular situations.

Topic Refinement and Review Protocol

Topics for the Closing the Quality Gap series were solicited from the portfolio leads at AHRQ. The nominations included a brief background and context; the importance and/or rationale for the topic; the focus or population of interest; relevant outcomes; and references to recent or ongoing work. Among the topics that were nominated, the following considerations were made in selection for inclusion in the series: the ability to focus and clarify the topic area appropriately; relevance to quality improvement and a systems approach; applicability to the Evidence-based Practice Center program/amenable to systematic review; the potential for duplication and/or overlap with other known or ongoing work; relevance and potential impact in improving care; and fit of the topics as a whole in reflecting the AHRQ portfolios.

Literature Search Strategy

Comprehensive search strategies were developed through an analysis of studies known to be eligible for this review and related systematic reviews. These strategies combined controlled vocabulary terms (e.g., MeSH, EMTREE terms) with free-text terms (Appendix B, Detailed Search Strategies).

In order to ensure that the search was comprehensive without capturing articles that did not apply to certain targets, separate searches were developed to capture studies on: (1) palliative care/patients with advanced and serious illness and quality improvement (Appendix Table B.1); (2) cancer/cancer care and communication/distress/pain management (Appendix Table B.2); and (3) nursing homes and care planning/pain management (Appendix Table B.3). Each of the search strategies was developed using PubMed and comparable searches were developed using the following databases: CINAHL, PsycINFO, Cochrane, and DARE (Appendix Tables B.1B.3). We reviewed reference lists of included articles for potentially relevant studies. We also identified systematic reviews that might contain relevant articles and reviewed these reference lists as well. Searches were updated through December 31, 2011, and were downloaded to the ProCite® reference database.

Gray Literature

We sought supplemental publications from our technical expert panel, including requests for any “gray literature” sources, and reviewed these publications as well as their references for potentially eligible articles. We did not conduct any additional gray literature searches, since another systematic review in palliative care found that addition of the gray literature did not affect the results.48

Inclusion and Exclusion Criteria

We included studies on seriously ill patients and those with advanced disease that met the population definition, including studies on pediatric and geriatric populations. We also included studies with outcomes related to the families/caregivers of these patients. Patients with all conditions (e.g., cancer, heart failure, end-stage lung disease, dementia, and frailty) were included.

Since there are high-quality studies in this field, we excluded all retrospective and uncontrolled studies of quality improvement interventions. We excluded individual studies published before 2000 because the nature of both quality improvement and palliative care practice has changed substantially since that time; and the populations served by hospice care were also markedly different before 2000. In addition, the pre-2000 data have been thoroughly addressed in a previous AHRQ Evidence-based Practice Center report9 and an extensive National Institute for Clinical Excellence (United Kingdom) report.8,49 In each section of the report, we describe previous systematic reviews that addressed the pre-2000 literature. Our search was not limited to English-language studies, since a significant proportion of the palliative care quality improvement studies have been conducted in non-US settings. We did not identify any eligible studies not published in English.

We included any timing of followup, including after-death interviews with families/caregivers. We addressed all settings, both inpatient and outpatient, as well as quality improvement interventions in inpatient or outpatient hospice or palliative care programs.

The detailed PICOTS criteria used for Key Questions 1 and 2 inclusion/exclusion for articles in this topic area follows:


We defined the relevant population as “…seriously ill patients and those with advanced disease (such as persons living with advanced cancer or intensive care unit patients at high risk of dying), who are unlikely to be cured, recover, or stabilize” (adapted from the National Consensus Project41).


We included studies evaluating health care and palliative care interventions, including quality improvement interventions, such as patient education and self-management and provider audit and feedback.


We included all comparators. For most studies, this was usual health care, but some studies tested interventions that were added on to usual hospice or palliative care.

Outcomes Measures for Each Key Question

For both Key Questions, we included all relevant patient or family/caregiver outcomes, including:

  • Patient and family satisfaction/perceptions of palliative care.
  • Patient symptoms, needs, distress, and quality of life.
  • Health care utilization, such as hospital admissions or do-not-resuscitate orders (but not costs).
  • Quality of care measures, such as timeliness of response to pain and other symptoms.
  • Family/caregiver psychosocial symptoms, support, needs, quality of life, and grief/bereavement.

We excluded studies that did not report measurements of any of these outcomes or that only had outcomes not related directly to the target populations (e.g., staff knowledge or perceptions of care).


We included any timing of followup, including after-death interviews with families/caregivers.


We addressed all settings, both inpatient and outpatient, with a specific focus on the nursing home setting (primary) and hospice program setting (specialty), as underlined in the Analytic Framework (see column 4 in Figure 1).

Technical Expert Panel

We discussed areas of focus with the Technical Expert Panel, which included a number of stakeholder perspectives. These included researchers in the field of hospice and palliative care; representatives of different disciplines (e.g., social work, nursing) and key settings (nursing home, hospice, and intensive care); and payers.

Study Selection

Abstract Screen

Each abstract was independently screened by two reviewers. These reviewers included a trained article screener and a content expert. An abstract was excluded at this level if it was not a study of a health care or palliative care intervention, did not address palliative care populations, did not include patient and/or family outcomes in the results, or was not an included study design. (Appendix C, Abstract Review Form).

Abstracts were promoted to be screened using full text article if both reviewers agreed that the abstract could apply to one or more of the Key Questions. An abstract could be excluded for different reasons by the two reviewers. Disagreements about the eligibility of an abstract were resolved by discussion between the two reviewers or by adjudication of a third reviewer; this person was either a trained screener or a principle investigator on the project.

Article Screen

Full text articles underwent another independent review by paired investigators to determine whether they should be included in the full data abstraction (see Appendix C, Article Inclusion/Exclusion Form). If articles were deemed to have applicable information, they were included in the data abstraction. Articles could be excluded at this level for the same set of reasons used at the abstract screen level.

Articles were promoted to data abstraction if both reviewers agreed. An article could be excluded for different reasons by the two reviewers. Disagreements about the eligibility of an article were resolved by discussion between the two reviewers or by adjudication of a third reviewer.

All screening was completed using the Distiller SR web-based systematic review software (Evidence Partners, Ontario, CA).

Data Abstraction

No forms were used for data abstraction in this systematic review. Due to the nature of the data (narrative), the senior investigators agreed that the data should be abstracted directly to tables. We used a consecutive two-reviewer process to abstract data from the included articles. In this process, a research assistant abstracted data directly to tables, and this data was checked by a senior investigator. Periodically, senior investigators cross-checked the work of the other senior investigators to ensure that abstractions were done appropriately. Reviewers were not masked to the articles’ authors, institutions, or journal. Disagreements that could not be resolved between the reviewers were resolved through consensus adjudication at team meetings.

For all articles, reviewers extracted information on general study and population characteristics: population, disease, study settings, single vs. multi center, study design and description of the intervention. For study design, we classified all articles as RCTs and non-randomized controlled studies (almost all of which were pre-post or concurrent non-randomized controlled studies). Data abstracted on the interventions included: integrative vs. consultative, target of the intervention, and quality improvement elements. Outcomes included: sample size, disparities, specific outcome measure, other outcome measures and harms and benefits. We abstracted effect size whenever possible, and both numerical and statistical results. Since forms were not used for this phase of data abstraction, the elements that were abstracted can be seen in the evidence tables (Appendix E).

Risk of Bias Assessment of Individual Studies

We used a tool implemented successfully in past Evidence-based Practice Center projects, including the Cochrane Collaboration Tool for Assessing Risk of Bias from the Cochrane Handbook for Systematic Reviews of Interventions for assessing randomized controlled trials50. Data abstraction forms were not developed for this phase of the review and elements of the data abstraction can be found in the evidence tables.

We assessed the risk of bias and appropriateness of all studies that met our eligibility criteria, following the guidance contained in chapter 6 of the AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews (hereafter, Methods Guide).51 We used a limited number of key criteria that are most appropriate for each study design and that are most important for determining the validity of the studies. After the pool of included articles in this review was determined, the core team of investigators determined that the Cochrane Collaboration tool50 was most appropriate for all risk of bias assessments. Although we considered assessing risk of bias separately for non-randomized studies, we were unable to identify any validated tools that worked well in this literature and could be compared to risk of bias tools designed for randomized trials. The quality of individual studies was classified as “good,” “fair,” or “poor” based on the degree to which the studies adhered to the defined criteria. We did not factor in the quality score for blinding of the intervention; we did this because blinding of patients and personnel was generally not feasible in these interventions, but blinding of outcomes assessors would have been possible. A low risk of bias was assessed if six or more of the items were scored as a “yes.” A medium risk of bias was assessed if four or five of the items were scored as a “yes” or “unclear”. A high risk of bias was assessed if zero to three items were scored as a “yes.”


To assess applicability, we used criteria stipulated in the Methods Guide1,51 and input from the expert panel concerning what criteria would be most useful to stakeholders. We addressed applicability in two ways. First, we assessed studies to ensure that they included a relevant palliative care population and outcome, as defined in the methods section on the population. For example, an intervention to improve advance directive completion by healthy patients might not translate well for ill cancer patients. An outcome of improved adherence to chemotherapy may not necessarily translate into improved quality of life. Secondly, to evaluate applicability for included studies, we extracted the relevant patient population (e.g., cancer type, stage, etc.) and setting (e.g., size, teaching vs. community hospital) information from each study in the evidence tables. For example, an intervention study on improving pain management in cancer patients may not translate well to the frail elderly, for whom the treatment of pain is very different. An intervention study that is successful in a hospital setting likely will not translate well to the nursing home setting. Finally, we abstracted details from articles about feasibility or setting-specific issues that could be relevant to translation to other settings.

Strength of the Body of Evidence

At the completion of our review, we assessed the quantity, quality, and consistency of the body of available evidence addressing Key Questions 1 and 2. We used the GRADE Working Group criteria adapted by AHRQ in its Methods Guide51 and published in the Journal of Clinical Epidemiology.52 We considered the strength of the study designs, with RCTs having the highest level of evidence, followed by non-randomized studies. If an outcome was evaluated by at least one randomized controlled trial in addition to non-randomized studies, our evidence grade was based on the randomized controlled trials, followed by the quality of the non-randomized studies. If an outcome was not evaluated in any randomized controlled trial, our evidence grade is based on the best available non-randomized study.

We assessed the strength of the best available evidence, including the risk of bias in relevant studies, as well as aspects of consistency, directness, and precision as described in the Methods Guide.51 52 As described in the Methods Guide, consistency was graded based on both the direction and range of effect size, including the percentage of studies with a statistically significant effect on the outcome. Directness was graded based on the whether the measured outcome was directly related to patient-centered outcomes. In particular, health care utilization may be related to patient-centered outcomes, such as improved satisfaction with communication, but this relationship has not been established in the palliative care research literature, and utilization does not measure patient-centeredness directly. This outcome was therefore graded as indirect. Precision was graded only if the measurement tools and reporting of outcomes was homogenous enough to allow for quantitative synthesis; if not, precision was graded as not applicable. The direction of effect was listed for each (improvement with intervention if any of the studies showed improvement). The overall strength of evidence was then graded for each outcome, based on all of these different dimensions. For each outcome of interest, two investigators graded the major outcomes for each Key Question.

Unless otherwise noted, all strength of evidence assessments are for improvements to the outcome of interest.

Data Synthesis

We organized the review by target of the intervention, including a category for interventions that focused on multiple targets or targets other than the primary ones for this review, and evaluated each target category for applicability to each Key Question. Since many studies did not report effect size but only a p value, for each category, we calculated the percentage of studies with a statistically significant improvement in outcomes with the intervention compared to control. We also checked that all other studies did not report significant results in the opposite direction. Because the results of higher-quality studies might differ from those of lower-quality studies (more likely to have methodological issues and lower sample size), we evaluated for potential differences in results in two ways. In the grading process, we compared the strength of the evidence for both RCTs and non-RCTs. Also, because interventions on the organizational level often cannot be conducted as an RCT and these studies would therefore not be graded as high-quality, as a sensitivity analysis, for the target of continuity, we determined whether results of the evidence synthesis were different when including only the high- and medium-quality studies. To determine whether quantitative synthesis was appropriate, we evaluated the clinical and methodological diversity of studies, measurement tools, and outcome reporting, and assessed for evidence of selective outcome reporting and reporting of effect sizes or confidence intervals.

Peer Review and Public Commentary

Experts in palliative care in nursing home, hospice, and intensive care unit settings, as well as experts from the disciplines of nursing and social work and individuals representing stakeholder and user communities were invited to provide external peer review of this CER; AHRQ and an associate editor also provided comments. The draft report was posted on the AHRQ website for 4 weeks to elicit public comment. We addressed all reviewer comments, revising the text as appropriate, and documented everything in a disposition of comments report that will be made available 3 months after the Agency posts the final CER on the AHRQ website.

Image introductionf1
PubReader format: click here to try


  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (4.3M)

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...