Format

Send to

Choose Destination
J Pain Symptom Manage. 2018 Sep;56(3):406-413.e3. doi: 10.1016/j.jpainsymman.2018.06.003. Epub 2018 Jun 12.

Inter-Rater Agreement of Intensivists Evaluating the Goal Concordance of Preference-Sensitive ICU Interventions.

Author information

1
Outcomes After Critical Illness and Surgery Group, Johns Hopkins University, Baltimore, Maryland, USA; Division of Pulmonary and Critical Care Medicine, School of Medicine, Johns Hopkins University, Baltimore, Maryland, USA; Department of Epidemiology, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, Maryland, USA. Electronic address: turnbull@jhmi.edu.
2
Division of Pulmonary and Critical Care Medicine, School of Medicine, Johns Hopkins University, Baltimore, Maryland, USA.
3
Outcomes After Critical Illness and Surgery Group, Johns Hopkins University, Baltimore, Maryland, USA; Department of Biostatistics, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, Maryland, USA.
4
Department of Anesthesiology and Critical Care, Anne Arundel Medical Center, Annapolis, Maryland, USA.
5
Outcomes After Critical Illness and Surgery Group, Johns Hopkins University, Baltimore, Maryland, USA.
6
Division of Pulmonary and Critical Care Medicine, Harborview Medical Center, University of Washington, Seattle, Washington, USA; Cambia Palliative Care Center of Excellence, University of Washington, Seattle, Washington, USA.

Abstract

CONTEXT:

Goal-concordant care has been identified as an important outcome of advance care planning and shared decision-making initiatives. However, validated methods for measuring goal concordance are needed.

OBJECTIVES:

To estimate the inter-rater reliability of senior critical care fellows rating the goal concordance of preference-sensitive interventions performed in intensive care units (ICUs) while considering patient-specific circumstances as described in a previously proposed methodology.

METHODS:

We identified ICU patients receiving preference-sensitive interventions in three adult ICUs at Johns Hopkins Hospital. A simulated cohort was created by randomly assigning each patient one of 10 sets of goals and preferences about limiting life support. Critical care fellows then independently reviewed patient charts and answered two questions: 1) Is this patient's goal achievable? and 2) Will performing this intervention help achieve the patient's goal? When the answer to both questions was yes, the intervention was rated as goal concordant. Inter-rater agreement was summarized by estimating intraclass correlation coefficient using mixed-effects models.

RESULTS:

Six raters reviewed the charts of 201 patients. Interventions were rated as goal concordant 22%-92% of the time depending on the patient's goal-limitation combination. Percent agreement between pairs of raters ranged from 59% to 86%. The intraclass correlation coefficient for ratings of goal concordance was 0.50 (95% CI 0.31-0.69) and was robust to patient age, gender, ICU, severity of illness, and lengths of stay.

CONCLUSION:

Inter-rater agreement between intensivists using a standardized methodology to evaluate the goal concordance of preference-sensitive ICU interventions was moderate. Further testing is needed before this methodology can be recommended as a clinical research outcome.

KEYWORDS:

Goal-concordant care; critical care outcomes; critical illness; interobserver variability; outcome measures; patient-centered care; psychometrics

PMID:
29902555
PMCID:
PMC6456035
DOI:
10.1016/j.jpainsymman.2018.06.003
[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for Elsevier Science Icon for PubMed Central
Loading ...
Support Center