Format

Send to

Choose Destination
Crit Care. 2015 Aug 13;19:285. doi: 10.1186/s13054-015-0999-1.

Why the C-statistic is not informative to evaluate early warning scores and what metrics to use.

Author information

1
Healthcare Systems Engineering Program, Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, 200 First Street SW, Rochester, MN, 55905, USA. RomeroBrufau.Santiago@mayo.edu.
2
Division of Health Care Policy and Research, Department of Health Sciences Research, Mayo Clinic, 200 First Street SW, Rochester, MN, 55905, USA. RomeroBrufau.Santiago@mayo.edu.
3
Healthcare Systems Engineering Program, Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, 200 First Street SW, Rochester, MN, 55905, USA. Huddleston.jeanne@mayo.edu.
4
Division of Health Care Policy and Research, Department of Health Sciences Research, Mayo Clinic, 200 First Street SW, Rochester, MN, 55905, USA. Huddleston.jeanne@mayo.edu.
5
Division of Hospital Internal Medicine, Mayo Clinic, 200 First Street SW, Rochester, MN, 55905, USA. Huddleston.jeanne@mayo.edu.
6
Kaiser Permanente Division of Research, 2000 Broadway Avenue, 032 R01, Oakland, CA, 94612, USA. Gabriel.Escobar@kp.org.
7
Division of General Internal Medicine, Mayo Clinic College of Medicine, 200 First Street SW, Rochester, MN, 55905, USA. mliebow@mayo.edu.

Abstract

Metrics typically used to report the performance of an early warning score (EWS), such as the area under the receiver operator characteristic curve or C-statistic, are not useful for pre-implementation analyses. Because physiological deterioration has an extremely low prevalence of 0.02 per patient-day, these metrics can be misleading. We discuss the statistical reasoning behind this statement and present a novel alternative metric more adequate to operationalize an EWS. We suggest that pre-implementation evaluation of EWSs should include at least two metrics: sensitivity; and either the positive predictive value, number needed to evaluate, or estimated rate of alerts. We also argue the importance of reporting each individual cutoff value.

PMID:
26268570
PMCID:
PMC4535737
DOI:
10.1186/s13054-015-0999-1
[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for BioMed Central Icon for PubMed Central
Loading ...
Support Center