Format

Send to

Choose Destination
Stat Biosci. 2015 Oct 1;7(2):282-295. Epub 2014 Aug 23.

The Net Reclassification Index (NRI): a Misleading Measure of Prediction Improvement Even with Independent Test Data Sets.

Author information

1
Biostatistics and Biomathematics Program, Fred Hutchinson Cancer Research Center, 1100 Fairview Ave N, M2B500, Seattle, WA 98109 USA.
2
The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd. Houston, TX 77030 USA.
3
Department of Biostatistics, University of Copenhagen, Oster Farimsgade 5, Denmark.

Abstract

The Net Reclassification Index (NRI) is a very popular measure for evaluating the improvement in prediction performance gained by adding a marker to a set of baseline predictors. However, the statistical properties of this novel measure have not been explored in depth. We demonstrate the alarming result that the NRI statistic calculated on a large test dataset using risk models derived from a training set is likely to be positive even when the new marker has no predictive information. A related theoretical example is provided in which an incorrect risk function that includes an uninformative marker is proven to erroneously yield a positive NRI. Some insight into this phenomenon is provided. Since large values for the NRI statistic may simply be due to use of poorly fitting risk models, we suggest caution in using the NRI as the basis for marker evaluation. Other measures of prediction performance improvement, such as measures derived from the ROC curve, the net benefit function and the Brier score, cannot be large due to poorly fitting risk functions.

KEYWORDS:

biomarkers; classification; diagnostic test; receiver operating characteristic; risk prediction

Supplemental Content

Full text links

Icon for PubMed Central
Loading ...
Support Center