Format

Send to

Choose Destination
Stat Med. 2014 Aug 30;33(19):3405-14. doi: 10.1002/sim.5804. Epub 2013 Apr 2.

A note on the evaluation of novel biomarkers: do not rely on integrated discrimination improvement and net reclassification index.

Author information

1
Department of Biostatistics, University of Copenhagen, Denmark.

Abstract

The 'integrated discrimination improvement' (IDI) and the 'net reclassification index' (NRI) are statistics proposed as measures of the incremental prognostic impact that a new biomarker will have when added to an existing prediction model for a binary outcome. By design, both measures were meant to be intuitively appropriate, and the IDI and NRI formulae do look intuitively plausible. Both have become increasingly popular. We shall argue, however, that their use is not always safe. If IDI and NRI are used to measure gain in prediction performance, then poorly calibrated models may appear advantageous, and in a simulation study, even the model that actually generates the data (and hence is the best possible model) can be improved on without adding measured information. We illustrate these shortcomings in actual cancer data as well as by Monte Carlo simulations. In these examples, we contrast IDI and NRI with the area under ROC and the Brier score. Unlike IDI and NRI, these traditional measures have the characteristic that prognostic performance cannot be accidentally or deliberately inflated.

KEYWORDS:

IDI; NRI; biomarker; prediction; prognostic models; proper scoring rules

PMID:
23553436
DOI:
10.1002/sim.5804
[Indexed for MEDLINE]

Supplemental Content

Loading ...
Support Center