Format

Send to

Choose Destination
See comment in PubMed Commons below
Eur J Epidemiol. 2013 Jan;28(1):25-33. doi: 10.1007/s10654-012-9744-0. Epub 2012 Nov 20.

Assessing improvement in disease prediction using net reclassification improvement: impact of risk cut-offs and number of risk categories.

Author information

1
Department of Molecular Epidemiology, German Institute of Human Nutrition Potsdam-Rehbruecke, Arthur-Scheunert-Allee 114-116, 14558 Nuthetal, Germany. Kristin.Muehlenbruch@dife.de

Abstract

Net reclassification improvement (NRI) has received much attention for comparing risk prediction models, and might be preferable over the area under the receiver operating characteristics (ROC) curve to indicate changes in predictive ability. We investigated the influence of the choice of risk cut-offs and number of risk categories on the NRI. Using data of the European Prospective Investigation into Cancer and Nutrition-Potsdam study, three diabetes prediction models were compared according to ROC area and NRI with varying cut-offs for two and three risk categories and varying numbers of risk categories. When compared with a basic model, including age, anthropometry, and hypertension status, a model extension by waist circumference improved discrimination from 0.720 to 0.831 (0.111 [0.097-0.125]) while increase in ROC-AUC from 0.831 to 0.836 (0.006 [0.002-0.009]) indicated moderate improvement when additionally considering diet and physical activity. However, NRI based on these two model comparisons varied with varying cut-offs for two (range: 5.59-23.20%; -0.79 to 4.09%) and three risk categories (20.37-40.15%; 1.22-4.34%). This variation was more pronounced in the model extension showing a larger difference in ROC-AUC. NRI increased with increasing numbers of categories from minimum NRIs of 18.41 and 0.46% to approximately category-free NRIs of 79.61 and 19.22%, but not monotonically. There was a similar pattern for this increase in both model comparisons. In conclusion, the choice of risk cut-offs and number of categories has a substantial impact on NRI. A limited number of categories should only be used if categories have strong clinical importance.

PMID:
23179629
DOI:
10.1007/s10654-012-9744-0
[Indexed for MEDLINE]
PubMed Commons home

PubMed Commons

0 comments
How to join PubMed Commons

    Supplemental Content

    Full text links

    Icon for Springer
    Loading ...
    Support Center