Table 5Evaluations of risk prediction models

Study DesignTool(s)ParticipantsOutcome measure(s), timing ascertainmentAlgorithm performanceConclusions
Spiegelman8 1994GMn=115,17212 y BC incidence Calibration
O, E E/O (95% CI)
O=2,396, E=3,196
E/O=1.33 (1.28–1.39)

Discrimination
Correlation coefficient
(Pearson) 0.67
(Spearman) 0.04
Significant overestimation of overall BC risk

Modest discriminatory accuracy
Rockhill9 2001MGMn=82,1095 y BC risk Calibration
O, E E/O (95% CI)
O=1,354, E=1,273.42
E/O=0.94 (0.89–0.99)

Discrimination
AUC (95% CI)
0.58 (0.56–0.60)

RR=(95% CI)
2.83 (2.19–3.65)

Sensitivity, specificity
(cut point=1.67% 5 y risk):
Se=0.44, Sp=0.66
Fairly well calibrated model

Modest discriminatory accuracy
Chlebowski13 2007MGMn=147,916
Mean age 63 y (range 50–79 y)
“Ethnically diverse”
Excluded:
History of BC, mastectomy
Suspicious baseline mammogram
<5 y followup
5 y invasive BC risk, assessed by annual or semi-annual ascertainment by mail or telephone questionnaire

Cancer verified through pathology reports
Calibration
O, E E/O
O=3,236, E=2,562
E/O=0.79
(p<0.001)

Discrimination
AUC (95% CI)
0.58 (0.56–0.60)
Poorly calibrated, underestimated number of invasive BCs in 5 years by about 20%

Modest discriminatory accuracy
Adams-Campbell10 2007MGM
GM-B
n=1,450
Age: 21–69 y
Enrolled 1995
Diagnosed with BC 1995-2003, aged ≥35 y (cases)
Age-matched, no BC by 2003
5 y invasive BC risk, assessed by biennial questionnaires Discrimination
Sensitivity, specificity
(cut point=1.7% 5 y risk)

MGM
Se=0.18, Sp=0.86

MGM-B
Se=0.04, Sp= 0.97
The MGM and MGM-B perform poorly at predicting risk of invasive BC in African American women

Limited discriminatory accuracy
Boyle11 2004
Secondary analysis of RCT data
172
IT-GM
MGM
n=5,383
Women had hysterectomies and no benign breast disease
5 y BC incidence Calibration
O, E E/O (95% CI)

IT-GM
O=79, E=82.5
E/O=0.92 (0.68–1.16)

MGM
O=79, E=88.4
E/O=0.86 (0.64–1.08)

Discrimination
AUC

IT-GM
0.58
Reasonably well calibrated

Modest discriminatory accuracy in the population studied

Data were missing on atypical hyperplasia on biopsies
Decarli12 2006IT-GM
MGM
n=10,031
Age: 35–64
5 y invasive
BC risk

Method of ascertainment not reported in this paper.
Calibration
O, E E/O (95% CI)

IT-GM
O=194, E=186.11
E/O=0.96 (0.84–1.11)

MGM
O=194, E=180.1
E/O=0.93 (0.81–1.08)

Discrimination
Average age-specific
AUC (95% CI)

IT-GM
0.59 (0.54–0.63)

MGM
0.58 (0.55–0.63)
MGM and IT-GM both well calibrated

Modest discriminatory accuracy in the population studied
Gail7 2007CAREn=14,059
Age:≥50
5 y age-specific invasive BC risk Calibration
O/E ratio (95% CI):1.08 (0.97–1.20)

Discrimination
Unweighted average age-specific AUC (95% CI): 0.555 (0.535–0.575)
Good calibration, very modest discriminatory accuracy.
Kim14 2004HCRIn=52,668
Females
Age: median 52
10 y colon cancer incidence Calibration
O/E(95% CI)

Overall O/E not reported

By HCRI category:
2.10≤RR≤5.10
1.79 (0.89–2.70)

1.10≤RR≤2.10
1.39 (1.12–1.67)

0.90≤RR≤1.10
1.00 (0.66–1.34)

0.50≤RR≤0.90
0.89 (0.68–1.09)

0.20≤RR≤0.50
0.58 (0.37–0.79)

Discrimination
AUC (95% CI): 0.67 (0.64–0.70)
Good calibration, moderate discriminatory power
Kim 14 2004HCRIn=38,953
Males
Age: median 51
10 y colon cancer incidence Calibration
O/E(95% CI)

Overall O/E not reported

By HCRI category:
2.10≤RR≤5.10
2.35 (1.12–3.59)

1.10≤RR≤2.10
1.34 (1.02–1.66)

0.90≤RR≤1.10
1.01 (0.64–1.39)

0.50≤RR≤0.90
0.75 (0.56–0.95)

0.20≤RR≤0.50
0.83 (0.60–1.07)

Discrimination
AUC (95% CI): 0.71 (0.68–0.74)
Poor calibration (possibly due to potential misclassification - older cohort at baseline), moderate discrimination

Abbreviations: 1DR=first degree relative; AUC=area under the curve; BC=Breast cancer; CI=confidence interval; CRC=Colorectal cancer; E=Expected events; GM=Gail Model; GM-B=GM for blacks/African Americans; IT-GM=GM for Italian population; MGM=Modified GM; O=Observed events; NHS= National Health Service; RCT=Randomized controlled trial; RR=Relative risk; Se=sensitivity; SEER=Surveillance, Epidemiology and End Results; Sp=specificity; y=years;

1

Performance by other cut-off points also reported

From: 3, Results

Cover of Clinical Utility of Cancer Family History Collection in Primary Care
Clinical Utility of Cancer Family History Collection in Primary Care.
Evidence Reports/Technology Assessments, No. 179.
Wilson B, Qureshi N, Little J, et al.

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.