Format

Send to

Choose Destination
See comment in PubMed Commons below
Acad Radiol. 2013 Jul;20(7):908-14. doi: 10.1016/j.acra.2013.04.002.

Validation of Monte Carlo estimates of three-class ideal observer operating points for normal data.

Author information

  • 1The Department of Physiology, 303 E Superior St., Northwestern University, Chicago, IL 60611, USA. darrin.c.edwards@gmail.com

Abstract

RATIONALE AND OBJECTIVES:

Traditional two-class receiver operating characteristic (ROC) analysis is inadequate for the complete evaluation of observer performance in tasks with more than two classes.

MATERIALS AND METHODS:

Here, a Monte Carlo estimation method for operating point coordinates on a three-class ROC surface is developed and compared with analytically calculated coordinates in two special cases: (1) univariate and (2) restricted bivariate trinormal underlying data.

RESULTS:

In both cases, the statistical estimates were found to be good in the sense that the analytical values lay within the 95% confidence interval of the estimated values about 95% of the time.

CONCLUSIONS:

The statistical estimation method should be key in the development of a pragmatic performance metric for evaluation of observers in classification tasks with three or more classes.

Copyright © 2013 AUR. Published by Elsevier Inc. All rights reserved.

[PubMed - indexed for MEDLINE]
Free PMC Article
PubMed Commons home

PubMed Commons

0 comments
How to join PubMed Commons

    Supplemental Content

    Full text links

    Icon for Elsevier Science Icon for PubMed Central
    Loading ...
    Write to the Help Desk