Send to:

Choose Destination
See comment in PubMed Commons below
Stat Methods Med Res. 2015 Feb;24(1):68-106. doi: 10.1177/0962280214537390. Epub 2014 Jun 11.

Quantitative imaging biomarkers: a review of statistical methods for computer algorithm comparisons.

Author information

  • 1Cleveland Clinic Foundation, Cleveland, OH, USA
  • 2Cornell University, Ithaca, NY, USA.
  • 3National Institutes of Health, Rockville, MD, USA.
  • 4Cleveland Clinic Foundation, Cleveland, OH, USA.
  • 5Elucid Bioimaging Inc., Wenham, MA, USA.
  • 6University of California, Los Angeles, CA, USA.
  • 7Duke University, Durham, NC, USA.
  • 8University of Wisconsin-Madison, Madison, WI, USA.
  • 9University of Chicago, Chicago, IL, USA.
  • 10Food and Drug Administration/CDRH, Silver Spring, MD, USA.
  • 11Biostatistics Consulting, LLC, Kensington, MD, USA.
  • 12MGH/Harvard Medical School, Boston, MA, USA.
  • 13George Washington University, NW Washington, DC, USA.
  • 14University of Washington, Seattle, WA, USA.
  • 15University of South Florida, Tampa, FL, USA.
  • 16H. Moffitt Cancer Center, Tampa, FL, USA.
  • 17Columbia University, New York, NY, USA.


Quantitative biomarkers from medical images are becoming important tools for clinical diagnosis, staging, monitoring, treatment planning, and development of new therapies. While there is a rich history of the development of quantitative imaging biomarker (QIB) techniques, little attention has been paid to the validation and comparison of the computer algorithms that implement the QIB measurements. In this paper we provide a framework for QIB algorithm comparisons. We first review and compare various study designs, including designs with the true value (e.g. phantoms, digital reference images, and zero-change studies), designs with a reference standard (e.g. studies testing equivalence with a reference standard), and designs without a reference standard (e.g. agreement studies and studies of algorithm precision). The statistical methods for comparing QIB algorithms are then presented for various study types using both aggregate and disaggregate approaches. We propose a series of steps for establishing the performance of a QIB algorithm, identify limitations in the current statistical literature, and suggest future directions for research.

© The Author(s) 2014 Reprints and permissions:


agreement; bias; image metrics; imaging biomarkers; precision; quantitative imaging; repeatability; reproducibility

[PubMed - indexed for MEDLINE]
Free PMC Article
PubMed Commons home

PubMed Commons

How to join PubMed Commons

    Supplemental Content

    Full text links

    Icon for HighWire Icon for PubMed Central
    Loading ...
    Write to the Help Desk