Format

Send to

Choose Destination
J Digit Imaging. 2013 Aug;26(4):714-20. doi: 10.1007/s10278-012-9557-4.

Modeling perceptual similarity measures in CT images of focal liver lesions.

Author information

1
Electrical Engineering Department, Stanford University, James H. Clark Center, 318 Campus Drive S-324, Stanford, CA 94305, USA. jesscaf@stanford.edu

Abstract

MOTIVATION:

A gold standard for perceptual similarity in medical images is vital to content-based image retrieval, but inter-reader variability complicates development. Our objective was to develop a statistical model that predicts the number of readers (N) necessary to achieve acceptable levels of variability.

MATERIALS AND METHODS:

We collected 3 radiologists' ratings of the perceptual similarity of 171 pairs of CT images of focal liver lesions rated on a 9-point scale. We modeled the readers' scores as bimodal distributions in additive Gaussian noise and estimated the distribution parameters from the scores using an expectation maximization algorithm. We (a) sampled 171 similarity scores to simulate a ground truth and (b) simulated readers by adding noise, with standard deviation between 0 and 5 for each reader. We computed the mean values of 2-50 readers' scores and calculated the agreement (AGT) between these means and the simulated ground truth, and the inter-reader agreement (IRA), using Cohen's Kappa metric.

RESULTS:

IRA for the empirical data ranged from =0.41 to 0.66. For between 1.5 and 2.5, IRA between three simulated readers was comparable to agreement in the empirical data. For these values , AGT ranged from =0.81 to 0.91. As expected, AGT increased with N, ranging from =0.83 to 0.92 for N = 2 to 50, respectively, with =2.

CONCLUSION:

Our simulations demonstrated that for moderate to good IRA, excellent AGT could nonetheless be obtained. This model may be used to predict the required N to accurately evaluate similarity in arbitrary size datasets.

PMID:
23254627
PMCID:
PMC3705003
DOI:
10.1007/s10278-012-9557-4
[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for Springer Icon for PubMed Central
Loading ...
Support Center