Logo of jclinpathJournal of Clinical PathologyVisit this articleSubmit a manuscriptReceive email alertsContact usBMJ
J Clin Pathol. May 2001; 54(5): 385–390.
PMCID: PMC1731421

Observer accuracy in estimating proportions in images: implications for the semiquantitative assessment of staining reactions and a proposal for a new system

Abstract

Background—The results of immunohistochemical staining are often assessed by semiquantitative scoring. However, these scoring systems are usually non-standardised and there has been little evaluation of the accuracy and reliability of this subjective assessment.

Aims—To assess the accuracy of observer estimation of proportions of objects in an image.

Methods—Images were generated that contained known proportions of pink squares in grids of 50 x 50 and 100 x 100 squares. Observers were shown each image for five seconds in random order and either estimated the proportion of pink squares or selected the image (from a pair of images) that contained the greater proportion of pink squares. The observers were four consultant histopathologists, seven trainee histopathologists, and six control non-histopathologists.

Results—The raw estimations of proportions showed a close correlation with the real proportions, with correlation coefficients of 0.94 and 0.95 for consultant and trainee histopathologists on the 50 x 50 grids. However, the performance in the comparison task was much higher, with an almost perfect classification for grids of equal size even when the proportions only differed by 5%.

Conclusions—Histopathologists can estimate proportions of objects in an image with a reasonable degree of accuracy in this abstract test system. All observers, whether histopathologists or not, can discriminate between proportions that are only 5% different in equal sized image grids. This suggests that the generation and use of carefully calibrated reference images could greatly improve the accuracy and reliability of semiquantitative scoring of immunohistochemical or any other staining.

Key Words: scoring • immunohistochemistry • semiquantitative • interobserver agreement • κ statistics

Full Text

The Full Text of this article is available as a PDF (288K).

Selected References

These references are in PubMed. This may not be the complete list of references from this article.
  • Barnes DM, Millis RR, Beex LV, Thorpe SM, Leake RE. Increased use of immunohistochemistry for oestrogen receptor measurement in mammary carcinoma: the need for quality assurance. Eur J Cancer. 1998 Oct;34(11):1677–1682. [PubMed]
  • McNicol AM, Richmond JA. Optimizing immunohistochemistry: antigen retrieval and signal amplification. Histopathology. 1998 Feb;32(2):97–103. [PubMed]
  • Jimenez RE, Wallis T, Tabasczka P, Visscher DW. Determination of Her-2/Neu status in breast carcinoma: comparative analysis of immunohistochemistry and fluorescent in situ hybridization. Mod Pathol. 2000 Jan;13(1):37–45. [PubMed]
  • Matkowskyj KA, Schonfeld D, Benya RV. Quantitative immunohistochemistry by measuring cumulative signal strength using commercially available software photoshop and matlab. J Histochem Cytochem. 2000 Feb;48(2):303–312. [PubMed]
  • Lohmann CM, League AA, Clark WS, Lawson D, DeRose PB, Cohen C. Bcl-2: bax and bcl-2: Bcl-x ratios by image cytometric quantitation of immunohistochemical expression in ovarian carcinoma: correlation with prognosis. Cytometry. 2000 Feb 15;42(1):61–66. [PubMed]
  • Ordi J, Schammel DP, Rasekh L, Tavassoli FA. Sertoliform endometrioid carcinomas of the ovary: a clinicopathologic and immunohistochemical study of 13 cases. Mod Pathol. 1999 Oct;12(10):933–940. [PubMed]
  • Bates AW, Baithun SI. Secondary neoplasms of the bladder are histological mimics of nontransitional cell primary tumours: clinicopathological and histological features of 282 cases. Histopathology. 2000 Jan;36(1):32–40. [PubMed]
  • McCluggage WG, Maxwell P, Hamilton PW, Jasani B. High metallothionein expression is associated with features predictive of aggressive behaviour in endometrial carcinoma. Histopathology. 1999 Jan;34(1):51–55. [PubMed]
  • Cross SS. Grading and scoring in histopathology. Histopathology. 1998 Aug;33(2):99–106. [PubMed]
  • Wang S, Saboorian MH, Frenkel E, Hynan L, Gokaslan ST, Ashfaq R. Laboratory assessment of the status of Her-2/neu protein and oncogene in breast cancer specimens: comparison of immunohistochemistry assay with fluorescence in situ hybridisation assays. J Clin Pathol. 2000 May;53(5):374–381. [PMC free article] [PubMed]
  • van Doorn HC, Burger CW, van der Valk P, Bonfrèr HM. Oestrogen, progesterone, and androgen receptors in ovarian neoplasia: correlation between immunohistochemical and biochemical receptor analyses. J Clin Pathol. 2000 Mar;53(3):201–205. [PMC free article] [PubMed]
  • Kaufmann O, Dietel M. Thyroid transcription factor-1 is the superior immunohistochemical marker for pulmonary adenocarcinomas and large cell carcinomas compared to surfactant proteins A and B. Histopathology. 2000 Jan;36(1):8–16. [PubMed]
  • Bànkfalvi A, Terpe HJ, Breukelmann D, Bier B, Rempe D, Pschadka G, Krech R, Lellè RJ, Boecker W. Immunophenotypic and prognostic analysis of E-cadherin and beta-catenin expression during breast carcinogenesis and tumour progression: a comparative study with CD44. Histopathology. 1999 Jan;34(1):25–34. [PubMed]
  • Pernick NL, DaSilva M, Gangi MD, Crissman J, Adsay V. "Histiocytic markers" in melanoma. Mod Pathol. 1999 Nov;12(11):1072–1077. [PubMed]
  • Bonatz G, Lüttes J, Hamann S, Mettler L, Jonat W, Parwaresch R. Immunohistochemical assessment of p170 provides prognostic information in endometrial carcinoma. Histopathology. 1999 Jan;34(1):43–50. [PubMed]
  • Hall PA, Going JJ. Predicting the future: a critical appraisal of cancer prognosis studies. Histopathology. 1999 Dec;35(6):489–494. [PubMed]
  • Rhodes A, Jasani B, Barnes DM, Bobrow LG, Miller KD. Reliability of immunohistochemical demonstration of oestrogen receptors in routine practice: interlaboratory variance in the sensitivity of detection and evaluation of scoring systems. J Clin Pathol. 2000 Feb;53(2):125–130. [PMC free article] [PubMed]
  • Rhodes A, Jasani B, Balaton AJ, Miller KD. Immunohistochemical demonstration of oestrogen and progesterone receptors: correlation of standards achieved on in house tumours with that achieved on external quality assessment material in over 150 laboratories from 26 countries. J Clin Pathol. 2000 Apr;53(4):292–301. [PMC free article] [PubMed]
  • Cross SS, Betmouni S, Burton JL, Dubé AK, Feeley KM, Holbrook MR, Landers RJ, Lumb PB, Stephenson TJ. What levels of agreement can be expected between histopathologists assigning cases to discrete nominal categories? A study of the diagnosis of hyperplastic and adenomatous colorectal polyps. Mod Pathol. 2000 Sep;13(9):941–944. [PubMed]
  • Ramsay AD. Errors in histopathology reporting: detection and avoidance. Histopathology. 1999 Jun;34(6):481–490. [PubMed]
  • Cross SS. Kappa statistics as indicators of quality assurance in histopathology and cytopathology. J Clin Pathol. 1996 Jul;49(7):597–599. [PMC free article] [PubMed]
  • Walker AJ, Cross SS, Harrison RF. Visualisation of biomedical datasets by use of growing cell structure networks: a novel diagnostic classification technique. Lancet. 1999 Oct 30;354(9189):1518–1521. [PubMed]

Figures and Tables

An external file that holds a picture, illustration, etc.
Object name is 00176.f1.jpg
Figure 1 Examples of the test images. (A) A 50 x 50 grid in which 40% of the squares are coloured pink. (B) Two 100 x 100 grids in a comparison image; the left hand grid has 50% pink squares, the right hand grid has 45%. Despite the small difference ...
An external file that holds a picture, illustration, etc.
Object name is 00176.f2.jpg
Figure 2 Scattergrams for the estimated proportion of pink squares in a 50 x 50 grid for (A) the consultant histopathologists, (B) the trainee histopathologists, and (C) controls. The regression line and correlation coefficient were derived using the ...
An external file that holds a picture, illustration, etc.
Object name is 00176.f3.jpg
Figure 3 Scattergrams for the estimated proportion of pink squares in a 100 x 100 grid for (A) the consultant histopathologists, (B) the trainee histopathologists, and (C) controls. The regression line and correlation coefficient were derived using the ...

Articles from Journal of Clinical Pathology are provided here courtesy of BMJ Group

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

  • PubMed
    PubMed
    PubMed citations for these articles

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...