Agreement between cytotechnologists and cytopathologists as a new measure of cytopathologist performance in gynecologic cytology

Background: Although objective measures of cytotechnologist (CT) and cytopathologist (CP) performance exist, challenges remain. Two assumptions deserve examination: CPs' interpretations are correct, and CTs and CPs render interpretations independently of each other. This study presents a CT-CP interpretation comparison and provides insight into these assumptions.

Methods: Every gynecologic cytology specimen examined by both a CT and a CP from December 2004 to March 2015 was extracted from the laboratory information system; glandular interpretations were excluded. Excel and SAS were used for CT-CP pair analysis. CT-CP pairs with fewer than 32 specimens (the lowest quartile) were excluded. For the remaining CT-CP pairs, 30 specimens or 10% of the specimens (whichever was higher) were randomly selected for comparison by a weighted κ statistic. κ values greater than 0.6 represented good agreement within CT-CP pairs.

Results: This study evaluated 7116 of 53,241 gynecologic cytology specimens (13.4%) that received CT and CP interpretations. This resulted in 155 pair-specific κ values from 15 CTs and 16 CPs. In aggregate, the κ values had a mean of 0.64, a standard deviation of 0.14, a median of 0.65, and a range of 0.27 to 0.91. Nine CTs exhibited good agreement in the majority of their pair-specific κ values with CPs (high-concordance CTs; 88 pair-specific κ values). This allowed us to identify outlier CPs who did not demonstrate good agreement with high-concordance CTs (16 of 88 pair-specific κ values [18.2%]).

Conclusions: Laboratories can use this κ to determine when CP levels of agreement with CTs depart from those of their peers. Adding this to established metrics can give a more nuanced impression of CP performance. Cancer Cytopathol 2017;125:576-80. © 2017 American Cancer Society.