Comparison of quantitative internal and external measures of performance for trainees in cytopathology fellowships

Introduction: Cytopathology fellowships need measures to assess performance of fellows. We sought to compare several internal quantitative assessment metrics in our fellowship with external metrics, such as performance on the American Society of Cytopathology (ASC) Progressive Evaluation of Competency (PEC) examination and United States Medical Licensing Examination (USMLE).

Methods: Quantitative parameters generated from our laboratory information system (LIS) on cytopathology fellows were evaluated over 6 years, including case volume and diagnostic discrepancies, in addition to ASC PEC and USMLE scores. For discrepancy reports, interpretations made by the fellow were compared with that of the cytopathologist, and classified as none (concordant), minor (<2-levels) or major (≥2-levels).

Results: We evaluated internal and external metrics on 13 fellows over 6 years. The program average diagnostic concordance rate was 89.9%, with an average major discrepancy rate of 1.5%, and an average monthly case volume of 260 cases. More fellows with above-average ASC PEC performance showed above-average concordant diagnoses and lower case volume, while below-average PEC scores were seen more often with higher major discrepancy rates. More fellows with above-average USMLE scores had higher case volumes, while low USMLE scores showed a trend towards higher major discrepancy rates.

Conclusion: Our fellowship program has used a variety of internal and external measures of performance for cytopathology fellows. Although the findings show no statistically significant finding correlating performance, these quantitative parameters generated from our LIS were helpful to identify areas of improvement, facilitate comparison to peers, and provide case volume documentation.