Abstract
OBJECTIVE:
The aim of the study was to examine the reliability (interexaminer consistency) of the American Board of Psychiatry and Neurology (ABPN) Part II (oral) examination in psychiatry.
METHOD:
Grades were assigned independently by two examiners who observed the same examination in a 1-year cycle (1,422 candidates, two examinations each). The consistency between these pairs of grades (pass, condition, fail) was analyzed using a weighted kappa statistic.
RESULTS:
There was perfect agreement between examiners in 67% of examinations, minor disagreement in 26%, and major disagreement in 7% (weighted kappa = 0.54-0.56).
CONCLUSIONS:
The Part II ABPN examination demonstrates fair to good reliability as measured by interexaminer consistency. Development of more explicit grading criteria should further improve examiner agreement in future examinations.