Introduction: There is a need to improve the validity of performance assessments and to develop better ways of identifying and assessing what students actually do in practice. Incorporating patients' assessments into OSCEs has the potential to offer both an expert assessment of aspects of the doctor-patient interaction and improve validity. Therefore, we held a trial using simulated patient (SP) assessments in history-taking, explaining and communication skills stations in third year OSCEs.
Methods: SPs made two separate ratings of each student they saw in the OSCE. Examiners graded students using checklists and an overall 'borderline' grade. SP and examiners' marks were subject to statistical analysis.
Results: The reliability of the SP ratings was .77. The reliability of the SP borderline grades was .68. The reliability of the ratings and grades combined was .86. SPs reached consensus on the characteristics of high and low performing students.
Conclusions: SP assessments are reliable. Statistical analysis demonstrated that SPs and clinicians are assessing different aspects of students' performance. We concluded that, due to our approach to working with SPs, their assessments increased validity.