Format

Send to:

Choose Destination
See comment in PubMed Commons below
J Surg Educ. 2013 Mar-Apr;70(2):189-92. doi: 10.1016/j.jsurg.2012.11.002.

Assessment in orthopedic training-an analysis of rating consistency by using an objective structured examination video.

Author information

  • 1Department of Orthopaedic Surgery, Chang Gung Memorial Hospital, Chang Gung University Collge of Medicine, Taiwan, Republlic of China. alvinchen@adm.cgmh.org.tw

Abstract

OBJECTIVES:

This study describes a fast and efficient method that uses a prevalidated videotape of an objective structured clinical examination (OSCE) in a fracture scenario to evaluate raters and to measure the consistency of raters from different subspecialties and with varying levels of seniority.

STUDY DESIGN:

We videotaped clinical scenarios for the purpose of evaluating residents' communication and clinical assessment skills. All orthopedic staff used prevalidated checklists to assess residents' performance in the videotape at 3 different time points. Cronbach's α was calculated to evaluate the internal consistency of the OSCE checklist construct. Kendall's W and KR-20 were used to investigate rater agreement. Expert validity was calculated to compare OSCE experts with the present raters.

RESULTS:

A high Cronbach's α for the 23-item scale regarding global assessment in all 3 tests confirmed construct validity. Kendall's W showed only moderate interrater reliability. KR-20 was 0.96 for the pretest, 0.968 for the posttest, and 0.892 for the long-term test, indicating high internal consistency. The p-value for expert validity was 0.626 (independent t-test, n.s.).

CONCLUSIONS:

This efficient and fast video-based assessment of raters was reliable and yielded satisfactory rater consistency and some evidence for validity.

Crown Copyright © 2013. Published by Elsevier Inc. All rights reserved.

PMID:
23427962
[PubMed - indexed for MEDLINE]
PubMed Commons home

PubMed Commons

0 comments
How to join PubMed Commons

    Supplemental Content

    Full text links

    Icon for Elsevier Science
    Loading ...
    Write to the Help Desk