Format

Send to

Choose Destination
Surgery. 2012 Oct;152(4):517-24; discussion 524-7. doi: 10.1016/j.surg.2012.07.004. Epub 2012 Aug 28.

A template for reliable assessment of resident operative performance: assessment intervals, numbers of cases and raters.

Author information

1
Department of Surgery, Southern Illinois University School of Medicine, Springfield, IL, USA. rwilliams@siumed.edu

Abstract

BACKGROUND:

Operative performance rating (OPR) instruments have been developed to assess operative performance (OP). To guide program implementation, this study determined: 1) Appropriate intervals for OP progress decisions, 2) Number of OPRs and raters required per interval to achieve reproducible results.

METHODS:

21 surgeons rated 897 OPs (3 procedures) by 36 residents. Six-month PGY intervals were compared to determine length of stable operative performance intervals. Variance component analyses established rating factor importance. Generalizability analyses and decision studies determined number of OPRs required for reproducible OP decisions (reliabilities = 0.80).

RESULTS:

Resident OPRs are stable across single PGY years. 2.3 OPRs/resident/month provided a dependable basis for annual or semi-annual resident OP decisions. Results were similar for all procedures and training years. Rater idiosyncrasies accounted for most score variation (63% when interaction effects involving rater idiosyncrasies were included). Resident ability was the next most important source of variation (12%). Procedure was a less important source (5%).

CONCLUSION:

Annual resident OP decisions are supported. 2.3 OPRs per month provide a dependable basis for judging resident OP. These numbers are sufficient regardless of training year or procedure mix though efforts should be made to balance procedure mix. Multiple raters should rate each resident to control for rater idiosyncrasies.

PMID:
22935094
DOI:
10.1016/j.surg.2012.07.004
[Indexed for MEDLINE]

Supplemental Content

Full text links

Icon for Elsevier Science
Loading ...
Support Center