Abstract

INTRODUCTION:

Systems for assessing surgical trainee competence must be practical, reliable, and valid. We developed a novel system, the Surgical Training and Assessment Tool (STAT), for longitudinal competency assessments of surgical trainees' operative performances. We hypothesized the tool would be both practical and reliable within an academic surgical oncology training program.

METHODS:

Three surgical qualities of our primary interest (knowledge, skill, and independence) and the key technical maneuvers of approximately 200 surgical oncology procedures were defined and organized into hierarchical menus and loaded into a secure, web-based database. After every training case, trainee and attending surgeon electronically submitted evaluations of the trainee's performance, along with comments, and an overall grade. Data on system use and scores were analyzed.

RESULTS:

Over the first 14 months of use at a university-based surgical oncology fellowship program, 1,029 assessments were recorded (528 attending surgeon, 501 trainee self-assessments). Median time to complete each assessment was 39 s (range 9-532 s, mean 60 s). Knowledge, skill, and independence assessments each demonstrated strong correlation with overall competency grade (Pearson correlations 0.60, 0.76, and 0.69, respectively). Multiple linear regression analysis showed all to be significant predictors of the overall grade (model R (2) = 0.63; test of predictive significance p < 0.001 for each).

CONCLUSIONS:

STAT is a novel system for tracking and assessing trainee operative performance, which is easily integrated into the workflow of an academic surgical oncology department. Our analysis suggests that it is a practical and reliable instrument; its validity is promising and warrants further study.