Format

Send to

Choose Destination
See comment in PubMed Commons below
Ann Surg. 2017 Oct 23. doi: 10.1097/SLA.0000000000002562. [Epub ahead of print]

A Novel Method of Setting Performance Standards in Surgery Using Patient Outcomes.

Author information

1
*International Centre for Surgical Safety, Keenan Centre for Biomedical Science, St. Michael's Hospital, Toronto, Ontario, Canada †Department of Surgery, University of Toronto, Toronto, Ontario, Canada.

Abstract

OBJECTIVE:

To describe a novel, outcome-based method of standard setting that differentiates between clinical outcomes rather than arbitrary educational goals.

BACKGROUND:

Standard setting methods used in assessments of procedural skill are currently not evidence-driven or outcome-based. This represents a potential obstacle for the broad implementation of these evaluations in summative assessments such as certification and credentialing.

METHODS:

The concept is based on deriving a receiver operating characteristic curve from a regression model that incorporates measures of intraoperative surgeon performance and confounding patient characteristics. This allows the creation of a performance standard that best predicts a clinically significant outcome of interest. The discovery cohort used to create the predictive model was derived from pilot data that used the Global Evaluative Assessment of Robotic Skill assessment tool to predict patient urinary continence 3 months following robotic-assisted radical prostatectomy.

RESULTS:

A receiver operating characteristic curve with an area under the curve of 0.75 was created from predicted probability statistic generated by the predictive model. We chose a predicted probability of 0.35, based on an optimal tradeoff in sensitivity and specificity (Youden Index). Rearranging the regression equation, we determined the performance score required to predict a 35%, patient-adjusted probability of postoperative urinary incontinence.

CONCLUSIONS:

This novel methodology is context, patient, and assessment-specific. Current standard setting methods do not account for the heterogeneity of the clinical environment. Workplace-based assessments in competency-based medical education require standards that are credible to the educator and the trainee. High-stakes assessments must ensure that surgeons have been evaluated to a standard that prioritizes satisfactory patient outcomes and safety.

PubMed Commons home

PubMed Commons

0 comments
How to join PubMed Commons

    Supplemental Content

    Full text links

    Icon for Wolters Kluwer
    Loading ...
    Support Center