Am J Surg. 2013 Feb;205(2):137-40. doi: 10.1016/j.amjsurg.2012.09.002. Epub 2012 Dec 13.

Assessing clinical judgment using the Script Concordance test: the importance of using specialty-specific experts to develop the scoring key.

Petrucci AM¹, Nouh T, Boutros M, Gagnon R, Meterissian SH.

Author information

1: Department of Surgery, McGill University, Montreal, Quebec, Canada.

Abstract

BACKGROUND:

The Script Concordance test (SCT) assesses clinical judgment. The purpose of this study was to determine whether a specialty-specific scoring key improves the validity of the SCT.

METHODS:

Thirty experts from 6 general surgery disciplines answered questions pertaining to their area of expertise. We created a scoring key of 5 amalgamated expert panel members. The answers of 227 general surgery residents were analyzed.

RESULTS:

The optimized test had a reliability level (Cronbach α) of .81. Scores increased progressively throughout all levels of training, with R5s scoring higher than R4s (R1, 42.7 ± 7.1; R2, 47.6 ± 7.5; R3, 48.7 ± 6.7; R4, 49.8 ± 7.7; R5, 52.9 ± 9.3). The average score of juniors (R1s + R2s, 45.1 ± 7.6) was significantly lower (P < .001) than seniors (R3s + R4s + R5s, 50.4 ± 8.0).

CONCLUSIONS:

We showed that specialty-specific experts must be used to develop the scoring key. This has important implications in the application of the SCT on a wider level.