Send to

Choose Destination
Ann Emerg Med. 2017 Sep;70(3):338-344.e3. doi: 10.1016/j.annemergmed.2016.12.004. Epub 2017 Feb 24.

Structured Clinical Decision Aids Are Seldom Compared With Subjective Physician Judgment, and Are Seldom Superior.

Author information

Department of Emergency Medicine, University of California, Los Angeles, CA. Electronic address:
Robert Wood Johnson Clinical Scholars Program, Yale University School of Medicine, New Haven, CT.
Department of Emergency Medicine, University of California, Los Angeles, CA.



We determine how often studies that evaluate the performance of an aid for decisionmaking, be it a simple laboratory or imaging test or a complex multielement decision instrument, compare the aid's performance to independent, unaided physician judgment.


This was a cross-sectional survey of all Original Research and Brief Research Report articles in Annals of Emergency Medicine from 1998 to 2015. We included all articles that evaluated the performance of an aid for decisionmaking in assisting a physician with a decision about testing, treatment, diagnosis, or disposition. Two authors independently characterized the intent and purpose of each aid for decisionmaking, determined whether each study had a comparison to unaided physician judgment within the article or in a separate article, and recorded the result of that comparison.


One hundred seventy-one (8.3%) of 2,060 research articles studied the performance characteristics of an aid for decisionmaking, 48 of which were formal clinical decision instruments. Forty of the 171 studies retrospectively analyzed existing databases and therefore could not assess physician judgment. Investigators compared the aid for decisionmaking to physician judgment in 11% (15/131) of the prospective studies, including 15% (6/41) of studies that evaluated a formal clinical decision instrument. For 9 articles that had no comparison to physician judgment, we found 6 unique external publications that compared that aid to physician clinical judgment. The decision aid was superior to clinical judgment in 2 of the 21 studies that contained a comparison.


Physician judgment is infrequently assessed when the performance of an aid for decisionmaking is evaluated, and, when reported, the decision aid seldom outperformed physician judgment.

[Indexed for MEDLINE]

Supplemental Content

Full text links

Icon for Elsevier Science
Loading ...
Support Center