Format

Send to

Choose Destination
BMJ. 2015 Jul 8;351:h3480. doi: 10.1136/bmj.h3480.

Evaluation of symptom checkers for self diagnosis and triage: audit study.

Author information

1
Department of Health Care Policy, Harvard Medical School, Boston, MA 02115, USA.
2
Division of General Medicine and Primary Care, Brigham and Women's Hospital & Harvard Medical School, Boston, MA, USA.
3
Division of Infectious Diseases, Boston Children's Hospital, Boston, MA, USA RAND Corporation, Boston, MA, USA.
4
Department of Health Care Policy, Harvard Medical School, Boston, MA 02115, USA Division of General Internal Medicine and Primary Care, Beth Israel Deaconess Medical Center, Boston, MA, USA.

Abstract

OBJECTIVE:

To determine the diagnostic and triage accuracy of online symptom checkers (tools that use computer algorithms to help patients with self diagnosis or self triage).

DESIGN:

Audit study.

SETTING:

Publicly available, free symptom checkers.

PARTICIPANTS:

23 symptom checkers that were in English and provided advice across a range of conditions. 45 standardized patient vignettes were compiled and equally divided into three categories of triage urgency: emergent care required (for example, pulmonary embolism), non-emergent care reasonable (for example, otitis media), and self care reasonable (for example, viral upper respiratory tract infection).

MAIN OUTCOME MEASURES:

For symptom checkers that provided a diagnosis, our main outcomes were whether the symptom checker listed the correct diagnosis first or within the first 20 potential diagnoses (n=770 standardized patient evaluations). For symptom checkers that provided a triage recommendation, our main outcomes were whether the symptom checker correctly recommended emergent care, non-emergent care, or self care (n=532 standardized patient evaluations).

RESULTS:

The 23 symptom checkers provided the correct diagnosis first in 34% (95% confidence interval 31% to 37%) of standardized patient evaluations, listed the correct diagnosis within the top 20 diagnoses given in 58% (55% to 62%) of standardized patient evaluations, and provided the appropriate triage advice in 57% (52% to 61%) of standardized patient evaluations. Triage performance varied by urgency of condition, with appropriate triage advice provided in 80% (95% confidence interval 75% to 86%) of emergent cases, 55% (47% to 63%) of non-emergent cases, and 33% (26% to 40%) of self care cases (P<0.001). Performance on appropriate triage advice across the 23 individual symptom checkers ranged from 33% (95% confidence interval 19% to 48%) to 78% (64% to 91%) of standardized patient evaluations.

CONCLUSIONS:

Symptom checkers had deficits in both triage and diagnosis. Triage advice from symptom checkers is generally risk averse, encouraging users to seek care for conditions where self care is reasonable.

PMID:
26157077
PMCID:
PMC4496786
[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for HighWire Icon for PubMed Central
Loading ...
Support Center