Format

Send to

Choose Destination
J Biomed Inform. 2014 Apr;48:66-72. doi: 10.1016/j.jbi.2013.11.010. Epub 2013 Dec 7.

Development of a clinician reputation metric to identify appropriate problem-medication pairs in a crowdsourced knowledge base.

Author information

1
The University of Texas School of Biomedical Informatics at Houston, 7000 Fannin St., Ste. 600, Houston, TX 70030, USA. Electronic address: amccoy1@tulane.edu.
2
Brigham and Women's Hospital, Harvard Medical School, 1620 Tremont St., Boston, MA 02115, USA. Electronic address: awright5@partners.org.
3
The University of Texas School of Biomedical Informatics at Houston, 7000 Fannin St., Ste. 600, Houston, TX 70030, USA. Electronic address: deevakar.rogith@uth.tmc.edu.
4
The University of Texas School of Biomedical Informatics at Houston, 7000 Fannin St., Ste. 600, Houston, TX 70030, USA. Electronic address: safa.fathiamini@uth.tmc.edu.
5
The University of Texas Medical School at Houston, 6410 Fannin St., Ste. 1100, Houston, TX 77030, USA. Electronic address: allison.ottenbacher@nih.gov.
6
The University of Texas School of Biomedical Informatics at Houston, 7000 Fannin St., Ste. 600, Houston, TX 70030, USA. Electronic address: dean.f.sittig@uth.tmc.edu.

Abstract

BACKGROUND:

Correlation of data within electronic health records is necessary for implementation of various clinical decision support functions, including patient summarization. A key type of correlation is linking medications to clinical problems; while some databases of problem-medication links are available, they are not robust and depend on problems and medications being encoded in particular terminologies. Crowdsourcing represents one approach to generating robust knowledge bases across a variety of terminologies, but more sophisticated approaches are necessary to improve accuracy and reduce manual data review requirements.

OBJECTIVE:

We sought to develop and evaluate a clinician reputation metric to facilitate the identification of appropriate problem-medication pairs through crowdsourcing without requiring extensive manual review.

APPROACH:

We retrieved medications from our clinical data warehouse that had been prescribed and manually linked to one or more problems by clinicians during e-prescribing between June 1, 2010 and May 31, 2011. We identified measures likely to be associated with the percentage of accurate problem-medication links made by clinicians. Using logistic regression, we created a metric for identifying clinicians who had made greater than or equal to 95% appropriate links. We evaluated the accuracy of the approach by comparing links made by those physicians identified as having appropriate links to a previously manually validated subset of problem-medication pairs.

RESULTS:

Of 867 clinicians who asserted a total of 237,748 problem-medication links during the study period, 125 had a reputation metric that predicted the percentage of appropriate links greater than or equal to 95%. These clinicians asserted a total of 2464 linked problem-medication pairs (983 distinct pairs). Compared to a previously validated set of problem-medication pairs, the reputation metric achieved a specificity of 99.5% and marginally improved the sensitivity of previously described knowledge bases.

CONCLUSION:

A reputation metric may be a valuable measure for identifying high quality clinician-entered, crowdsourced data.

KEYWORDS:

Crowdsourcing; Electronic health records; Knowledge bases; Medical records; Problem-oriented

PMID:
24321170
PMCID:
PMC4026169
DOI:
10.1016/j.jbi.2013.11.010
[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for Elsevier Science Icon for PubMed Central
Loading ...
Support Center