Format

Send to

Choose Destination
See comment in PubMed Commons below
AMIA Annu Symp Proc. 2013 Nov 16;2013:721-30. eCollection 2013.

Optimized dual threshold entity resolution for electronic health record databases--training set size and active learning.

Author information

1
School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX.
2
School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX. ; Department of Bioinformatics and Computational Biology, MD Anderson Cancer Center, Houston, TX.
3
School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX. ; UT Houston - Memorial Hermann Center for Healthcare Quality & Safety, Houston, TX.
4
School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX. ; Division of General Internal Medicine, Department of Internal Medicine, Medical School, The University of Texas Health Science Center at Houston, Houston, TX.

Abstract

Clinical databases may contain several records for a single patient. Multiple general entity-resolution algorithms have been developed to identify such duplicate records. To achieve optimal accuracy, algorithm parameters must be tuned to a particular dataset. The purpose of this study was to determine the required training set size for probabilistic, deterministic and Fuzzy Inference Engine (FIE) algorithms with parameters optimized using the particle swarm approach. Each algorithm classified potential duplicates into: definite match, non-match and indeterminate (i.e., requires manual review). Training sets size ranged from 2,000-10,000 randomly selected record-pairs. We also evaluated marginal uncertainty sampling for active learning. Optimization reduced manual review size (Deterministic 11.6% vs. 2.5%; FIE 49.6% vs. 1.9%; and Probabilistic 10.5% vs. 3.5%). FIE classified 98.1% of the records correctly (precision=1.0). Best performance required training on all 10,000 randomly-selected record-pairs. Active learning achieved comparable results with 3,000 records. Automated optimization is effective and targeted sampling can reduce the required training set size.

PMID:
24551372
PMCID:
PMC3900213
[Indexed for MEDLINE]
Free PMC Article
PubMed Commons home

PubMed Commons

0 comments
How to join PubMed Commons

    Supplemental Content

    Full text links

    Icon for PubMed Central
    Loading ...
    Support Center