Format

Send to

Choose Destination
J Biomed Inform. 2015 Dec;58:60-69. doi: 10.1016/j.jbi.2015.08.019. Epub 2015 Sep 16.

Comparison of machine learning classifiers for influenza detection from emergency department free-text reports.

Author information

1
Department of Biomedical Informatics, University of Pittsburgh School of Medicine, 5607 Baum Boulevard, Pittsburgh, PA, United States.
2
Department of Biomedical Informatics, University of Pittsburgh School of Medicine, 5607 Baum Boulevard, Pittsburgh, PA, United States; Intelligent System Program, University of Pittsburgh Dietrich School of Arts and Sciences, 210 South Bouquet Street, Pittsburgh, PA, United States.
3
Department of Biomedical Informatics, University of Pittsburgh School of Medicine, 5607 Baum Boulevard, Pittsburgh, PA, United States; Intelligent System Program, University of Pittsburgh Dietrich School of Arts and Sciences, 210 South Bouquet Street, Pittsburgh, PA, United States. Electronic address: tsui2@pitt.edu.

Abstract

Influenza is a yearly recurrent disease that has the potential to become a pandemic. An effective biosurveillance system is required for early detection of the disease. In our previous studies, we have shown that electronic Emergency Department (ED) free-text reports can be of value to improve influenza detection in real time. This paper studies seven machine learning (ML) classifiers for influenza detection, compares their diagnostic capabilities against an expert-built influenza Bayesian classifier, and evaluates different ways of handling missing clinical information from the free-text reports. We identified 31,268 ED reports from 4 hospitals between 2008 and 2011 to form two different datasets: training (468 cases, 29,004 controls), and test (176 cases and 1620 controls). We employed Topaz, a natural language processing (NLP) tool, to extract influenza-related findings and to encode them into one of three values: Acute, Non-acute, and Missing. Results show that all ML classifiers had areas under ROCs (AUC) ranging from 0.88 to 0.93, and performed significantly better than the expert-built Bayesian model. Missing clinical information marked as a value of missing (not missing at random) had a consistently improved performance among 3 (out of 4) ML classifiers when it was compared with the configuration of not assigning a value of missing (missing completely at random). The case/control ratios did not affect the classification performance given the large number of training cases. Our study demonstrates ED reports in conjunction with the use of ML and NLP with the handling of missing value information have a great potential for the detection of infectious diseases.

KEYWORDS:

Bayesian; Case detection; Emergency department reports; Influenza; Machine learning

PMID:
26385375
PMCID:
PMC4684714
DOI:
10.1016/j.jbi.2015.08.019
[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for Elsevier Science Icon for PubMed Central
Loading ...
Support Center