Format

Send to

Choose Destination
J Digit Imaging. 2019 Feb;32(1):30-37. doi: 10.1007/s10278-018-0105-8.

Large Scale Semi-Automated Labeling of Routine Free-Text Clinical Records for Deep Learning.

Author information

1
Department of Radiology and Biomedical Imaging, University of California, San Francisco, CA, USA. hari.trivedi@gmail.com.
2
Institute for Computational Health Sciences, University of California, San Francisco, CA, USA.
3
University of California School of Medicine, San Francisco, CA, USA.
4
Department of Radiology and Biomedical Imaging, University of California, San Francisco, CA, USA.
5
Department of Pathology, University of California, San Francisco, CA, USA.

Abstract

Breast cancer is a leading cause of cancer death among women in the USA. Screening mammography is effective in reducing mortality, but has a high rate of unnecessary recalls and biopsies. While deep learning can be applied to mammography, large-scale labeled datasets, which are difficult to obtain, are required. We aim to remove many barriers of dataset development by automatically harvesting data from existing clinical records using a hybrid framework combining traditional NLP and IBM Watson. An expert reviewer manually annotated 3521 breast pathology reports with one of four outcomes: left positive, right positive, bilateral positive, negative. Traditional NLP techniques using seven different machine learning classifiers were compared to IBM Watson's automated natural language classifier. Techniques were evaluated using precision, recall, and F-measure. Logistic regression outperformed all other traditional machine learning classifiers and was used for subsequent comparisons. Both traditional NLP and Watson's NLC performed well for cases under 1024 characters with weighted average F-measures above 0.96 across all classes. Performance of traditional NLP was lower for cases over 1024 characters with an F-measure of 0.83. We demonstrate a hybrid framework using traditional NLP techniques combined with IBM Watson to annotate over 10,000 breast pathology reports for development of a large-scale database to be used for deep learning in mammography. Our work shows that traditional NLP and IBM Watson perform extremely well for cases under 1024 characters and can accelerate the rate of data annotation.

KEYWORDS:

Artificial intelligence; Deep learning; IBM Watson; Machine learning; Mammography; Natural language processing (NLP); Pathology

PMID:
30128778
DOI:
10.1007/s10278-018-0105-8

Supplemental Content

Full text links

Icon for Springer
Loading ...
Support Center