Send to

Choose Destination
PLoS One. 2018 Nov 21;13(11):e0207749. doi: 10.1371/journal.pone.0207749. eCollection 2018.

Machine learning model combining features from algorithms with different analytical methodologies to detect laboratory-event-related adverse drug reaction signals.

Jeong E1, Park N1,2, Choi Y1,3, Park RW1,3, Yoon D1,3.

Author information

Department of Biomedical Informatics, Ajou University School of Medicine, Suwon, Gyeonggi-do, Republic of Korea.
College of Pharmacy, Ewha Womans University, Seoul, Republic of Korea.
Department of Biomedical Sciences, Ajou University Graduate School of Medicine, Suwon, Gyeonggi-do, Republic of Korea.



The importance of identifying and evaluating adverse drug reactions (ADRs) has been widely recognized. Many studies have developed algorithms for ADR signal detection using electronic health record (EHR) data. In this study, we propose a machine learning (ML) model that enables accurate ADR signal detection by integrating features from existing algorithms based on inpatient EHR laboratory results.


To construct an ADR reference dataset, we extracted known drug-laboratory event pairs represented by a laboratory test from the EU-SPC and SIDER databases. All possible drug-laboratory event pairs, except known ones, are considered unknown. To detect a known drug-laboratory event pair, three existing algorithms-CERT, CLEAR, and PACE-were applied to 21-year inpatient EHR data. We also constructed ML models (based on random forest, L1 regularized logistic regression, support vector machine, and a neural network) that use the intermediate products of the CERT, CLEAR, and PACE algorithms as inputs and determine whether a drug-laboratory event pair is associated. For performance comparison, we evaluated the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), F1-measure, and area under receiver operating characteristic (AUROC).


All measures of ML models outperformed those of existing algorithms with sensitivity of 0.593-0.793, specificity of 0.619-0.796, NPV of 0.645-0.727, PPV of 0.680-0.777, F1-measure of 0.629-0.709, and AUROC of 0.737-0.816. Features related to change or distribution of shape were considered important for detecting ADR signals.


Improved performance of ML models indicated that applying our model to EHR data is feasible and promising for detecting more accurate and comprehensive ADR signals.

[Indexed for MEDLINE]
Free PMC Article

Supplemental Content

Full text links

Icon for Public Library of Science Icon for PubMed Central
Loading ...
Support Center