Format

Send to

Choose Destination
Semin Arthritis Rheum. 2019 Aug;49(1):84-90. doi: 10.1016/j.semarthrit.2019.01.002. Epub 2019 Jan 4.

Identifying lupus patients in electronic health records: Development and validation of machine learning algorithms and application of rule-based algorithms.

Author information

1
Division of Rheumatology, Allergy, and Immunology, Department of Medicine, Massachusetts General Hospital, Harvard Medical School, 55 Fruit Street, Bulfinch 165, Boston, MA 02114, United States. Electronic address: AMJorge@mgh.harvard.edu.
2
Research Information Systems and Computing, Partners Healthcare, United States.
3
Division of Rheumatology and Immunology, Vanderbilt University Medical Center, United States.
4
Harvard T.H. Chan School of Public Health, United States.
5
Research Information Systems and Computing, Partners Healthcare, United States; Department of Biomedical Informatics, Harvard Medical School, United States.
6
Division of Rheumatology, Immunology, and Allergy, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, United States; Department of Biomedical Informatics, Harvard Medical School, United States.
7
Department of Biomedical Informatics, Vanderbilt University Medical Center, United States.
8
Division of Rheumatology, Immunology, and Allergy, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, United States.

Abstract

OBJECTIVE:

To utilize electronic health records (EHRs) to study SLE, algorithms are needed to accurately identify these patients. We used machine learning to generate data-driven SLE EHR algorithms and assessed performance of existing rule-based algorithms.

METHODS:

We randomly selected subjects with ≥ 1 SLE ICD-9/10 codes from our EHR and identified gold standard definite and probable SLE cases by chart review, based on 1997 ACR or 2012 SLICC Classification Criteria. From a training set, we extracted coded and narrative concepts using natural language processing and generated algorithms using penalized logistic regression to classify definite or definite/probable SLE. We assessed predictive characteristics in internal and external cohort validations. We also tested performance characteristics of published rule-based algorithms with pre-specified permutations of ICD-9 codes, laboratory tests and medications in our EHR.

RESULTS:

At a specificity of 97%, our machine learning coded algorithm for definite SLE had 90% positive predictive value (PPV) and 64% sensitivity and for definite/probable SLE, 92% PPV and 47% sensitivity. In the external validation, at 97% specificity, the definite/probable algorithm had 94% PPV and 60% sensitivity. Adding NLP concepts did not improve performance metrics. The PPVs of published rule-based algorithms ranged from 45-79% in our EHR.

CONCLUSION:

Our machine learning SLE algorithms performed well in internal and external validation. Rule-based SLE algorithms did not transport as well to our EHR. Unique EHR characteristics, clinical practices and research goals regarding the desired sensitivity and specificity of the case definition must be considered when applying algorithms to identify SLE patients.

KEYWORDS:

Algorithms; Bioinformatics; Electronic health records; Systemic lupus erythematosus

PMID:
30665626
PMCID:
PMC6609504
[Available on 2020-08-01]
DOI:
10.1016/j.semarthrit.2019.01.002

Supplemental Content

Full text links

Icon for Elsevier Science
Loading ...
Support Center