Validation of automated data abstraction for SCCM discovery VIRUS COVID-19 registry: practical EHR export pathways (VIRUS-PEEP)

Diana J Valencia Morales; Vikas Bansal; Smith F Heavner; Janna C Castro; Mayank Sharma; Aysun Tekin; Marija Bogojevic; Simon Zec; Nikhil Sharma; Rodrigo Cartin-Ceba; Rahul S Nanchal; Devang K Sanghavi; Abigail T La Nou; Syed A Khan; Katherine A Belden; Jen-Ting Chen; Roman R Melamed; Imran A Sayed; Ronald A Reilkoff; Vitaly Herasevich; Juan Pablo Domecq Garces; Allan J Walkey; Karen Boman; Vishakha K Kumar; Rahul Kashyap

doi:10.3389/fmed.2023.1089087

Validation of automated data abstraction for SCCM discovery VIRUS COVID-19 registry: practical EHR export pathways (VIRUS-PEEP)

Front Med (Lausanne). 2023 Oct 4:10:1089087. doi: 10.3389/fmed.2023.1089087. eCollection 2023.

Authors

Diana J Valencia Morales¹, Vikas Bansal², Smith F Heavner³, Janna C Castro⁴, Mayank Sharma¹, Aysun Tekin¹, Marija Bogojevic¹, Simon Zec¹, Nikhil Sharma², Rodrigo Cartin-Ceba⁵, Rahul S Nanchal⁶, Devang K Sanghavi⁷, Abigail T La Nou⁸, Syed A Khan⁹, Katherine A Belden¹⁰, Jen-Ting Chen¹¹, Roman R Melamed¹², Imran A Sayed¹³, Ronald A Reilkoff¹⁴, Vitaly Herasevich¹, Juan Pablo Domecq Garces², Allan J Walkey¹⁵, Karen Boman¹⁶, Vishakha K Kumar¹⁶, Rahul Kashyap¹

Affiliations

¹ Division of Critical Care Medicine, Department of Anesthesiology and Perioperative Care, Mayo Clinic, Rochester, MN, United States.
² Division of Nephrology and Critical Care Medicine, Department of Internal Medicine, Mayo Clinic, Rochester, MN, United States.
³ CURE Drug Repurposing Collaboratory, Critical Path Institute, Tucson, AZ, United States.
⁴ Department of Information Technology, Mayo Clinic, Scottsdale, AZ, United States.
⁵ Division of Critical Care Medicine, Department of Pulmonary Medicine, Mayo Clinic, Scottsdale, AZ, United States.
⁶ Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Medical College of Wisconsin, Milwaukee, WI, United States.
⁷ Department of Critical Care Medicine, Mayo Clinic Florida, Jacksonville, FL, United States.
⁸ Department of Critical Care Medicine, Mayo Clinic Health System, Eau Claire, WI, United States.
⁹ Department of Critical Care Medicine, Mayo Clinic Health System, Mankato, MN, United States.
¹⁰ Division of Infectious Diseases, Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, PA, United States.
¹¹ Division of Critical Care Medicine, Department of Internal Medicine, Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY, United States.
¹² Department of Critical Care Medicine, Abbott Northwestern Hospital, Allina Health, Minneapolis, MN, United States.
¹³ Department of Pediatrics, Children's Hospital of Colorado, University of Colorado Anschutz Medical Campus, Colorado Springs, CO, United States.
¹⁴ Division of Pulmonary, Allergy, Critical Care and Sleep Medicine, Department of Internal Medicine, University of Minnesota Medical School, Edina, MN, United States.
¹⁵ Division of Pulmonary, Allergy, Critical Care and Sleep Medicine, Department of Medicine, Evans Center of Implementation and Improvement Sciences, Boston University School of Medicine, Boston, MA, United States.
¹⁶ Society of Critical Care Medicine, Mount Prospect, IL, United States.

Abstract

Background: The gold standard for gathering data from electronic health records (EHR) has been manual data extraction; however, this requires vast resources and personnel. Automation of this process reduces resource burdens and expands research opportunities.

Objective: This study aimed to determine the feasibility and reliability of automated data extraction in a large registry of adult COVID-19 patients.

Materials and methods: This observational study included data from sites participating in the SCCM Discovery VIRUS COVID-19 registry. Important demographic, comorbidity, and outcome variables were chosen for manual and automated extraction for the feasibility dataset. We quantified the degree of agreement with Cohen's kappa statistics for categorical variables. The sensitivity and specificity were also assessed. Correlations for continuous variables were assessed with Pearson's correlation coefficient and Bland-Altman plots. The strength of agreement was defined as almost perfect (0.81-1.00), substantial (0.61-0.80), and moderate (0.41-0.60) based on kappa statistics. Pearson correlations were classified as trivial (0.00-0.30), low (0.30-0.50), moderate (0.50-0.70), high (0.70-0.90), and extremely high (0.90-1.00).

Measurements and main results: The cohort included 652 patients from 11 sites. The agreement between manual and automated extraction for categorical variables was almost perfect in 13 (72.2%) variables (Race, Ethnicity, Sex, Coronary Artery Disease, Hypertension, Congestive Heart Failure, Asthma, Diabetes Mellitus, ICU admission rate, IMV rate, HFNC rate, ICU and Hospital Discharge Status), and substantial in five (27.8%) (COPD, CKD, Dyslipidemia/Hyperlipidemia, NIMV, and ECMO rate). The correlations were extremely high in three (42.9%) variables (age, weight, and hospital LOS) and high in four (57.1%) of the continuous variables (Height, Days to ICU admission, ICU LOS, and IMV days). The average sensitivity and specificity for the categorical data were 90.7 and 96.9%.

Conclusion and relevance: Our study confirms the feasibility and validity of an automated process to gather data from the EHR.

Keywords: COVID-19; VIRUS COVID-19 registry; data automation; electronic health records; validation.

Copyright © 2023 Valencia Morales, Bansal, Heavner, Castro, Sharma, Tekin, Bogojevic, Zec, Sharma, Cartin-Ceba, Nanchal, Sanghavi, La Nou, Khan, Belden, Chen, Melamed, Sayed, Reilkoff, Herasevich, Domecq Garces, Walkey, Boman, Kumar and Kashyap.

Grants and funding

The VIRUS: COVID-19 Registry was supported, in part, by the Gordon and Betty Moore Foundation, and Janssen Research & Development, LLC. They have no role in data gathering, analysis, interpretation, and writing.