Send to

Choose Destination
Front Genet. 2019 Jun 28;10:617. doi: 10.3389/fgene.2019.00617. eCollection 2019.

MildInt: Deep Learning-Based Multimodal Longitudinal Data Integration Framework.

Lee G1,2, Kang B1, Nho K3,4, Sohn KA1, Kim D2,5,6.

Author information

Department of Software and Computer Engineering, Ajou University, Suwon, South Korea.
Biomedical & Translational Informatics Institute, Geisinger, Danville, PA, United States.
Center for Computational Biology and Bioinformatics, Indiana University School of Medicine, Indianapolis, IN, United States.
Center for Neuroimaging, Department of Radiology and Imaging Sciences, Indiana University School of Medicine, Indianapolis, IN, United States.
Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States.
Institute for Biomedical Informatics, University of Pennsylvania, Philadelphia, PA, United States.


As large amounts of heterogeneous biomedical data become available, numerous methods for integrating such datasets have been developed to extract complementary knowledge from multiple domains of sources. Recently, a deep learning approach has shown promising results in a variety of research areas. However, applying the deep learning approach requires expertise for constructing a deep architecture that can take multimodal longitudinal data. Thus, in this paper, a deep learning-based python package for data integration is developed. The python package deep learning-based multimodal longitudinal data integration framework (MildInt) provides the preconstructed deep learning architecture for a classification task. MildInt contains two learning phases: learning feature representation from each modality of data and training a classifier for the final decision. Adopting deep architecture in the first phase leads to learning more task-relevant feature representation than a linear model. In the second phase, linear regression classifier is used for detecting and investigating biomarkers from multimodal data. Thus, by combining the linear model and the deep learning model, higher accuracy and better interpretability can be achieved. We validated the performance of our package using simulation data and real data. For the real data, as a pilot study, we used clinical and multimodal neuroimaging datasets in Alzheimer's disease to predict the disease progression. MildInt is capable of integrating multiple forms of numerical data including time series and non-time series data for extracting complementary features from the multimodal dataset.


Alzheimer’s disease; data integration; gated recurrent unit; multimodal deep learning; python package

Supplemental Content

Full text links

Icon for Frontiers Media SA Icon for PubMed Central
Loading ...
Support Center