Development and external validation of a stability machine learning model to identify wake-up stroke onset time from MRI

Objectives: To develop and externally validate a machine learning (ML) model based on diffusion-weighted imaging (DWI) and fluid-attenuated inversion recovery (FLAIR) to identify the onset time of wake-up stroke from MRI.

Methods: DWI and FLAIR images of stroke patients within 24 h of clear symptom onset in our hospital (dataset 1, n = 410) and another hospital (dataset 2, n = 177) were included. Seven ML models based on dataset 1 were developed to estimate the stroke onset time for binary classification (≤ 4.5 h or > 4.5 h): Random Forest (RF), support vector machine with kernel (svmLinear) or radial basis function kernel (svmRadial), Bayesian (Bayes), K-nearest neighbor (KNN), adaptive boosting (AdaBoost), and neural network (NNET). ROC analysis and RSD were performed to evaluate the performance and stability of the ML models, respectively, and dataset 2 was externally validated to evaluate the model generalization ability using ROC analysis.

Results: svmRadial achieved the best performance with the highest AUC and accuracy (AUC: 0.896, accuracy: 0.878), and was the most stable (RSD% of AUC: 0.08, RSD% of accuracy: 0.06). The svmRadial model was then selected as the final model, and the AUC of the svmRadial model for predicting the onset time external validation was 0.895, with 0.825 accuracy.

Conclusions: The svmRadial model using DWI + FLAIR is the most stable and generalizable for identifying the onset time of wake-up stroke patients within 4.5 h of symptom onset.

Key points: • Machining learning model helps clinicians to identify wake-up stroke patients within 4.5 h of symptom onset. • A prospective study showed that svmRadial model based on DWI + FLAIR was the most stable in predicting the stroke onset time. • External validation showed that svmRadial model has good generalization ability in predicting the stroke onset time.