Artificial intelligence (AI)-based decision support improves reproducibility of tumor response assessment in neuro-oncology: An international multi-reader study

Philipp Vollmuth; Martha Foltyn; Raymond Y Huang; Norbert Galldiks; Jens Petersen; Fabian Isensee; Martin J van den Bent; Frederik Barkhof; Ji Eun Park; Yae Won Park; Sung Soo Ahn; Gianluca Brugnara; Hagen Meredig; Rajan Jain; Marion Smits; Whitney B Pope; Klaus Maier-Hein; Michael Weller; Patrick Y Wen; Wolfgang Wick; Martin Bendszus

doi:10.1093/neuonc/noac189

Artificial intelligence (AI)-based decision support improves reproducibility of tumor response assessment in neuro-oncology: An international multi-reader study

Neuro Oncol. 2023 Mar 14;25(3):533-543. doi: 10.1093/neuonc/noac189.

Authors

Philipp Vollmuth¹, Martha Foltyn¹, Raymond Y Huang², Norbert Galldiks^{3

4

5}, Jens Petersen⁶, Fabian Isensee⁶, Martin J van den Bent⁷, Frederik Barkhof^{8

9}, Ji Eun Park¹⁰, Yae Won Park¹¹, Sung Soo Ahn¹¹, Gianluca Brugnara¹, Hagen Meredig¹, Rajan Jain¹², Marion Smits¹³, Whitney B Pope¹⁴, Klaus Maier-Hein⁶, Michael Weller¹⁵, Patrick Y Wen¹⁶, Wolfgang Wick^{17

18}, Martin Bendszus¹

Affiliations

¹ Department of Neuroradiology, Heidelberg University Hospital, Heidelberg, Germany.
² Department of Radiology, Brigham and Women's Hospital, Boston, Massachusetts, USA.
³ Department of Neurology, Faculty of Medicine, University Hospital Cologne, University of Cologne, Cologne, Germany.
⁴ Institute of Neuroscience and Medicine (INM-3, -4), Research Center Juelich, Juelich, Germany.
⁵ Center for Integrated Oncology (CIO), Universities of Aachen, Bonn, Cologne, and Duesseldorf, Germany.
⁶ Department of Medical Image Computing (MIC), German Cancer Research Center (DKFZ), Heidelberg, Germany.
⁷ Brain Tumor Center, Erasmus MC Cancer Institute, Rotterdam, the Netherlands.
⁸ Department of Radiology & Nuclear Medicine, Amsterdam UMC, Vrije Universiteit, Amsterdam, the Netherlands.
⁹ Institutes of Neurology & Centre for Medical Image Computing, University College London, London, UK.
¹⁰ Department of Radiology and Research Institute of Radiology, Asan Medical Centre, University of Ulsan College of Medicine, Seoul, Republic of Korea.
¹¹ Department of Radiology and Research Institute of Radiological Science and Center for Clinical Imaging Data Science, Yonsei University College of Medicine, Seoul, Republic of Korea.
¹² Department of Radiology, New York University School of Medicine, New York, New York, USA.
¹³ Department of Radiology and Nuclear Medicine, Erasmus MC, Rotterdam, the Netherlands.
¹⁴ Department of Radiological Sciences, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, California, USA.
¹⁵ Department of Neurology, University Hospital and University of Zurich, Zurich, Switzerland.
¹⁶ Center for Neuro-oncology, Dana-Farber Cancer Institute, Boston, Massachusetts, USA.
¹⁷ Neurology Clinic, Heidelberg University Hospital, Heidelberg, Germany.
¹⁸ Clinical Cooperation Unit Neurooncology, German Cancer Consortium (DKTK) within the German Cancer Research Center (DKFZ), Heidelberg, Germany.

Abstract

Background: To assess whether artificial intelligence (AI)-based decision support allows more reproducible and standardized assessment of treatment response on MRI in neuro-oncology as compared to manual 2-dimensional measurements of tumor burden using the Response Assessment in Neuro-Oncology (RANO) criteria.

Methods: A series of 30 patients (15 lower-grade gliomas, 15 glioblastoma) with availability of consecutive MRI scans was selected. The time to progression (TTP) on MRI was separately evaluated for each patient by 15 investigators over two rounds. In the first round the TTP was evaluated based on the RANO criteria, whereas in the second round the TTP was evaluated by incorporating additional information from AI-enhanced MRI sequences depicting the longitudinal changes in tumor volumes. The agreement of the TTP measurements between investigators was evaluated using concordance correlation coefficients (CCC) with confidence intervals (CI) and P-values obtained using bootstrap resampling.

Results: The CCC of TTP-measurements between investigators was 0.77 (95% CI = 0.69,0.88) with RANO alone and increased to 0.91 (95% CI = 0.82,0.95) with AI-based decision support (P = .005). This effect was significantly greater (P = .008) for patients with lower-grade gliomas (CCC = 0.70 [95% CI = 0.56,0.85] without vs. 0.90 [95% CI = 0.76,0.95] with AI-based decision support) as compared to glioblastoma (CCC = 0.83 [95% CI = 0.75,0.92] without vs. 0.86 [95% CI = 0.78,0.93] with AI-based decision support). Investigators with less years of experience judged the AI-based decision as more helpful (P = .02).

Conclusions: AI-based decision support has the potential to yield more reproducible and standardized assessment of treatment response in neuro-oncology as compared to manual 2-dimensional measurements of tumor burden, particularly in patients with lower-grade gliomas. A fully-functional version of this AI-based processing pipeline is provided as open-source (https://github.com/NeuroAI-HD/HD-GLIO-XNAT).

Keywords: Artificial intelligence (AI)-based decision support; RANO; tumor response assessment; tumor volumetry.

Publication types

Research Support, Non-U.S. Gov't

MeSH terms

Artificial Intelligence
Brain Neoplasms* / diagnostic imaging
Brain Neoplasms* / pathology
Brain Neoplasms* / therapy
Glioblastoma* / pathology
Glioma* / diagnostic imaging
Glioma* / pathology
Glioma* / therapy
Humans
Reproducibility of Results

Grants and funding

P50 CA211015/CA/NCI NIH HHS/United States