Artificial intelligence (AI)-based decision support improves reproducibility of tumor response assessment in neuro-oncology: An international multi-reader study

Philipp Vollmuth, Martha Foltyn, Raymond Y. Huang, Norbert Galldiks, Jens Petersen, Fabian Isensee, Martin J. van den Bent, Frederik Barkhof, Ji Eun Park, Yae Won Park, Sung Soo Ahn, Gianluca Brugnara, Hagen Meredig, Rajan Jain, Marion Smits, Whitney B. Pope, Klaus Maier-Hein, Michael Weller, Patrick Y. Wen, Wolfgang WickMartin Bendszus

Research output: Contribution to journalArticleAcademicpeer-review

15 Citations (Scopus)


BACKGROUND: To assess whether artificial intelligence (AI)-based decision support allows more reproducible and standardized assessment of treatment response on MRI in neuro-oncology as compared to manual 2-dimensional measurements of tumor burden using the Response Assessment in Neuro-Oncology (RANO) criteria. METHODS: A series of 30 patients (15 lower-grade gliomas, 15 glioblastoma) with availability of consecutive MRI scans was selected. The time to progression (TTP) on MRI was separately evaluated for each patient by 15 investigators over two rounds. In the first round the TTP was evaluated based on the RANO criteria, whereas in the second round the TTP was evaluated by incorporating additional information from AI-enhanced MRI sequences depicting the longitudinal changes in tumor volumes. The agreement of the TTP measurements between investigators was evaluated using concordance correlation coefficients (CCC) with confidence intervals (CI) and P-values obtained using bootstrap resampling. RESULTS: The CCC of TTP-measurements between investigators was 0.77 (95% CI = 0.69,0.88) with RANO alone and increased to 0.91 (95% CI = 0.82,0.95) with AI-based decision support (P = .005). This effect was significantly greater (P = .008) for patients with lower-grade gliomas (CCC = 0.70 [95% CI = 0.56,0.85] without vs. 0.90 [95% CI = 0.76,0.95] with AI-based decision support) as compared to glioblastoma (CCC = 0.83 [95% CI = 0.75,0.92] without vs. 0.86 [95% CI = 0.78,0.93] with AI-based decision support). Investigators with less years of experience judged the AI-based decision as more helpful (P = .02). CONCLUSIONS: AI-based decision support has the potential to yield more reproducible and standardized assessment of treatment response in neuro-oncology as compared to manual 2-dimensional measurements of tumor burden, particularly in patients with lower-grade gliomas. A fully-functional version of this AI-based processing pipeline is provided as open-source (
Original languageEnglish
Pages (from-to)533-543
Number of pages11
Issue number3
Publication statusPublished - 14 Mar 2023


  • Artificial intelligence (AI)-based decision support
  • RANO
  • tumor response assessment
  • tumor volumetry

Cite this