1
|
Pemberton HG, Wu J, Kommers I, Müller DMJ, Hu Y, Goodkin O, Vos SB, Bisdas S, Robe PA, Ardon H, Bello L, Rossi M, Sciortino T, Nibali MC, Berger MS, Hervey-Jumper SL, Bouwknegt W, Van den Brink WA, Furtner J, Han SJ, Idema AJS, Kiesel B, Widhalm G, Kloet A, Wagemakers M, Zwinderman AH, Krieg SM, Mandonnet E, Prados F, de Witt Hamer P, Barkhof F, Eijgelaar RS. Multi-class glioma segmentation on real-world data with missing MRI sequences: comparison of three deep learning algorithms. Sci Rep 2023; 13:18911. [PMID: 37919354 PMCID: PMC10622563 DOI: 10.1038/s41598-023-44794-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2023] [Accepted: 10/12/2023] [Indexed: 11/04/2023] Open
Abstract
This study tests the generalisability of three Brain Tumor Segmentation (BraTS) challenge models using a multi-center dataset of varying image quality and incomplete MRI datasets. In this retrospective study, DeepMedic, no-new-Unet (nn-Unet), and NVIDIA-net (nv-Net) were trained and tested using manual segmentations from preoperative MRI of glioblastoma (GBM) and low-grade gliomas (LGG) from the BraTS 2021 dataset (1251 in total), in addition to 275 GBM and 205 LGG acquired clinically across 12 hospitals worldwide. Data was split into 80% training, 5% validation, and 15% internal test data. An additional external test-set of 158 GBM and 69 LGG was used to assess generalisability to other hospitals' data. All models' median Dice similarity coefficient (DSC) for both test sets were within, or higher than, previously reported human inter-rater agreement (range of 0.74-0.85). For both test sets, nn-Unet achieved the highest DSC (internal = 0.86, external = 0.93) and the lowest Hausdorff distances (10.07, 13.87 mm, respectively) for all tumor classes (p < 0.001). By applying Sparsified training, missing MRI sequences did not statistically affect the performance. nn-Unet achieves accurate segmentations in clinical settings even in the presence of incomplete MRI datasets. This facilitates future clinical adoption of automated glioma segmentation, which could help inform treatment planning and glioma monitoring.
Collapse
Affiliation(s)
- Hugh G Pemberton
- Centre for Medical Image Computing (CMIC), University College London, London, UK
- Neuroradiological Academic Unit, UCL Queen Square Institute of Neurology, University College London, London, UK
| | - Jiaming Wu
- Centre for Medical Image Computing (CMIC), University College London, London, UK
| | - Ivar Kommers
- Neurosurgical Center Amsterdam, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands
| | - Domenique M J Müller
- Neurosurgical Center Amsterdam, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands
| | - Yipeng Hu
- Centre for Medical Image Computing (CMIC), University College London, London, UK
| | - Olivia Goodkin
- Centre for Medical Image Computing (CMIC), University College London, London, UK
- Neuroradiological Academic Unit, UCL Queen Square Institute of Neurology, University College London, London, UK
| | - Sjoerd B Vos
- Centre for Medical Image Computing (CMIC), University College London, London, UK
- Neuroradiological Academic Unit, UCL Queen Square Institute of Neurology, University College London, London, UK
| | - Sotirios Bisdas
- Neuroradiological Academic Unit, UCL Queen Square Institute of Neurology, University College London, London, UK
| | - Pierre A Robe
- Department of Neurology & Neurosurgery, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Hilko Ardon
- Department of Neurosurgery, St. Elisabeth Hospital, Tilburg, The Netherlands
| | - Lorenzo Bello
- Neurosurgical Oncology Unit, Department of Oncology and Hemato-Oncology, Università degli Studi di Milano, Milan, Italy
| | - Marco Rossi
- Neurosurgical Oncology Unit, Department of Oncology and Hemato-Oncology, Università degli Studi di Milano, Milan, Italy
| | - Tommaso Sciortino
- Neurosurgical Oncology Unit, Department of Oncology and Hemato-Oncology, Università degli Studi di Milano, Milan, Italy
| | - Marco Conti Nibali
- Neurosurgical Oncology Unit, Department of Oncology and Hemato-Oncology, Università degli Studi di Milano, Milan, Italy
| | - Mitchel S Berger
- Department of Neurological Surgery, University of California, San Francisco, CA, USA
| | - Shawn L Hervey-Jumper
- Department of Neurological Surgery, University of California, San Francisco, CA, USA
| | - Wim Bouwknegt
- Department of Neurosurgery, Medical Center Slotervaart, Amsterdam, The Netherlands
| | | | - Julia Furtner
- Department of Biomedical Imaging and Image-Guided Therapy, Medical University Vienna, Vienna, Austria
| | - Seunggu J Han
- Department of Neurological Surgery, Stanford University, Stanford, USA
| | - Albert J S Idema
- Department of Neurosurgery, Northwest Clinics, Alkmaar, The Netherlands
| | - Barbara Kiesel
- Department of Neurosurgery, Medical University Vienna, Vienna, Austria
| | - Georg Widhalm
- Department of Neurosurgery, Medical University Vienna, Vienna, Austria
| | - Alfred Kloet
- Department of Neurosurgery, Medical Center Haaglanden, The Hague, The Netherlands
| | - Michiel Wagemakers
- Department of Neurosurgery, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands
| | - Aeilko H Zwinderman
- Department of Clinical Epidemiology and Biostatistics, Academic Medical Center, Amsterdam, The Netherlands
| | - Sandro M Krieg
- TUM-Neuroimaging Center, Klinikum rechts der Isar, Technische Universität München, Munich, Germany
- Department of Neurosurgery, Klinikum rechts der Isar, Technische Universität München, Munich, Germany
| | | | - Ferran Prados
- Centre for Medical Image Computing (CMIC), University College London, London, UK
- Department of Neuroinflammation, Faculty of Brain Sciences, Queen Square MS Centre, UCL Institute of Neurology, University College London, London, UK
- e-Health Center, Universitat Oberta de Catalunya, Barcelona, Spain
| | - Philip de Witt Hamer
- Neurosurgical Center Amsterdam, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands
| | - Frederik Barkhof
- Centre for Medical Image Computing (CMIC), University College London, London, UK
- Neuroradiological Academic Unit, UCL Queen Square Institute of Neurology, University College London, London, UK
- Radiology & Nuclear Medicine, VU University Medical Center, Amsterdam, the Netherlands
| | - Roelant S Eijgelaar
- Neurosurgical Center Amsterdam, Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands.
| |
Collapse
|
2
|
Helland RH, Ferles A, Pedersen A, Kommers I, Ardon H, Barkhof F, Bello L, Berger MS, Dunås T, Nibali MC, Furtner J, Hervey-Jumper S, Idema AJS, Kiesel B, Tewari RN, Mandonnet E, Müller DMJ, Robe PA, Rossi M, Sagberg LM, Sciortino T, Aalders T, Wagemakers M, Widhalm G, Witte MG, Zwinderman AH, Majewska PL, Jakola AS, Solheim O, Hamer PCDW, Reinertsen I, Eijgelaar RS, Bouget D. Segmentation of glioblastomas in early post-operative multi-modal MRI with deep neural networks. Sci Rep 2023; 13:18897. [PMID: 37919325 PMCID: PMC10622432 DOI: 10.1038/s41598-023-45456-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2023] [Accepted: 10/19/2023] [Indexed: 11/04/2023] Open
Abstract
Extent of resection after surgery is one of the main prognostic factors for patients diagnosed with glioblastoma. To achieve this, accurate segmentation and classification of residual tumor from post-operative MR images is essential. The current standard method for estimating it is subject to high inter- and intra-rater variability, and an automated method for segmentation of residual tumor in early post-operative MRI could lead to a more accurate estimation of extent of resection. In this study, two state-of-the-art neural network architectures for pre-operative segmentation were trained for the task. The models were extensively validated on a multicenter dataset with nearly 1000 patients, from 12 hospitals in Europe and the United States. The best performance achieved was a 61% Dice score, and the best classification performance was about 80% balanced accuracy, with a demonstrated ability to generalize across hospitals. In addition, the segmentation performance of the best models was on par with human expert raters. The predicted segmentations can be used to accurately classify the patients into those with residual tumor, and those with gross total resection.
Collapse
Affiliation(s)
- Ragnhild Holden Helland
- Department of Health Research, SINTEF Digital, 7465, Trondheim, Norway.
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, NO-7491, Trondheim, Norway.
| | - Alexandros Ferles
- Cancer Center Amsterdam, Brain Tumor Center, Amsterdam University Medical Centers, 1081 HV, Amsterdam, The Netherlands
- Department of Radiology and Nuclear Medicine, Amsterdam University Medical Centers, Vrije Universiteit, 1081 HV, Amsterdam, The Netherlands
| | - André Pedersen
- Department of Health Research, SINTEF Digital, 7465, Trondheim, Norway
| | - Ivar Kommers
- Cancer Center Amsterdam, Brain Tumor Center, Amsterdam University Medical Centers, 1081 HV, Amsterdam, The Netherlands
- Department of Neurosurgery, Amsterdam University Medical Centers, Vrije Universiteit, 1081 HV, Amsterdam, The Netherlands
| | - Hilko Ardon
- Department of Neurosurgery, Twee Steden Hospital, 5042 AD, Tilburg, The Netherlands
| | - Frederik Barkhof
- Department of Radiology and Nuclear Medicine, Amsterdam University Medical Centers, Vrije Universiteit, 1081 HV, Amsterdam, The Netherlands
- Institutes of Neurology and Healthcare Engineering, University College London, London, WC1E 6BT, UK
| | - Lorenzo Bello
- Neurosurgical Oncology Unit, Department of Oncology and Hemato-oncology, Humanitas Research Hospital, Università Degli Studi di Milano, 20122, Milan, Italy
| | - Mitchel S Berger
- Department of Neurological Surgery, University of California San Francisco, San Francisco, CA, 94143, USA
| | - Tora Dunås
- Department of Clinical Neuroscience, Institute of Neuroscience and Physiology, Sahlgrenska Academy, University of Gothenburg, 405 30, Gothenburg, Sweden
| | | | - Julia Furtner
- Department of Biomedical Imaging and Image-guided Therapy, Medical University Vienna, 1090, Vienna, Austria
- Research Center for Medical Image Analysis and Artificial Intelligence (MIAAI), Faculty of Medicine and Dentistry, Danube Private University, 3500, Krems, Austria
| | - Shawn Hervey-Jumper
- Department of Neurological Surgery, University of California San Francisco, San Francisco, CA, 94143, USA
| | - Albert J S Idema
- Department of Neurosurgery, Northwest Clinics, 1815 JD, Alkmaar, The Netherlands
| | - Barbara Kiesel
- Department of Neurosurgery, Medical University Vienna, 1090, Vienna, Austria
| | - Rishi Nandoe Tewari
- Department of Neurosurgery, Haaglanden Medical Center, 2512 VA, The Hague, The Netherlands
| | - Emmanuel Mandonnet
- Department of Neurological Surgery, Hôpital Lariboisière, 75010, Paris, France
| | - Domenique M J Müller
- Cancer Center Amsterdam, Brain Tumor Center, Amsterdam University Medical Centers, 1081 HV, Amsterdam, The Netherlands
- Department of Neurosurgery, Amsterdam University Medical Centers, Vrije Universiteit, 1081 HV, Amsterdam, The Netherlands
| | - Pierre A Robe
- Department of Neurology and Neurosurgery, University Medical Center Utrecht, 3584 CX, Utrecht, The Netherlands
| | - Marco Rossi
- Department of Medical Biotechnology and Translational Medicine, Università Degli Studi di Milano, 20122, Milan, Italy
| | - Lisa M Sagberg
- Department of Neurosurgery, St. Olavs hospital, Trondheim University Hospital, 7030, Trondheim, Norway
- Department of Public Health and Nursing, Norwegian University of Science and Technology, 7491, Trondheim, Norway
| | | | - Tom Aalders
- Department of Neurosurgery, Isala, 8025 AB, Zwolle, The Netherlands
| | - Michiel Wagemakers
- Department of Neurosurgery, University Medical Center Groningen, University of Groningen, 9713 GZ, Groningen, The Netherlands
| | - Georg Widhalm
- Department of Neurosurgery, Medical University Vienna, 1090, Vienna, Austria
| | - Marnix G Witte
- Department of Radiation Oncology, The Netherlands Cancer Institute, 1066 CX, Amsterdam, The Netherlands
| | - Aeilko H Zwinderman
- Department of Clinical Epidemiology and Biostatistics, Amsterdam University Medical Centers, University of Amsterdam, 1105 AZ, Amsterdam, The Netherlands
| | - Paulina L Majewska
- Department of Neurology and Neurosurgery, University Medical Center Utrecht, 3584 CX, Utrecht, The Netherlands
- Department of Neuromedicine and Movement Science, Norwegian University of Science and Technology, 7491, Trondheim, Norway
| | - Asgeir S Jakola
- Department of Clinical Neuroscience, Institute of Neuroscience and Physiology, Sahlgrenska Academy, University of Gothenburg, 405 30, Gothenburg, Sweden
- Department of Neurosurgery, Sahlgrenska University Hospital, Gothenburg, Sweden
| | - Ole Solheim
- Department of Neurology and Neurosurgery, University Medical Center Utrecht, 3584 CX, Utrecht, The Netherlands
- Department of Neuromedicine and Movement Science, Norwegian University of Science and Technology, 7491, Trondheim, Norway
| | - Philip C De Witt Hamer
- Cancer Center Amsterdam, Brain Tumor Center, Amsterdam University Medical Centers, 1081 HV, Amsterdam, The Netherlands
- Department of Neurosurgery, Amsterdam University Medical Centers, Vrije Universiteit, 1081 HV, Amsterdam, The Netherlands
| | - Ingerid Reinertsen
- Department of Health Research, SINTEF Digital, 7465, Trondheim, Norway
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, NO-7491, Trondheim, Norway
| | - Roelant S Eijgelaar
- Cancer Center Amsterdam, Brain Tumor Center, Amsterdam University Medical Centers, 1081 HV, Amsterdam, The Netherlands
- Department of Neurosurgery, Amsterdam University Medical Centers, Vrije Universiteit, 1081 HV, Amsterdam, The Netherlands
| | - David Bouget
- Department of Health Research, SINTEF Digital, 7465, Trondheim, Norway
| |
Collapse
|