1
|
Wahlig SG, Nedelec P, Weiss DA, Rudie JD, Sugrue LP, Rauschecker AM. 3D U-Net for automated detection of multiple sclerosis lesions: utility of transfer learning from other pathologies. Front Neurosci 2023; 17:1188336. [PMID: 37965219 PMCID: PMC10641790 DOI: 10.3389/fnins.2023.1188336] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2023] [Accepted: 09/26/2023] [Indexed: 11/16/2023] Open
Abstract
Background and purpose Deep learning algorithms for segmentation of multiple sclerosis (MS) plaques generally require training on large datasets. This manuscript evaluates the effect of transfer learning from segmentation of another pathology to facilitate use of smaller MS-specific training datasets. That is, a model trained for detection of one type of pathology was re-trained to identify MS lesions and active demyelination. Materials and methods In this retrospective study using MRI exams from 149 patients spanning 4/18/2014 to 7/8/2021, 3D convolutional neural networks were trained with a variable number of manually-segmented MS studies. Models were trained for FLAIR lesion segmentation at a single timepoint, new FLAIR lesion segmentation comparing two timepoints, and enhancing (actively demyelinating) lesion segmentation on T1 post-contrast imaging. Models were trained either de-novo or fine-tuned with transfer learning applied to a pre-existing model initially trained on non-MS data. Performance was evaluated with lesionwise sensitivity and positive predictive value (PPV). Results For single timepoint FLAIR lesion segmentation with 10 training studies, a fine-tuned model demonstrated improved performance [lesionwise sensitivity 0.55 ± 0.02 (mean ± standard error), PPV 0.66 ± 0.02] compared to a de-novo model (sensitivity 0.49 ± 0.02, p = 0.001; PPV 0.32 ± 0.02, p < 0.001). For new lesion segmentation with 30 training studies and their prior comparisons, a fine-tuned model demonstrated similar sensitivity (0.49 ± 0.05) and significantly improved PPV (0.60 ± 0.05) compared to a de-novo model (sensitivity 0.51 ± 0.04, p = 0.437; PPV 0.43 ± 0.04, p = 0.002). For enhancement segmentation with 20 training studies, a fine-tuned model demonstrated significantly improved overall performance (sensitivity 0.74 ± 0.06, PPV 0.69 ± 0.05) compared to a de-novo model (sensitivity 0.44 ± 0.09, p = 0.001; PPV 0.37 ± 0.05, p = 0.001). Conclusion By fine-tuning models trained for other disease pathologies with MS-specific data, competitive models identifying existing MS plaques, new MS plaques, and active demyelination can be built with substantially smaller datasets than would otherwise be required to train new models.
Collapse
Affiliation(s)
- Stephen G. Wahlig
- Center for Intelligent Imaging (ci), Department of Radiology and Biomedical Imaging, University of California, San Francisco, San Francisco, CA, United States
| | - Pierre Nedelec
- Center for Intelligent Imaging (ci), Department of Radiology and Biomedical Imaging, University of California, San Francisco, San Francisco, CA, United States
| | - David A. Weiss
- Center for Intelligent Imaging (ci), Department of Radiology and Biomedical Imaging, University of California, San Francisco, San Francisco, CA, United States
| | - Jeffrey D. Rudie
- Center for Intelligent Imaging (ci), Department of Radiology and Biomedical Imaging, University of California, San Francisco, San Francisco, CA, United States
- Department of Radiology, University of California, San Diego, San Diego, CA, United States
| | - Leo P. Sugrue
- Center for Intelligent Imaging (ci), Department of Radiology and Biomedical Imaging, University of California, San Francisco, San Francisco, CA, United States
| | - Andreas M. Rauschecker
- Center for Intelligent Imaging (ci), Department of Radiology and Biomedical Imaging, University of California, San Francisco, San Francisco, CA, United States
| |
Collapse
|
2
|
Spagnolo F, Depeursinge A, Schädelin S, Akbulut A, Müller H, Barakovic M, Melie-Garcia L, Bach Cuadra M, Granziera C. How far MS lesion detection and segmentation are integrated into the clinical workflow? A systematic review. Neuroimage Clin 2023; 39:103491. [PMID: 37659189 PMCID: PMC10480555 DOI: 10.1016/j.nicl.2023.103491] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2023] [Accepted: 08/04/2023] [Indexed: 09/04/2023]
Abstract
INTRODUCTION Over the past few years, the deep learning community has developed and validated a plethora of tools for lesion detection and segmentation in Multiple Sclerosis (MS). However, there is an important gap between validating models technically and clinically. To this end, a six-step framework necessary for the development, validation, and integration of quantitative tools in the clinic was recently proposed under the name of the Quantitative Neuroradiology Initiative (QNI). AIMS Investigate to what extent automatic tools in MS fulfill the QNI framework necessary to integrate automated detection and segmentation into the clinical neuroradiology workflow. METHODS Adopting the systematic Cochrane literature review methodology, we screened and summarised published scientific articles that perform automatic MS lesions detection and segmentation. We categorised the retrieved studies based on their degree of fulfillment of QNI's six-steps, which include a tool's technical assessment, clinical validation, and integration. RESULTS We found 156 studies; 146/156 (94%) fullfilled the first QNI step, 155/156 (99%) the second, 8/156 (5%) the third, 3/156 (2%) the fourth, 5/156 (3%) the fifth and only one the sixth. CONCLUSIONS To date, little has been done to evaluate the clinical performance and the integration in the clinical workflow of available methods for MS lesion detection/segmentation. In addition, the socio-economic effects and the impact on patients' management of such tools remain almost unexplored.
Collapse
Affiliation(s)
- Federico Spagnolo
- Translational Imaging in Neurology (ThINK) Basel, Department of Biomedical Engineering, Faculty of Medicine, University Hospital Basel and University of Basel, Basel, Switzerland; Department of Neurology, University Hospital Basel, Basel, Switzerland; Research Center for Clinical Neuroimmunology and Neuroscience Basel (RC2NB), University Hospital Basel and University of Basel, Basel, Switzerland; MedGIFT, Institute of Informatics, School of Management, HES-SO Valais-Wallis University of Applied Sciences and Arts Western Switzerland, Sierre, Switzerland
| | - Adrien Depeursinge
- MedGIFT, Institute of Informatics, School of Management, HES-SO Valais-Wallis University of Applied Sciences and Arts Western Switzerland, Sierre, Switzerland; Nuclear Medicine and Molecular Imaging Department, Lausanne University Hospital (CHUV) and University of Lausanne, Lausanne, Switzerland
| | - Sabine Schädelin
- Translational Imaging in Neurology (ThINK) Basel, Department of Biomedical Engineering, Faculty of Medicine, University Hospital Basel and University of Basel, Basel, Switzerland; Clinical Trial Unit, Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland
| | - Aysenur Akbulut
- Translational Imaging in Neurology (ThINK) Basel, Department of Biomedical Engineering, Faculty of Medicine, University Hospital Basel and University of Basel, Basel, Switzerland; Ankara University School of Medicine, Ankara, Turkey
| | - Henning Müller
- MedGIFT, Institute of Informatics, School of Management, HES-SO Valais-Wallis University of Applied Sciences and Arts Western Switzerland, Sierre, Switzerland; The Sense Research and Innovation Center, Lausanne and Sion, Switzerland
| | - Muhamed Barakovic
- Translational Imaging in Neurology (ThINK) Basel, Department of Biomedical Engineering, Faculty of Medicine, University Hospital Basel and University of Basel, Basel, Switzerland; Department of Neurology, University Hospital Basel, Basel, Switzerland; Research Center for Clinical Neuroimmunology and Neuroscience Basel (RC2NB), University Hospital Basel and University of Basel, Basel, Switzerland
| | - Lester Melie-Garcia
- Translational Imaging in Neurology (ThINK) Basel, Department of Biomedical Engineering, Faculty of Medicine, University Hospital Basel and University of Basel, Basel, Switzerland; Department of Neurology, University Hospital Basel, Basel, Switzerland; Research Center for Clinical Neuroimmunology and Neuroscience Basel (RC2NB), University Hospital Basel and University of Basel, Basel, Switzerland
| | - Meritxell Bach Cuadra
- CIBM Center for Biomedical Imaging, Lausanne, Switzerland; Radiology Department, Lausanne University Hospital (CHUV) and University of Lausanne, Lausanne, Switzerland
| | - Cristina Granziera
- Translational Imaging in Neurology (ThINK) Basel, Department of Biomedical Engineering, Faculty of Medicine, University Hospital Basel and University of Basel, Basel, Switzerland; Department of Neurology, University Hospital Basel, Basel, Switzerland; Research Center for Clinical Neuroimmunology and Neuroscience Basel (RC2NB), University Hospital Basel and University of Basel, Basel, Switzerland.
| |
Collapse
|
3
|
Elbaz M, Yassin S, Magdy S, Elbadawy E, Mohamed A, Elwahash H. Novel framework for Detecting Multiple Sclerosis using Hybrid models. 2022 32nd International Conference on Computer Theory and Applications (ICCTA) 2022. [DOI: 10.1109/iccta58027.2022.10206298] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/01/2023]
Affiliation(s)
- Mostafa Elbaz
- Kafrelsheikh University,Faculty of Computers and Informatics,Department of Computer Science,kafrelsheikh,Egypt
| | - Sherouk Yassin
- Kafrelsheikh University,Faculty of Computers and Informatics,Department of bio-informatics,kafrelsheikh,Egypt
| | - Sherief Magdy
- Kafrelsheikh University,Faculty of Computers and Informatics,Department of bio-informatics,kafrelsheikh,Egypt
| | - Eman Elbadawy
- Kafrelsheikh University,Faculty of Computers and Informatics,Department of bio-informatics,kafrelsheikh,Egypt
| | - Asmaa Mohamed
- Kafrelsheikh University,Faculty of Computers and Informatics,Department of bio-informatics,kafrelsheikh,Egypt
| | - Hitham Elwahash
- Kafrelsheikh University,Faculty of Computers and Informatics,Department of Computer Science,kafrelsheikh,Egypt
| |
Collapse
|
4
|
Nguyen TH, Vaussy A, Le Gaudu V, Aboab J, Espinoza S, Curajos I, Heron E, Habas C. The brainstem in multiple sclerosis: MR identification of tracts and nuclei damage. Insights Imaging 2021; 12:151. [PMID: 34674050 PMCID: PMC8531176 DOI: 10.1186/s13244-021-01101-7] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2021] [Accepted: 08/23/2021] [Indexed: 01/04/2023] Open
Abstract
Objective To evaluate the 3D Fast Gray Acquisition T1 Inversion Recovery (FGATIR) sequence for MRI identification of brainstem tracts and nuclei damage in multiple sclerosis (MS) patients. Methods From april to december 2020, 10 healthy volunteers and 50 patients with remitted-relapsing MS (58% female, mean age 36) underwent MR imaging in the Neuro-imaging department of the C.H.N.O. des Quinze-Vingts, Paris, France. MRI was achieved on a 3 T system (MAGNETOM Skyra) using a 64-channel coil. 3D FGATIR sequence was first performed on healthy volunteers to classify macroscopically identifiable brainstem structures. Then, FGATIR was assessed in MS patients to locate brainstem lesions detected with Proton Density/T2w (PD/T2w) sequence. Results In healthy volunteers, FGATIR allowed a precise visualization of tracts and nuclei according to their myelin density. Including FGATIR in MR follow-up of MS patients helped to identify structures frequently involved in the inflammatory process. Most damaged tracts were the superior cerebellar peduncle and the transverse fibers of the pons. Most frequently affected nuclei were the vestibular nuclei, the trigeminal tract, the facial nerve and the solitary tract. Conclusion Combination of FGATIR and PD/T2w sequences opened prospects to define MS elective injury in brainstem tracts and nuclei, with particular lesion features suggesting variations of the inflammatory process within brainstem structures. In a further study, hypersignal quantification and microstructure information should be evaluated using relaxometry and diffusion tractography. Technical improvements would bring novel parameters to train an artificial neural network for accurate automated labeling of MS lesions within the brainstem.
Collapse
Affiliation(s)
- Thien Huong Nguyen
- Department of Neuro Imaging, C.H.N.O. des Quinze- Vingts, Paris, France.
| | | | - Violette Le Gaudu
- Department of Neuro Imaging, C.H.N.O. des Quinze- Vingts, Paris, France
| | - Jennifer Aboab
- Department of Internal Medicine, C.H.N.O. des Quinze-Vingts, Paris, France
| | - Sophie Espinoza
- Department of Neuro Imaging, C.H.N.O. des Quinze- Vingts, Paris, France
| | - Irina Curajos
- Department of Neuro Imaging, C.H.N.O. des Quinze- Vingts, Paris, France
| | - Emmanuel Heron
- Department of Internal Medicine, C.H.N.O. des Quinze-Vingts, Paris, France
| | - Christophe Habas
- Department of Neuro Imaging, C.H.N.O. des Quinze- Vingts, Paris, France
| |
Collapse
|
5
|
Moazami F, Lefevre-Utile A, Papaloukas C, Soumelis V. Machine Learning Approaches in Study of Multiple Sclerosis Disease Through Magnetic Resonance Images. Front Immunol 2021; 12:700582. [PMID: 34456913 PMCID: PMC8385534 DOI: 10.3389/fimmu.2021.700582] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2021] [Accepted: 07/26/2021] [Indexed: 11/13/2022] Open
Abstract
Multiple sclerosis (MS) is one of the most common autoimmune diseases which is commonly diagnosed and monitored using magnetic resonance imaging (MRI) with a combination of clinical manifestations. The purpose of this review is to highlight the main applications of Machine Learning (ML) models and their performance in the MS field using MRI. We reviewed the articles of the last decade and grouped them based on the applications of ML in MS using MRI data into four categories: 1) Automated diagnosis of MS, 2) Prediction of MS disease progression, 3) Differentiation of MS stages, 4) Differentiation of MS from similar disorders.
Collapse
Affiliation(s)
- Faezeh Moazami
- Université de Paris, Institut de Recherche Saint-Louis, Institut National de la Santé et de la Recherche Médicale (INSERM) U976, Hôpital Saint-Louis, Paris, France
| | - Alain Lefevre-Utile
- Université de Paris, Institut de Recherche Saint-Louis, Institut National de la Santé et de la Recherche Médicale (INSERM) U976, Hôpital Saint-Louis, Paris, France.,Université Paris-Saclay, Saint Aubin, France.,Assistance Publique Hopitaux de Paris (APHP), General Pediatric and Pediatric Emergency Department, Jean Verdier Hospital, Bondy, France
| | - Costas Papaloukas
- Department of Biological Applications and Technology, University of Ioannina, Ioannina, Greece
| | - Vassili Soumelis
- Université de Paris, Institut de Recherche Saint-Louis, Institut National de la Santé et de la Recherche Médicale (INSERM) U976, Hôpital Saint-Louis, Paris, France.,Assistance Publique Hopitaux de Paris (APHP), Hôpital Saint-Louis, Immunology-Histocompatibility Department, Paris, France
| |
Collapse
|