Predicting outcome in clinically isolated syndrome using machine learning

A systematic review of the application of machine-learning algorithms in multiple sclerosis

INTRODUCTION

The applications of artificial intelligence, and in particular automatic learning or "machine learning" (ML), constitute both a challenge and a great opportunity in numerous scientific, technical, and clinical disciplines. Specific applications in the study of multiple sclerosis (MS) have been no exception, and constitute an area of increasing interest in recent years.

OBJECTIVE

We present a systematic review of the application of ML algorithms in MS.

MATERIALS AND METHODS

We used the PubMed search engine, which allows free access to the MEDLINE medical database, to identify studies including the keywords "machine learning" and "multiple sclerosis." We excluded review articles, studies written in languages other than English or Spanish, and studies that were mainly technical and did not specifically apply to MS. The final selection included 76 articles, and 38 were rejected.

CONCLUSIONS

After the review process, we established 4 main applications of ML in MS: 1) classifying MS subtypes; 2) distinguishing patients with MS from healthy controls and individuals with other diseases; 3) predicting progression and response to therapeutic interventions; and 4) other applications. Results found to date have shown that ML algorithms may offer great support for health professionals both in clinical settings and in research into MS.

Prognostic models for predicting clinical disease progression, worsening and activity in people with multiple sclerosis

BACKGROUND

Multiple sclerosis (MS) is a chronic inflammatory disease of the central nervous system that affects millions of people worldwide. The disease course varies greatly across individuals and many disease-modifying treatments with different safety and efficacy profiles have been developed recently. Prognostic models evaluated and shown to be valid in different settings have the potential to support people with MS and their physicians during the decision-making process for treatment or disease/life management, allow stratified and more precise interpretation of interventional trials, and provide insights into disease mechanisms. Many researchers have turned to prognostic models to help predict clinical outcomes in people with MS; however, to our knowledge, no widely accepted prognostic model for MS is being used in clinical practice yet.

OBJECTIVES

To identify and summarise multivariable prognostic models, and their validation studies for quantifying the risk of clinical disease progression, worsening, and activity in adults with MS.

SEARCH METHODS

We searched MEDLINE, Embase, and the Cochrane Database of Systematic Reviews from January 1996 until July 2021. We also screened the reference lists of included studies and relevant reviews, and references citing the included studies.

SELECTION CRITERIA

We included all statistically developed multivariable prognostic models aiming to predict clinical disease progression, worsening, and activity, as measured by disability, relapse, conversion to definite MS, conversion to progressive MS, or a composite of these in adult individuals with MS. We also included any studies evaluating the performance of (i.e. validating) these models. There were no restrictions based on language, data source, timing of prognostication, or timing of outcome.

DATA COLLECTION AND ANALYSIS

Pairs of review authors independently screened titles/abstracts and full texts, extracted data using a piloted form based on the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS), assessed risk of bias using the Prediction Model Risk Of Bias Assessment Tool (PROBAST), and assessed reporting deficiencies based on the checklist items in Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD). The characteristics of the included models and their validations are described narratively. We planned to meta-analyse the discrimination and calibration of models with at least three external validations outside the model development study but no model met this criterion. We summarised between-study heterogeneity narratively but again could not perform the planned meta-regression.

MAIN RESULTS

We included 57 studies, from which we identified 75 model developments, 15 external validations corresponding to only 12 (16%) of the models, and six author-reported validations. Only two models were externally validated multiple times. None of the identified external validations were performed by researchers independent of those that developed the model. The outcome was related to disease progression in 39 (41%), relapses in 8 (8%), conversion to definite MS in 17 (18%), and conversion to progressive MS in 27 (28%) of the 96 models or validations. The disease and treatment-related characteristics of included participants, and definitions of considered predictors and outcome, were highly heterogeneous amongst the studies. Based on the publication year, we observed an increase in the percent of participants on treatment, diversification of the diagnostic criteria used, an increase in consideration of biomarkers or treatment as predictors, and increased use of machine learning methods over time. Usability and reproducibility All identified models contained at least one predictor requiring the skills of a medical specialist for measurement or assessment. Most of the models (44; 59%) contained predictors that require specialist equipment likely to be absent from primary care or standard hospital settings. Over half (52%) of the developed models were not accompanied by model coefficients, tools, or instructions, which hinders their application, independent validation or reproduction. The data used in model developments were made publicly available or reported to be available on request only in a few studies (two and six, respectively). Risk of bias We rated all but one of the model developments or validations as having high overall risk of bias. The main reason for this was the statistical methods used for the development or evaluation of prognostic models; we rated all but two of the included model developments or validations as having high risk of bias in the analysis domain. None of the model developments that were externally validated or these models' external validations had low risk of bias. There were concerns related to applicability of the models to our research question in over one-third (38%) of the models or their validations. Reporting deficiencies Reporting was poor overall and there was no observable increase in the quality of reporting over time. The items that were unclearly reported or not reported at all for most of the included models or validations were related to sample size justification, blinding of outcome assessors, details of the full model or how to obtain predictions from it, amount of missing data, and treatments received by the participants. Reporting of preferred model performance measures of discrimination and calibration was suboptimal.

AUTHORS' CONCLUSIONS

The current evidence is not sufficient for recommending the use of any of the published prognostic prediction models for people with MS in clinical routine today due to lack of independent external validations. The MS prognostic research community should adhere to the current reporting and methodological guidelines and conduct many more state-of-the-art external validation studies for the existing or newly developed models.

Subject-specific whole-brain parcellations of nodes and boundaries are modulated differently under 10 Hz rTMS

Repetitive transcranial magnetic stimulation (rTMS) has gained considerable importance in the treatment of neuropsychiatric disorders, including major depression. However, it is not yet understood how rTMS alters brain's functional connectivity. Here we report changes in functional connectivity captured by resting state functional magnetic resonance imaging (rsfMRI) within the first hour after 10 Hz rTMS. We apply subject-specific parcellation schemes to detect changes (1) in network nodes, where the strongest functional connectivity of regions is observed, and (2) in network boundaries, where functional transitions between regions occur. We use support vector machine (SVM), a widely used machine learning algorithm that is robust and effective, for the classification and characterization of time intervals of changes in node and boundary maps. Our results reveal that changes in connectivity at the boundaries are slower and more complex than in those observed in the nodes, but of similar magnitude according to accuracy confidence intervals. These results were strongest in the posterior cingulate cortex and precuneus. As network boundaries are indeed under-investigated in comparison to nodes in connectomics research, our results highlight their contribution to functional adjustments to rTMS.

Accounting for uncertainty in training data to improve machine learning performance in predicting new disease activity in early multiple sclerosis

Introduction

Machine learning (ML) has great potential for using health data to predict clinical outcomes in individual patients. Missing data are a common challenge in training ML algorithms, such as when subjects withdraw from a clinical study, leaving some samples with missing outcome labels. In this study, we have compared three ML models to determine whether accounting for label uncertainty can improve a model's predictions.

Methods

We used a dataset from a completed phase-III clinical trial that evaluated the efficacy of minocycline for delaying the conversion from clinically isolated syndrome to multiple sclerosis (MS), using the McDonald 2005 diagnostic criteria. There were a total of 142 participants, and at the 2-year follow-up 81 had converted to MS, 29 remained stable, and 32 had uncertain outcomes. In a stratified 7-fold cross-validation, we trained three random forest (RF) ML models using MRI volumetric features and clinical variables to predict the conversion outcome, which represented new disease activity within 2 years of a first clinical demyelinating event. One RF was trained using subjects with the uncertain labels excluded (RF_exclude), another RF was trained using the entire dataset but with assumed labels for the uncertain group (RF_naive), and a third, a probabilistic RF (PRF, a type of RF that can model label uncertainty) was trained on the entire dataset, with probabilistic labels assigned to the uncertain group.

Results

Probabilistic random forest outperformed both the RF models with the highest AUC (0.76, compared to 0.69 for RF_exclude and 0.71 for RF_naive) and F1-score (86.6% compared to 82.6% for RF_exclude and 76.8% for RF_naive).

Conclusion

Machine learning algorithms capable of modeling label uncertainty can improve predictive performance in datasets in which a substantial number of subjects have unknown outcomes.

Patterns of inflammation, microstructural alterations, and sodium accumulation define multiple sclerosis subtypes after 15 years from onset

Introduction

Conventional MRI is routinely used for the characterization of pathological changes in multiple sclerosis (MS), but due to its lack of specificity is unable to provide accurate prognoses, explain disease heterogeneity and reconcile the gap between observed clinical symptoms and radiological evidence. Quantitative MRI provides measures of physiological abnormalities, otherwise invisible to conventional MRI, that correlate with MS severity. Analyzing quantitative MRI measures through machine learning techniques has been shown to improve the understanding of the underlying disease by better delineating its alteration patterns.

Methods

In this retrospective study, a cohort of healthy controls (HC) and MS patients with different subtypes, followed up 15 years from clinically isolated syndrome (CIS), was analyzed to produce a multi-modal set of quantitative MRI features encompassing relaxometry, microstructure, sodium ion concentration, and tissue volumetry. Random forest classifiers were used to train a model able to discriminate between HC, CIS, relapsing remitting (RR) and secondary progressive (SP) MS patients based on these features and, for each classification task, to identify the relative contribution of each MRI-derived tissue property to the classification task itself.

Results and discussion

Average classification accuracy scores of 99 and 95% were obtained when discriminating HC and CIS vs. SP, respectively; 82 and 83% for HC and CIS vs. RR; 76% for RR vs. SP, and 79% for HC vs. CIS. Different patterns of alterations were observed for each classification task, offering key insights in the understanding of MS phenotypes pathophysiology: atrophy and relaxometry emerged particularly in the classification of HC and CIS vs. MS, relaxometry within lesions in RR vs. SP, sodium ion concentration in HC vs. CIS, and microstructural alterations were involved across all tasks.

Present and future of the diagnostic work-up of multiple sclerosis: the imaging perspective

In recent years, the use of magnetic resonance imaging (MRI) for the diagnostic work-up of multiple sclerosis (MS) has evolved considerably. The 2017 McDonald criteria show high sensitivity and accuracy in predicting a second clinical attack in patients with a typical clinically isolated syndrome and allow an earlier diagnosis of MS. They have been validated, are evidence-based, simplify the clinical use of MRI criteria and improve MS patients' management. However, to limit the risk of misdiagnosis, they should be applied by expert clinicians only after the careful exclusion of alternative diagnoses. Recently, new MRI markers have been proposed to improve diagnostic specificity for MS and reduce the risk of misdiagnosis. The central vein sign and chronic active lesions (i.e., paramagnetic rim lesions) may increase the specificity of MS diagnostic criteria, but further effort is necessary to validate and standardize their assessment before implementing them in the clinical setting. The feasibility of subpial demyelination assessment and the clinical relevance of leptomeningeal enhancement evaluation in the diagnostic work-up of MS appear more limited. Artificial intelligence tools may capture MRI attributes that are beyond the human perception, and, in the future, artificial intelligence may complement human assessment to further ameliorate the diagnostic work-up and patients' classification. However, guidelines that ensure reliability, interpretability, and validity of findings obtained from artificial intelligence approaches are still needed to implement them in the clinical scenario. This review provides a summary of the most recent updates regarding the application of MRI for the diagnosis of MS.

Prediction of the information processing speed performance in multiple sclerosis using a machine learning approach in a large multicenter magnetic resonance imaging data set

Many patients with multiple sclerosis (MS) experience information processing speed (IPS) deficits, and the Symbol Digit Modalities Test (SDMT) has been recommended as a valid screening test. Magnetic resonance imaging (MRI) has markedly improved the understanding of the mechanisms associated with cognitive deficits in MS. However, which structural MRI markers are the most closely related to cognitive performance is still unclear. We used the multicenter 3T-MRI data set of the Italian Neuroimaging Network Initiative to extract multimodal data (i.e., demographic, clinical, neuropsychological, and structural MRIs) of 540 MS patients. We aimed to assess, through machine learning techniques, the contribution of brain MRI structural volumes in the prediction of IPS deficits when combined with demographic and clinical features. We trained and tested the eXtreme Gradient Boosting (XGBoost) model following a rigorous validation scheme to obtain reliable generalization performance. We carried out a classification and a regression task based on SDMT scores feeding each model with different combinations of features. For the classification task, the model trained with thalamus, cortical gray matter, hippocampus, and lesions volumes achieved an area under the receiver operating characteristic curve of 0.74. For the regression task, the model trained with cortical gray matter and thalamus volumes, EDSS, nucleus accumbens, lesions, and putamen volumes, and age reached a mean absolute error of 0.95. In conclusion, our results confirmed that damage to cortical gray matter and relevant deep and archaic gray matter structures, such as the thalamus and hippocampus, is among the most relevant predictors of cognitive performance in MS.

Evaluating Magnetic Resonance Diffusion Properties Together with Brain Volumetry May Predict Progression to Multiple Sclerosis

RATIONALE AND OBJECTIVES

Although the gold standard in predicting future progression from clinically isolated syndrome (CIS) to clinically definite multiple sclerosis (CDMS) consists in the McDonald criteria, efforts are being made to employ various advanced MRI techniques for predicting clinical progression. This study's main aim was to evaluate the predictive power of diffusion tensor imaging (DTI) of the brain and brain volumetry to distinguish between patients having CIS with future progression to CDMS from those without progression during the following 2 years and to compare those parameters with conventional MRI evaluation.

MATERIALS AND METHODS

All participants underwent an MRI scan of the brain. DTI and volumetric data were processed and various parameters were compared between the study groups.

RESULTS

We found significant differences between the subgroups of patients differing by future progression to CDMS in most of those DTI and volumetric parameters measured. Fractional anisotropy of water diffusion proved to be the strongest predictor of clinical conversion among all parameters evaluated, demonstrating also higher specificity compared to evaluation of conventional MRI images according to McDonald criteria.

CONCLUSION

Conclusion: Our results provide evidence that the evaluation of DTI parameters together with brain volumetry in patients with early-stage CIS may be useful in predicting conversion to CDMS within the following 2 years of the disease course.

Multiple sclerosis diagnosis and phenotype identification by multivariate classification of in vivo frontal cortex metabolite profiles

Multiple sclerosis (MS) is a heterogeneous autoimmune disease for which diagnosis continues to rely on subjective clinical judgment over a battery of tests. Proton magnetic resonance spectroscopy (¹H MRS) enables the noninvasive in vivo detection of multiple small-molecule metabolites and is therefore in principle a promising means of gathering information sufficient for multiple sclerosis diagnosis and subtype classification. Here we show that supervised classification using ¹H-MRS-visible normal-appearing frontal cortex small-molecule metabolites alone can indeed differentiate individuals with progressive MS from control (held-out validation sensitivity 79% and specificity 68%), as well as between relapsing and progressive MS phenotypes (held-out validation sensitivity 84% and specificity 74%). Post hoc assessment demonstrated the disproportionate contributions of glutamate and glutamine to identifying MS status and phenotype, respectively. Our finding establishes ¹H MRS as a viable means of characterizing progressive multiple sclerosis disease status and paves the way for continued refinement of this method as an auxiliary or mainstay of multiple sclerosis diagnostics.

Prediction of Conversion from CIS to Clinically Definite Multiple Sclerosis Using Convolutional Neural Networks

Multiple sclerosis (MS) is a chronic neurological disease of the central nervous system (CNS). Early diagnosis of MS is highly desirable as treatments are more effective in preventing MS-related disability when given in the early stages of the disease. The main aim of this research is to predict the occurrence of a second MS-related clinical event, which indicates the conversion of clinically isolated syndrome (CIS) to clinically definite MS (CDMS). In this study, we apply a branch of artificial intelligence known as deep learning and develop a fully automated algorithm primed with convolutional neural network (CNN) that has the ability to learn from MRI scan features. The basic architecture of our algorithm is that of the VGG16 CNN model, but amended such that it can handle MRI DICOM images. A dataset comprised of scans acquired using two different scanners was used for the purposes of verification of the algorithm. A group of 49 patients had volumetric MRI scans taken at onset of the disease and then again one year later using one of the two scanners. In total, this yielded 7360 images which were then used for training, validation, and testing of the algorithm. Initially, these raw images were taken through 4 steps of preprocessing. In order to boost the efficiency of the process, we pretrained our algorithm using the publicly available ADNI dataset used to classify Alzheimer's disease. Finally, we used our preprocessed dataset to train and test the algorithm. Clinical evaluation conducted a year after the first time point revealed that 26 of the 49 patients had converted to CDMS, while the remaining 23 had not. Results of testing showed that our algorithm was able to predict the clinical results with an accuracy of 88.8% and with an area under the curve (AUC) of 91%. A highly accurate algorithm was developed using CNN approach to reliably predict conversion of patients with CIS to CDMS using MRI data from two different scanners.

Prediction of high and low disease activity in early MS patients using multiple kernel learning identifies importance of lateral ventricle intensity

Background

Lack of easy-to-interpret disease activity prediction methods in early MS can lead to worse patient prognosis.

Objectives

Using machine learning (multiple kernel learning – MKL) models, we assessed the prognostic value of various clinical and MRI measures for disease activity.

Methods

Early MS patients ( n = 148) with at least two associated clinical and MRI visits were investigated. T2-weighted MRIs were cropped to contain mainly the lateral ventricles (LV). High disease activity was defined as surpassing NEDA-3 Criteria more than once per year. Clinical demographic, MRI-extracted image-derived phenotypes (IDP), and MRI data were used as inputs for separate kernels to predict future disease activity with MKL. Model performance was compared using bootstrapped effect size analysis of mean differences.

Results

A total of 681 visits were included, where 81 (55%) patients had high disease activity in a combined end point measure using all follow-up visits. MKL model discrimination performance was moderate (AUC ≥ 0.62); however, modelling with combined clinical and cropped LV kernels gave the highest prediction performance (AUC = 0.70).

Conclusions

MRIs contain valuable information on future disease activity, especially in and around the LV. MKL techniques for combining different data types can be used for the prediction of disease activity in a relatively small MS cohort.

Role of artificial intelligence in MS clinical practice

•

For medical applications, machine learning (including deep learning) are the most commonly used artificial intelligence (AI) approaches.

•

It can improve multiple sclerosis (MS) diagnosis, prognostication and treatment monitoring.

•

Thanks to AI, MRI and cognitive phenotypes of MS patients were identified.

•

AI can shorten MRI protocols for MS, allowing the application of advanced techniques.

•

It can reduce the human effort for MRI analysis, especially for lesion segmentation.

Machine learning (ML) and its subset, deep learning (DL), are branches of artificial intelligence (AI) showing promising findings in the medical field, especially when applied to imaging data. Given the substantial role of MRI in the diagnosis and management of patients with multiple sclerosis (MS), this disease is an ideal candidate for the application of AI techniques. In this narrative review, we are going to discuss the potential applications of AI for MS clinical practice, together with their limitations. Among their several advantages, ML algorithms are able to automate repetitive tasks, to analyze more data in less time and to achieve higher accuracy and reproducibility than the human counterpart. To date, these algorithms have been applied to MS diagnosis, prognosis, disease and treatment monitoring. Other fields of application have been improvement of MRI protocols as well as automated lesion and tissue segmentation. However, several challenges remain, including a better understanding of the information selected by AI algorithms, appropriate multicenter and longitudinal validations of results and practical aspects regarding hardware and software integration. Finally, one cannot overemphasize the paramount importance of human supervision, in order to optimize the use and take full advantage of the potential of AI approaches.

Prognosis of a second clinical event from baseline MRI in patients with a CIS: a multicenter study using a machine learning approach

PURPOSE

To predict the occurrence of a second clinical event in patients with a CIS suggestive of MS, from baseline magnetic resonance imaging (MRI), by means of a pattern recognition approach.

METHODS

Two hundred sixty-six patients with a CIS were recruited from four participating centers. Over a follow-up of 3 years, 130 patients had a second clinical episode and 136 did not. Grey matter and white matter T1-hypointensities masks segmented from 3D T1-weighted images acquired on 3 T scanners were used as features for the classification approach. Differences between CIS that remained CIS and those that developed a second event were assessed at a global level and at a regional level, arranging the regions according to their contribution to the classification model.

RESULTS

All classification metrics were around or even below 50% for both global and regional approaches. Accuracies did not change when T1-hypointensity maps were added to the model; just the specificity was increased up to 80%. Among the 30 regions with the largest contribution, 26 were grey matter and 4 were white matter regions. For grey matter, regions contributing showed either a larger or a smaller volume in the group of patients that remained CIS, compared to those with a second event. The volume of T1-hypointensities was always larger for the group that presented a second event.

CONCLUSIONS

Prediction of a second clinical event in CIS patients from baseline MRI seems to present a highly heterogeneous pattern, leading to very low classification accuracies. Adding the T1-hypointensity maps does not seem to improve the accuracy of the classification model.

Dynamic topology analysis for spatial patterns of multifocal lesions on MRI

Quantitatively analysing the spatial patterns of multifocal lesions on clinical MRI is an important step towards a better understanding of the disease and for precision medicine, which is yet to be properly explored by feature engineering and deep learning methods. Network science addresses this issue by explicitly modeling the inter-lesion topology. However, the construction of the informative graph with optimal edge sparsity and quantification of community graph structures are the current challenges in network science. In this paper, we address these challenges with a novel Dynamic Topology Analysis framework on the basis of persistent homology, aiming to investigate the predictive values of global geometry and local clusters of multifocal lesions. Firstly, Dynamic Hierarchical Network is proposed to construct informative global and community-level topology over multi-scale networks from sparse to dense. Multi-scale global topology is constructed with a nested sequence of Rips complexes, from which a new K-simplex Filtration is designed to generate a higher-level topological abstraction for community identification based on the connectivity of k-simplices in the Rips Complex. Secondly, to quantify multi-scale community structures, we design a new Decomposed Community Persistence algorithm to track the dynamic evolution of communities, and then summarise the evolutionary communities incorporated with a customisable descriptor. The quantified community features are encapsulated with global geometric invariants for topological pattern analysis. The proposed framework was evaluated on both diagnostic differentiation and prognostic prediction for multiple sclerosis that is a typical multifocal disease, and achieved ROC_AUC 0.875 and 0.767, respectively, outperforming seven state-of-the-art persistent homology methods and the reported performance of six feature engineering and deep learning methods.

Assessment of Artificial Intelligence Automatic Multiple Sclerosis Lesion Delineation Tool for Clinical Use

PURPOSE

To implement and validate an existing algorithm for automatic delineation of white matter lesions on magnetic resonance imaging (MRI) in patients with multiple sclerosis (MS) on a local single-center dataset.

METHODS

We implemented a white matter hyperintensity segmentation model, based on a 2D convolutional neural network, using the conventional T2-weighted fluid attenuated inversion recovery (FLAIR) MRI sequence as input. The model was adapted for delineation of MS lesions by further training on a local dataset of 93 MS patients with a total of 3040 lesions. A quantitative evaluation was performed on ten test patients, in which model-generated masks were compared to manually delineated masks from two expert delineators. A subsequent qualitative evaluation of the implemented model was performed by two expert delineators, in which generated delineation masks on a clinical dataset of 53 patients were rated acceptable (< 10% errors) or unacceptable (> 10% errors) based on the total number of true lesions.

RESULTS

The quantitative evaluation resulted in an average accuracy score (F1) of 0.71, recall of 0.77 and dice similarity coefficient of 0.62. Our implemented model obtained the highest scores in all three metrics, when compared to three out of the box lesion segmentation models. In the clinical evaluation an average of 94% of our 53 model-generated masks were rated acceptable.

CONCLUSION

After adaptation to our local dataset, the implemented segmentation model was able to delineate MS lesions with a high clinical value as rated by delineation experts while outperforming popular out of the box applications. This serves as a promising step towards implementation of automatic lesion delineation in our MS clinic.

Machine learning classifier to identify clinical and radiological features relevant to disability progression in multiple sclerosis

Objectives

To evaluate the accuracy of a data-driven approach, such as machine learning classification, in predicting disability progression in MS.

Methods

We analyzed structural brain images of 163 subjects diagnosed with MS acquired at two different sites. Participants were followed up for 2–6 years, with disability progression defined according to the expanded disability status scale (EDSS) increment at follow-up. T2-weighted lesion load (T2LL), thalamic and cerebellar gray matter (GM) volumes, fractional anisotropy of the normal appearing white matter were calculated at baseline and included in supervised machine learning classifiers. Age, sex, phenotype, EDSS at baseline, therapy and time to follow-up period were also included. Classes were labeled as stable or progressed disability. Participants were randomly chosen from both sites to build a sample including 50% patients showing disability progression and 50% patients being stable. One-thousand machine learning classifiers were applied to the resulting sample, and after testing for overfitting, classifier confusion matrix, relative metrics and feature importance were evaluated.

Results

At follow-up, 36% of participants showed disability progression. The classifier with the highest resulting metrics had accuracy of 0.79, area under the true positive versus false positive rates curve of 0.81, sensitivity of 0.90 and specificity of 0.71. T2LL, thalamic volume, disability at baseline and administered therapy were identified as important features in predicting disability progression. Classifiers built on radiological features had higher accuracy than those built on clinical features.

Conclusions

Disability progression in MS may be predicted via machine learning classifiers, mostly evaluating neuroradiological features.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00415-021-10605-7.

Machine Learning Use for Prognostic Purposes in Multiple Sclerosis

The course of multiple sclerosis begins with a relapsing-remitting phase, which evolves into a secondarily progressive form over an extremely variable period, depending on many factors, each with a subtle influence. To date, no prognostic factors or risk score have been validated to predict disease course in single individuals. This is increasingly frustrating, since several treatments can prevent relapses and slow progression, even for a long time, although the possible adverse effects are relevant, in particular for the more effective drugs. An early prediction of disease course would allow differentiation of the treatment based on the expected aggressiveness of the disease, reserving high-impact therapies for patients at greater risk. To increase prognostic capacity, approaches based on machine learning (ML) algorithms are being attempted, given the failure of other approaches. Here we review recent studies that have used clinical data, alone or with other types of data, to derive prognostic models. Several algorithms that have been used and compared are described. Although no study has proposed a clinically usable model, knowledge is building up and in the future strong tools are likely to emerge.

A systematic review of the application of machine-learning algorithms in multiple sclerosis

INTRODUCTION

The applications of artificial intelligence, and in particular automatic learning or "machine learning" (ML), constitute both a challenge and a great opportunity in numerous scientific, technical, and clinical disciplines. Specific applications in the study of multiple sclerosis (MS) have been no exception, and constitute an area of increasing interest in recent years.

OBJECTIVE

We present a systematic review of the application of ML algorithms in MS.

MATERIALS AND METHODS

We used the PubMed search engine, which allows free access to the MEDLINE medical database, to identify studies including the keywords "machine learning" and "multiple sclerosis." We excluded review articles, studies written in languages other than English or Spanish, and studies that were mainly technical and did not specifically apply to MS. The final selection included 76 articles, and 38 were rejected.

CONCLUSIONS

After the review process, we established 4 main applications of ML in MS: 1) classifying MS subtypes; 2) distinguishing patients with MS from healthy controls and individuals with other diseases; 3) predicting progression and response to therapeutic interventions; and 4) other applications. Results found to date have shown that ML algorithms may offer great support for health professionals both in clinical settings and in research into MS.

MR Diffusion Properties of Cervical Spinal Cord as a Predictor of Progression to Multiple Sclerosis in Patients with Clinically Isolated Syndrome

BACKGROUND AND PURPOSE

This study's aim was to investigate diffusion properties of the cervical spinal cord in patients with clinically isolated syndrome (CIS) through analysis of diffusion tensor imaging (DTI) data and thereby to assess the capacity of this technique for predicting the progression of CIS to clinically definite multiple sclerosis (CDMS).

METHODS

The study groups were comprised of 47 patients with CIS (15 of them with progression to CDMS within 2 years of follow-up) and 57 asymptomatic controls. All patients and controls had undergone magnetic resonance imaging (MRI) of the cervical spine including DTI and brain MRI. Methodological approaches included histogram analysis of the cervical cord's diffusion parameters and evaluation of T2 hyperintense lesions of the spinal cord and brain. All parameters were compared between the study groups. Sensitivity and specificity calculations were then performed with a view to predicting conversion to CDMS.

RESULTS

The patient subgroups defined by progression to CDMS differed significantly in values of fractional anisotropy (FA) kurtosis measured within white matter (WM) and normal-appearing WM (NAWM). The same parameters also differed significantly when patients with progression to CDMS were compared to healthy controls. Receiver operating characteristic (ROC) analysis revealed sensitivity and specificity of FA kurtosis of WM and NAWM of 93% and 72%, respectively, in terms of predicting CIS to CDMS progression.

CONCLUSION

This study presents evidence that histogram analysis of diffusion parameters of the cervical spinal cord in patients with CIS may be helpful in predicting conversion to CDMS.

Model-Driven Decision Making in Multiple Sclerosis Research: Existing Works and Latest Trends

Multiple sclerosis (MS) is a neurological disorder that strikes the central nervous system. Due to the complexity of this disease, healthcare sectors are increasingly in need of shared clinical decision-making tools to provide practitioners with insightful knowledge and information about MS. These tools ought to be comprehensible by both technical and non-technical healthcare audiences. To aid this cause, this literature review analyzes the state-of-the-art decision support systems (DSSs) in MS research with a special focus on model-driven decision-making processes. The review clusters common methodologies used to support the decision-making process in classifying, diagnosing, predicting, and treating MS. This work observes that the majority of the investigated DSSs rely on knowledge-based and machine learning (ML) approaches, so the utilization of ontology and ML in the MS domain is observed to extend the scope of this review. Finally, this review summarizes the state-of-the-art DSSs, discusses the methods that have commonalities, and addresses the future work of applying DSS technologies in the MS field.

Multiple sclerosis (MS) is a disorder that strikes the central nervous system of the human body. This article reviews state-of-the-art decision support systems (DSSs) in MS research, as recent studies have highlighted the importance of DSSs in the medical realm. However, the utilization of decision support systems for MS remains an open challenge. A special focus in this article is given to model-driven DSSs, which uses knowledge representation to simplify the complex process for decision makers. We find that most investigated studies use knowledge-based and machine learning approaches. Based on the literature review, we suggest some future work of applying DSSs in the MS domain. Potential future directions should focus on applying DSS technologies to understand the MS patterns, etiology, effects on the quality-of-life, and correlations with other disorders.

The emerging role of artificial intelligence in multiple sclerosis imaging

BACKGROUND

Computer-aided diagnosis can facilitate the early detection and diagnosis of multiple sclerosis (MS) thus enabling earlier interventions and a reduction in long-term MS-related disability. Recent advancements in the field of artificial intelligence (AI) have led to the improvements in the classification, quantification and identification of diagnostic patterns in medical images for a range of diseases, in particular, for MS. Importantly, data generated using AI techniques are analyzed automatically, which compares favourably with labour-intensive and time-consuming manual methods.

OBJECTIVE

The aim of this review is to assist MS researchers to understand current and future developments in the AI-based diagnosis and prognosis of MS.

METHODS

We will investigate a variety of AI approaches and various classifiers and compare the current state-of-the-art techniques in relation to lesion segmentation/detection and prognosis of disease. After briefly describing the magnetic resonance imaging (MRI) techniques commonly used, we will describe AI techniques used for the detection of lesions and MS prognosis.

RESULTS

We then evaluate the clinical maturity of these AI techniques in relation to MS.

CONCLUSION

Finally, future research challenges are identified in a bid to encourage further improvements of the methods.

What do we currently know about the clinically isolated syndrome suggestive of multiple sclerosis? An update

Multiple sclerosis (MS) is a chronic, demyelinating, not fully understood disease of the central nervous system. The first demyelinating clinical episode is called clinically isolated syndrome (CIS) suggestive of MS. Although the most common manifestations of CIS are long tracts dysfunction and unilateral optic neuritis, it can also include isolated brainstem syndromes, cerebellar involvement, and polysymptomatic clinical image. Recently, the frequency of CIS diagnosis has decreased due to the more sensitive and less specific 2017 McDonald criteria compared with the revisions from 2010. Not all patients with CIS develop MS. The risk of conversion can be estimated based on many predictive factors including epidemiological, ethnical, clinical, biochemical, radiological, immunogenetic, and other markers. The management of CIS is nowadays widely discussed among clinicians and neuroscientists. To date, interferons, glatiramer acetate, teriflunomide, cladribine, and some other agents have been evaluated in randomized, placebo-controlled, double-blind studies relying on large groups of patients with the first demyelinating event. All of these drugs were shown to have beneficial effects in patients with CIS and might be used routinely in the future. The goal of this article is to explore the most relevant topics regarding CIS as well as to provide the most recent information in the field. The review presents CIS definition, classification, clinical image, predictive factors, and management. What is more, this is one of very few reviews summarizing the topic in the light of the 2017 McDonald criteria.

Considering patient clinical history impacts performance of machine learning models in predicting course of multiple sclerosis

Multiple Sclerosis (MS) progresses at an unpredictable rate, but predictions on the disease course in each patient would be extremely useful to tailor therapy to the individual needs. We explore different machine learning (ML) approaches to predict whether a patient will shift from the initial Relapsing-Remitting (RR) to the Secondary Progressive (SP) form of the disease, using only "real world" data available in clinical routine. The clinical records of 1624 outpatients (207 in the SP phase) attending the MS service of Sant'Andrea hospital, Rome, Italy, were used. Predictions at 180, 360 or 720 days from the last visit were obtained considering either the data of the last available visit (Visit-Oriented setting), comparing four classical ML methods (Random Forest, Support Vector Machine, K-Nearest Neighbours and AdaBoost) or the whole clinical history of each patient (History-Oriented setting), using a Recurrent Neural Network model, specifically designed for historical data. Missing values were handled by removing either all clinical records presenting at least one missing parameter (Feature-saving approach) or the 3 clinical parameters which contained missing values (Record-saving approach). The performances of the classifiers were rated using common indicators, such as Recall (or Sensitivity) and Precision (or Positive predictive value). In the visit-oriented setting, the Record-saving approach yielded Recall values from 70% to 100%, but low Precision (5% to 10%), which however increased to 50% when considering only predictions for which the model returned a probability above a given "confidence threshold". For the History-oriented setting, both indicators increased as prediction time lengthened, reaching values of 67% (Recall) and 42% (Precision) at 720 days. We show how "real world" data can be effectively used to forecast the evolution of MS, leading to high Recall values and propose innovative approaches to improve Precision towards clinically useful values.

A systematic review of the applications of artificial intelligence and machine learning in autoimmune diseases

Autoimmune diseases are chronic, multifactorial conditions. Through machine learning (ML), a branch of the wider field of artificial intelligence, it is possible to extract patterns within patient data, and exploit these patterns to predict patient outcomes for improved clinical management. Here, we surveyed the use of ML methods to address clinical problems in autoimmune disease. A systematic review was conducted using MEDLINE, embase and computers and applied sciences complete databases. Relevant papers included "machine learning" or "artificial intelligence" and the autoimmune diseases search term(s) in their title, abstract or key words. Exclusion criteria: studies not written in English, no real human patient data included, publication prior to 2001, studies that were not peer reviewed, non-autoimmune disease comorbidity research and review papers. 169 (of 702) studies met the criteria for inclusion. Support vector machines and random forests were the most popular ML methods used. ML models using data on multiple sclerosis, rheumatoid arthritis and inflammatory bowel disease were most common. A small proportion of studies (7.7% or 13/169) combined different data types in the modelling process. Cross-validation, combined with a separate testing set for more robust model evaluation occurred in 8.3% of papers (14/169). The field may benefit from adopting a best practice of validation, cross-validation and independent testing of ML models. Many models achieved good predictive results in simple scenarios (e.g. classification of cases and controls). Progression to more complex predictive models may be achievable in future through integration of multiple data types.

MRI-based prediction of conversion from clinically isolated syndrome to clinically definite multiple sclerosis using SVM and lesion geometry

Neuroanatomical pattern classification using support vector machines (SVMs) has shown promising results in classifying Multiple Sclerosis (MS) patients based on individual structural magnetic resonance images (MRI). To determine whether pattern classification using SVMs facilitates predicting conversion to clinically definite multiple sclerosis (CDMS) from clinically isolated syndrome (CIS). We used baseline MRI data from 364 patients with CIS, randomised to interferon beta-1b or placebo. Non-linear SVMs and 10-fold cross-validation were applied to predict converters/non-converters (175/189) at two years follow-up based on clinical and demographic data, lesion-specific quantitative geometric features and grey-matter-to-whole-brain volume ratios. We applied linear SVM analysis and leave-one-out cross-validation to subgroups of converters (n = 25) and non-converters (n = 44) based on cortical grey matter segmentations. Highest prediction accuracies of 70.4% (p = 8e-5) were reached with a combination of lesion-specific geometric (image-based) and demographic/clinical features. Cortical grey matter was informative for the placebo group (acc.: 64.6%, p = 0.002) but not for the interferon group. Classification based on demographic/clinical covariates only resulted in an accuracy of 56% (p = 0.05). Overall, lesion geometry was more informative in the interferon group, EDSS and sex were more important for the placebo cohort. Alongside standard demographic and clinical measures, both lesion geometry and grey matter based information can aid prediction of conversion to CDMS.

Modified connectivity of vulnerable brain nodes in multiple sclerosis, their impact on cognition and their discriminative value

Brain structural network modifications in multiple sclerosis (MS) seem to be clinically relevant. The discriminative ability of those changes to identify MS patients or their cognitive status remains unknown. Therefore, this study aimed to investigate connectivity changes in MS patients related to their cognitive status, and to define an automatic classification method to classify subjects as patients and healthy volunteers (HV) or as cognitively preserved (CP) and impaired (CI) patients. We analysed structural brain connectivity in 45 HV and 188 MS patients (104 CP and 84 CI). A support vector machine with k-fold cross-validation was built using the graph metrics features that best differentiate the groups (p < 0.05). Local efficiency (LE) and node strength (NS) network properties showed the largest differences: 100% and 69.7% of nodes had reduced LE and NS in CP patients compared to HV. Moreover, 55.3% and 57.9% of nodes had decreased LE and NS in CI compared to CP patients, in associative multimodal areas. The classification method achieved an accuracy of 74.8–77.2% to differentiate patients from HV, and 59.9–60.8% to discriminate CI from CP patients. Structural network integrity is widely reduced and worsens as cognitive function declines. Central network properties of vulnerable nodes can be useful to classify MS patients.

Quantifying the Metabolic Signature of Multiple Sclerosis by in vivo Proton Magnetic Resonance Spectroscopy: Current Challenges and Future Outlook in the Translation From Proton Signal to Diagnostic Biomarker

Proton magnetic resonance spectroscopy (¹H-MRS) offers a growing variety of methods for querying potential diagnostic biomarkers of multiple sclerosis in living central nervous system tissue. For the past three decades, ¹H-MRS has enabled the acquisition of a rich dataset suggestive of numerous metabolic alterations in lesions, normal-appearing white matter, gray matter, and spinal cord of individuals with multiple sclerosis, but this body of information is not free of seeming internal contradiction. The use of ¹H-MRS signals as diagnostic biomarkers depends on reproducible and generalizable sensitivity and specificity to disease state that can be confounded by a multitude of influences, including experiment group classification and demographics; acquisition sequence; spectral quality and quantifiability; the contribution of macromolecules and lipids to the spectroscopic baseline; spectral quantification pipeline; voxel tissue and lesion composition; T₁ and T₂ relaxation; B₁ field characteristics; and other features of study design, spectral acquisition and processing, and metabolite quantification about which the experimenter may possess imperfect or incomplete information. The direct comparison of ¹H-MRS data from individuals with and without multiple sclerosis poses a special challenge in this regard, as several lines of evidence suggest that experimental cohorts may differ significantly in some of these parameters. We review the existing findings of in vivo¹H-MRS on central nervous system metabolic abnormalities in multiple sclerosis and its subtypes within the context of study design, spectral acquisition and processing, and metabolite quantification and offer an outlook on technical considerations, including the growing use of machine learning, by future investigations into diagnostic biomarkers of multiple sclerosis measurable by ¹H-MRS.

Machine learning in secondary progressive multiple sclerosis: an improved predictive model for short-term disability progression

Background

Enhanced prediction of progression in secondary progressive multiple sclerosis (SPMS) could improve clinical trial design. Machine learning (ML) algorithms are methods for training predictive models with minimal human intervention.

Objective

To evaluate individual and ensemble model performance built using decision tree (DT)-based algorithms compared to logistic regression (LR) and support vector machines (SVMs) for predicting SPMS disability progression.

Methods

SPMS participants (n = 485) enrolled in a 2-year placebo-controlled (negative) trial assessing the efficacy of MBP8298 were classified as progressors if a 6-month sustained increase in Expanded Disability Status Scale (EDSS) (≥1.0 or ≥0.5 for a baseline of ≤5.5 or ≥6.0 respectively) was observed. Variables included EDSS, Multiple Sclerosis Functional Composite component scores, T2 lesion volume, brain parenchymal fraction, disease duration, age, and sex. Area under the receiver operating characteristic curve (AUC) was the primary outcome for model evaluation.

Results

Three DT-based models had greater AUCs (61.8%, 60.7%, and 60.2%) than independent and ensemble SVM (52.4% and 51.0%) and LR (49.5% and 51.1%).

Conclusion

SPMS disability progression was best predicted by non-parametric ML. If confirmed, ML could select those with highest progression risk for inclusion in SPMS trial cohorts and reduce the number of low-risk individuals exposed to experimental therapies.

SVM recursive feature elimination analyses of structural brain MRI predicts near-term relapses in patients with clinically isolated syndromes suggestive of multiple sclerosis

•

RFE-SVMs predict future outcome of CIS patients with conservative accuracy estimates between 64.9% and 88.1%.

•

Recursive feature selection improves classification performance compared to using all information.

•

Relevant features include regional WM lesion load and GM density, as well as the type of CIS onset.

•

Cross-validation introduces positive bias on accuracy estimate.

Machine learning classification is an attractive approach to automatically differentiate patients from healthy subjects, and to predict future disease outcomes. A clinically isolated syndrome (CIS) is often the first presentation of multiple sclerosis (MS), but it is difficult at onset to predict who will have a second relapse and hence convert to clinically definite MS. In this study, we thus aimed to distinguish CIS converters from non-converters at onset of a CIS, using recursive feature elimination and weight averaging with support vector machines. We also sought to assess the influence of cohort size and cross-validation methods on the accuracy estimate of the classification.

We retrospectively collected 400 patients with CIS from six European MAGNIMS MS centres. Patients underwent brain MRI at onset of a CIS according to local standard-of-care protocols. The diagnosis of clinically definite MS at one-year follow-up was the standard against which the accuracy of the model was tested. For each patient, we derived MRI-based features, such as grey matter probability, white matter lesion load, cortical thickness, and volume of specific cortical and white matter regions. Features with little contribution to the classification model were removed iteratively through an interleaved sample bootstrapping and feature averaging approach. Classification of CIS outcome at one-year follow-up was performed with 2-fold, 5-fold, 10-fold and leave-one-out cross-validation for each centre cohort independently and in all patients together.

The estimated classification accuracy across centres ranged from 64.9% to 88.1% using 2-fold cross-validation and from 73% to 92.9% using leave-one-out cross-validation. The classification accuracy estimate was higher in single-centre, smaller data sets than in combinations of data sets, being the lowest when all patients were merged together.

Regional MRI features such as WM lesions, grey matter probability in the thalamus and the precuneus or cortical thickness in the cuneus and inferior temporal gyrus predicted the occurrence of a second relapse in patients at onset of a CIS using support vector machines. The increased accuracy estimate of the classification achieved with smaller and single-centre samples may indicate a model bias (overfitting) when data points were limited, but also more homogeneous. We provide an overview of classifier performance from a range of cross-validation schemes to give insight into the variability across schemes. The proposed recursive feature elimination approach with weight averaging can be used both in single- and multi-centre data sets in order to bridge the gap between group-level comparisons and making predictions for individual patients.

Uncovering convolutional neural network decisions for diagnosing multiple sclerosis on conventional MRI using layer-wise relevance propagation

Machine learning-based imaging diagnostics has recently reached or even surpassed the level of clinical experts in several clinical domains. However, classification decisions of a trained machine learning system are typically non-transparent, a major hindrance for clinical integration, error tracking or knowledge discovery. In this study, we present a transparent deep learning framework relying on 3D convolutional neural networks (CNNs) and layer-wise relevance propagation (LRP) for diagnosing multiple sclerosis (MS), the most widespread autoimmune neuroinflammatory disease. MS is commonly diagnosed utilizing a combination of clinical presentation and conventional magnetic resonance imaging (MRI), specifically the occurrence and presentation of white matter lesions in T2-weighted images. We hypothesized that using LRP in a naive predictive model would enable us to uncover relevant image features that a trained CNN uses for decision-making. Since imaging markers in MS are well-established this would enable us to validate the respective CNN model. First, we pre-trained a CNN on MRI data from the Alzheimer's Disease Neuroimaging Initiative (n = 921), afterwards specializing the CNN to discriminate between MS patients (n = 76) and healthy controls (n = 71). Using LRP, we then produced a heatmap for each subject in the holdout set depicting the voxel-wise relevance for a particular classification decision. The resulting CNN model resulted in a balanced accuracy of 87.04% and an area under the curve of 96.08% in a receiver operating characteristic curve. The subsequent LRP visualization revealed that the CNN model focuses indeed on individual lesions, but also incorporates additional information such as lesion location, non-lesional white matter or gray matter areas such as the thalamus, which are established conventional and advanced MRI markers in MS. We conclude that LRP and the proposed framework have the capability to make diagnostic decisions of CNN models transparent, which could serve to justify classification decisions for clinical review, verify diagnosis-relevant features and potentially gather new disease knowledge.

Classification of radiologically isolated syndrome and clinically isolated syndrome with machine-learning techniques

BACKGROUND AND PURPOSE

The unanticipated detection by magnetic resonance imaging (MRI) in the brain of asymptomatic subjects of white matter lesions suggestive of multiple sclerosis (MS) has been named radiologically isolated syndrome (RIS). As the difference between early MS [i.e. clinically isolated syndrome (CIS)] and RIS is the occurrence of a clinical event, it is logical to improve detection of the subclinical form without interfering with MRI as there are radiological diagnostic criteria for that. Our objective was to use machine-learning classification methods to identify morphometric measures that help to discriminate patients with RIS from those with CIS.

METHODS

We used a multimodal 3-T MRI approach by combining MRI biomarkers (cortical thickness, cortical and subcortical grey matter volume, and white matter integrity) of a cohort of 17 patients with RIS and 17 patients with CIS for single-subject level classification.

RESULTS

The best proposed models to predict the diagnosis of CIS and RIS were based on the Naive Bayes, Bagging and Multilayer Perceptron classifiers using only three features: the left rostral middle frontal gyrus volume and the fractional anisotropy values in the right amygdala and right lingual gyrus. The Naive Bayes obtained the highest accuracy [overall classification, 0.765; area under the receiver operating characteristic (AUROC), 0.782].

CONCLUSIONS

A machine-learning approach applied to multimodal MRI data may differentiate between the earliest clinical expressions of MS (CIS and RIS) with an accuracy of 78%.

Intelligent Computer Systems for Multiple Sclerosis Diagnosis: a Systematic Review of Reasoning Techniques and Methods

OBJECTIVE

Intelligent computer systems are used in diagnosing Multiple Sclerosis and help physicians in the accurate and timely diagnosis of the disease. This study focuses on a review of different reasoning techniques and methods used in intelligent systems to diagnose MS and analyze the application and efficiency of different reasoning methods in order to find the most efficient and applicable methods and techniques for MS diagnosis.

METHODS

A complete research was carried out on articles in various electronic databases based on Mesh vocabulary. 85 articles out of 614 articles published in English between 2000 to 2018 were analyzed, 30 of which have been selected based on inclusion criteria such as system scope and domain, full description of reasoning method and system evaluation.

RESULTS

Results indicate that different reasoning methods are used unintelligent systems of MS diagnosis. In 27% of the studies, the rule-based method was used, in 20% the fuzzy logic method, in 18%the artificial neural network method, and in 35% other reasoning methods were used. The average sensitivity, specificity and accuracy of reasoning methods were0.91, 0.77, and 0.86, respectively.

CONCLUSIONS

Rule-based, fuzzy-logic and artificial neural network methods have had more applications in intelligent systems for the diagnosis of MS, respectively. The highest rate of sensitivity and accuracy indexes is associated to the neural network reasoning method at 0.97 and 0.99, respectively .In the fuzzy logic method, the Kappa rate has been reported as one, which shows full conformity between software diagnosis and the physician's decision .In some articles, in order to remove the limitations of the methods and enhance their efficiency, combinations of different methods are used.

Predicting conversion from clinically isolated syndrome to multiple sclerosis-An imaging-based machine learning approach

Magnetic resonance imaging (MRI) scans play a pivotal role in the evaluation of patients presenting with a clinically isolated syndrome (CIS), as these may depict brain lesions suggestive of an inflammatory cause. We hypothesized that it is possible to predict the conversion from CIS to multiple sclerosis (MS) based on the baseline MRI scan by studying image features of these lesions.

We analyzed 84 patients diagnosed with CIS from a prospective observational single center cohort. The patients were followed up for at least three years. Conversion to MS was defined according to the 2010 McDonald criteria. Brain lesions were segmented based on 3D FLAIR and 3D T1 images. We generated brain lesion masks by a computer assisted manual segmentation. We also generated a set of automated segmentations using the Lesion Segmentation Toolbox for SPM to assess the influence of different segmentation methods. Shape and brightness features were automatically calculated from the segmented masks and used as input data to train an oblique random forest classifier. Prediction accuracies of the resulting model were validated through a three-fold cross-validation.

Conversion from CIS to MS occurred in 66 of 84 patients (79%). The conversion or non-conversion was predicted correctly in 71 patients based on shape features derived from the computer assisted manual segmentation masks (84.5% accuracy). This predictor was more accurate than predicting conversion using dissemination in space at baseline according to the 2010 McDonald criteria (75% accuracy). While shape features strongly contributed to the accuracy of the predictor, including intensity features did not further improve performance.

As patients who convert to definite MS benefit from early treatment, an early classification model is highly desirable. Our study shows that shape parameters of lesions can contribute to predicting the future course of CIS patients more accurately.

•

A random forest tool can help to identify patients who convert from clinical isolated syndrome into multiple sclerosis (MS).

•

The classifier is driven by shape features of lesions in the first MR scan.

•

The found shape features reflect the typical ovoid growth of MS lesions.

MRI in predicting conversion to multiple sclerosis within 1 year

OBJECTIVES

Most patients diagnosed with multiple sclerosis (MS) present with a clinically isolated syndrome (CIS). We aimed to verify previously reported imaging and clinical findings, and to identify new MRI findings that might serve as prognostic factors for a second clinical episode or a change in the MRI scan during the first year following a CIS.

MATERIALS AND METHODS

We identified from our medical records, 46 individuals who presented with an episode of CIS, which was followed clinically and with imaging studies. A neuroradiologist blinded to the clinical data reviewed the images and recorded the number of lesions, lesion location, and the largest longitudinal diameter of the lesion.

RESULTS

One year after the first MRI, 25 (54%) patients had progressed to MS. The clinical presentation of those who were and were not diagnosed with MS was predominantly motor or sensory deficit. Patients with lesions that were temporal, occipital, or perpendicular to the corpus callosum at the first episode were more likely to have recurrence. Individuals with a combination of more than 13 lesions, with maximal lesion length greater than 0.75 cm, and a lesion perpendicular to the corpus callosum, had a 19 times higher chance of conversion MS during the following year.

CONCLUSIONS

Assessment of the number of lesions, lesion location, and maximal lesion size can predict the risk to develop another clinical episode or a new lesion/new enhancement in MRI during the year after CIS. For patients with a higher risk of recurrence, we recommend closer follow-up.

Structural neuroimaging as clinical predictor: A review of machine learning applications

In this paper, we provide an extensive overview of machine learning techniques applied to structural magnetic resonance imaging (MRI) data to obtain clinical classifiers. We specifically address practical problems commonly encountered in the literature, with the aim of helping researchers improve the application of these techniques in future works. Additionally, we survey how these algorithms are applied to a wide range of diseases and disorders (e.g. Alzheimer's disease (AD), Parkinson's disease (PD), autism, multiple sclerosis, traumatic brain injury, etc.) in order to provide a comprehensive view of the state of the art in different fields.

Random support vector machine cluster analysis of resting-state fMRI in Alzheimer's disease

Early diagnosis is critical for individuals with Alzheimer's disease (AD) in clinical practice because its progress is irreversible. In the existing literature, support vector machine (SVM) has always been applied to distinguish between AD and healthy controls (HC) based on neuroimaging data. But previous studies have only used a single SVM to classify AD and HC, and the accuracy is not very high and generally less than 90%. The method of random support vector machine cluster was proposed to classify AD and HC in this paper. From the Alzheimer's Disease Neuroimaging Initiative database, the subjects including 25 AD individuals and 35 HC individuals were obtained. The classification accuracy could reach to 94.44% in the results. Furthermore, the method could also be used for feature selection and the accuracy could be maintained at the level of 94.44%. In addition, we could also find out abnormal brain regions (inferior frontal gyrus, superior frontal gyrus, precentral gyrus and cingulate cortex). It is worth noting that the proposed random support vector machine cluster could be a new insight to help the diagnosis of AD.

Collaboration between a human group and artificial intelligence can improve prediction of multiple sclerosis course: a proof-of-principle study

Background: Multiple sclerosis has an extremely variable natural course. In most patients, disease starts with a relapsing-remitting (RR) phase, which proceeds to a secondary progressive (SP) form. The duration of the RR phase is hard to predict, and to date predictions on the rate of disease progression remain suboptimal. This limits the opportunity to tailor therapy on an individual patient's prognosis, in spite of the choice of several therapeutic options.

Approaches to improve clinical decisions, such as collective intelligence of human groups and machine learning algorithms are widely investigated.

Methods: Medical students and a machine learning algorithm predicted the course of disease on the basis of randomly chosen clinical records of patients that attended at the Multiple Sclerosis service of Sant'Andrea hospital in Rome.

Results: A significant improvement of predictive ability was obtained when predictions were combined with a weight that depends on the consistence of human (or algorithm) forecasts on a given clinical record.

Conclusions: In this work we present proof-of-principle that human-machine hybrid predictions yield better prognoses than machine learning algorithms or groups of humans alone. To strengthen and generalize this preliminary result, we propose a crowdsourcing initiative to collect prognoses by physicians on an expanded set of patients.

Collaboration between a human group and artificial intelligence can improve prediction of multiple sclerosis course: a proof-of-principle study

Background: Multiple sclerosis has an extremely variable natural course. In most patients, disease starts with a relapsing-remitting (RR) phase, which proceeds to a secondary progressive (SP) form. The duration of the RR phase is hard to predict, and to date predictions on the rate of disease progression remain suboptimal. This limits the opportunity to tailor therapy on an individual patient's prognosis, in spite of the choice of several therapeutic options. Approaches to improve clinical decisions, such as collective intelligence of human groups and machine learning algorithms are widely investigated. Methods: Medical students and a machine learning algorithm predicted the course of disease on the basis of randomly chosen clinical records of patients that attended at the Multiple Sclerosis service of Sant'Andrea hospital in Rome. Results: A significant improvement of predictive ability was obtained when predictions were combined with a weight that depends on the consistence of human (or algorithm) forecasts on a given clinical record. Conclusions: In this work we present proof-of-principle that human-machine hybrid predictions yield better prognoses than machine learning algorithms or groups of humans alone. To strengthen and generalize this preliminary result, we propose a crowdsourcing initiative to collect prognoses by physicians on an expanded set of patients.

Gray matter MRI differentiates neuromyelitis optica from multiple sclerosis using random forest

OBJECTIVE

We tested whether brain gray matter (GM) imaging measures can differentiate between multiple sclerosis (MS) and neuromyelitis optica (NMO) using random-forest classification.

METHODS

Ninety participants (25 patients with MS, 30 patients with NMO, and 35 healthy controls [HCs]) were studied in Tehran, Iran, and 54 (24 patients with MS, 20 patients with NMO, and 10 HCs) in Padua, Italy. Participants underwent brain T1 and T2/fluid-attenuated inversion recovery MRI. Volume, thickness, and surface of 50 cortical GM regions and volumes of the deep GM nuclei were calculated and used to construct 3 random-forest models to classify patients as either NMO or MS, and separate each patient group from HCs. Clinical diagnosis was the gold standard against which the accuracy was calculated.

RESULTS

The classifier distinguished patients with MS, who showed greater atrophy especially in deep GM, from those with NMO with an average accuracy of 74% (sensitivity/specificity: 77/72; p < 0.01). When we used thalamic volume (the most discriminating GM measure) together with the white matter lesion volume, the accuracy of the classification of MS vs NMO was 80%. The classifications of MS vs HCs and NMO vs HCs achieved higher accuracies (92% and 88%).

CONCLUSIONS

GM imaging biomarkers, automatically obtained from clinical scans, can be used to distinguish NMO from MS, even in a 2-center setting, and may facilitate the differential diagnosis in clinical practice.

CLASSIFICATION OF EVIDENCE

This study provides Class II evidence that GM imaging biomarkers can distinguish patients with NMO from those with MS.

Structural Brain Network Characteristics Can Differentiate CIS from Early RRMS

Focal demyelinated lesions, diffuse white matter (WM) damage, and gray matter (GM) atrophy influence directly the disease progression in patients with multiple sclerosis. The aim of this study was to identify specific characteristics of GM and WM structural networks in subjects with clinically isolated syndrome (CIS) in comparison to patients with early relapsing-remitting multiple sclerosis (RRMS). Twenty patients with CIS, 33 with RRMS, and 40 healthy subjects were investigated using 3 T-MRI. Diffusion tensor imaging was applied, together with probabilistic tractography and fractional anisotropy (FA) maps for WM and cortical thickness correlation analysis for GM, to determine the structural connectivity patterns. A network topology analysis with the aid of graph theoretical approaches was used to characterize the network at different community levels (modularity, clustering coefficient, global, and local efficiencies). Finally, we applied support vector machines (SVM) to automatically discriminate the two groups. In comparison to CIS subjects, patients with RRMS were found to have increased modular connectivity and higher local clustering, highlighting increased local processing in both GM and WM. Both groups presented increased modularity and clustering coefficients in comparison to healthy controls. SVM algorithms achieved 97% accuracy using the clustering coefficient as classifier derived from GM and 65% using WM from probabilistic tractography and 67% from modularity of FA maps to differentiate between CIS and RRMS patients. We demonstrate a clear increase of modular and local connectivity in patients with early RRMS in comparison to CIS and healthy subjects. Based only on a single anatomic scan and without a priori information, we developed an automated and investigator-independent paradigm that can accurately discriminate between patients with these clinically similar disease entities, and could thus complement the current dissemination-in-time criteria for clinical diagnosis.

Classification, diagnosis, and differential diagnosis of multiple sclerosis

PURPOSE OF REVIEW

The increasing availability of effective therapies for multiple sclerosis as well as research demonstrating the benefits of early treatment highlights the importance of expedient and accurate multiple sclerosis diagnosis. This review will discuss the classification, diagnosis, and differential diagnosis of multiple sclerosis.

RECENT FINDINGS

An international panel of multiple sclerosis experts, the MS Phenotype Group, recently revised the multiple sclerosis phenotypic classifications and published their recommendations in 2014. Recent research developments have helped improve the accuracy of multiple sclerosis diagnosis, especially with regard to differentiating multiple sclerosis from neuromyelitis optica spectrum disorders.

SUMMARY

Current multiple sclerosis phenotypic classifications include relapsing-remitting multiple sclerosis, clinically isolated syndrome, radiologically isolated syndrome, primary-progressive multiple sclerosis, and secondary-progressive multiple sclerosis. The McDonald 2010 diagnostic criteria provide formal guidelines for the diagnosis of relapsing-remitting multiple sclerosis and primary-progressive multiple sclerosis. These require demonstration of dissemination in space and time, with consideration given to both clinical findings and imaging data. The criteria also require that there exist no better explanation for the patient's presentation. The clinical history, examination, and MRI should be most consistent with multiple sclerosis, including the presence of features typical for the disease as well as the absence of features that suggest an alternative cause, for a diagnosis of multiple sclerosis to be proposed.

Update on clinically isolated syndrome

Optic neuritis, myelitis and brainstem syndrome accompanied by a symptomatic MRI T2 or FLAIR hyperintensity and T1 hypointensity are highly suggestive of multiple sclerosis (MS) in young adults. They are called "clinically isolated syndrome" (CIS) and correspond to the typical first multiple sclerosis (MS) episode, especially when associated with other asymptomatic demyelinating lesions, without clinical, radiological and immunological sign of differential diagnosis. After a CIS, the delay of apparition of a relapse, which corresponds to the conversion to clinically definite MS (CDMS), varies from several months to more than 10 years (10-15% of cases, generally called benign RRMS). This delay is generally associated with the number and location of demyelinating lesions of the brain and spinal cord and the results of CSF analysis. Several studies comparing different MRI criteria for dissemination in space and dissemination in time of demyelinating lesions, two hallmarks of MS, provided enough substantial data to update diagnostic criteria for MS after a CIS. In the last revision of the McDonald's criteria in 2010, diagnostic criteria were simplified and now the diagnosis can be made by a single initial scan that proves the presence of active asymptomatic lesions (with gadolinium enhancement) and of unenhanced lesions. However, time to conversion remains highly unpredictable for a given patient and CIS can remain isolated, especially for idiopathic unilateral optic neuritis or myelitis. Univariate analyses of clinical, radiological, biological or electrophysiological characteristics of CIS patients in small series identified numerous risk factors of rapid conversion to MS. However, large series of CIS patients analyzing several characteristics of CIS patients and the influence of disease modifying therapies brought important information about the risk of CDMS or RRMS over up to 20 years of follow-up. They confirmed the importance of the initial MRI pattern of demyelinating lesions and of CSF oligoclonal bands. Available treatments of MS (immunomodulators or immunosuppressants) have also shown unequivocal efficacy to slow the conversion to RRMS after a CIS, but they could be unnecessary for patients with benign RRMS. Beyond diagnostic criteria, knowledge of established and potential risk factors of conversion to MS and of disability progression is essential for CIS patients' follow-up and initiation of disease modifying therapies.