A Novel Public MR Image Dataset of Multiple Sclerosis Patients With Lesion Segmentations Based on Multi-rater Consensus

Enabling new insights from old scans by repurposing clinical MRI archives for multiple sclerosis research

Magnetic resonance imaging (MRI) biomarkers are vital for multiple sclerosis (MS) clinical research and trials but quantifying them requires multi-contrast protocols and limits the use of abundant single-contrast hospital archives. We developed MindGlide, a deep learning model to extract brain region and white matter lesion volumes from any single MRI contrast. We trained MindGlide on 4247 brain MRI scans from 2934 MS patients across 592 scanners, and externally validated it using 14,952 scans from 1,001 patients in two clinical trials (primary-progressive MS and secondary-progressive MS trials) and a routine-care MS dataset. The model outperformed two state-of-the-art models when tested against expert-labelled lesion volumes. In clinical trials, MindGlide detected treatment effects on T2-lesion accrual and cortical and deep grey matter volume loss. In routine-care data, T2-lesion volume increased with moderate-efficacy treatment but remained stable with high-efficacy treatment. MindGlide uniquely enables quantitative analysis of archival single-contrast MRIs, unlocking insights from untapped hospital datasets.

Guided reconstruction with conditioned diffusion models for unsupervised anomaly detection in brain MRIs

The application of supervised models to clinical screening tasks is challenging due to the need for annotated data for each considered pathology. Unsupervised Anomaly Detection (UAD) is an alternative approach that aims to identify any anomaly as an outlier from a healthy training distribution. A prevalent strategy for UAD in brain MRI involves using generative models to learn the reconstruction of healthy brain anatomy for a given input image. As these models should fail to reconstruct unhealthy structures, the reconstruction errors indicate anomalies. However, a significant challenge is to balance the accurate reconstruction of healthy anatomy and the undesired replication of abnormal structures. While diffusion models have shown promising results with detailed and accurate reconstructions, they face challenges in preserving intensity characteristics, resulting in false positives. We propose conditioning the denoising process of diffusion models with additional information derived from a latent representation of the input image. We demonstrate that this conditioning allows for accurate and local adaptation to the general input intensity distribution while avoiding the replication of unhealthy structures. We compare the novel approach to different state-of-the-art methods and for different data sets. Our results show substantial improvements in the segmentation performance, with the Dice score improved by 11.9%, 20.0%, and 44.6%, for the BraTS, ATLAS and MSLUB data sets, respectively, while maintaining competitive performance on the WMH data set. Furthermore, our results indicate effective domain adaptation across different MRI acquisitions and simulated contrasts, an important attribute for general anomaly detection methods. The code for our work is available at https://github.com/FinnBehrendt/Conditioned-Diffusion-Models-UAD.

Deep Learning techniques to detect and analysis of multiple sclerosis through MRI: A systematic literature review

Deep learning (DL) techniques represent a rapidly advancing field within artificial intelligence, gaining significant prominence in the detection and analysis of various medical conditions through the analysis of medical data. This study presents a systematic literature review (SLR) focused on deep learning methods for the detection and analysis of multiple sclerosis (MS) using magnetic resonance imaging (MRI). The initial search identified 401 articles, which were rigorously screened, a selection of 82 highly relevant studies. These selected studies primarily concentrate on key areas such as multiple sclerosis, deep learning, convolutional neural networks (CNN), lesion segmentation, and classification, reflecting their alignment with the current state of the art. This review comprehensively examines diverse deep-learning approaches for MS detection and analysis, offering a valuable resource for researchers. Additionally, it presents key insights by summarizing these DL techniques for MS detection and analysis using MRI in a structured tabular format.

Textural analysis and artificial intelligence as decision support tools in the diagnosis of multiple sclerosis - a systematic review

Introduction

Magnetic resonance imaging (MRI) is conventionally used for the detection and diagnosis of multiple sclerosis (MS), often complemented by lumbar puncture-a highly invasive method-to validate the diagnosis. Additionally, MRI is periodically repeated to monitor disease progression and treatment efficacy. Recent research has focused on the application of artificial intelligence (AI) and radiomics in medical image processing, diagnosis, and treatment planning.

Methods

A review of the current literature was conducted, analyzing the use of AI models and texture analysis for MS lesion segmentation and classification. The study emphasizes common models, including U-Net, Support Vector Machine, Random Forest, and K-Nearest Neighbors, alongside their evaluation metrics.

Results

The analysis revealed a fragmented research landscape, with significant variation in model architectures and performance. Evaluation metrics such as Accuracy, Dice score, and Sensitivity are commonly employed, with some models demonstrating robustness across multi-center datasets. However, most studies lack validation in clinical scenarios.

Discussion

The absence of consensus on the optimal model for MS lesion segmentation highlights the need for standardized methodologies and clinical validation. Future research should prioritize clinical trials to establish the real-world applicability of AI-driven decision support tools. This review provides a comprehensive overview of contemporary advancements in AI and radiomics for analyzing and monitoring emerging MS lesions in MRI.

Structural-based uncertainty in deep learning across anatomical scales: Analysis in white matter lesion segmentation

This paper explores uncertainty quantification (UQ) as an indicator of the trustworthiness of automated deep-learning (DL) tools in the context of white matter lesion (WML) segmentation from magnetic resonance imaging (MRI) scans of multiple sclerosis (MS) patients. Our study focuses on two principal aspects of uncertainty in structured output segmentation tasks. First, we postulate that a reliable uncertainty measure should indicate predictions likely to be incorrect with high uncertainty values. Second, we investigate the merit of quantifying uncertainty at different anatomical scales (voxel, lesion, or patient). We hypothesize that uncertainty at each scale is related to specific types of errors. Our study aims to confirm this relationship by conducting separate analyses for in-domain and out-of-domain settings. Our primary methodological contributions are (i) the development of novel measures for quantifying uncertainty at lesion and patient scales, derived from structural prediction discrepancies, and (ii) the extension of an error retention curve analysis framework to facilitate the evaluation of UQ performance at both lesion and patient scales. The results from a multi-centric MRI dataset of 444 patients demonstrate that our proposed measures more effectively capture model errors at the lesion and patient scales compared to measures that average voxel-scale uncertainty values. We provide the UQ protocols code at https://github.com/Medical-Image-Analysis-Laboratory/MS_WML_uncs.

Decoding the best automated segmentation tools for vascular white matter hyperintensities in the aging brain: a clinician's guide to precision and purpose

White matter hyperintensities of vascular origin (WMH) are commonly found in individuals over 60 and increase in prevalence with age. The significance of WMH is well-documented, with strong associations with cognitive impairment, risk of stroke, mental health, and brain structure deterioration. Consequently, careful monitoring is crucial for the early identification and management of individuals at risk. Luckily, WMH are detectable and quantifiable on standard MRI through visual assessment scales, but it is time-consuming and has high rater variability. Addressing this issue, the main aim of our study is to decipher the utility of quantitative measures of WMH, assessed with automatic tools, in establishing risk profiles for cerebrovascular deterioration. For this purpose, first, we work to determine the most precise WMH segmentation open access tool compared to clinician manual segmentations (LST-LPA, LST-LGA, SAMSEG, and BIANCA), offering insights into methodology and usability to balance clinical precision with practical application. The results indicated that supervised algorithms (LST-LPA and BIANCA) were superior, particularly in detecting small WMH, and can improve their consistency when used in parallel with unsupervised tools (LST-LGA and SAMSEG). Additionally, to investigate the behavior and real clinical utility of these tools, we tested them in a real-world scenario (N = 300; age > 50 y.o. and MMSE > 26), proposing an imaging biomarker for moderate vascular damage. The results confirmed its capacity to effectively identify individuals at risk comparing the cognitive and brain structural profiles of cognitively healthy adults above and below the resulted threshold.

Perceptual super-resolution in multiple sclerosis MRI

Introduction

Magnetic resonance imaging (MRI) is crucial for diagnosing and monitoring of multiple sclerosis (MS) as it is used to assess lesions in the brain and spinal cord. However, in real-world clinical settings, MRI scans are often acquired with thick slices, limiting their utility for automated quantitative analyses. This work presents a single-image super-resolution (SR) reconstruction framework that leverages SR convolutional neural networks (CNN) to enhance the through-plane resolution of structural MRI in people with MS (PwMS).

Methods

Our strategy involves the supervised fine-tuning of CNN architectures, guided by a content loss function that promotes perceptual quality, as well as reconstruction accuracy, to recover high-level image features.

Results

Extensive evaluation with MRI data of PwMS shows that our SR strategy leads to more accurate MRI reconstructions than competing methods. Furthermore, it improves lesion segmentation on low-resolution MRI, approaching the performance achievable with high-resolution images.

Discussion

Results demonstrate the potential of our SR framework to facilitate the use of low-resolution retrospective MRI from real-world clinical settings to investigate quantitative image-based biomarkers of MS.

Consensus of algorithms for lesion segmentation in brain MRI studies of multiple sclerosis

Segmentation of multiple sclerosis (MS) lesions on brain MRI scans is crucial for diagnosis, disease and treatment monitoring but is a time-consuming task. Despite several automated algorithms have been proposed, there is still no consensus on the most effective method. Here, we applied a consensus-based framework to improve lesion segmentation on T1-weighted and FLAIR scans. The framework is designed to combine publicly available state-of-the-art deep learning models, by running multiple segmentation tasks before merging the outputs of each algorithm. To assess the effectiveness of the approach, we applied it to MRI datasets from two different centers, including a private and a public dataset, with 131 and 30 MS patients respectively, with manually segmented lesion masks available. No further training was performed for any of the included algorithms. Overlap and detection scores were improved, with Dice increasing by 4-8% and precision by 3-4% respectively for the private and public dataset. High agreement was obtained between estimated and true lesion load (ρ = 0.92 and ρ = 0.97) and count (ρ = 0.83 and ρ = 0.94). Overall, this framework ensures accurate and reliable results, exploiting complementary features and overcoming some of the limitations of individual algorithms.

Contrast-Enhancing Lesion Segmentation in Multiple Sclerosis: A Deep Learning Approach Validated in a Multicentric Cohort

The detection of contrast-enhancing lesions (CELs) is fundamental for the diagnosis and monitoring of patients with multiple sclerosis (MS). This task is time-consuming and suffers from high intra- and inter-rater variability in clinical practice. However, only a few studies proposed automatic approaches for CEL detection. This study aimed to develop a deep learning model that automatically detects and segments CELs in clinical Magnetic Resonance Imaging (MRI) scans. A 3D UNet-based network was trained with clinical MRI from the Swiss Multiple Sclerosis Cohort. The dataset comprised 372 scans from 280 MS patients: 162 showed at least one CEL, while 118 showed no CELs. The input dataset consisted of T1-weighted before and after gadolinium injection, and FLuid Attenuated Inversion Recovery images. The sampling strategy was based on a white matter lesion mask to confirm the existence of real contrast-enhancing lesions. To overcome the dataset imbalance, a weighted loss function was implemented. The Dice Score Coefficient and True Positive and False Positive Rates were 0.76, 0.93, and 0.02, respectively. Based on these results, the model developed in this study might well be considered for clinical decision support.

Artificial Intelligence and Multiple Sclerosis

In this paper, we analyse the different advances in artificial intelligence (AI) approaches in multiple sclerosis (MS). AI applications in MS range across investigation of disease pathogenesis, diagnosis, treatment, and prognosis. A subset of AI, Machine learning (ML) models analyse various data sources, including magnetic resonance imaging (MRI), genetic, and clinical data, to distinguish MS from other conditions, predict disease progression, and personalize treatment strategies. Additionally, AI models have been extensively applied to lesion segmentation, identification of biomarkers, and prediction of outcomes, disease monitoring, and management. Despite the big promises of AI solutions, model interpretability and transparency remain critical for gaining clinician and patient trust in these methods. The future of AI in MS holds potential for open data initiatives that could feed ML models and increasing generalizability, the implementation of federated learning solutions for training the models addressing data sharing issues, and generative AI approaches to address challenges in model interpretability, and transparency. In conclusion, AI presents an opportunity to advance our understanding and management of MS. AI promises to aid clinicians in MS diagnosis and prognosis improving patient outcomes and quality of life, however ensuring the interpretability and transparency of AI-generated results is going to be key for facilitating the integration of AI into clinical practice.

Decoding the Best Automated Segmentation Tools for Vascular White Matter Hyperintensities in the Aging Brain: A Clinician's Guide to Precision and Purpose

Cerebrovascular damage from small vessel disease (SVD) occurs in healthy and pathological aging. SVD markers, such as white matter hyperintensities (WMH), are commonly found in individuals over 60 and increase in prevalence with age. WMHs are detectable on standard MRI by adhering to the STRIVE criteria. Currently, visual assessment scales are used in clinical and research scenarios but is time-consuming and has rater variability, limiting its practicality. Addressing this issue, our study aimed to determine the most precise WMH segmentation software, offering insights into methodology and usability to balance clinical precision with practical application. This study employed a dataset comprising T1, FLAIR, and DWI images from 300 cognitively healthy older adults. WMHs in this cohort were evaluated using four automated neuroimaging tools: Lesion Prediction Algorithm (LPA) and Lesion Growth Algorithm (LGA) from Lesion Segmentation Tool (LST), Sequence Adaptive Multimodal Segmentation (SAMSEG), and Brain Intensity Abnormalities Classification Algorithm (BIANCA). Additionally, clinicians manually segmented WMHs in a subsample of 45 participants to establish a gold standard. The study assessed correlations with the Fazekas scale, algorithm performance, and the influence of WMH volume on reliability. Results indicated that supervised algorithms were superior, particularly in detecting small WMHs, and can improve their consistency when used in parallel with unsupervised tools. The research also proposed a biomarker for moderate vascular damage, derived from the top 95th percentile of WMH volume in healthy individuals aged 50 to 60. This biomarker effectively differentiated subgroups within the cohort, correlating with variations in brain structure and behavior.

LST-AI: A deep learning ensemble for accurate MS lesion segmentation

Automated segmentation of brain white matter lesions is crucial for both clinical assessment and scientific research in multiple sclerosis (MS). Over a decade ago, we introduced an engineered lesion segmentation tool, LST. While recent lesion segmentation approaches have leveraged artificial intelligence (AI), they often remain proprietary and difficult to adopt. As an open-source tool, we present LST-AI, an advanced deep learning-based extension of LST that consists of an ensemble of three 3D U-Nets. LST-AI explicitly addresses the imbalance between white matter (WM) lesions and non-lesioned WM. It employs a composite loss function incorporating binary cross-entropy and Tversky loss to improve segmentation of the highly heterogeneous MS lesions. We train the network ensemble on 491 MS pairs of T1-weighted and FLAIR images, collected in-house from a 3T MRI scanner, and expert neuroradiologists manually segmented the utilized lesion maps for training. LST-AI also includes a lesion location annotation tool, labeling lesions as periventricular, infratentorial, and juxtacortical according to the 2017 McDonald criteria, and, additionally, as subcortical. We conduct evaluations on 103 test cases consisting of publicly available data using the Anima segmentation validation tools and compare LST-AI with several publicly available lesion segmentation models. Our empirical analysis shows that LST-AI achieves superior performance compared to existing methods. Its Dice and F1 scores exceeded 0.62, outperforming LST, SAMSEG (Sequence Adaptive Multimodal SEGmentation), and the popular nnUNet framework, which all scored below 0.56. Notably, LST-AI demonstrated exceptional performance on the MSSEG-1 challenge dataset, an international WM lesion segmentation challenge, with a Dice score of 0.65 and an F1 score of 0.63-surpassing all other competing models at the time of the challenge. With increasing lesion volume, the lesion detection rate rapidly increased with a detection rate of >75% for lesions with a volume between 10 mm3 and 100 mm3. Given its higher segmentation performance, we recommend that research groups currently using LST transition to LST-AI. To facilitate broad adoption, we are releasing LST-AI as an open-source model, available as a command-line tool, dockerized container, or Python script, enabling diverse applications across multiple platforms.

LST-AI: a Deep Learning Ensemble for Accurate MS Lesion Segmentation

Automated segmentation of brain white matter lesions is crucial for both clinical assessment and scientific research in multiple sclerosis (MS). Over a decade ago, we introduced an engineered lesion segmentation tool, LST. While recent lesion segmentation approaches have leveraged artificial intelligence (AI), they often remain proprietary and difficult to adopt. As an open-source tool, we present LST-AI, an advanced deep learning-based extension of LST that consists of an ensemble of three 3D-UNets. LST-AI explicitly addresses the imbalance between white matter (WM) lesions and non-lesioned WM. It employs a composite loss function incorporating binary cross-entropy and Tversky loss to improve segmentation of the highly heterogeneous MS lesions. We train the network ensemble on 491 MS pairs of T1w and FLAIR images, collected in-house from a 3T MRI scanner, and expert neuroradiologists manually segmented the utilized lesion maps for training. LST-AI additionally includes a lesion location annotation tool, labeling lesion location according to the 2017 McDonald criteria (periventricular, infratentorial, juxtacortical, subcortical). We conduct evaluations on 103 test cases consisting of publicly available data using the Anima segmentation validation tools and compare LST-AI with several publicly available lesion segmentation models. Our empirical analysis shows that LST-AI achieves superior performance compared to existing methods. Its Dice and F1 scores exceeded 0.62, outperforming LST, SAMSEG (Sequence Adaptive Multimodal SEGmentation), and the popular nnUNet framework, which all scored below 0.56. Notably, LST-AI demonstrated exceptional performance on the MSSEG-1 challenge dataset, an international WM lesion segmentation challenge, with a Dice score of 0.65 and an F1 score of 0.63-surpassing all other competing models at the time of the challenge. With increasing lesion volume, the lesion detection rate rapidly increased with a detection rate of >75% for lesions with a volume between 10mm3 and 100mm3. Given its higher segmentation performance, we recommend that research groups currently using LST transition to LST-AI. To facilitate broad adoption, we are releasing LST-AI as an open-source model, available as a command-line tool, dockerized container, or Python script, enabling diverse applications across multiple platforms.

Self-Supervised Super-Resolution for Anisotropic MR Images with and Without Slice Gap

Magnetic resonance (MR) images are often acquired as multi-slice volumes to reduce scan time and motion artifacts while improving signal-to-noise ratio. These slices often are thicker than their in-plane resolution and sometimes are acquired with gaps between slices. Such thick-slice image volumes (possibly with gaps) can impact the accuracy of volumetric analysis and 3D methods. While many super-resolution (SR) methods have been proposed to address thick slices, few have directly addressed the slice gap scenario. Furthermore, data-driven methods are sensitive to domain shift due to the variability of resolution, contrast in acquisition, pathology, and differences in anatomy. In this work, we propose a self-supervised SR technique to address anisotropic MR images with and without slice gap. We compare against competing methods and validate in both signal recovery and downstream task performance on two open-source datasets and show improvements in all respects. Our code publicly available at https://gitlab.com/iacl/smore.

Psychophysical Evaluation of Visual vs. Computer-Aided Detection of Brain Lesions on Magnetic Resonance Images

BACKGROUND

Magnetic resonance imaging (MRI) diagnosis is usually performed by analyzing contrast-weighted images, where pathology is detected once it reached a certain visual threshold. Computer-aided diagnosis (CAD) has been proposed as a way for achieving higher sensitivity to early pathology.

PURPOSE

To compare conventional (i.e., visual) MRI assessment of artificially generated multiple sclerosis (MS) lesions in the brain's white matter to CAD based on a deep neural network.

STUDY TYPE

Prospective.

POPULATION

A total of 25 neuroradiologists (15 males, age 39 ± 9, 9 ± 9.8 years of experience) independently assessed all synthetic lesions.

FIELD STRENGTH/SEQUENCE

A 3.0 T, T₂ -weighted multi-echo spin-echo (MESE) sequence.

ASSESSMENT

MS lesions of varying severity levels were artificially generated in healthy volunteer MRI scans by manipulating T₂ values. Radiologists and a neural network were tasked with detecting these lesions in a series of 48 MR images. Sixteen images presented healthy anatomy and the rest contained a single lesion at eight increasing severity levels (6%, 9%, 12%, 15%, 18%, 21%, 25%, and 30% elevation in T₂ ). True positive (TP) rates, false positive (FP) rates, and odds ratios (ORs) were compared between radiological diagnosis and CAD across the range lesion severity levels.

STATISTICAL TESTS

Diagnostic performance of the two approaches was compared using z-tests on TP rates, FP rates, and the logarithm of ORs across severity levels. A P-value <0.05 was considered statistically significant.

RESULTS

ORs of identifying pathology were significantly higher for CAD vis-à-vis visual inspection for all lesions' severity levels. For a 6% change in T₂ value (lowest severity), radiologists' TP and FP rates were not significantly different (P = 0.12), while the corresponding CAD results remained statistically significant.

DATA CONCLUSION

CAD is capable of detecting the presence or absence of more subtle lesions with greater precision than the representative group of 25 radiologists chosen in this study.

LEVEL OF EVIDENCE

1 TECHNICAL EFFICACY: Stage 3.

Diffusion models in medical imaging: A comprehensive survey

Denoising diffusion models, a class of generative models, have garnered immense interest lately in various deep-learning problems. A diffusion probabilistic model defines a forward diffusion stage where the input data is gradually perturbed over several steps by adding Gaussian noise and then learns to reverse the diffusion process to retrieve the desired noise-free data from noisy data samples. Diffusion models are widely appreciated for their strong mode coverage and quality of the generated samples in spite of their known computational burdens. Capitalizing on the advances in computer vision, the field of medical imaging has also observed a growing interest in diffusion models. With the aim of helping the researcher navigate this profusion, this survey intends to provide a comprehensive overview of diffusion models in the discipline of medical imaging. Specifically, we start with an introduction to the solid theoretical foundation and fundamental concepts behind diffusion models and the three generic diffusion modeling frameworks, namely, diffusion probabilistic models, noise-conditioned score networks, and stochastic differential equations. Then, we provide a systematic taxonomy of diffusion models in the medical domain and propose a multi-perspective categorization based on their application, imaging modality, organ of interest, and algorithms. To this end, we cover extensive applications of diffusion models in the medical domain, including image-to-image translation, reconstruction, registration, classification, segmentation, denoising, 2/3D generation, anomaly detection, and other medically-related challenges. Furthermore, we emphasize the practical use case of some selected approaches, and then we discuss the limitations of the diffusion models in the medical domain and propose several directions to fulfill the demands of this field. Finally, we gather the overviewed studies with their available open-source implementations at our GitHub.1 We aim to update the relevant latest papers within it regularly.

Multiple sclerosis lesion segmentation: revisiting weighting mechanisms for federated learning

Background and introduction

Federated learning (FL) has been widely employed for medical image analysis to facilitate multi-client collaborative learning without sharing raw data. Despite great success, FL's applications remain suboptimal in neuroimage analysis tasks such as lesion segmentation in multiple sclerosis (MS), due to variance in lesion characteristics imparted by different scanners and acquisition parameters.

Methods

In this work, we propose the first FL MS lesion segmentation framework via two effective re-weighting mechanisms. Specifically, a learnable weight is assigned to each local node during the aggregation process, based on its segmentation performance. In addition, the segmentation loss function in each client is also re-weighted according to the lesion volume for the data during training.

Results

The proposed method has been validated on two FL MS segmentation scenarios using public and clinical datasets. Specifically, the case-wise and voxel-wise Dice score of the proposed method under the first public dataset is 65.20 and 74.30, respectively. On the second in-house dataset, the case-wise and voxel-wise Dice score is 53.66, and 62.31, respectively.

Discussions and conclusions

The Comparison experiments on two FL MS segmentation scenarios using public and clinical datasets have demonstrated the effectiveness of the proposed method by significantly outperforming other FL methods. Furthermore, the segmentation performance of FL incorporating our proposed aggregation mechanism can achieve comparable performance to that from centralized training with all the raw data.

Longitudinal Assessment of Multiple Sclerosis Lesion Load With Synthetic Magnetic Resonance Imaging-A Multicenter Validation Study

INTRODUCTION

Double inversion recovery (DIR) has been validated as a sensitive magnetic resonance imaging (MRI) contrast in multiple sclerosis (MS). Deep learning techniques can use basic input data to generate synthetic DIR (synthDIR) images that are on par with their acquired counterparts. As assessment of longitudinal MRI data is paramount in MS diagnostics, our study's purpose is to evaluate the utility of synthDIR longitudinal subtraction imaging for detection of disease progression in a multicenter data set of MS patients.

METHODS

We implemented a previously established generative adversarial network to synthesize DIR from input T1-weighted and fluid-attenuated inversion recovery (FLAIR) sequences for 214 MRI data sets from 74 patients and 5 different centers. One hundred and forty longitudinal subtraction maps of consecutive scans (follow-up scan-preceding scan) were generated for both acquired FLAIR and synthDIR. Two readers, blinded to the image origin, independently quantified newly formed lesions on the FLAIR and synthDIR subtraction maps, grouped into specific locations as outlined in the McDonald criteria.

RESULTS

Both readers detected significantly more newly formed MS-specific lesions in the longitudinal subtractions of synthDIR compared with acquired FLAIR (R1: 3.27 ± 0.60 vs 2.50 ± 0.69 [ P = 0.0016]; R2: 3.31 ± 0.81 vs 2.53 ± 0.72 [ P < 0.0001]). Relative gains in detectability were most pronounced in juxtacortical lesions (36% relative gain in lesion counts-pooled for both readers). In 5% of the scans, synthDIR subtraction maps helped to identify a disease progression missed on FLAIR subtraction maps.

CONCLUSIONS

Generative adversarial networks can generate high-contrast DIR images that may improve the longitudinal follow-up assessment in MS patients compared with standard sequences. By detecting more newly formed MS lesions and increasing the rates of detected disease activity, our methodology promises to improve clinical decision-making.

Spatially informed Bayesian neural network for neurodegenerative diseases classification

Magnetic resonance imaging (MRI) plays an increasingly important role in the diagnosis and prognosis of neurodegenerative diseases. One field of extensive clinical use of MRI is the accurate and automated classification of degenerative disorders. Most of current classification studies either do not mirror medical practice where patients may exhibit early stages of the disease, comorbidities, or atypical variants, or they are not able to produce probabilistic predictions nor account for uncertainty. Also, the spatial heterogeneity of the brain alterations caused by neurodegenerative processes is not usually considered, despite the spatial configuration of the neuronal loss is a characteristic hallmark for each disorder. In this article, we propose a classification technique that incorporates uncertainty and spatial information for distinguishing between healthy subjects and patients from four distinct neurodegenerative diseases: Alzheimer's disease, mild cognitive impairment, Parkinson's disease, and Multiple Sclerosis. We introduce a spatially informed Bayesian neural network (SBNN) that combines a three-dimensional neural network to extract neurodegeneration features from MRI, Bayesian inference to account for uncertainty in diagnosis, and a spatially informed MRI image using hidden Markov random fields to encode cerebral spatial information. The SBNN model demonstrates that classification accuracy increases up to 25% by including a spatially informed MRI scan. Furthermore, the SBNN provides a robust probabilistic diagnosis that resembles clinical decision-making and can account for the heterogeneous medical presentations of neurodegenerative disorders.

Open-source dataset reveals relationship between walking bout duration and fall risk classification performance in persons with multiple sclerosis

Falls are frequent and associated with morbidity in persons with multiple sclerosis (PwMS). Symptoms of MS fluctuate, and standard biannual clinical visits cannot capture these fluctuations. Remote monitoring techniques that leverage wearable sensors have recently emerged as an approach sensitive to disease variability. Previous research has shown that fall risk can be identified from walking data collected by wearable sensors in controlled laboratory conditions however this data may not be generalizable to variable home environments. To investigate fall risk and daily activity performance from remote data, we introduce a new open-source dataset featuring data collected from 38 PwMS, 21 of whom are identified as fallers and 17 as non-fallers based on their six-month fall history. This dataset contains inertial-measurement-unit data from eleven body locations collected in the laboratory, patient-reported surveys and neurological assessments, and two days of free-living sensor data from the chest and right thigh. Six-month (n = 28) and one-year repeat assessment (n = 15) data are also available for some patients. To demonstrate the utility of these data, we explore the use of free-living walking bouts for characterizing fall risk in PwMS, compare these data to those collected in controlled environments, and examine the impact of bout duration on gait parameters and fall risk estimates. Both gait parameters and fall risk classification performance were found to change with bout duration. Deep learning models outperformed feature-based models using home data; the best performance was observed with all bouts for deep-learning and short bouts for feature-based models when evaluating performance on individual bouts. Overall, short duration free-living walking bouts were found to be the least similar to laboratory walking, longer duration free-living walking bouts provided more significant differences between fallers and non-fallers, and an aggregation of all free-living walking bouts yields the best performance in fall risk classification.

Unsupervised brain imaging 3D anomaly detection and segmentation with transformers

Pathological brain appearances may be so heterogeneous as to be intelligible only as anomalies, defined by their deviation from normality rather than any specific set of pathological features. Amongst the hardest tasks in medical imaging, detecting such anomalies requires models of the normal brain that combine compactness with the expressivity of the complex, long-range interactions that characterise its structural organisation. These are requirements transformers have arguably greater potential to satisfy than other current candidate architectures, but their application has been inhibited by their demands on data and computational resources. Here we combine the latent representation of vector quantised variational autoencoders with an ensemble of autoregressive transformers to enable unsupervised anomaly detection and segmentation defined by deviation from healthy brain imaging data, achievable at low computational cost, within relative modest data regimes. We compare our method to current state-of-the-art approaches across a series of experiments with 2D and 3D data involving synthetic and real pathological lesions. On real lesions, we train our models on 15,000 radiologically normal participants from UK Biobank and evaluate performance on four different brain MR datasets with small vessel disease, demyelinating lesions, and tumours. We demonstrate superior anomaly detection performance both image-wise and pixel/voxel-wise, achievable without post-processing. These results draw attention to the potential of transformers in this most challenging of imaging tasks.

Learning disentangled representations in the imaging domain

Disentangled representation learning has been proposed as an approach to learning general representations even in the absence of, or with limited, supervision. A good general representation can be fine-tuned for new target tasks using modest amounts of data, or used directly in unseen domains achieving remarkable performance in the corresponding task. This alleviation of the data and annotation requirements offers tantalising prospects for applications in computer vision and healthcare. In this tutorial paper, we motivate the need for disentangled representations, revisit key concepts, and describe practical building blocks and criteria for learning such representations. We survey applications in medical imaging emphasising choices made in exemplar key works, and then discuss links to computer vision applications. We conclude by presenting limitations, challenges, and opportunities.

Validation of an automatic tool for the rapid measurement of brain atrophy and white matter hyperintensity: QyScore®

OBJECTIVES

QyScore® is an imaging analysis tool certified in Europe (CE marked) and the US (FDA cleared) for the automatic volumetry of grey and white matter (GM and WM respectively), hippocampus (HP), amygdala (AM), and white matter hyperintensity (WMH). Here we compare QyScore® performances with the consensus of expert neuroradiologists.

METHODS

Dice similarity coefficient (DSC) and the relative volume difference (RVD) for GM, WM volumes were calculated on 50 3DT1 images. DSC and the F1 metrics were calculated for WMH on 130 3DT1 and FLAIR images. For each index, we identified thresholds of reliability based on current literature review results. We hypothesized that DSC/F1 scores obtained using QyScore® markers would be higher than the threshold. In contrast, RVD scores would be lower. Regression analysis and Bland-Altman plots were obtained to evaluate QyScore® performance in comparison to the consensus of three expert neuroradiologists.

RESULTS

The lower bound of the DSC/F1 confidence intervals was higher than the threshold for the GM, WM, HP, AM, and WMH, and the higher bounds of the RVD confidence interval were below the threshold for the WM, GM, HP, and AM. QyScore®, compared with the consensus of three expert neuroradiologists, provides reliable performance for the automatic segmentation of the GM and WM volumes, and HP and AM volumes, as well as WMH volumes.

CONCLUSIONS

QyScore® represents a reliable medical device in comparison with the consensus of expert neuroradiologists. Therefore, QyScore® could be implemented in clinical trials and clinical routine to support the diagnosis and longitudinal monitoring of neurological diseases.

KEY POINTS

• QyScore® provides reliable automatic segmentation of brain structures in comparison with the consensus of three expert neuroradiologists. • QyScore® automatic segmentation could be performed on MRI images using different vendors and protocols of acquisition. In addition, the fast segmentation process saves time over manual and semi-automatic methods. • QyScore® could be implemented in clinical trials and clinical routine to support the diagnosis and longitudinal monitoring of neurological diseases.

Automatic segmentation of white matter hyperintensities: validation and comparison with state-of-the-art methods on both Multiple Sclerosis and elderly subjects

•

Automatic segmentation of MS lesions and age-related WMH from 3D T1 and T2-FLAIR.

•

Comparison to consensus show improved performance of WHASA-3D compared to WHASA.

•

WHASA-3D outperforms available state-of-the-art methods with their default settings.

•

WHASA-3D could be a useful tool for clinical practice and clinical trials.

Different types of white matter hyperintensities (WMH) can be observed through MRI in the brain and spinal cord, especially Multiple Sclerosis (MS) lesions for patients suffering from MS and age-related WMH for subjects with cognitive disorders and/or elderly people. To better diagnose and monitor the disease progression, the quantitative evaluation of WMH load has proven to be useful for clinical routine and trials. Since manual delineation for WMH segmentation is highly time-consuming and suffers from intra and inter observer variability, several methods have been proposed to automatically segment either MS lesions or age-related WMH, but none is validated on both WMH types. Here, we aim at proposing the White matter Hyperintensities Automatic Segmentation Algorithm adapted to 3D T2-FLAIR datasets (WHASA-3D), a fast and robust automatic segmentation tool designed to be implemented in clinical practice for the detection of both MS lesions and age-related WMH in the brain, using both 3D T1-weighted and T2-FLAIR images. In order to increase its robustness for MS lesions, WHASA-3D expands the original WHASA method, which relies on the coupling of non-linear diffusion framework and watershed parcellation, where regions considered as WMH are selected based on intensity and location characteristics, and finally refined with geodesic dilation. The previous validation was performed on 2D T2-FLAIR and subjects with cognitive disorders and elderly subjects. 60 subjects from a heterogeneous database of dementia patients, multiple sclerosis patients and elderly subjects with multiple MRI scanners and a wide range of lesion loads were used to evaluate WHASA and WHASA-3D through volume and spatial agreement in comparison with consensus reference segmentations. In addition, a direct comparison on the MS database with six available supervised and unsupervised state-of-the-art WMH segmentation methods (LST-LGA and LPA, Lesion-TOADS, lesionBrain, BIANCA and nicMSlesions) with default and optimised settings (when feasible) was conducted. WHASA-3D confirmed an improved performance with respect to WHASA, achieving a better spatial overlap (Dice) (0.67 vs 0.63), a reduced absolute volume error (AVE) (3.11 vs 6.2 mL) and an increased volume agreement (intraclass correlation coefficient, ICC) (0.96 vs 0.78). Compared to available state-of-the-art algorithms on the MS database, WHASA-3D outperformed both unsupervised and supervised methods when used with their default settings, showing the highest volume agreement (ICC = 0.95) as well as the highest average Dice (0.58). Optimising and/or retraining LST-LGA, BIANCA and nicMSlesions, using a subset of the MS database as training set, resulted in improved performances on the remaining testing set (average Dice: LST-LGA default/optimized = 0.41/0.51, BIANCA default/optimized = 0.22/0.39, nicMSlesions default/optimized = 0.17/0.63, WHASA-3D = 0.58). Evaluation and comparison results suggest that WHASA-3D is a reliable and easy-to-use method for the automated segmentation of white matter hyperintensities, for both MS lesions and age-related WMH. Further validation on larger datasets would be useful to confirm these first findings.

Domain- and task-specific transfer learning for medical segmentation tasks

BACKGROUND AND OBJECTIVES

Transfer learning is a valuable approach to perform medical image segmentation in settings with limited cases available for training convolutional neural networks (CNN). Both the source task and the source domain influence transfer learning performance on a given target medical image segmentation task. This study aims to assess transfer learning-based medical segmentation task performance for various source task and domain combinations.

METHODS

CNNs were pre-trained on classification, segmentation, and self-supervised tasks on two domains: natural images and T1 brain MRI. Next, these CNNs were fine-tuned on three target T1 brain MRI segmentation tasks: stroke lesion, MS lesions, and brain anatomy segmentation. In all experiments, the CNN architecture and transfer learning strategy were the same. The segmentation accuracy on all target tasks was evaluated using the mIOU or Dice coefficients. The detection accuracy was evaluated for the stroke and MS lesion target tasks only.

RESULTS

CNNs pre-trained on a segmentation task on the same domain as the target tasks resulted in higher or similar segmentation accuracy compared to other source task and domain combinations. Pre-training a CNN on ImageNet resulted in a comparable, but not consistently higher lesion detection rate, despite the amount of training data used being 10 times larger.

CONCLUSIONS

This study suggests that optimal transfer learning for medical segmentation is achieved with a similar task and domain for pre-training. As a result, CNNs can be effectively pre-trained on smaller datasets by selecting a source domain and task similar to the target domain and task.

Deep Learning in Large and Multi-Site Structural Brain MR Imaging Datasets

Large, multi-site, heterogeneous brain imaging datasets are increasingly required for the training, validation, and testing of advanced deep learning (DL)-based automated tools, including structural magnetic resonance (MR) image-based diagnostic and treatment monitoring approaches. When assembling a number of smaller datasets to form a larger dataset, understanding the underlying variability between different acquisition and processing protocols across the aggregated dataset (termed “batch effects”) is critical. The presence of variation in the training dataset is important as it more closely reflects the true underlying data distribution and, thus, may enhance the overall generalizability of the tool. However, the impact of batch effects must be carefully evaluated in order to avoid undesirable effects that, for example, may reduce performance measures. Batch effects can result from many sources, including differences in acquisition equipment, imaging technique and parameters, as well as applied processing methodologies. Their impact, both beneficial and adversarial, must be considered when developing tools to ensure that their outputs are related to the proposed clinical or research question (i.e., actual disease-related or pathological changes) and are not simply due to the peculiarities of underlying batch effects in the aggregated dataset. We reviewed applications of DL in structural brain MR imaging that aggregated images from neuroimaging datasets, typically acquired at multiple sites. We examined datasets containing both healthy control participants and patients that were acquired using varying acquisition protocols. First, we discussed issues around Data Access and enumerated the key characteristics of some commonly used publicly available brain datasets. Then we reviewed methods for correcting batch effects by exploring the two main classes of approaches: Data Harmonization that uses data standardization, quality control protocols or other similar algorithms and procedures to explicitly understand and minimize unwanted batch effects; and Domain Adaptation that develops DL tools that implicitly handle the batch effects by using approaches to achieve reliable and robust results. In this narrative review, we highlighted the advantages and disadvantages of both classes of DL approaches, and described key challenges to be addressed in future studies.

Lesion Volume Quantification Using Two Convolutional Neural Networks in MRIs of Multiple Sclerosis Patients

Background: Multiple sclerosis (MS) is a neurologic disease of the central nervous system which affects almost three million people worldwide. MS is characterized by a demyelination process that leads to brain lesions, allowing these affected areas to be visualized with magnetic resonance imaging (MRI). Deep learning techniques, especially computational algorithms based on convolutional neural networks (CNNs), have become a frequently used algorithm that performs feature self-learning and enables segmentation of structures in the image useful for quantitative analysis of MRIs, including quantitative analysis of MS. To obtain quantitative information about lesion volume, it is important to perform proper image preprocessing and accurate segmentation. Therefore, we propose a method for volumetric quantification of lesions on MRIs of MS patients using automatic segmentation of the brain and lesions by two CNNs. Methods: We used CNNs at two different moments: the first to perform brain extraction, and the second for lesion segmentation. This study includes four independent MRI datasets: one for training the brain segmentation models, two for training the lesion segmentation model, and one for testing. Results: The proposed brain detection architecture using binary cross-entropy as the loss function achieved a 0.9786 Dice coefficient, 0.9969 accuracy, 0.9851 precision, 0.9851 sensitivity, and 0.9985 specificity. In the second proposed framework for brain lesion segmentation, we obtained a 0.8893 Dice coefficient, 0.9996 accuracy, 0.9376 precision, 0.8609 sensitivity, and 0.9999 specificity. After quantifying the lesion volume of all patients from the test group using our proposed method, we obtained a mean value of 17,582 mm3. Conclusions: We concluded that the proposed algorithm achieved accurate lesion detection and segmentation with reproducibility corresponding to state-of-the-art software tools and manual segmentation. We believe that this quantification method can add value to treatment monitoring and routine clinical evaluation of MS patients.

Joint MRI T1 Unenhancing and Contrast-enhancing Multiple Sclerosis Lesion Segmentation with Deep Learning in OPERA Trials

Background Deep learning-based segmentation could facilitate rapid and reproducible T1 lesion load assessments, which is crucial for disease management in multiple sclerosis (MS). T1 unenhancing and contrast-enhancing lesions in MS are those that enhance or do not enhance after administration of a gadolinium-based contrast agent at T1-weighted MRI. Purpose To develop deep learning models for automated assessment of T1 unenhancing and contrast-enhancing lesions; to investigate if joint training improved performance; to reproduce a known ocrelizumab treatment response; and to evaluate the association of baseline T1-weighted imaging metrics with clinical outcomes in relapsing MS clinical trials. Materials and Methods Joint and individual deep learning models (U-Nets) were developed retrospectively on multimodal MRI data sets from large multicenter OPERA trials of relapsing MS (August 2011 to May 2015). The joint model included cross-network connections and a combined loss function. Models were trained on OPERA I data sets with three-fold cross-validation. OPERA II data sets were the internal test set. Dice coefficients, lesion true-positive and false-positive rates, and areas under the receiver operating characteristic curve (AUCs) were used to evaluate model performance. Association of baseline imaging metrics with clinical outcomes was assessed with Cox proportional hazards models. Results A total of 796 patients (3030 visits; mean age, 37 years ± 9; 521 women) from the OPERA II trial were evaluated. The joint model achieved a mean Dice coefficient of 0.77 and 0.74, lesion true-positive rate of 0.88 and 0.86, and lesion false-positive rate of 0.04 and 0.19 for T1 contrast-enhancing and T1 unenhancing lesion segmentation, respectively. Joint training improved performance for smaller T1 contrast-enhancing lesions (≤0.06 mL; individual training AUC: 0.75; joint training AUC: 0.87; P < .001). A significant ocrelizumab treatment effect (P < .001) was seen in reducing the mean number of T1 contrast-enhancing lesions at 24, 48, and 96 weeks (manual assessment at 24 weeks: 10 lesions in 366 patients with ocrelizumab, 141 lesions in 355 patients with interferon, 93% reduction; manual assessment at 48 weeks: six lesions in 355 patients with ocrelizumab, 150 lesions in 317 patients with interferon, 96% reduction; manual assessment at 96 weeks: five lesions in 340 patients with ocrelizumab, 157 lesions in 294 patients with interferon, 97% reduction; joint model assessment at 24 weeks: 19 lesions in 365 patients with ocrelizumab, 128 lesions in 354 patients with interferon, 86% reduction; joint model assessment at 48 weeks: 14 lesions in 355 patients with ocrelizumab, 121 lesions in 317 patients with interferon, 90% reduction; joint model assessment at 96 weeks: 10 lesions in 340 patients with ocrelizumab, 144 lesions in 294 patients with interferon, 94% reduction) and the mean number of new T1 unenhancing lesions across all follow-up examinations (manual assessment: 504 lesions in 1060 visits for ocrelizumab-treated patients, 1438 lesions in 965 visits for interferon-treated patients, 68% reduction; joint model assessment: 205 lesions in 1053 visits for ocrelizumab-treated patients, 661 lesions in 957 visits for interferon-treated patients, 78% reduction). Baseline T1 unenhancing total lesion volume was associated with clinical outcomes (manual hazard ratio [HR]: 1.12, P = .02; joint model HR: 1.11, P = .03). Conclusion Joint architecture and training improved segmentation of MRI T1 contrast-enhancing multiple sclerosis lesions, and both deep learning models had sufficiently high performance to detect an ocrelizumab treatment response consistent with manual assessments. ClinicalTrials.gov: NCT01247324 and NCT01412333 © RSNA, 2021 Online supplemental material is available for this article. See also the editorial by Talbott in this issue.

Using Convolutional Encoder Networks to Determine the Optimal Magnetic Resonance Image for the Automatic Segmentation of Multiple Sclerosis

Multiple Sclerosis (MS) is a neuroinflammatory demyelinating disease that affects over 2,000,000 individuals worldwide. It is characterized by white matter lesions that are identified through the segmentation of magnetic resonance images (MRIs). Manual segmentation is very time-intensive because radiologists spend a great amount of time labeling T1-weighted, T2-weighted, and FLAIR MRIs. In response, deep learning models have been created to reduce segmentation time by automatically detecting lesions. These models often use individual MRI sequences as well as combinations, such as FLAIR2, which is the multiplication of FLAIR and T2 sequences. Unlike many other studies, this seeks to determine an optimal MRI sequence, thus reducing even more time by not having to obtain other MRI sequences. With this consideration in mind, four Convolutional Encoder Networks (CENs) with different network architectures (U-Net, U-Net++, Linknet, and Feature Pyramid Network) were used to ensure that the optimal MRI applies to a wide array of deep learning models. Each model had used a pretrained ResNeXt-50 encoder in order to conserve memory and to train faster. Training and testing had been performed using two public datasets with 30 and 15 patients. Fisher’s exact test was used to evaluate statistical significance, and the automatic segmentation times were compiled for the top two models. This work determined that FLAIR is the optimal sequence based on Dice Similarity Coefficient (DSC) and Intersection over Union (IoU). By using FLAIR, the U-Net++ with the ResNeXt-50 achieved a high DSC of 0.7159.

Enhanced Magnetic Resonance Image Synthesis with Contrast-Aware Generative Adversarial Networks

A magnetic resonance imaging (MRI) exam typically consists of the acquisition of multiple MR pulse sequences, which are required for a reliable diagnosis. With the rise of generative deep learning models, approaches for the synthesis of MR images are developed to either synthesize additional MR contrasts, generate synthetic data, or augment existing data for AI training. While current generative approaches allow only the synthesis of specific sets of MR contrasts, we developed a method to generate synthetic MR images with adjustable image contrast. Therefore, we trained a generative adversarial network (GAN) with a separate auxiliary classifier (AC) network to generate synthetic MR knee images conditioned on various acquisition parameters (repetition time, echo time, and image orientation). The AC determined the repetition time with a mean absolute error (MAE) of 239.6 ms, the echo time with an MAE of 1.6 ms, and the image orientation with an accuracy of 100%. Therefore, it can properly condition the generator network during training. Moreover, in a visual Turing test, two experts mislabeled 40.5% of real and synthetic MR images, demonstrating that the image quality of the generated synthetic and real MR images is comparable. This work can support radiologists and technologists during the parameterization of MR sequences by previewing the yielded MR contrast, can serve as a valuable tool for radiology training, and can be used for customized data generation to support AI training.

Autoencoders for unsupervised anomaly segmentation in brain MR images: A comparative study

Deep unsupervised representation learning has recently led to new approaches in the field of Unsupervised Anomaly Detection (UAD) in brain MRI. The main principle behind these works is to learn a model of normal anatomy by learning to compress and recover healthy data. This allows to spot abnormal structures from erroneous recoveries of compressed, potentially anomalous samples. The concept is of great interest to the medical image analysis community as it i) relieves from the need of vast amounts of manually segmented training data-a necessity for and pitfall of current supervised Deep Learning-and ii) theoretically allows to detect arbitrary, even rare pathologies which supervised approaches might fail to find. To date, the experimental design of most works hinders a valid comparison, because i) they are evaluated against different datasets and different pathologies, ii) use different image resolutions and iii) different model architectures with varying complexity. The intent of this work is to establish comparability among recent methods by utilizing a single architecture, a single resolution and the same dataset(s). Besides providing a ranking of the methods, we also try to answer questions like i) how many healthy training subjects are needed to model normality and ii) if the reviewed approaches are also sensitive to domain shift. Further, we identify open challenges and provide suggestions for future community efforts and research directions.

Deep-Learning-Based Segmentation and Localization of White Matter Hyperintensities on Magnetic Resonance Images

White matter magnetic resonance hyperintensities of presumed vascular origin, which could be widely observed in elderly people, and has significant importance in multiple neurological studies. Quantitative measurement usually relies heavily on manual or semi-automatic delineation and intuitive localization, which is time-consuming and observer-dependent. Current automatic quantification methods focus mainly on the segmentation, but the spatial distribution of lesions plays a vital role in clinical diagnosis. In this study, we implemented four segmentation algorithms and compared the performances quantitatively and qualitatively on two open-access datasets. The location-specific analysis was conducted sequentially on 213 clinical patients with cerebral ischemia and lacune. The experimental results suggest that our deep-learning-based model has the potential to be integrated into the clinical workflow.

Limited One-time Sampling Irregularity Map (LOTS-IM) for Automatic Unsupervised Assessment of White Matter Hyperintensities and Multiple Sclerosis Lesions in Structural Brain Magnetic Resonance Images

We present the application of limited one-time sampling irregularity map (LOTS-IM): a fully automatic unsupervised approach to extract brain tissue irregularities in magnetic resonance images (MRI), for quantitatively assessing white matter hyperintensities (WMH) of presumed vascular origin, and multiple sclerosis (MS) lesions and their progression. LOTS-IM generates an irregularity map (IM) that represents all voxels as irregularity values with respect to the ones considered "normal". Unlike probability values, IM represents both regular and irregular regions in the brain based on the original MRI's texture information. We evaluated and compared the use of IM for WMH and MS lesions segmentation on T2-FLAIR MRI with the state-of-the-art unsupervised lesions' segmentation method, Lesion Growth Algorithm from the public toolbox Lesion Segmentation Toolbox (LST-LGA), with several well established conventional supervised machine learning schemes and with state-of-the-art supervised deep learning methods for WMH segmentation. In our experiments, LOTS-IM outperformed unsupervised method LST-LGA on WMH segmentation, both in performance and processing speed, thanks to the limited one-time sampling scheme and its implementation on GPU. Our method also outperformed supervised conventional machine learning algorithms (i.e., support vector machine (SVM) and random forest (RF)) and deep learning algorithms (i.e., deep Boltzmann machine (DBM) and convolutional encoder network (CEN)), while yielding comparable results to the convolutional neural network schemes that rank top of the algorithms developed up to date for this purpose (i.e., UResNet and UNet). LOTS-IM also performed well on MS lesions segmentation, performing similar to LST-LGA. On the other hand, the high sensitivity of IM on depicting signal change deems suitable for assessing MS progression, although care must be taken with signal changes not reflective of a true pathology.

Practical Priors for Bayesian Inference of Latent Biomarkers

Latent biomarkers are quantities that strongly relate to patient's disease diagnosis and prognosis, but are difficult to measure or even not directly observable. The objective of this study was to develop, analyze and validate new priors for Bayesian inference of such biomarkers. Theoretical analysis revealed a relationship between the estimates inferred from the model and the true values of measured quantities, and the impact of the priors. This led to a new prior encoding scheme that incorporates objectively measurable domain knowledge, i.e. by performing two measurements with a reference method, which imply scale of the prior distribution. Second, priors on parameters of systematic error are non-informative, which enables biomarker estimation from a set of different quantities. Analysis showed that the volume of nucleus basalis of Meynert, which is reduced in early stages of Alzheimer's dementia and Parkinson's disease, is inter-related and could be inferred from compartmental brain volume measurements performed on routine clinical MR scans. Another experiment showed that total lesion load, associated to future disability progression in multiple sclerosis patients, could be inferred from lesion volume measurements based on multiple automated MR scan segmentations. Besides, figures of merit derived from the estimates could, without comparing against reference gold standard segmentations, identify the best performing lesion segmentation method. The proposed new priors substantially simplify the application of Bayesian inference for latent biomarkers and thus open an avenue for clinical implementation of new biomarkers, which may ultimately advance the evidence-based medicine.

High-dimensional detection of imaging response to treatment in multiple sclerosis

Changes on brain imaging may precede clinical manifestations or disclose disease progression opaque to conventional clinical measures. Where, as in multiple sclerosis, the pathological process has a complex anatomical distribution, such changes are not easily detected by low-dimensional models in common use. This hinders our ability to detect treatment effects, both in the management of individual patients and in interventional trials. Here we compared the ability of conventional models to detect an imaging response to treatment against high-dimensional models incorporating a wide multiplicity of imaging factors. We used fully-automated image analysis to extract 144 regional, longitudinal trajectories of pre- and post- treatment changes in brain volume and disconnection in a cohort of 124 natalizumab-treated patients. Low- and high-dimensional models of the relationship between treatment and the trajectories of change were built and evaluated with machine learning, quantifying performance with receiver operating characteristic curves. Simulations of randomised controlled trials enrolling varying numbers of patients were used to quantify the impact of dimensionality on statistical efficiency. Compared to existing methods, high-dimensional models were superior in treatment response detection (area under the receiver operating characteristic curve = 0.890 [95% CI = 0.885–0.895] vs. 0.686 [95% CI = 0.679–0.693], P < 0.01]) and in statistical efficiency (achieved statistical power = 0.806 [95% CI = 0.698–0.872] vs. 0.508 [95% CI = 0.403–0.593] with number of patients enrolled = 50, at α = 0.01). High-dimensional models based on routine, clinical imaging can substantially enhance the detection of the imaging response to treatment in multiple sclerosis, potentially enabling more accurate individual prediction and greater statistical efficiency of randomised controlled trials.

Predicting conversion from clinically isolated syndrome to multiple sclerosis-An imaging-based machine learning approach

Magnetic resonance imaging (MRI) scans play a pivotal role in the evaluation of patients presenting with a clinically isolated syndrome (CIS), as these may depict brain lesions suggestive of an inflammatory cause. We hypothesized that it is possible to predict the conversion from CIS to multiple sclerosis (MS) based on the baseline MRI scan by studying image features of these lesions.

We analyzed 84 patients diagnosed with CIS from a prospective observational single center cohort. The patients were followed up for at least three years. Conversion to MS was defined according to the 2010 McDonald criteria. Brain lesions were segmented based on 3D FLAIR and 3D T1 images. We generated brain lesion masks by a computer assisted manual segmentation. We also generated a set of automated segmentations using the Lesion Segmentation Toolbox for SPM to assess the influence of different segmentation methods. Shape and brightness features were automatically calculated from the segmented masks and used as input data to train an oblique random forest classifier. Prediction accuracies of the resulting model were validated through a three-fold cross-validation.

Conversion from CIS to MS occurred in 66 of 84 patients (79%). The conversion or non-conversion was predicted correctly in 71 patients based on shape features derived from the computer assisted manual segmentation masks (84.5% accuracy). This predictor was more accurate than predicting conversion using dissemination in space at baseline according to the 2010 McDonald criteria (75% accuracy). While shape features strongly contributed to the accuracy of the predictor, including intensity features did not further improve performance.

As patients who convert to definite MS benefit from early treatment, an early classification model is highly desirable. Our study shows that shape parameters of lesions can contribute to predicting the future course of CIS patients more accurately.

•

A random forest tool can help to identify patients who convert from clinical isolated syndrome into multiple sclerosis (MS).

•

The classifier is driven by shape features of lesions in the first MR scan.

•

The found shape features reflect the typical ovoid growth of MS lesions.

Locally adaptive magnetic resonance intensity models for unsupervised segmentation of multiple sclerosis lesions

Multiple sclerosis (MS) is a neurological disease characterized by focal lesions and morphological changes in the brain captured on magnetic resonance (MR) images. However, extraction of the corresponding imaging markers requires accurate segmentation of normal-appearing brain structures (NABS) and the lesions in MR images. On MR images of healthy brains, the NABS can be accurately captured by MR intensity mixture models, which, in combination with regularization techniques, such as in Markov random field (MRF) models, are known to give reliable NABS segmentation. However, on MR images that also contain abnormalities such as MS lesions, obtaining an accurate and reliable estimate of NABS intensity models is a challenge. We propose a method for automated segmentation of normal-appearing and abnormal structures in brain MR images that is based on a locally adaptive NABS model, a robust model parameters estimation method, and an MRF-based segmentation framework. Experiments on multisequence brain MR images of 30 MS patients show that, compared to whole-brain MR intensity model and compared to four popular unsupervised lesion segmentation methods, the proposed method increases the accuracy of MS lesion segmentation.